Facility for configuring GitHub Copilot file system directory permissions outside the workspace

[garretwilson]

As described in #282209, I have provided GitHub Copilot instructions that allow the LLM to find source code for my GlobalMentor libraries so that it can decipher which utilities are available and how to use them. Those directories are outside of the workspace path

But this is a slippery slope. I'm leery of the LLM suddenly deciding it wants to peek inside my personal files, in other folders I didn't mention.

We need a way to specify which directory trees in the file system are in-limits, off-limits, and read-only to the LLM. This would apply (especially!) to the terminal as well.

It is useless having individual restricted workspaces, or even disabling GitHub Copilot in some workspace, if the LLM has free reign of the entire file system once it's in unleashed in a single workspace!

[garretwilson]

The LLM even mentioned this loophole:

My read_file tool blocked direct file reads from outside the workspace (which is expected), but terminal commands like cat, grep, and sed worked fine for reading.

[garretwilson]

I actually consider this a significant security issue that many people may not be aware of. How many days is it appropriate that I wait for a fix or some sort of mitigation from GitHub before I report this more widely? This is a sincere question; please let me know.

[garretwilson]

While it's difficult to prevent this sort of thing altogether, this could be made orders of magnitude safer with a few simple tweaks:

1. Create the facility for configuring allowed directories as requested here.
2. Parse the requested command line and normalize it as explained in GitHub Copilot auto-approve not recognizing commands because of CLI option order. #281779.
3. Normalize any paths, including resolving to the current working directory.
4. Compare the paths against the configured directories.
5. Check the command to see if it read-only (the most common-commands are easy) or destructive.
6. Confirm that the read/write request matches the read/write permissions of that path.

None of this is difficult in the least. It has several corner cases (path manipulation is tricky); but it's nothing that hasn't been implemented 100 times, and some well-written unit tests can easily validate correctness. It would address the most common use cases and prevent all but the most conniving tricks by the LLM.

[Tyriar]

We have a feature called content exclusions that does exactly this, the problem is it doesn't work for certain tools such as the terminal tool. Right now the only path detection we do is for file redirection. I don't think we can ever achieve full content exclusion support this reliably without some sort of sandboxing (which we are exploring in #280236), so if that's important to you (or your company) you can disable the terminal tool, which of course will limit chat's capabilities quite a bit.

There is also the chat.tools.edits.autoApprove setting which includes a bunch of globs with rules, these don't work yet in the terminal but will soon after #274013 is done. These will only apply to those paths we detect though, it's not clear to me how far we're going to push path in command line detection.

[garretwilson]

I don't think we can ever achieve full content exclusion support this reliably without some sort of sandboxing …

Perhaps, but that majorly misses the point of prohibiting the most common, obvious things that the an LLM might accidentally do (or half-heartedly try to do on purpose).

Look, five years from now if the LLM decides it really wants to screw things up, the LLMs then will be so advanced it will probably be close to impossible for us to prevent. That's not what this ticket is about. It's about preventing the 99% of obvious access to places we don't want it to access. Things that I could easily eyeball if I stopped and looked at every cat or grep going by, but if I did that, it would make a large part of the tool worthless.

These will only apply to those paths we detect though, it's not clear to me how far we're going to push path in command line detection.

Again, don't lose sight of the order-of-magnitude better 99.9% or whatever prevention of access. Don't fall into the trap of, "we can't prevent every single possible bad access, so let's not even try".

The steps I outlined are super easy and straightforward. They probably take care of 99% of the cases. They were just off the top of my head; a little more thinking and they would probably take care of 99.9% percent or whatever it is.

The LLM wants to grep something somewhere else. We can parse a command line request. We know what grep means. We know where the paths are in a grep command, and even if we didn't, we would recognize that something like /usr/local/super-private is a path. We know how to normalize paths; it's been re-implemented 100 times. We know how to resolve a path to a working directory, etc.

The 99.99% case of catching known commands with obvious paths is straightforward and easy. What is the problem with implementing that immediately if it brings an order of magnitude increase in security?

P.S. Sorry if I'm talking to you as if you're an LLM. I've been talking to Claude all day! 😅

[garretwilson]

… or heck, if nothing else, just expose a isAllowed(llm:Llm, tool:Tool, parameters:ToolParameters) method/plugin, and I'll implement my own checks (those I've already outlined) to check if each tool request is allowed.

Now that I think of it, this is something we need anyway. Should I create a separate feature request ticket for it?