Has anyone else tried sandboxing KoboldCPP?

[TFWol]

Since AI stuff is pretty popular I've been messing with it sandboxed since I'm unsure about the various models and what they could possibly do to a system. In fact, I refuse to risk running them unsandboxed despite how hilarious I find them.

I've had success running most projects, but I can't figure out what settings would make this one work without it breaking:
https://github.com/LostRuins/koboldcpp
Examples from: https://github.com/LostRuins/koboldcpp/releases/download/v1.33/koboldcpp.exe

Running in via CPU only will work (not ideal since that's super slow, as expected)

When attempting to offload it to GPU, it will throw a Windows Error 0xc000001d

I'm wondering if anyone has played around with the same stuff. I'm leaning heavily on 'No', but I thought I'd ask.

[DavidXanatos]

have you tried a green box?

[TFWol]

I just gave it a quick stab. It won't crash this time, but it can't find the GPU to offload to.

For reference, I used .\Koboldcpp.exe --noavx2 --debugmode --threads 5 --usecublas --gpulayers 100

The expected entry should state it found a GPU like this example output I found at their repo:

ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 3090
llama_model_load_internal: using CUDA for GPU acceleration

It may just be a glitch with their program since I'm using a weird combo of high-end GPU and a cpu without AVX2. I'll probably open an Issue with them and see what they say.

[TFWol]

Found the problem. Looks like I was on the right track about AVX2. It's nothing to do with Sandboxie.