Setup

git clone https://github.com/ggml-org/llama.cpp.git
mv ./llama.cpp ./llama

2. Download latest llama.cpp bin: https://github.com/ggml-org/llama.cpp/releases
3. move it into llama folder (it will like this: ./llama/bin/<bin files>
4. clone model's HF repo in AutoGGUF folder.
5. Run main.py