Shell-GPT with local deepseek-coder-v2 working config

[JohnRDOrazio]

After the Deepseek coder was announced promising amazing performance and quality in code generation, I was curious to try it out locally. I found this youtube video that explained a little bit of how to install deepseek-coder-v2 with shell-gpt and ollama / litellm. However in the video the author mentioned that he was having trouble getting it to work correctly locally, and in the end just resorted to using the platform API.

I have finally been able to do a bit more testing and have succeeded in getting a local deepseek-coder-v2 to work with Shell GPT, here are the correct configurations:

CHAT_CACHE_PATH=/tmp/chat_cache
CACHE_PATH=/tmp/cache
CHAT_CACHE_LENGTH=100
CACHE_LENGTH=100
REQUEST_TIMEOUT=60
DEFAULT_MODEL=ollama/deepseek-coder-v2
DEFAULT_COLOR=magenta
ROLE_STORAGE_PATH=~/.config/shell_gpt/roles
DEFAULT_EXECUTE_SHELL_CMD=false
DISABLE_STREAMING=false
CODE_THEME=dracula
OPENAI_FUNCTIONS_PATH=~/.config/shell_gpt/functions
OPENAI_USE_FUNCTIONS=true
SHOW_FUNCTIONS_OUTPUT=false
API_BASE_URL=http://127.0.0.1:11434
PRETTIFY_MARKDOWN=true
USE_LITELLM=true
SHELL_INTERACTION=true
OS_NAME=auto
SHELL_NAME=auto
OPENAI_API_KEY=_deepseek-api-key-here_

Make sure to:

1. set DEFAULT_MODEL to ollama/deepseek-coder-v2
2. set API_BASE_URL to http://127.0.0.1:11434 (or the IP:PORT your ollama instance is running on)
3. set USE_LITELLM to true
4. paste your Deepseek API key as the value of the OPENAI_API_KEY environment variable

You should now be able to run sgpt hello and get an answer from your local deepseek-coder-v2!
https://www.johnromanodorazio.com/en/2024/11/how-to-install-shell-gpt-with-deepseek-coder-v2/