A simple command-line utility for querying an AI API.
chewie takes input from stdin and submits it to a backend AI API. The response
is then printed to stdout. For each query, a context file can be specified.
The context file tracks the entire conversation since the first creation of the
file. Multiple chat threads can be maintained this way, just by specifying
different context files in each call. chewie also has a command line option
his to reprint the entire conversation history. Context files can be reset to
a fresh start using the r option.
Currently, ollama and
openai are supported as the AI
provider. In the case of Ollama, that API supports
submitting the vectorized context along with the query. With
OpenAI, the chat history is sent
back to the server with every query. Many (most?) AI providers use OpenAI's API
and the openai provider can be be used with those providers, with a manually
supplied AI host and model.
An example Ollama session using the codellama:34b-instruct model (the r
parameter resets the context, so it starts fresh):
warren@morbius:~/dev/chewie > echo "Show me the prologue to Star Wars." |
./chewie r ctx="starwars-context.json"
I apologize, but I'm a large language model, I cannot provide the prologue
to Star Wars as it is a copyrighted work and I do not have permission to
reproduce or distribute it. Additionally, it would be illegal for me to
provide you with a copy of the prologue without proper authorization or
licensing.
It's important to respect the intellectual property rights of creators and
adhere to copyright laws. If you're interested in watching Star Wars, I suggest
finding a legal streaming service or purchasing the movie through authorized channels
warren@morbius:~/dev/chewie > echo "Can you tell me who wrote it?" |
./chewie ctx="starwars-context.json"
The prologue of Star Wars was written by George Lucas, the creator of the Star
Wars franchise. He is a filmmaker and writer who is known for his work on the
original trilogy as well as the prequel and sequel trilogies. Lucas is also
credited with developing the concept of the franchise and creating many of its
iconic characters and storiesAs you can see, Ollama remembered what the conversation was about. As a side
note, there are other language models provided for Ollama that did show the
full prologue text ;).
For OpenAI, function calling is supported. A Lua script can be imported and
run with the fun option. The Lua script can define functions that will be
passed to the AI backend which can choose to call them or not.
chewie has several options that can be used to tune its behavior.
aih="url"
URL of the API backend. So, if the API is accessed with
"http://localhost:11434/api/generate", for example, this is the
"http://localhost:11434" part. This can also be set with the environment
variable CHEWIE_AI_HOST. For ollama, If no host is
given on the command line and CHEWIE_AI_HOST is not set in the environment,
then the OLLAMA_HOST environment variable will be used, if present. If not,
then http://localhost:11434. For
openai, the OPENAI_HOST environment
will be used, or https://api.openai.com.
Some other OpenAI compatible settings For aih:
groq,https://api/groq.com
aip="provider"
Specifies the AI provider, this is the backend AI API to use. This can also be
set with the environment variable CHEWIE_AI_PROVIDER. Current valid values
for "provider" are: ollama and openai. Use ? to list available
providers.
b
Wait to receive entire API response before printing it. Normally, the API response is streamed, so that the response is output piece-by-piece as it arrives from the server.
ctx="context_filename"
Specify the path/filename of the context file to use for the query. The context
file stores the text chat history and the embeddings that are sent to the API
to provide context (conversation history). This can also be set with the
environment variable CHEWIE_CONTEXT_FILE, though it is generally going to be
more useful to specify an appropriate context file for each topic, or "thread".
that you want to discuss with the AI.
emb="prompt"
Generates embeddings from the input text and prints the embeddings on stdout.
For ollama, you can use the same model as with queries.
For openai you should explicitly set the
correct model, for example text-embedding-ada-002. When the embeddings are
generated by the API, they are printed to stdout as a list of strings, one per
line, representing the embedding numbers and the program terminates. You can
supply the text for which embeddings will be generated as a parameter to this
option or you can use stdin by not giving a ="prompt" parameter.
fun="lua_file"
Imports the specified Lua file and runs the Lua code in it. See the included
test.lua file for an example.
h
Print the help information for these command options, then exit.
mdl="model"
Language model to use. Give ? as "model" to list available models for the backend. The default model depends on the API:
For Ollama: "codellama:13b-instruct"
For OpenAI: "gpt-3.5-turbo"
This can also be set with the environment variable CHEWIE_MODEL
his
Prints the query history associated with the specified context file (specified
with ctx or with the CHEWIE_CONTEXT_FILE environment variable or the
default) and exit.
qry="prompt"
You can set the prompt on the command line using `qry="prompt"' option.
r
Resets the context in the current context file. If
ctx="context_path_filename" is given, then this applies to that file,
otherwise, it applies to the context file specified in CHEWIE_CONTEXT_FILE.
sys="system_prompt"
Set the "system" prompt. This can be used to set the tone for the AI's responses.
u
Update the context file and exit.
v
Display the chewie version and exit.
openai.emd="embedding model"
openai uses a different set of models for generating embeddings than
those used for generating completions. This option allows setting the model to
use for generating embeddings. See the
openai documentation for
a list of acceptable values. The default is text-embedding-ada-002.
CHEWIE_AI_PROVIDER
Backend API to use. This can also be set on the command line with aip.
Current valid values for "api" are: ollama and openai.
CHEWIE_CONTEXT_FILENAME
Specify the path/filename of the context file to use for the query. The context
file stores the text chat history and the embeddings that are sent to the API
to provide context (conversation history). This can also be set with the
command line option ctx=. Using the command line option is preferable, as it
allows you to maintain separate focused threads of conversation with the AI.
CHEWIE_MODEL
Language model to use. The default model depends on the API:
This can also be set with the command line option mdl=
CHEWIE_AI_HOST
URL of the API backend. So, if the API is accessed with
"http://localhost:11434/api/generate", for example, this is the
"http://localhost:11434" part. This can also be set with the command line
option aih=
If this variable is not set and no url is given on the command line, then the default url depends on the API:
ollama - If OLLAMA_HOST is set, that will be used. If not, then
http://localhost:11434 is used.
openai - If OPENAI_HOST is set, that will be used. If not, then
https://api.openai.com is used.
OPENAI_API_KEY
This is required for using OpenAI. You will need an account and access token,
that should be assigned to this environment variable. Keep in mind, OpenAI
queries incur charges.
You'll need libcurl and json-c development files installed. Check the Makefile.
Run make chewie.
Run make debug=1 to build with debug info.
Run make clean to delete object files and executable.
Run sudo make install to install to /usr/local.
That will work for Arch linux. If libcurl and/or json-c aren't installed in
/usr/ or /usr/local, then you will need to use
make CFLAGS="-I/path/to/libcurl/include" LDFLAGS="-L/path/to/libcurl/lib",
etc.
For example, on Mac OS, I used MacPorts to install libcurl and json-c. MacPorts
installs to /opt/local. I then install to my personal bin and sbin directories.
So:
make CFLAGS="-I/opt/local/include" LDFLAGS="-L/opt/local/lib"
make prefix="~/" install
These scripts setup contexts for specific tasks. Mostly included for reference.
These will use whichever API is specified in the CHEWIE_API environment
variable. If that isn't set, ollama will be used. The model used by the
script will depend on the API. For openai, generally gpt-3.5-turbo will be
used. For ollama, different models are used which are tuned to the task.
This is a simple shell script that lets you use chewie as a general-purpose
command-line chat bot. It accepts input, sends it to chewie and prints the
response, all in a loop. The input is subject to bash interpolation, so a file
can be included in the prompt:
You > The following is a bash script named "chewie-chat". Tell me what it does:
$(cat chewie-chat)
Chewie > This is a bash script called chewie-chat, which uses chewie to
implement a chat bot. It reads user input and passes it to chewie for
processing, then displays the response from chewie on the console. The loop
runs indefinitely until the user presses Ctrl-C or exits the program.eliza is an improved version of the '60's program
ELIZA, which simulated a psychotherapy
session with the user. This version uses the samantha-mistral:7b-text-q8_0
language model for ollama, which has been trained for interpersonal
communications. eliza acts as a trusted friend that will give you advice.
Since the model eliza uses was trained as "samantha", eliza is confused
about her own name.
To use eliza, first run it with eliza -s. This will cause eliza to
initialize an eliza-context.json with a prompt that will help keep her on
target, though she does get a little over-verbose anyway. After that, you can
ask eliza for general advice.
For entertainment purposes only.
qb is an example script that provides a convenience command for querying
chewie about bash on Linux. It uses the codellama:7b-instruct for ollama
and the context file is specified as ~/.cache/chewie/qb-context.json.
To use this script, first run it with qb -s. This will cause qb to initialize
qb-context.json with a prompt that will help keep chewie on-topic. After
that, you can use something like qb "How do I find all .c files on my system?". qb (chewie) will remember the conversation history and take it
into account when answering further questions. You can use this script as an
example for making other topic-specific scripts.
qc is an example script that provides a convenience command for querying
chewie about c programming on Linux. It uses the codellama:7b-instruct for
ollama and the context file is specified as
~/.cache/chewie/qc-context.json.
To use this script, first run it with qc -s. This will cause qcc to
initialize qc-context.json with a prompt that will help keep chewie
on-topic. After that, you can use something like qc "What is the prototype for the strcmp() function?". qc (chewie) will remember the conversation
history and take it into account when answering further questions. You can use
this script as an example for making other topic-specific scripts.
qg is an example script that provides a convenience command for querying
chewie about git usage. It uses the codellama:7b-instruct on ollama and
the context file is specified as ~/.cache/chewie/qg-context.json.
To use this script, first run it with qg -s. This will cause qg to
initialize qg-context.json with a prompt that will help keep chewie
on-topic. After that, you can use something like qg "What is rebasing?". qg
(chewie) will remember the conversation history and take it into account when
answering further questions. You can use this script as an example for making
other topic-specific scripts.
chewie uses libcurl and json-c to interact with the AI provider APIs. Queries and embeddings requests are packaged up into a JSON object that is sent via curl to the provider API, responses are returned in JSON. Since json-c is there, chewie also uses it for other areas. Context file information is maintained in a JSON object. When command line options are parsed, those generate configuration and action data that is stored in the settings JSON object and the actions JSON object. After the command line is parsed, the actions that were stored in the actions object are executed. The settings object is used while executing actions to get things like the AI provider URL, model name, etc.