docs: Added initial documentation for how to configure and launch llama stack #3450
Conversation
omaryashraf5
requested review from
ashwinb,
bbrowning,
ehhuang,
hardikjshah,
leseb,
mattf,
raghotham,
reluctantfuturist,
slekkala1,
terrytangyuan and
yanxi0830
as code owners
meta-cla
bot
added
the
CLA Signed
| -v ~/.llama:/root/.llama \ | ||
| --network=host \ | ||
| llamastack/distribution-starter \ | ||
| --env OLLAMA_URL=http://localhost:11434 |
There was a problem hiding this comment.
--env here and docker -e later. we provide multiple ways to do the same thing. let's stick to -e here.
There was a problem hiding this comment.
Thanks, @mattf , I ran the command again with -e and it worked for me.
| pip install llama-stack | ||
| ``` | ||
|
|
||
| 2. **Install Provider Dependencies** (as needed): |
There was a problem hiding this comment.
this is nearly impossible for a user to get right, which is why we do it in llama stack build
this step should be removed.
| "model": "llama3.1:8b", | ||
| "messages": [{"role": "user", "content": "Hello!"}] | ||
| }' | ||
| ``` |
There was a problem hiding this comment.
verify w/ client -
uv run --with llama-stack-client llama-stack-client providers list
There was a problem hiding this comment.
Thanks, @mattf Updated that section.
|
|
||
| ### Common Configuration Issues | ||
|
|
||
| #### Files Provider Missing |
There was a problem hiding this comment.
I will remove it for now but I will verify.
| llamastack/distribution-starter | ||
| ``` | ||
|
|
||
| 2. **Module Not Found Errors**: |
There was a problem hiding this comment.
@mattf This applies to cases where users are missing a few dependencies, such as Faiss. However, since these should be included in the Llama stack build and we discourage users from installing individual packages, I could remove this section.
|
|
||
| 1. **Install Llama Stack**: | ||
| ```bash | ||
| pip install llama-stack |
There was a problem hiding this comment.
The current quick start guide uses uv but misses this step. I dont know if its my place to suggest anything, but uv equivalent would be uv init in a new folder and then uv add llama-stack Every other command can then be run with uv run infront of it.
There was a problem hiding this comment.
good idea. there's a debate around if uv is a dev tool or an end-user tool. imho, anyone who uses pip should be using uv pip. uv run --with llama-stack ... makes things pretty easy too.
There was a problem hiding this comment.
I will add steps for using uv as well.
- Resolved file location conflicts: moved docs to new structure - Updated documentation to follow new docs/docs/ structure - Removed outdated docs/source/getting_started/index.md
- Added link to configuring_and_launching_llama_stack.md in Quick Links section - Provides users easy access to detailed Docker and manual setup instructions - Fills important gap between Quick Start and Contributing guides
|
@omaryashraf5 what's the status on this one? Thanks |
|
@leseb I have added all the change requests from my end. I just need to resolve the recent conflicts. |
|
|
||
| This guide walks you through the two primary methods for setting up and running Llama Stack: using Docker containers and configuring the server manually. | ||
|
|
||
| ## Method 1: Using the Starter Docker Container |
There was a problem hiding this comment.
let's structure from the approach that needs the least infrastructure & knowledge to the most -
Prerequesites
- Ollama running at http://localhost:11434 (include link to ollama getting started docs)
export OLLAMA_URL=http://localhost:11434
Using llama stack CLI
pip install llama-stack
llama stack build --distro starter --image-type venv --run
Using docker or podman
...
There was a problem hiding this comment.
Thanks, @mattf I have restructured the doc.
5 participants
What does this PR do?
Added documentation for how to configure and run llama stack using:
1- Docker container approach
2- Manual server configuration approach.
Added information about Prerequisites, dependencies, troubleshooting tips, how to verify and test the server with curl commands and how to set environment variables.
Test Plan
Tested these commands on llama-stack version 0.2.21