Draft: Ollama API with Podman Compose #93
Conversation
Make some changes required to use podman instead of docker
docs/ai.md
Outdated
| ports: | ||
| - 11434:11434 | ||
| volumes: | ||
| - ./ollama_v:/root/.ollama:z |
There was a problem hiding this comment.
Permission denied in podman without :z
docs/ai.md
Outdated
| volumes: | ||
| - ./ollama_v:/root/.ollama:z | ||
| devices: | ||
| - nvidia.com/gpu=all |
There was a problem hiding this comment.
This might work with docker-compose as well. If so we can probably unify the 2 blocks (:z worked with docker-compose)
There was a problem hiding this comment.
Does not work for docker
Error response from daemon: could not select device driver "cdi" with capabilities: []
|
We could also just put a quadlet in there and tell people to paste it into the right file? I'm thinking, if we're going podman, we should go full podman/quadlet which is what they prefer. We should also leave the docker example there too, if someone needs to get something done and they need ollama they shouldn't have to learn podman that same time, so offering both feels great. What do you think? |
|
@castrojo I think that makes sense. I suspect that it will still need to run as root, but I'll give it a try and report back. |
split podman from the docker section, also add a quadlet version
| ContainerName=ollama | ||
| AutoUpdate=yes | ||
| PublishPort=11434:11434 | ||
| Volume=./ollama_v:/root/.ollama:z |
There was a problem hiding this comment.
Not sure what the volume path should be in the quadlet
|
OK so the quadlet kind of works.
|
WIP for using podman-compose instead of docker compose with the Ollama API docs.
Needs someone else to verify before merging these updates. I have only tested on my Aurora install with Nvidia GPU.
Nvidia GPU passthrough seems to need
sudountil this bug is fixed containers/podman#19338