This role will install ollama, a tool to run Large Language Models locally, and ollama-ui, a simple web interface for ollama.
See meta/main.yml
# playbook.yml
- hosts: my.CHANGEME.org
roles:
- nodiscc.xsrv.common # (optional) base server setup, hardening, firewall, bruteforce prevention
- nodiscc.xsrv.monitoring # (optional) server monitoring
- nodiscc.xsrv.apache # (required in the default configuration) webserver/reverse proxy, SSL certificates
- nodiscc.xsrv.ollama
# required variables
# host_vars/my.CHANGEME.org/my.CHANGEME.org.yml
ollama_ui_fqdn: "llm.CHANGEME.org"
# ansible-vault edit host_vars/my.CHANGEME.org/my.CHANGEME.org.vault.yml
ollama_username: "CHANGEME"
ollama_password: "CHANGEME"See defaults/main.yml for all configuration variables.
Access the web interface at https://{{ ollama_ui_fqdn }} and connect the web interface to the ollama server instance, by setting the Hostname value to https://{{ ollama_ui_fqdn }}/api/. Access to the web interface is protected by the ollama_username/ollama_password credentials.
You can also use the command-line interface by SSHing (xsrv shell) to the server and running ollama run MODEL_NAME.
Models can be downloaded from the ollama library. Set the ollama_models configuration variable and deploy the role to pull models, or run ollama pull MODEL_NAME from the command line. The orca-mini model is pulled by default and should run with acceptable performance on a medium-sized machine (4GB RAM, 4vCPU). More advanced models require more resources to run with decent performance. See the ollama README for more information.
See the included rsnapshot configuration for the backup role and the ollama_backup_models configuration variable. By default, automatic backups of downloaded models are disabled.
ollama - setup ollama Large Language Model server
ollama-ui - setup ollama-ui web interface for ollama