Note: This is a limited, basic, and incomplete implementation of a simple DM helper AI bot for Fifth Edition.
It was built to support the Rapid Prototyping with AI talk I am presenting at Prompt Engineering Conference 2023.
Watch the Presentation: Rapid Prototyping with AI
We'll load in the Fifth Edition Systems Reference Document (SRD) in Markdown from OldManUmby/DND.SRD.Wiki and then load a few documents into a local Chroma vector database to power the retrieval augmented generation (RAG) for our bot.
Gygax.AI.Demo.mp4
Note: Apologies for the low-quality, but part of the talk was focused on working with low-powered hardware and my 2012 Macbook Pro was struggling to keep up with recording, running the LLM and ChromaDB Docker Container, and the NextJS development server.
- Clone the repo:
git clone git@github.com:DVDAGames/local-tabletop-ai-demo.git
orgh repo clone DVDAGames/local-tabletop-ai-demo
- Move into the project's directory:
cd local-tabletop-ai-demo
- Install the dependencies:
npm install
Note: There is apostinstall
hook that will clone thechroma-core/chroma
repo into.data/chroma
so that you can run it as a Docker container. This method was what was available at the time this project was started - there may be other ways to run ChromaDB in a container now. - Create the
gygax
model:ollama create gygax -f Modelfile
- Start the Vector DB:
npm run chroma:start
- Embed the SRD content into the DB:
npm run srd
Note: Depending on your hardware, this could take a while. You can adjust which documents are embedded via thescripts/dnd_srd.mjs
file. - Start the development server:
npm run dev
- Chat with Gygax AI: http://localhost:3000
Note: Local models, especially small, quantized ones like orca-mini:7b-v3
can be weird. I've tried to reduce that with some adjustment to the bot's settings and some prompt engineering, but your experience may vary and gygax
might still go off the rails a little bit.
After you stop the server with a SIGINT
or SIGTERM
(e.g. CTRL+C
), you can run npm run chroma:stop
to stop the ChromaDB container.
Here's the abstract for the Rapid Prototyping with AI session:
ChatGPT can help you brainstorm. GitHub's Copilot can be a great rubber duck for your codebase. But, what if you want to build something new?
In this talk we'll explore how to leverage current tooling to explore open source models, quickly spin up a model with a persona and instructions, and even build out an MVP with the help of a bot empowered with the ability to write and execute code.
You'll learn about some exciting new tools as well as some tips and tricks for prototyping your next idea using language models.