From 2cf3d4a534f7a9a91e84ba6b699da9a313b55bd2 Mon Sep 17 00:00:00 2001 From: Yiping Kang Date: Wed, 11 Sep 2024 13:48:49 -0400 Subject: [PATCH] 449 Assignment Updates --- .../learn/tutorial/1_setting-up-jac-cloud.md | 53 ++++--- .../tutorial/2_building-a-rag-chatbot.md | 132 ++++++++++-------- .../3_rag-dialogue-routing-chatbot.md | 108 +++++++------- 3 files changed, 165 insertions(+), 128 deletions(-) diff --git a/jac/support/jac-lang.org/docs/learn/tutorial/1_setting-up-jac-cloud.md b/jac/support/jac-lang.org/docs/learn/tutorial/1_setting-up-jac-cloud.md index 5f91a9bbdd..f64b2145a6 100644 --- a/jac/support/jac-lang.org/docs/learn/tutorial/1_setting-up-jac-cloud.md +++ b/jac/support/jac-lang.org/docs/learn/tutorial/1_setting-up-jac-cloud.md @@ -1,12 +1,10 @@ # Setting Up Your Jac Cloud Application -Jac Cloud is a jaclang plugin that bootstraps your jac application into a running web server. It allows you to serve your jac code as a REST API and interact with it from any client that can make HTTP requests. To set up Jac Cloud, you need to install the `jac-cloud` python package using pip: +Jac Cloud is a jaclang plugin that bootstraps your jac application into a running web server. It allows you to serve your jac code as a REST API and interact with it from any client that can make HTTP requests. To set up Jac Cloud, you need to install the `jaclang` and `jac-cloud` python package using pip: ```bash -pip install jac-cloud +pip install jaclang==0.7.18 jac-cloud==0.1.1 ``` -**Note**: jac-cloud requires jaclang version 0.7.18 or later. Make sure you have the latest version of jaclang installed. - ## Setting Up Your Database Like most API servers, jac-cloud requires a database to store data persistently. Jac-cloud uses mongoDB as the database engine. You will need to have a running mongodb service. You can do this in several ways: @@ -48,6 +46,12 @@ docker pull mongodb/mongodb-community-server:latest docker run --name mongodb -p 27017:27017 -d mongodb/mongodb-community-server:latest --replSet my-rs ``` This command will start a mongoDB container with the name `mongodb` and expose port `27017` to the host machine. + +To check that the docker is up and running, run `docker ps` to get the lists of running container and you should see the following: +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +d289c01c3f1c mongodb/mongodb-community-server:latest "python3 /usr/local/…" 12 seconds ago Up 11 seconds 0.0.0.0:27017->27017/tcp mongodb +``` - Install the mongo shell to connect to the mongoDB container. You can install the mongo shell by following the instructions [here](https://www.mongodb.com/docs/mongodb-shell/install/). - Connect to the MongoDB Deployment with mongosh ```bash @@ -55,14 +59,17 @@ mongosh --port 27017 ``` - **First time only**: The first time you start the mongodb, do the following two quick steps - Run the command `mongosh` in another terminal to open the mongo shell. - - In the mongo shell, run the following command to initiate the replica set: + - In the mongo shell, run the following command to initiate the replica set. This command will initiate the replica set with the default configuration. Feel free to learn more about mongodb replica set [here](https://docs.mongodb.com/manual/tutorial/deploy-replica-set/). ```bash rs.initiate({_id: "my-rs", members: [{_id: 0, host: "localhost"}]}) ``` - This command will initiate the replica set with the default configuration. You can customize the configuration as needed. - - Run `Exit` to exit the mongo shell. + We should see the following output: + ``` + { ok: 1 } + ``` + - Run `exit` to exit the mongo shell. -Additional setup instructions can be found [here](https://docs.mongodb.com/manual/tutorial/deploy-replica-set/). +You have successfully set up a running MongoDB service our application can use as the database. ## Installing your VSCode Extension To make your development experience easier, you should install the jac extension for Visual Studio Code. This extension provides syntax highlighting, code snippets, and other features to help you write Jac Cloud code more efficiently. You can install the extension from the Visual Studio Code marketplace [here](https://marketplace.visualstudio.com/items?itemName=jaseci-labs.jaclang-extension). @@ -70,14 +77,14 @@ To make your development experience easier, you should install the jac extension ![Jac Extension](images/1_vscode.png) ## Your First Jac Cloud Application -Now that you have your database set up, you can start building your first Jac Cloud application. Create a new file called `app.jac` and add the following code: +Now that you have your database set up, you can start building your first Jac application. Create a new file called `server.jac` and add the following code: ```jac walker interact { can return_message with `root entry { report { - "message": "Hello, world!" - } + "response": "Hello, world!" + }; } } @@ -86,8 +93,8 @@ walker interact_with_body { can return_message with `root entry { report { - "message": "Hello, " + self.name + "!" - } + "response": "Hello, " + self.name + "!" + }; } } ``` @@ -106,7 +113,9 @@ This command starts the Jac Cloud server with the database host set to `mongodb: Now, before we can fully test the API, it is important to know that by default, Jac Cloud requires authentication to access the API. So we need to create a user and get an access token to access the API. You can do this using the Swagger UI or by making HTTP requests. We will show you how to do this using HTTP requests. -To do this, we need to run the following command: +For this tutorial, we use `curl` to send API requests. You can also use tools like Postman or Insomnia to faciliate this. + +Keep the previous terminal with the `jac serve` process running and open a new terminal. In the new termminal, run the following command to register a new user with the email `test@gmail.com` and password `password`. ```bash curl --location 'http://localhost:8000/user/register' \ @@ -118,7 +127,12 @@ curl --location 'http://localhost:8000/user/register' \ }' ``` -This command will create a user with the username `test@mail.com` and password `password`. Next, we'll need to login and get the access token. To do this, run the following command: +We should see +```json +{"message":"Successfully Registered!"} +``` + +Next, we'll need to login and get the access token. To do this, run the following command: ```bash curl --location 'http://localhost:8000/user/login' \ @@ -130,14 +144,15 @@ curl --location 'http://localhost:8000/user/login' \ }' ``` -You should see a response similar to this: +We should see a response similar to this: ```json {"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjY2ZGYzN2Y0MjIzNDM2N2QxZDMzMDE1MSIsImVtYWlsIjoidGVzdEBtYWlsLmNvbSIsInJvb3RfaWQiOiI2NmRmMzdmNDIyMzQzNjdkMWQzMzAxNTAiLCJpc19hY3RpdmF0ZWQiOnRydWUsImV4cGlyYXRpb24iOjE3MjYwMzAyNDUsInN0YXRlIjoiZGlCQnJOMHMifQ.oFQ5DuUBwzGVedmk4ktesFIelZR0JH8xx7zU4L_Vu3k","user":{"id":"66df37f42234367d1d330151","email":"test@mail.com","root_id":"66df37f42234367d1d330150","is_activated":true,"expiration":1726030245,"state":"diBBrN0s"}} ``` -Now that you have your access token copy the access token and use it to access the API. You can now test the API using the following command: +Save the `token` value. This will be our access token to authenticate with the API for subsequen requests. +Let's now test the `interact` API we created earlier: ```bash curl -X POST http://localhost:8000/walker/interact -H "Authorization: Bearer " ``` @@ -148,6 +163,6 @@ Replace `` with the access token you received. This command will return t {"status":200,"reports":[{"response":"Hello, world!"}]} ``` -You can also do this in the browser by visiting the Swagger docs `http://localhost:8000/docs` and adding the `Authorization` header with the value `Bearer ACCESS TOKEN`. +You can also do this in the browser by visiting the Swagger docs `http://localhost:8000/docs` and adding the `Authorization` header with the value `Bearer ACCESS TOKEN`. -That's it! You have successfully set up your Jac Cloud application and served your first API. In the [next](2_building-a-rag-chatbot.md) part we will learn how to build a simple conversational agent using Jac Cloud. Stay tuned! \ No newline at end of file +That's it! You have successfully set up your Jac application and served your first API. In the [next](2_building-a-rag-chatbot.md) part we will learn how to build a simple conversational agent using Jac. \ No newline at end of file diff --git a/jac/support/jac-lang.org/docs/learn/tutorial/2_building-a-rag-chatbot.md b/jac/support/jac-lang.org/docs/learn/tutorial/2_building-a-rag-chatbot.md index db89e8e38b..5ddddee2bb 100644 --- a/jac/support/jac-lang.org/docs/learn/tutorial/2_building-a-rag-chatbot.md +++ b/jac/support/jac-lang.org/docs/learn/tutorial/2_building-a-rag-chatbot.md @@ -2,17 +2,17 @@ Now that we have a jac application served up, let's build a simple chatbot using Retrieval Augmented Generation (RAG) with Jac Cloud and Streamlit as our frontend interface. ### Preparation / Installation -Make sure you have all the dependencies installed in your environment. If you don't have it installed, you can install it using pip: +There are a couple of additional dependenices we need here ```bash -pip install jaclang mtllm mtllm[openai] jaclang_streamlit jac-cloud requests langchain_community chromadb langchain pypdf +pip install mtllm==0.3.2 jac-streamlit==0.0.3 langchain==0.1.16 langchain_community==0.0.34 chromadb==0.5.0 pypdf==4.2.0 ``` ## Building a Streamlit Interface Before we begin building out our chatbot, let's first build a simple GUI to interact with the chatbot. Streamlit offers several Chat elements, enabling you to build Graphical User Interfaces (GUIs) for conversational agents or chatbots. Leveraging session state along with these elements allows you to construct anything from a basic chatbot to a more advanced, ChatGPT-like experience using purely Python code. -Luckily for us, jaclang has a plugin for streamlit that allows us to build web applications with streamlit using jaclang. In this part of the tutorial, we will build a frontend for our conversational agent using streamlit. You can find more information about the `jaclang_streamlit` plugin [here](https://github.com/Jaseci-Labs/jaclang/blob/main/support/plugins/streamlit/README.md). +Luckily for us, jaclang has a plugin for streamlit that allows us to build web applications with streamlit using jaclang. In this part of the tutorial, we will build a frontend for our conversational agent using streamlit. You can find more information about the `jac-streamlit` plugin [here](https://github.com/Jaseci-Labs/jaclang/blob/main/support/plugins/streamlit/README.md). -First, let's create a new file called `client.jac` in the root directory of your project. This file will contain the code for the frontend chat interface. +First, let's create a new file called `client.jac`. This file will contain the code for the frontend chat interface. We start by importing the necessary modules in Jac: @@ -30,21 +30,20 @@ import:py requests; Now let's define a function bootstrap_frontend, which accepts a token for authentication and builds the chat interface. ```jac -can bootstrap_frontend (token:str) { +can bootstrap_frontend (token: str) { st.write("Welcome to your Demo Agent!"); # Initialize chat history if "messages" not in st.session_state { st.session_state.messages = []; } - +} ``` - `st.write()` adds a welcome message to the app. - `st.session_state` is used to persist data across user interactions. Here, we're using it to store the chat history (`messages`). - -Now, let's update the function such that when the page reloads or updates, the previous chat messages are reloaded from `st.session_state.messages`. +Now, let's update the function such that when the page reloads or updates, the previous chat messages are reloaded from `st.session_state.messages`. Add the following to `bootstrap_frontend` ```jac for message in st.session_state.messages { @@ -68,14 +67,26 @@ Next, let's capture user input using `st.chat_input()`. This is where users can with st.chat_message("user") { st.markdown(prompt); } + } ``` - `st.chat_input()` waits for the user to type a message and submit it. - Once the user submits a message, it's appended to the session state's message history and immediately displayed on the screen. -Now we handle the backend interaction. After the user submits a message, the assistant responds. This involves sending the user's message to the backend and displaying the response. +Now we handle the interaction with the backend server. After the user submits a message, the assistant responds. This involves sending the user's message to the backend, receiving a response from the backend and displaying it. + +Add the following to `bootstrap_frontend`. ```jac + if prompt := st.chat_input("What is up?") { + # Add user message to chat history + st.session_state.messages.append({"role": "user", "content": prompt}); + + # Display user message in chat message container + with st.chat_message("user") { + st.markdown(prompt); + } + # Display assistant response in chat message container with st.chat_message("assistant") { @@ -94,14 +105,14 @@ Now we handle the backend interaction. After the user submits a message, the ass } } } -} ``` - The user's input (`prompt`) is sent to the backend using a POST request to the `/walker/interact` endpoint. +- The `interact` walker, as we created in the last chapter, just returns `Hello World!` for now. This will change as we build out our chatbot. + - `message` and `session_id` are not yet utilized at this point. They will come into play later in this chapter. - The response from the backend is then displayed using `st.write()`, and the assistant's message is stored in the session state. -- The `session_id` is hardcoded here for simplicity, but you can make it dynamic based on your application’s needs. -Lastly, we'll define the entry point, which authenticates the user and retrieves the token needed for the bootstrap_frontend function. +Lastly, we'll define the entry point of `client.jac`. Think `main` function of a python program. We authenticates the user and retrieves the token needed for the `bootstrap_frontend` function. ```jac with entry { @@ -153,7 +164,7 @@ Now you can run the frontend using the following command: jac streamlit client.jac ``` -If your server is still running, you can chat with your assistant using the streamlit interface. The response will only be "Hello, world!" for now, but we will update it to use the RAG module shortly. +If your server is still running, you can chat with your assistant using the streamlit interface. The response will only be "Hello, world!" for now, but we will update it to be a fully working chatbot next. Now let's move on to building the RAG module. @@ -173,12 +184,12 @@ First, let's add a file called `rag.jac` to our project. This file will contain Jac allows you to import Python libraries, making it easy to integrate existing libraries such as langchain, langchain_community, and more. In this RAG engine, we need document loaders, text splitters, embedding functions, and vector stores. ```jac +import: py os; import: py from langchain_community.document_loaders {PyPDFDirectoryLoader} import: py from langchain_text_splitters {RecursiveCharacterTextSplitter} import: py from langchain.schema.document {Document} import: py from langchain_community.embeddings.ollama {OllamaEmbeddings} import: py from langchain_community.vectorstores.chroma {Chroma} -import: py os; ``` - `PyPDFDirectoryLoader` is used to load documents from a directory. @@ -189,17 +200,23 @@ import: py os; Now let's define the `rag_engine` object that will handle the retrieval and generation of responses. The object will have two properties: `file_path` for the location of documents and `chroma_path` for the location of the vector store. ```jac -obj rag_engine { - has file_path:str="docs"; - has chroma_path:str = "chroma"; +obj RagEngine { + has file_path: str = "docs"; + has chroma_path: str = "chroma"; +} ``` -The object will have a `postinit` method that runs automatically upon initialization, loading documents, splitting them into chunks, and adding them to the vector database (Chroma). +Note: `obj` works similarly as dataclasses in Python. + +We will now build out this `RagEngine` object by adding relevant abilities. Abilities, as annotated by the `can` keyword, are analogus to member methods of a Python class. The `can` abilities in the following code snippets shoudld be added inside the `RagEngine` object scope. + + +The object will have a `postinit` method that runs automatically upon initialization, loading documents, splitting them into chunks, and adding them to the vector database (Chroma). `postinit` works similarly to the `__post_init__` in Python dataclasses. ```jac can postinit { - documents:list = self.load_documents(); - chunks:list = self.split_documents(documents); + documents: list = self.load_documents(); + chunks: list = self.split_documents(documents); self.add_to_chroma(chunks); } ``` @@ -240,7 +257,7 @@ Next, let's define the `get_embedding_function` method. The `get_embedding_funct Now, each chunk of text needs a unique identifier to ensure that it can be referenced in the vector store. The `add_chunk_id` ability assigns IDs to each chunk, using the format `Page Source:Page Number:Chunk Index`. ```jac - can add_chunk_id(chunks:str) { + can add_chunk_id(chunks: str) { last_page_id = None; current_chunk_index = 0; @@ -295,15 +312,17 @@ Once the documents are split and chunk IDs are assigned, we add them to the Chro Next, the `get_from_chroma` ability takes a query and returns the most relevant chunks based on similarity search. This is the core of retrieval-augmented generation, as the engine fetches chunks that are semantically similar to the query. ```jac - can get_from_chroma(query:str,chunck_nos:int=5) { - db = Chroma(persist_directory=self.chroma_path, embedding_function=self.get_embedding_function()); + can get_from_chroma(query: str,chunck_nos: int=5) { + db = Chroma( + persist_directory=self.chroma_path, + embedding_function=self.get_embedding_function() + ); results = db.similarity_search_with_score(query,k=chunck_nos); return results; } -} ``` -To summarize, we define an object called `rag_engine` with two properties: `file_path` and `chroma_path`. The `file_path` property specifies the path to the directory containing the documents we want to retrieve responses from. The `chroma_path` property specifies the path to the directory containing the pre-trained embeddings. We will use these embeddings to retrieve candidate responses. +To summarize, we define an object called `RagEngine` with two properties: `file_path` and `chroma_path`. The `file_path` property specifies the path to the directory containing the documents we want to retrieve responses from. The `chroma_path` property specifies the path to the directory containing the pre-trained embeddings. We will use these embeddings to retrieve candidate responses. We define a few methods to load the documents, split them into chunks, and add them to the Chroma vector store. We also define a method to retrieve candidate responses based on a query. Let's break down the code: @@ -333,15 +352,14 @@ ollama serve You can add your documents to the `docs` directory. The documents should be in PDF format. You can add as many documents as you want to the directory. We've included a sample document in the `docs` directory for you to test. ### Setting up your LLM -Here we are going to use one of the magic features of jaclang called [MTLLM](https://jaseci-labs.github.io/mtllm/). MTTLM facilitates the integration of generative AI models, specifically Large Language Models (LLMs) into programming in an ultra seamless manner right from the Jaclang code. You should have the `mtllm` library installed in your environment. If you don't have it installed, you can install it using `pip install mtllm[OpenAI]`. +Here we are going to use one of the key features of jaclang called [MTLLM](https://jaseci-labs.github.io/mtllm/), or Meaning-typed LLM. MTTLM facilitates the integration of generative AI models, specifically Large Language Models (LLMs) into programming at the language level. -Create a new file called `server.jac` and add the following code: +We will create a new server code so delete the existing code in `server.jac` that we created in the last chapter and start from scratch and add the following. ```jac -import:py from mtllm.llms {OpenAI}; +import:py from mtllm.llms {OpenAI} glob llm = OpenAI(model_name='gpt-4o'); - ``` Here we use the OpenAI model gpt-4o as our Large Language Model (LLM). To use OpenAI you will need an API key. You can get an API key by signing up on the OpenAI website [here](https://platform.openai.com/). Once you have your API key, you can set it as an environment variable: @@ -359,7 +377,7 @@ ollama pull llama3.1 This will download the `llama3.1` model to your local machine and make it available for inference when you run the `ollama serve` command. If you want use Ollama replace your import statement with the following: ```jac -import:py from mtllm.llms {Ollama}; +import:py from mtllm.llms {Ollama} glob llm = Ollama(model_name='llama3.1'); ``` @@ -367,27 +385,28 @@ glob llm = Ollama(model_name='llama3.1'); Now that you have your LLM ready let's create a simple walker that uses the RAG module and MTLLM to generate responses to user queries. First, let's declare the global variables for MTLLM and the RAG engine. ```jac -glob llm = OpenAI(model_name='gpt-4o'); +import:jac from rag {RagEngine} glob RagEngine:rag_engine = rag_engine(); ``` - `llm`: This an MTLLM instance of the model utilized by jaclang whenever we make `by llm()` abilities. Here we are using OpenAI's GPT-4 model. -- `RagEngine`: This is an instance of the `rag_engine` object for document retrieval and processing. +- `rag_engine`: This is an instance of the `RagEngine` object for document retrieval and processing. -Next, let's define a node called `session` that stores the chat history and status of the session. The session node also has an ability called `llm_chat` that uses the MTLLM model to generate responses based on the chat history, agent role, and context. If you've ever worked with graphs before, you should be familiar with nodes and edges. Nodes are entities that store data, while edges are connections between nodes that represent relationships. In this case, `session` is node in our graph that stores the chat history and status of the session. +Next, let's define a node called `Session` that stores the chat history and status of the session. The session node also has an ability called `llm_chat` that uses the MTLLM model to generate responses based on the chat history, agent role, and context. If you've ever worked with graphs before, you should be familiar with nodes and edges. Nodes are entities that store data, while edges are connections between nodes that represent relationships. In this case, `Session` is node in our graph that stores the chat history and status of the session. ```jac -node session { +node Session { has id: str; has chat_history: list[dict]; has status: int = 1; can 'Respond to message using chat_history as context and agent_role as the goal of the agent' - llm_chat(message:'current message':str, - chat_history: 'chat history':list[dict], - agent_role:'role of the agent responding':str, - context:'retrieved context from documents':list - ) -> 'response':str by llm(); + llm_chat( + message:'current message':str, + chat_history: 'chat history':list[dict], + agent_role:'role of the agent responding':str, + context:'retrieved context from documents':list + ) -> 'response':str by llm(); } ``` @@ -406,13 +425,14 @@ walker interact { has session_id: str; can init_session with `root entry { - visit [-->](`?session)(?id == self.session_id) else { - session_node = here ++> session(id=self.session_id, chat_history=[], status=1); + visit [-->](`?Session)(?id == self.session_id) else { + session_node = here ++> Session(id=self.session_id, chat_history=[], status=1); print("Session Node Created"); visit session_node; } } +} ``` **Attributes:** @@ -423,21 +443,23 @@ walker interact { Now, let's define the `chat` ability which once the session is initialized, will handle interactions with the user and the document retrieval system. -```jac - can chat with session entry { +Add the following ability inside the scope of `interact` walker. +```jac + can chat with Session entry { here.chat_history.append({"role": "user", "content": self.message}); - - data = RagEngine.get_from_chroma(query=self.message); - response = here.llm_chat(message=self.message, chat_history=here.chat_history, agent_role="You are a conversation agent designed to help users with their queries based on the documents provided", context=data); + data = rag_engine.get_from_chroma(query=self.message); + response = here.llm_chat( + message=self.message, + chat_history=here.chat_history, + agent_role="You are a conversation agent designed to help users with their queries based on the documents provided", + context=data + ); here.chat_history.append({"role": "assistant", "content": response}); - report { - "response": response - }; + report {"response": response}; } -} ``` **Logic flow:** @@ -450,8 +472,8 @@ Now, let's define the `chat` ability which once the session is initialized, will To summarize: -- We define a `session` node that stores the chat history and status of the session. The session node also has an ability called `llm_chat` that uses the MTLLM model to generate responses based on the chat history, agent role, and context. -- We define a `interact` walker that initializes a session and generates responses to user queries. The walker uses the `RagEngine` object to retrieve candidate responses and the `llm_chat` ability to generate the final response. +- We define a `Session` node that stores the chat history and status of the session. The session node also has an ability called `llm_chat` that uses the MTLLM model to generate responses based on the chat history, agent role, and context. +- We define a `interact` walker that initializes a session and generates responses to user queries. The walker uses the `rag_engine` object to retrieve candidate responses and the `llm_chat` ability to generate the final response. You can now serve this code using Jac Cloud by running the following command: @@ -459,14 +481,14 @@ You can now serve this code using Jac Cloud by running the following command: DATABASE_HOST=mongodb://localhost:27017/?replicaSet=my-rs jac serve server.jac ``` -Now you can test out your chatbot using the streamlit interface. The chatbot will retrieve candidate responses from the documents and generate the final response using the MTLLM model. Ask any question related to the documents you added to the `docs` directory and see how the chatbot responds. +Now you can test out your chatbot using the client we created earlier. The chatbot will retrieve candidate responses from the documents and generate the final response using the MTLLM model. Ask any question related to the documents you added to the `docs` directory and see how the chatbot responds. You can also try testing out the updated endpoint using the swagger UI at `http://localhost:8000/docs` or using the following curl command: ```bash -curl -X POST http://localhost:8000/walkers/interact -d '{"message": "I am having major back pain, what can i do", "session_id": "123"} -H "Authorization: Bearer ACCESS TOKEN" +curl -X POST http://localhost:8000/walkers/interact -d '{"message": "I am having major back pain, what can i do", "session_id": "123"} -H "Authorization: Bearer " ``` -Remember to replace `ACCESS TOKEN` with the access token of your user. +Remember to replace `` with the access token you saved. Note you might need to re-login to get an updated token if the token has expired or the server has been restarted since. In the next part of the tutorial, we will enhance the chatbot by adding dialogue routing capabilities. We will direct the conversation to the appropriate dialogue model based on the user's input. \ No newline at end of file diff --git a/jac/support/jac-lang.org/docs/learn/tutorial/3_rag-dialogue-routing-chatbot.md b/jac/support/jac-lang.org/docs/learn/tutorial/3_rag-dialogue-routing-chatbot.md index 265680e38b..fa4eb578d5 100644 --- a/jac/support/jac-lang.org/docs/learn/tutorial/3_rag-dialogue-routing-chatbot.md +++ b/jac/support/jac-lang.org/docs/learn/tutorial/3_rag-dialogue-routing-chatbot.md @@ -5,7 +5,7 @@ In some cases, the user query that comes in may not be suitable for the RAG mode Let's modify the RAG chatbot we built in the previous part to include dialogue routing. We will add a simple dialogue router that will route the conversation to the appropriate dialogue model based on the user's input. -First, let's add a new enum `ChatType` to define the different dialogue states we will use: +First, let's add a new enum `ChatType` to define the different dialogue states we will use. Add this to `server.jac`. ```jac enum ChatType { @@ -20,95 +20,93 @@ enum ChatType { Next, we'll a new node to our graph called `router`. The `router` node is responsible for determining which type of model (RAG or QA) should be used to handle the query. ```jac -node router { +node Router { can 'route the query to the appropriate task type' classify(message:'query from the user to be routed.':str) -> ChatType by llm(method="Reason", temperature=0.0); } ``` -The `router` node has an ability called `classify` that takes the user query as input and classifies it into one of the `ChatType` states using the ` by llm` feature from MTLLM. So cool! 😎 +The `router` node has an ability called `classify` that takes the user query as input and classifies it into one of the `ChatType` states using the ` by llm` feature from MTLLM. -Next, we'll add a new walker called `infer`. The `infer` walker contains the logic for routing the user query to the appropriate dialogue model based on the classification from the `router` node. +Let's now define two different modes of chatbots that the router can direct the request to accordingly. ```jac -walker infer { - has message:str; - has chat_history: list[dict]; - - can init_router with `root entry { - visit [-->](`?router) else { - router_node = here ++> router(); - print("Router Node Created"); - router_node ++> rag_chat(); - router_node ++> qa_chat(); - visit router_node; - } - } -``` - -Here we have a new ability `init_router` that initializes the `router` node and creates and edge to two new nodes `rag_chat` and `qa_chat`. These nodes will be used to handle the RAG and QA models respectively. We'll define these nodes later. Let's finish the `infer` walker first. - -```jac - can route with router entry { - classification = here.classify(message = self.message); - - print("Classification:", classification); - - visit [-->](`?chat)(?chat_type==classification); - } -} -``` - -Now we add an ability `route` that classifies the user query using the `router` node and routes it to the appropriate node called `chat` based on the classification. Let's define the `chat` node next. - -```jac -node chat { +node Chat { has chat_type: ChatType; } -node rag_chat :chat: { +node RagChat :Chat: { has chat_type: ChatType = ChatType.RAG; can respond with infer entry { - print("Responding to the message"); can 'Respond to message using chat_history as context and agent_role as the goal of the agent' respond_with_llm( message:'current message':str, chat_history: 'chat history':list[dict], agent_role:'role of the agent responding':str, context:'retirved context from documents':list ) -> 'response':str by llm(); - data = RagEngine.get_from_chroma(query=here.message); - print("Data:", data); + data = rag_engine.get_from_chroma(query=here.message); here.response = respond_with_llm(here.message, here.chat_history, "You are a conversation agent designed to help users with their queries based on the documents provided", data); - print("Here:", here); } } -node qa_chat :chat: { +node QAChat :Chat: { has chat_type: ChatType = ChatType.QA; can respond with infer entry { - print("Responding to the message"); can 'Respond to message using chat_history as context and agent_role as the goal of the agent' respond_with_llm( message:'current message':str, chat_history: 'chat history':list[dict], agent_role:'role of the agent responding':str ) -> 'response':str by llm(); here.response = respond_with_llm(here.message, here.chat_history, agent_role="You are a conversation agent designed to help users with their queries"); - print("Here:", here); } +} +``` +We define two new nodes `RagChat` and `QAChat` that extend the `Chat` node. The `RagChat` node is used for the RAG model, and the `QAChat` node is used for a simple question-answering model. Both nodes have the ability `respond` that responds to the user query using the respective model. + +In the `RagChat` node, we have a new ability `respond_with_llm` that responds to the user query using the RAG model. The ability retrieves the relevant information from the documents and responds to the user query. In the `QAChat` node, we have a new ability `respond_with_llm` that responds to the user query using a simple question-answering model. + + +Next, we'll add a new walker called `infer`. The `infer` walker contains the logic for routing the user query to the appropriate dialogue model based on the classification from the `Router` node. + +```jac +walker infer { + has message:str; + has chat_history: list[dict]; + + can init_router with `root entry { + visit [-->](`?Router) else { + router_node = here ++> Router(); + router_node ++> RagChat(); + router_node ++> QAChat(); + visit router_node; + } + } } ``` -We define two new nodes `rag_chat` and `qa_chat` that extend the `chat` node. The `rag_chat` node is used for the RAG model, and the `qa_chat` node is used for a simple question-answering model. Both nodes have the ability `respond` that responds to the user query using the respective model. +Here we have a new ability `init_router` that initializes the `Router` node and creates and edge to two new nodes `RagChat` and `QAChat`. These nodes will be used to handle the RAG and QA models respectively. We'll define these nodes later. Let's finish the `infer` walker first. + +The following `can` abilities should be added to the `infer` walker scope. -In the `rag_chat` node, we have a new ability `respond_with_llm` that responds to the user query using the RAG model. The ability retrieves the relevant information from the documents and responds to the user query. In the `qa_chat` node, we have a new ability `respond_with_llm` that responds to the user query using a simple question-answering model. +We add an ability `route` that classifies the user query using the `Router` node and routes it to the appropriate node based on the classification. -Lastly, we'll update our `session` node. Instead of directly responding to the user query like before, the `session` will something super cool! 🤯 ```jac -node session { + can route with Router entry { + classification = here.classify(message = self.message); + visit [-->](`?Chat)(?chat_type==classification); + } +``` + +Lastly, we'll update our `Session` node. We will maintain a record of previous conversation histories so the chatbot can have a continuous back-and-forth conversation with memories of previous interactions. + +Add the following `chat` abilities to the `Session` node scope. + +```jac +node Session { can chat with interact entry { self.chat_history.append({"role": "user", "content": here.message}); response = infer(message=here.message, chat_history=self.chat_history) spawn root; @@ -121,19 +119,21 @@ node session { } ``` -In our updated `session` node, we have a new ability `chat` that is triggered by the `interact` walker. This means that when the interact walker traverses to the `session` node, it will trigger the `chat` ability. The `chat` ability will then spawns the `infer` walker on `root`. The `infer` walker will execute its logic to route the user query to the appropriate dialogue model based on the classification. The response from the dialogue model is then appended to the `infer` walker's object and reported back to the frontend. This is the magic of Data Spacial programming! 🪄 Super dope, right? 😎 +In our updated `Session` node, we have a new ability `chat` that is triggered by the `interact` walker. This means that when the interact walker traverses to the `Session` node, it will trigger the `chat` ability. The `chat` ability will then spawns the `infer` walker on `root`. The `infer` walker will execute its logic to route the user query to the appropriate dialogue model based on the classification. The response from the dialogue model is then appended to the `infer` walker's object and reported back to the frontend. This is the magic of Data Spacial programming! + +**Note*+: the `can chat with Session entry` ability needs to be removed from the `interact` walker. To summarize, here are the changes we made to our RAG chatbot to add dialogue routing capabilities: - Our first addition is the enum `ChatType`, which defines the different types of chat models we will use. We have two types: `RAG` and `QA`. `RAG` is for the RAG model, and `QA` is for a simple question-answering model. This will be used to classify the user query and route it to the appropriate model. -- Next we have a new node `router`, with the ability `classify`. The ability classifies the user query and route it to the appropriate model. +- Next we have a new node `Router`, with the ability `classify`. The ability classifies the user query and route it to the appropriate model. - We have a new walker `infer`, which has the ability `route`. The ability routes the user query to the appropriate model based on the classification. -- We have two new nodes `rag_chat` and `qa_chat`, which are the chat models for the RAG and QA models respectively. These nodes extend the `chat` node and have the ability `respond`. The ability responds to the user query using the respective model. -- In the `rag_chat` node, we have a new ability `respond_with_llm`, which responds to the user query using the RAG model. The ability retrieves the relevant information from the documents and responds to the user query. -- In the `qa_chat` node, we have a new ability `respond_with_llm`, which responds to the user query using a simple question-answering model. +- We have two new nodes `RagChat` and `QAChat`, which are the chat models for the RAG and QA models respectively. These nodes extend the `Chat` node and have the ability `respond`. The ability responds to the user query using the respective model. +- In the `RagChat` node, we have a new ability `respond_with_llm`, which responds to the user query using the RAG model. The ability retrieves the relevant information from the documents and responds to the user query. +- In the `QAChat` node, we have a new ability `respond_with_llm`, which responds to the user query using a simple question-answering model. - We update our `interact` walker to include the new `init_router` ability, which initializes the router node and routes the user query to the appropriate model. -- Lastly, we update the `session` node to have the ability `chat`, which is triggered by the when the `interact` walker is on the node. This ability spawns the `infer` walker and reports the response back to the frontend. +- Lastly, we update the `Session` node to have the ability `chat`, which is triggered by the when the `interact` walker is on the node. This ability spawns the `infer` walker and reports the response back to the frontend. Now that we have added dialogue routing capabilities to our RAG chatbot, we can test it out by running the following command: