| page_type | languages | products | name | urlFragment | description | azureDeploy | |||
|---|---|---|---|---|---|---|---|---|---|
sample |
|
|
Sample chat app using Azure Cosmos DB for NoSQL and Azure OpenAI Service |
chat-app |
Sample application that implements multiple chat threads using the Azure OpenAI "gpt-4" and Azure Cosmos DB for NoSQL for storage. |
This sample application combines Azure Cosmos DB with Azure OpenAI Service to build a simple AI-enabled Chat Application. The purpose of this application is to provide a simple demonstration of how to design a service to generate completions from user prompts and store the chat history prompts and completions from a generative AI application. The application is written in C# on .NET 8 with a Blazor Server front-end and is hosted on Azure Web Apps.
Individual chat sessions (or conversations) are displayed and can be selected in the left-hand nav. Clicking on a session will show the messages that contain user prompts and OpenAI completions.
When a new prompt is sent to the Azure OpenAI Service, some or all of the conversation history is sent with it. This provides context allowing ChatGPT to respond as though it is having a conversation. The length of this conversation history can be configured from appsettings.json with the OpenAiMaxTokens value. When a user prompt is entered, the application will cycle from the most recent to the oldest prompts and completions, counting the tokens used for each. When it gets to the OpenAiMaxTokens it stops and returns the conversational history as far back as the token limit allows.
The "gpt-4" model used by this sample has a maximum of 4096 tokens. Token are used in both the request and reponse from the service. Overriding the maxConversationLength to values approaching maximum token value could result in completions that contain little to no text if all of it has been used in the request.
The history for all prompts and completions is stored in Azure Cosmos DB. Deleting a chat session in the UI will delete it's corresponding user prompts and completions.
The application summarizes the name of the chat session by asking ChatGPT to provide a one or two word summary of the first user prompt and completion. This allows you to easily identity different chat sessions.
Please note this is a sample application. It is intended to demonstrate how to use Azure Cosmos DB and Azure OpenAI Service ChatGPT together and capture user prompts and completions with an example of how to model data for a chat-based application. It also demonstrates how system prompts are used to define behavior for generation completions in a Generative AI scenario.
- Azure Subscription
- Subscription access to Azure OpenAI service. Start here to Request Access to Azure OpenAI Service
- Visual Studio, VS Code, or some editor if you want to edit or view the source for this sample.
If you do not expect to modify this solution and deploy merged changes, you can install using this step below.
- Click one of the Deploy to Azure buttons belows and follow the prompts in Azure Portal to deploy this solution. There is one that deploys a new Azure OpenAI account and a second that allows you to use an existing Azure OpenAI account.
- Click Fork in the upper right hand corner of this page to your account.
- From your forked version, open this `README.md`` file to change the path for the Deploy To Azure button to your local repository. Commit the change to your repo.
- Click Deploy to Azure below and follow the prompts in Azure Portal to deploy this solution.
- In the Custom Deployment blade in Azure Portal, modify App Git Repository to point to your forked GitHub repo for this solution. By default the value is
https://github.com/Azure-Samples/cosmosdb-chatgpt.git - If you deploy this solution without making either of these changes, you can update the repository by disconnecting and connecting an external git repository pointing to your fork from within Azure App Service Deployment Center blade in Azure Portal.
The provided ARM or Bicep Template will provision the following resources:
- Azure Cosmos DB account with database and container at 400 RU/s. This can optionally be configured to run on the Cosmos DB free tier if available for your subscription.
- Azure App service configured for CI/CD to your forked GitHub repository. This service can also be configured to run on App Service free tier.
- Azure OpenAI You must also specify a name for the deployment of the "gpt-4" model which is used by this application.
Note: You must have access to Azure Open AI service from your subscription before attempting to deploy this application.
All connection information for Azure Cosmos DB and Azure Open AI is zero-touch and injected as environment variables in the Azure App Service instance at deployment time.
Note: If you already have an Azure OpenAI account deployed and wish to use it with this application, use this template instead. You will be prompted to provide the name of the Azure OpenAI account, a key, and the name of the GPT 3.5 Turbo model used for completions.
- After deployment, go to the resource group for your deployment and open the Azure App Service in the Azure Portal. Click the web url to launch the website.
- Click + New Chat to create a new chat session.
- Type your question in the text box and press Enter.
- Create new sessions and ask more questions on different topics.
- View the token usage for the prompts and completions and the total for the session.
To remove all the resources used by this sample, you must first manually delete the deployed model within the Azure AI service (if you originally provisioned it using this sample). You can then delete the resource group for your deployment. This will delete all remaining resources.