The main objective of this project is to build a customized chatbot capable of answering questions using a customized database. That is, a data set that was not used during model training.
Image by upklyak on Freepik
In this project, a dataset containing 101 facts about the Premier League for the 2022/23 season was used, coming from the page theanalyst.com. This dataset was chosen because we aim to develop a tool that acts as an expert and can provide answers to questions related to this specific league and season.
To improve the question answering capability of our tool, we employ the Retrieval Augmented Generation technique. This technique complements the prompt with contextual information from the dataset, allowing the model to provide more accurate and relevant answers to the questions posed to it.
The chosen dataset was loaded into a pandas dataframe with a column called "text". This column should contain all of your text data, separated into at least 20 lines. To do this, a web scraping of the page was carried out 101 Best Premier League Facts of the 2022-23 Season and then transformed into a DataFrame so that embeddings could be created in the next step. The objective of this was to demonstrate the correct functioning of the project, since OpenAI's GPT-3.5 has information until 2021.
To perform the customized query, embeddings were created using the OpenAI API and saved in the csv file. With this, a customized search was carried out with an OpenAI Completion Model.
Para avaliar o resultado do projeto desenvolvido, foram realizadas três querys, cada uma com e sem contexto. Nesta etapa, obteu-se os seguintes resultados:
Answer without Context: It is not possible to accurately answer this question as the 2022/2023 Premier League season has not yet taken place.
Answer with Context: Man City won the Premier League in the 2022/2023 season.
Answer without Context: Harry Kane played for Manchester City in the 2022/2023 season.
Answer with Context: Harry Kane played for Tottenham Hotspur in the 2022/2023 season.
Answer without Context: I'm sorry, but I would need more information in order to provide a specific answer to your question. Please provide the name of the sport and teams involved in the match.
Answer with Context: Liverpool finished the match with the most competitive win, defeating Manchester United 7-0 in March 2024.
Question 1: It is evident that the model correctly responded after querying the custom database, indicating its ability to retrieve relevant information when necessary. Question 2: The model provided an incorrect response to the question when not provided with the context from the custom database. However, upon utilizing the custom database, it furnished a correct response, underscoring the significance of context in response accuracy. Question 3: It is evident that the model correctly responded after querying the custom database, indicating its ability to retrieve relevant information when necessary.
1 - Clone this repositoy
git clone https://github.com/Morsinaldo/GAIND-Custom-Chatbot.git
cd GAIND-Custom-Chatbot2 - Create the virtual environment
conda env create -f environment.ymlThis step uses Anaconda as the environment manager, but feel free to use another one of your choice. You also can use requirements.txt file to install the necessary libraries.
3 - Run the notebook file.
Important: You need to put your OpenAI key in the first cell to run the notebook.
# Environment variables
OPENAI_API_KEY = 'YOUR API KEY'