evaluating-rag-giskard.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f3cvnSVemTSU"
   },
   "source": [
    "# Evaluating a RAG solution with Giskard"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DUORXVgTmNYj"
   },
   "source": [
    "In the final part of this notebook, you will find the necessary code to evaluate the suitability of the responses provided by the Agent using Giskard."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KNW-LniD2I2o"
   },
   "source": [
    "#Installing libraries & Loading Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KEwdHiGFkK_S"
   },
   "source": [
    "We will download the dataset from the Hugging Face datasets library. It's a dataset with information about diseases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv, find_dotenv\n",
    "import os\n",
    "_ = load_dotenv(find_dotenv())\n",
    "\n",
    "OPENAI_API_KEY  = os.getenv('OPENAI_API_KEY') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "laSDMjqQXuj-",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "data = load_dataset(\"keivalya/MedQuad-MedicalQnADataset\", split='train')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 484
    },
    "id": "JnWZTcJiXzor",
    "outputId": "aa9e33d6-99a0-491f-94b1-4df6ab4ec902",
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = data.to_pandas()\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "hf7RQa6B5xpx",
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = data[0:100]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4FhUslovtiqn"
   },
   "source": [
    "As you can see, the medical information in the dataset is well-organized, and to someone like me, who is not an expert in the field, it appears to be quite valuable. This information could be a useful addition to any general medicine book to support primary care doctors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2sLkrHF6lWhM"
   },
   "source": [
    "Load the langchain libraries to load the document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cCBAlIb596wZ",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.document_loaders import DataFrameLoader\n",
    "from langchain.vectorstores import Chroma"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nGCkVX6xldOR"
   },
   "source": [
    "The Document is in the Answer column, and the others columns are Metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "JZX8SaTe99Uf",
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_loader = DataFrameLoader(data, page_content_column=\"Answer\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 642
    },
    "id": "HDWnTqRY-IDr",
    "outputId": "45e56803-62a0-4d42-9889-244744196848",
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_document = df_loader.load()\n",
    "display(df_document[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zPuTbSsXl3uF"
   },
   "source": [
    "We can chunk the documents. The size to which we want to split the document is a design decision. The larger it is, the larger the prompt will be, and the slower the Model's response process.\n",
    "\n",
    "We also need to consider the maximum prompt size and ensure that the document does not exceed it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8wogWV1_-lxh",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.text_splitter import CharacterTextSplitter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kg7SRkqO-f9x",
    "tags": []
   },
   "outputs": [],
   "source": [
    "text_splitter = CharacterTextSplitter(chunk_size=1250, chunk_overlap=100)\n",
    "texts = text_splitter.split_documents(df_document)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "y0dn9EfrKoeW"
   },
   "source": [
    "These warnings we see are because it can't perform the partition of the required size. This is because it waits for a page break to divide the text and does so when possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XaLWV_TzASSB",
    "outputId": "85ce0ec4-87d9-41c1-9386-0d905cb3fbca",
    "tags": []
   },
   "outputs": [],
   "source": [
    "first_doc = texts[1]\n",
    "print(first_doc.page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B2_Pt7N6Zg2X"
   },
   "source": [
    "### Initialize the Embedding Model and Vector DB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n8ROb8oMnRLD"
   },
   "source": [
    "We load the text-embedding-ada-002 model from OpenAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "U57x2_87YSpb",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "model_name = 'text-embedding-ada-002'\n",
    "#model_name = 'text-embedding-3-small'\n",
    "\n",
    "embed = OpenAIEmbeddings(\n",
    "    model=model_name,\n",
    "    openai_api_key=OPENAI_API_KEY\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cgTCwF7UMyNW"
   },
   "source": [
    "The execution of this cell may take 3 to 5 minutes. If you want it to be faster, you can reduce the number of records in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SEhQMQ8eCMj8",
    "tags": []
   },
   "outputs": [],
   "source": [
    "directory_cdb = 'chromadb/'\n",
    "chroma_db = Chroma.from_documents(\n",
    "    df_document, embed, persist_directory=directory_cdb\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KhjV-T8GoarF"
   },
   "source": [
    "We are going to create three objects.\n",
    "\n",
    "* The language model, which can be any of those from OpenAI, the most common being gpt-3.5.\n",
    "* The memory, responsible for keeping the prompt with all the necessary history.\n",
    "* The retrieval, used to obtain information stored in ChromaDB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zMRs9Klic5-Y",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain_openai import OpenAI\n",
    "from langchain.chains.conversation.memory import ConversationBufferWindowMemory\n",
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "llm=OpenAI(openai_api_key=OPENAI_API_KEY,\n",
    "           temperature=0.0)\n",
    "\n",
    "conversational_memory = ConversationBufferWindowMemory(\n",
    "    memory_key='chat_history',\n",
    "    k=4, #Number of messages stored in memory\n",
    "    return_messages=True #Must return the messages in the response.\n",
    ")\n",
    "\n",
    "qa = RetrievalQA.from_chain_type(\n",
    "    llm=llm,\n",
    "    chain_type=\"stuff\",\n",
    "    retriever=chroma_db.as_retriever()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ergrieE4o8qu"
   },
   "source": [
    "We can try the isolated Retrieval to see if the information it returns is relevant.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "id": "LaYSq0V-dxHw",
    "outputId": "63fdb5c9-fd35-4988-d48d-b76c15f69c21",
    "tags": []
   },
   "outputs": [],
   "source": [
    "qa.invoke(\"What is the main symptom of LCM?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pf9MXPeipEBO"
   },
   "source": [
    "Perfect! The information returned is exactly what we desired.\n",
    "\n",
    "## Creating the Agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FwCYrS4duqBW",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.agents import Tool, AgentExecutor\n",
    "\n",
    "#Defining the list of tool objects to be used by LangChain.\n",
    "tools = [\n",
    "    Tool(\n",
    "        name='Medical KB',\n",
    "        func=qa.run,\n",
    "        description=(\n",
    "            'use this tool when answering medical knowledge queries to get '\n",
    "            'more information about the topic'\n",
    "        )\n",
    "    )\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "JaKTzPUEvOoy",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.agents import initialize_agent, create_react_agent\n",
    "from langchain import hub\n",
    "\n",
    "prompt = hub.pull(\"hwchase17/react-chat\")\n",
    "agent = create_react_agent(\n",
    "    #agent='chat-conversational-react-description',\n",
    "    tools=tools,\n",
    "    llm=llm,\n",
    "    prompt=prompt\n",
    "    #verbose=True,\n",
    "    #max_iterations=3,\n",
    "    #early_stopping_method='generate',\n",
    "    #memory=conversational_memory\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "5sjTL1EU1vNW",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create an agent executor by passing in the agent and tools\n",
    "agent_executor = AgentExecutor(agent=agent,\n",
    "                               tools=tools,\n",
    "                               verbose=True,\n",
    "                               memory=conversational_memory,\n",
    "                               max_iterations=30,\n",
    "                               max_execution_time=600,\n",
    "                               #early_stopping_method='generate',\n",
    "                               handle_parsing_errors=True\n",
    "                               )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IlxUBWKcvzeP"
   },
   "source": [
    "### Using the Conversational Agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZZapCP4Pv2kz"
   },
   "source": [
    "To make queries we simply call the `agent` directly.\n",
    "\n",
    "First i will try a order not related to the Medical field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "85vipqC02deV",
    "outputId": "59548f2e-91d1-4b2a-c397-69ddd938c2d0",
    "tags": []
   },
   "outputs": [],
   "source": [
    "agent_executor.invoke({\"input\": \"Give me the area of square of 2x2\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8suStMR7G11e"
   },
   "source": [
    "Perfect, the model has responded without accessing the configured knowledge database.\n",
    "\n",
    "Now I will try with a question that is also not related to health."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9YEsxEuCVgMv",
    "outputId": "5ea10363-9721-4d39-c238-854fb0c2ee72",
    "tags": []
   },
   "outputs": [],
   "source": [
    "agent_executor.invoke({\"input\": \"Do you know who is Clark Kent?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lyd0XJ3CHjVb"
   },
   "source": [
    "It has not accessed either, as the model has been able to identify that it is not a question related to the database that LangChain provides.\n",
    "\n",
    "Now it's time to try with a question related to Medicine. Let's see if the model can understand that it should first look for information in the vector database at its disposal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Wtwgfuy158LV",
    "tags": []
   },
   "outputs": [],
   "source": [
    " agent_executor.memory.clear()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RJoAhy76vzAB",
    "outputId": "277a9922-caac-4185-9cc0-15a61fcb11e6",
    "tags": []
   },
   "outputs": [],
   "source": [
    "agent_executor.invoke({\"input\": \"\"\"I have a patient that can have Botulism,\n",
    "how can I confirm the diagnose?\"\"\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8lsaoF8nJNnR"
   },
   "source": [
    "Perfect, the most important thing for us is that it has been able to identify that it should go to the medical database to search for information about the symptoms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mQeicHTj2pmY",
    "outputId": "440daeda-d980-4d11-ea0e-bff5745035d3",
    "tags": []
   },
   "outputs": [],
   "source": [
    "agent_executor.invoke({\"input\": \"Is this an important illness?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "quf3qyI_aVz3"
   },
   "source": [
    "And the memory works perfectly. We can maintain a conversation, taking into account that the model knows the previous questions and answers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FQwHcAMi3e85"
   },
   "source": [
    "## Evaluating the solution with Giskard\n",
    "\n",
    "Install and Load the libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6Em-qkfK637x",
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install --default-timeout=100 \"giskard[llm]\"\n",
    "from giskard.rag import KnowledgeBase, generate_testset, evaluate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qXN4hGqDoMg1"
   },
   "source": [
    "Is necesary to create a Dataframe with just the column containing the information used to create the RAG system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 363
    },
    "id": "yeIMrHlhMJDF",
    "outputId": "8df35d55-9231-4c81-940a-7fbe03c3f367",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df_giskard = pd.DataFrame([d.page_content for d in df_document], columns=[\"text\"])\n",
    "df_giskard.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VK3hgB-MoWLj"
   },
   "source": [
    "Using the information from the dataset, we ask Giskard to create a Knowledge Base, which is nothing more than a set of questions along with their respective answers. Both the questions and answers are generated by OpenAI's most advanced model, which is why it requires our OpenAI key to be provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iPaw4w9c_Gbo",
    "tags": []
   },
   "outputs": [],
   "source": [
    "kb_giskard = KnowledgeBase(df_giskard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "a1375f0fef4d43318ce4d893fe00cee8",
      "047b1132836c43f789ee3ffa794fb78a",
      "79de9c8407214fe5b6336a06e4bb0349",
      "816cd7664bd143dba6cadc7e44ee1eb1",
      "0d5d96c8d3d44f789bb6a01b658c1d2a",
      "df6477401b054940900648bfddbb8fa8",
      "e24cd9cfbd3347fd8efab530ff2c035f",
      "44c19c4eed39471695be2866ae2bbc16",
      "5794c634f988490c819e1cca2b8fb3d9",
      "f461e8947b584ec8a9ea0d8e190d5f34",
      "a85754523d064103863f4e5bc78afcc1"
     ]
    },
    "id": "VcSnDxElKQmw",
    "outputId": "c19c5b13-e1be-4e4d-ab83-865fe2f8dc39",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The more questions you generate, the more you will be charged.\n",
    "test_questions = generate_testset(\n",
    "    kb_giskard,\n",
    "    num_questions=30,\n",
    "    agent_description=\"Medical assistant for diagnosis and treatment support.\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "rYH3J0MXTJl1",
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_test_questions = test_questions.to_pandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 552
    },
    "id": "rvHhO_oDThft",
    "outputId": "5271f070-bbef-4772-b6b3-9befb5bf5e14",
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_test_questions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "21tFRKfXC2wF"
   },
   "source": [
    "A function is created that will be called from Giskard's evaluate function. This function receives the question and returns the agent's response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# df_test_questions['reference_context'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "InRbIK2_V2HU",
    "tags": []
   },
   "outputs": [],
   "source": [
    "def use_agent(question, history=None):\n",
    "    return agent_executor.invoke({\"input\": question})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "5a72094bcf8849478e7f67534c82b986",
      "a8e8bf71c3ec48d48651815da43ed87b",
      "24465a52647144f68e1790097be7322f",
      "986027d02f924baa8ab8577b784c0ae0",
      "fc943a9d4af04f2d8fa3fb84f28e93c9",
      "79136cd9901f4c9897ab4aece1ed4dd9",
      "24dd887e6a6e463f83dda16487b3a3c4",
      "90ad8de1a70840acb49adc1ade25987f",
      "e0ebf112d8a5438f8011c58a67b46e91",
      "777dd928e66343bb853d84ce41fee52d",
      "cf3686a938fc4d5e837b508c05105252",
      "0467453ef02b44b184966eac732df1f8",
      "75ede69d28f34af2874d18b8027acb2c",
      "f71d882f15934e7ea939cdd7c2352c98",
      "effa5f9248464a649bc9f63c46c9aa1d",
      "f7424a8cd83940fe82e1be8813a7df35",
      "c941895af5614ff087907ec478808184",
      "0ec6f8c9dffe41d7aff4be8ee208556d",
      "30a0be6e8f5240c0b852d2633dd5827e",
      "d73162b994cd45168c10197d8d1bfeb8",
      "c3b15d415d254e10a9c79de92cc043c1",
      "f74fd76b49344088829f405db7e82e68"
     ]
    },
    "id": "D3fm-_7RU4y9",
    "outputId": "8086e9e6-1a79-4d95-95f7-44c5763d031c",
    "tags": []
   },
   "outputs": [],
   "source": [
    "report = evaluate(use_agent, testset=test_questions, knowledge_base=kb_giskard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 269
    },
    "id": "WNLuWCB2YJwO",
    "outputId": "748954b5-40c8-4346-e07e-f0753480cd78"
   },
   "outputs": [],
   "source": [
    "# Summary with the results.\n",
    "report.correctness_by_question_type()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 319
    },
    "id": "EsBU-rL4d1lM",
    "outputId": "2792852d-14d0-45bd-f1da-dc6c08c3b9e3"
   },
   "outputs": [],
   "source": [
    "# Obtaining the incorrect answers\n",
    "failures = report.get_failures()[:2]\n",
    "failures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 70
    },
    "id": "VBFOYCMTeOUd",
    "outputId": "cdba7efd-e15d-4447-ac18-bf6a2369b64e"
   },
   "outputs": [],
   "source": [
    "# Giskard explains the reasons why it considers the answers to be incorrect.\n",
    "failures['correctness_reason'].iloc[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gdyhyd6nJnwN"
   },
   "source": [
    "\n",
    "# Conclusions.\n",
    "The experiment has been a small success. The Vectorial database has been configured and filled with information from the dataset. A LangChain agent has been created, and it has been able to retrieve information from the database only when necessary. Don't forget that our ChatBot has memory.\n",
    "\n",
    "All of this in just a few lines of code!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ykg5TYA033yR"
   },
   "source": [
    "---"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "gpuType": "V100",
   "include_colab_link": true,
   "machine_shape": "hm",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0467453ef02b44b184966eac732df1f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_75ede69d28f34af2874d18b8027acb2c",
       "IPY_MODEL_f71d882f15934e7ea939cdd7c2352c98",
       "IPY_MODEL_effa5f9248464a649bc9f63c46c9aa1d"
      ],
      "layout": "IPY_MODEL_f7424a8cd83940fe82e1be8813a7df35"
     }
    },
    "047b1132836c43f789ee3ffa794fb78a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_df6477401b054940900648bfddbb8fa8",
      "placeholder": "​",
      "style": "IPY_MODEL_e24cd9cfbd3347fd8efab530ff2c035f",
      "value": "Generating questions: 100%"
     }
    },
    "0d5d96c8d3d44f789bb6a01b658c1d2a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0ec6f8c9dffe41d7aff4be8ee208556d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "24465a52647144f68e1790097be7322f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_90ad8de1a70840acb49adc1ade25987f",
      "max": 30,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e0ebf112d8a5438f8011c58a67b46e91",
      "value": 30
     }
    },
    "24dd887e6a6e463f83dda16487b3a3c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "30a0be6e8f5240c0b852d2633dd5827e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "44c19c4eed39471695be2866ae2bbc16": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5794c634f988490c819e1cca2b8fb3d9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5a72094bcf8849478e7f67534c82b986": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a8e8bf71c3ec48d48651815da43ed87b",
       "IPY_MODEL_24465a52647144f68e1790097be7322f",
       "IPY_MODEL_986027d02f924baa8ab8577b784c0ae0"
      ],
      "layout": "IPY_MODEL_fc943a9d4af04f2d8fa3fb84f28e93c9"
     }
    },
    "75ede69d28f34af2874d18b8027acb2c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c941895af5614ff087907ec478808184",
      "placeholder": "​",
      "style": "IPY_MODEL_0ec6f8c9dffe41d7aff4be8ee208556d",
      "value": "Correctness evaluation: 100%"
     }
    },
    "777dd928e66343bb853d84ce41fee52d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "79136cd9901f4c9897ab4aece1ed4dd9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "79de9c8407214fe5b6336a06e4bb0349": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_44c19c4eed39471695be2866ae2bbc16",
      "max": 30,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5794c634f988490c819e1cca2b8fb3d9",
      "value": 30
     }
    },
    "816cd7664bd143dba6cadc7e44ee1eb1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f461e8947b584ec8a9ea0d8e190d5f34",
      "placeholder": "​",
      "style": "IPY_MODEL_a85754523d064103863f4e5bc78afcc1",
      "value": " 30/30 [03:56&lt;00:00,  8.22s/it]"
     }
    },
    "90ad8de1a70840acb49adc1ade25987f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "986027d02f924baa8ab8577b784c0ae0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_777dd928e66343bb853d84ce41fee52d",
      "placeholder": "​",
      "style": "IPY_MODEL_cf3686a938fc4d5e837b508c05105252",
      "value": " 30/30 [01:11&lt;00:00,  1.25s/it]"
     }
    },
    "a1375f0fef4d43318ce4d893fe00cee8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_047b1132836c43f789ee3ffa794fb78a",
       "IPY_MODEL_79de9c8407214fe5b6336a06e4bb0349",
       "IPY_MODEL_816cd7664bd143dba6cadc7e44ee1eb1"
      ],
      "layout": "IPY_MODEL_0d5d96c8d3d44f789bb6a01b658c1d2a"
     }
    },
    "a85754523d064103863f4e5bc78afcc1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a8e8bf71c3ec48d48651815da43ed87b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_79136cd9901f4c9897ab4aece1ed4dd9",
      "placeholder": "​",
      "style": "IPY_MODEL_24dd887e6a6e463f83dda16487b3a3c4",
      "value": "Asking questions to the agent: 100%"
     }
    },
    "c3b15d415d254e10a9c79de92cc043c1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c941895af5614ff087907ec478808184": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cf3686a938fc4d5e837b508c05105252": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d73162b994cd45168c10197d8d1bfeb8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "df6477401b054940900648bfddbb8fa8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e0ebf112d8a5438f8011c58a67b46e91": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e24cd9cfbd3347fd8efab530ff2c035f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "effa5f9248464a649bc9f63c46c9aa1d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c3b15d415d254e10a9c79de92cc043c1",
      "placeholder": "​",
      "style": "IPY_MODEL_f74fd76b49344088829f405db7e82e68",
      "value": " 30/30 [00:58&lt;00:00,  2.43s/it]"
     }
    },
    "f461e8947b584ec8a9ea0d8e190d5f34": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f71d882f15934e7ea939cdd7c2352c98": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_30a0be6e8f5240c0b852d2633dd5827e",
      "max": 30,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d73162b994cd45168c10197d8d1bfeb8",
      "value": 30
     }
    },
    "f7424a8cd83940fe82e1be8813a7df35": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f74fd76b49344088829f405db7e82e68": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fc943a9d4af04f2d8fa3fb84f28e93c9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}