11_1_Instruction_fine_tuning__M.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "08982c92-81fd-4cce-9653-8d935e58f0cf"
      },
      "source": [
        "<p style=\"text-align:center\">\n",
        "    <a href=\"https://skills.network\" target=\"_blank\">\n",
        "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n",
        "    </a>\n",
        "</p>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9f19a4e7-07e9-4c56-b01d-e3e0eec35d94"
      },
      "source": [
        "#  Instruction-Tuning with LLMs\n",
        "\n",
        "\n",
        "Instruction-based fine-tuning, referred to as instruction GPT. It trains the language models to follow specific instructions and generate appropriate responses. For instruction-tuning, the dataset plays an important role as it provides structured examples of instructions, contexts, and responses, allowing the model to learn how to handle various tasks effectively. Instruction GPT often uses human feedback to refine and improve model performance; however, this lab doesn't cover this aspect.\n",
        "\n",
        "The context and instruction are concatenated to form a single input sequence that the model can understand and use to generate the correct response.\n",
        "\n",
        "#### Context and instruction\n",
        "\n",
        "\t•\tInstruction: A command to specify what the model should do\n",
        "\t•\tContext: Additional information or background required for performing the instruction\n",
        "\t•\tCombined input: The instruction and context combine together into a single input sequence\n",
        "    \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "71c39064-e6c5-4ad3-b504-cde763b7fc5c"
      },
      "source": [
        "Let's review certain examples for various templates:\n",
        "\n",
        "---\n",
        "#### Response template\n",
        "Template: `### Question: {question}\\n ### Answer: {answer}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Question: What is the capital of France?\n",
        "### Answer: Paris\n",
        "```\n",
        "\n",
        "---\n",
        "#### Conversation template\n",
        "\n",
        "Template: `### User: {user_input}\\n ### Bot: {bot_response}`\n",
        "Example:\n",
        "```\n",
        "### User: How are you today?\n",
        "### Bot: I'm doing great, thank you! How can I assist you today?\n",
        "```\n",
        "\n",
        "---\n",
        "#### Instruction and output template\n",
        "\n",
        "Template: `### Instruction: {instruction}\\n ### Output: {output}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Instruction: Translate the following sentence to Spanish: \"Hello, how are you?\"\n",
        "### Output: \"Hola, ¿cómo estás?\"\n",
        "```\n",
        "\n",
        "---\n",
        "#### Completion template\n",
        "\n",
        "Template: `{prompt} ### Completion: {completion}`\n",
        "Example:\n",
        "```\n",
        "Once upon a time in a faraway land, ### Completion: there lived a wise old owl who knew all the secrets of the forest.\n",
        "```\n",
        "\n",
        "#### Summarization template\n",
        "\n",
        "Template: `### Text: {text}\\n ### Summary: {summary}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Text: The quick brown fox jumps over the lazy dog.\n",
        "### Summary: A fox jumps over a dog.\n",
        "```\n",
        "\n",
        "---\n",
        "#### Dialogue template\n",
        "\n",
        "Template: `### Speaker 1: {utterance_1}\\n ### Speaker 2: {utterance_2}\\n ### Speaker 1: {utterance_3}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Speaker 1: Hi, what are you doing today?\n",
        "### Speaker 2: I'm going to the park.\n",
        "### Speaker 1: That sounds fun!\n",
        "```\n",
        "\n",
        "---\n",
        "#### Code generation template\n",
        "\n",
        "Template: `### Task: {task_description}\\n ### Code: {code_output}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Task: Write a function to add two numbers in Python.\n",
        "### Code: def add(a, b):\\n    return a + b\n",
        "```\n",
        "\n",
        "---\n",
        "#### Data analysis template\n",
        "\n",
        "Template: `### Analysis Task: {task_description}\\n ### Analysis: {analysis_output}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Analysis Task: Provide insights from the sales data of Q1 2022.\n",
        "### Analysis: The sales increased by 15% compared to Q4 2021, with the highest growth in the electronics category.\n",
        "```\n",
        "\n",
        "---\n",
        "#### Recipe template\n",
        "\n",
        "Template: `### Recipe Name: {recipe_name}\\n ### Ingredients: {ingredients}\\n ### Instructions: {instructions}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Recipe Name: Chocolate Chip Cookies\n",
        "### Ingredients: Flour, Sugar, Chocolate Chips, Butter, Eggs, Vanilla Extract\n",
        "### Instructions: Mix the dry ingredients, add the wet ingredients, fold in the chocolate chips, and bake at 350°F for 10-12 minutes.\n",
        "```\n",
        "\n",
        "---\n",
        "#### Explanation template\n",
        "\n",
        "Template: `### Concept: {concept}\\n ### Explanation: {explanation}`\n",
        "\n",
        "Example:\n",
        "```\n",
        "### Concept: Photosynthesis\n",
        "### Explanation: Photosynthesis is the process by which green plants use sunlight to synthesize nutrients from carbon dioxide and water.\n",
        "```\n",
        "\n",
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9a2f906f-27c9-4a72-81b4-2a64c86b7e84"
      },
      "source": [
        "## Objectives\n",
        "\n",
        "After completing this lab, you will be able to:\n",
        "\n",
        " - Understand the various types of templates including instruction-response, question-answering, summarization, code generation, dialogue, data analysis, and explanation and their applications for fine-tuning large language models (LLMs).\n",
        " - Create and apply different templates to fine-tune LLMs for various tasks.\n",
        " - Format datasets based on the created templates to prepare them for effective model training\n",
        " - Perform instruction fine-tuning using Hugging Face libraries and tools\n",
        " - Apply Low-Rank Adaptation (LoRA) techniques to fine-tune LLMs efficiently\n",
        " - Configure and use the SFTTrainer for supervised fine-tuning of instruction-following models\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cc809c7e-6730-4b73-bb9e-c5b847df5586"
      },
      "source": [
        "The concepts presented in this lab would apply to the other template formats as well.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8f962405-9853-4581-b39f-f186dacbeea9"
      },
      "source": [
        "# __Table of contents__\n",
        "\n",
        "<ol>\n",
        "    <li>\n",
        "        <a href=\"#Setup\">Setup</a>\n",
        "        <ol>\n",
        "            <li><a href=\"#Install-required-libraries\">Install required libraries</a></li>\n",
        "            <li><a href=\"#Import-required-libraries\">Import required libraries</a></li>\n",
        "            <li><a href=\"#Define-the-device\">Define the device</a></li>\n",
        "        </ol>\n",
        "    </li>\n",
        "    <li><a href=\"#Dataset-description\">Dataset description</a></li>\n",
        "    <li><a href=\"#Model-and-tokenizer\">Model and tokenizer</a></li>\n",
        "    <li><a href=\"#Preprocessing-the-data\">Preprocessing the data</a></li>\n",
        "    <li><a href=\"#Test-the-base-model\">Test the base model</a></li>\n",
        "        <ol>\n",
        "            <li><a href=\"#BLEU-score\">BLEU score</a></li>\n",
        "        </ol>\n",
        "    <li><a href=\"#Perform-instruction-fine-tuning-with-LoRA\">Perform instruction fine-tuning with LoRA</a></li>\n",
        "    <li><a href=\"#Exercises\">Exercises</a></li>\n",
        "</ol>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a42fb869-78c6-4223-b65d-dc24cc34e64f"
      },
      "source": [
        "# Setup\n",
        "\n",
        "### Install required libraries\n",
        "\n",
        "For this lab, use the following libraries, which are __not__ preinstalled in the Skills Network Labs environment. You can install libraries by running the code in the below cell.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8d5da844-9084-467a-be77-5b9b1dc16b7c",
        "outputId": "2230ff18-c6de-4c5a-9f74-c90886ba1d16"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m1.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.6/57.6 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.0/58.0 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[33mWARNING: The candidate selected for download or install is a yanked version: 'matplotlib' candidate (version 3.9.1 at https://files.pythonhosted.org/packages/32/ad/58902b481f5a294101a53ed964d68a2c2355d55622a8e9cb09fc3f517385/matplotlib-3.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.org/simple/matplotlib/) (requires-python:>=3.9))\n",
            "Reason for being yanked: The Windows wheels, under some conditions, caused segfaults in unrelated user code.  Due to this we deleted the Windows wheels to prevent these segfaults, however this caused greater disruption as pip then began to try (and fail) to build 3.9.1 from the sdist on Windows which impacted far more users.  Yanking the whole release is the only tool available to eliminate these failures without changes to on the user side.  The sdist, OSX wheel, and manylinux wheels are all functional and there are no critical bugs in the release.   Downstream packagers should not yank their builds of Matplotlib 3.9.1.  See https://github.com/matplotlib/matplotlib/issues/28551 for details.\u001b[0m\u001b[33m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m547.8/547.8 kB\u001b[0m \u001b[31m19.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m245.8/245.8 kB\u001b[0m \u001b[31m10.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m9.3/9.3 MB\u001b[0m \u001b[31m57.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m251.6/251.6 kB\u001b[0m \u001b[31m11.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.3/78.3 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.3/8.3 MB\u001b[0m \u001b[31m36.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.4/13.4 MB\u001b[0m \u001b[31m33.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m106.7/106.7 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m316.1/316.1 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.6/3.6 MB\u001b[0m \u001b[31m37.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m112.1/112.1 kB\u001b[0m \u001b[31m8.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m9.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m13.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "gcsfs 2024.10.0 requires fsspec==2024.10.0, but you have fsspec 2024.5.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ],
      "source": [
        "!pip install -qq datasets==2.20.0 trl==0.9.6 transformers==4.42.3 peft==0.11.1 tqdm==4.66.4 numpy==1.26.4 pandas==2.2.2 matplotlib==3.9.1 seaborn==0.13.2 scikit-learn==1.5.1 sacrebleu==2.4.2 evaluate==0.4.2"
      ],
      "execution_count": 1
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c1e7628c-08af-40ac-a542-f7338bf818b6"
      },
      "source": [
        "### Import required libraries\n",
        "\n",
        "The following code imports the required libraries.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "28a5ffd4-d7f1-4c7a-b3ab-27c1df57a8b8",
        "outputId": "02e09a8d-38a6-42ca-d051-14bc0c65b52b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'\n"
          ]
        }
      ],
      "source": [
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n",
        "warnings.simplefilter('ignore')\n",
        "\n",
        "from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline\n",
        "from datasets import load_dataset\n",
        "import torch\n",
        "from torch.utils.data import Dataset\n",
        "from tqdm import tqdm\n",
        "import evaluate\n",
        "from trl import SFTConfig, SFTTrainer, DataCollatorForCompletionOnlyLM\n",
        "\n",
        "from peft import get_peft_model, LoraConfig, TaskType\n",
        "\n",
        "import pickle\n",
        "import json\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from urllib.request import urlopen\n",
        "import io"
      ],
      "execution_count": 2
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e4fc2def-e871-4667-87aa-7799e785969b"
      },
      "source": [
        "### Define the device\n",
        "\n",
        "The below code will set your device to 'cuda' if your device is compatible with GPU, otherwise, you can use 'cpu'.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c7c853b9-4029-4437-a893-c4b85e083b48"
      },
      "outputs": [],
      "source": [
        "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')"
      ],
      "execution_count": 3
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6fa914f3-d215-4d04-9421-666f410cac1c"
      },
      "source": [
        "# Dataset description\n",
        "\n",
        "Use the below sentences to download the CodeAlpaca 20k dataset, a programming code dataset. This code is available [here](https://github.com/sahil280114/codealpaca?tab=readme-ov-file#data-release). The CodeAlpaca dataset contains the following elements:\n",
        "\n",
        "\n",
        "- `instruction`: **str**, describes the task the model should perform. Each of the 20K instructions is unique.\n",
        "- `input`: **str**, optional context or input for the task. For example, when the instruction is \"Amend the following SQL query to select distinct elements\", the input is the SQL query. Around 40% of the examples have an input.\n",
        "- `output`: **str**, the answer to the instruction as generated by text-davinci-003.\n",
        "\n",
        "The following code block downloads the training split from the CodeAlpaca-20k dataset:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 182,
          "referenced_widgets": [
            "cb71b09a16774e72aeec01c0b42f7d66",
            "96b5a5169e3741e2820d75cac5190494",
            "0d17230fabdc49c2991673954cffbaac",
            "d2157dd40a694e7981a45b3dcdc1ff38",
            "ac4a9b64af5847c6be5e40721f09a8b8",
            "469908a524f4474da3b481d7c63d34e6",
            "dd11fc531a0e4df5b8d67f2affbdb97f",
            "89790b7d33de4edf81a42725598ccfdc",
            "612150bc29284f29bc6c855ce0f56f83",
            "52813aed63804e14b08ee718e1e749e3",
            "50efc449012b4896a75fc88863329284",
            "6ef8949a8ea7469bbcaea6d519e85c27",
            "094f4fc7fc6b4944874ec4b8edbf857e",
            "a3a7f97cae8246f4afada61d26c62183",
            "d979a2231053453ca94f13a9e4b7131e",
            "99f29bb6eb464adc9f76a3278ee53464",
            "129dc7c02f714b0792b56489e8d2ce10",
            "f3c0c6a080c74770a9a1d0e6a4da108c",
            "751abdc766d4495ca14d169b667127b8",
            "c2dd774f5aa7461f95b645bc08f0d38e",
            "0bf7354018534fbf925cf52a92470558",
            "6cf18909c3bb409fb4fb7266f8d86351",
            "ed1ed0052c8045f8b23dca0169dca90a",
            "e6c5dec324e54e2d8d887d8a8cb224e4",
            "899a8a9246974ac9953e7768b6afc2a0",
            "e394ca8bdcc74b58aff34c1bdba7eeae",
            "848dd9f9a081460ba7097bf11ae8d5b2",
            "48314243197c4d48872d53c8ee5882ca",
            "be2b16450065479ea4347261c85a97ce",
            "bfa52b79af9c41fa967506dcf9dbbc19",
            "4d8a2d293aa242b5b12cb1d8a90b7114",
            "795a6bbe483a4cd9b65644c91d48dd42",
            "66b7b785275c4ea7bd13d325a462efaf"
          ]
        },
        "id": "3647f1eb-bb50-444e-a25e-ee1909e4c3c6",
        "outputId": "a2a2c8c6-cb72-4815-964d-724ba20971c3"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading readme:   0%|          | 0.00/677 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "cb71b09a16774e72aeec01c0b42f7d66"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading data:   0%|          | 0.00/3.45M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6ef8949a8ea7469bbcaea6d519e85c27"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating train split:   0%|          | 0/20022 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ed1ed0052c8045f8b23dca0169dca90a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['instruction', 'input', 'output'],\n",
              "    num_rows: 20022\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "dataset = load_dataset(\"lucasmccabe-lmi/CodeAlpaca-20k\", split=\"train\")\n",
        "dataset"
      ],
      "execution_count": 4
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8caf5936-a813-407d-bff3-b40d1a67178d"
      },
      "source": [
        "Let's look at the example in the dataset:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d331b3c1-061c-49fc-885f-5743edf05317",
        "outputId": "7125e9cc-d491-4a9f-ce74-c831d3bc214e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'instruction': 'Create a JavaScript code snippet to get a list of all the elements in an array with even index.',\n",
              " 'input': 'let arr = [1,2,3,4,5,6];',\n",
              " 'output': 'let evenIndexArr = arr.filter((elem, index) => index % 2 === 0);'}"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "dataset[1000]"
      ],
      "execution_count": 5
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4bc526ff-ee58-42ba-b4ed-5261c72a8b0b"
      },
      "source": [
        "To keep things simple let's just focus on the examples that do not have any `input`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "d27d92400e834384bab9a0b1a2a56576",
            "3dbdc0ec32ba473ca50c7fe836df924b",
            "7eb2ef695c4b4dd48f4cc82bdf80b7e8",
            "ff1501322ad349b4984eeaf83c145628",
            "d4cf71ae37f64f8c8a9f978141f77933",
            "884388e5313147dd885dac5d57ec0dd2",
            "4e07ff7572aa48cd86f5ea25866e6d66",
            "1997613e0a2a400a865690b3052c14b9",
            "2154d9b562cf49258e18370200ca9884",
            "2d0461f4983f489ca4164b9ebc666bfe",
            "e51dc15c3ac94fda8701d46a707db362"
          ]
        },
        "id": "e540f688-0772-4cb5-bd25-d5147eb6e2a0",
        "outputId": "b2c2103a-125b-4033-f4e2-9dfbadc3500d"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Filter:   0%|          | 0/20022 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d27d92400e834384bab9a0b1a2a56576"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "dataset = dataset.filter(lambda example: example[\"input\"] == '')"
      ],
      "execution_count": 6
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fe42e0a5-73ce-4fc2-82da-15f16f6b017c"
      },
      "source": [
        "The original CodeAlpaca dataset may not have been shuffled. The following line indicates how to shuffle a `datasets.arrow_dataset.Dataset()` object with a random seed:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1b9e9731-d0e2-4683-b745-83734684489c"
      },
      "outputs": [],
      "source": [
        "dataset = dataset.shuffle(seed=42)"
      ],
      "execution_count": 7
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5baf1539-6619-466a-9fe3-fb605e9cf9f3",
        "outputId": "f8f9ad46-52d8-4b56-d96c-f047e9753cfe"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['instruction', 'input', 'output'],\n",
              "    num_rows: 9764\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "dataset"
      ],
      "execution_count": 8
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6f9ce5f0-5c73-4303-ada0-c8bdea5fac25"
      },
      "source": [
        "The CodeAlpaca 20k dataset has a training and test set. You can split the original training data into a train and test set by assigning 80% of the data to the training set and 20% to the testing set.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7403b499-e4c7-4fbd-83b9-0dda749790ba",
        "outputId": "c70a7efb-2e30-49d8-c20b-3e6dbb9bf2db"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['instruction', 'input', 'output'],\n",
              "        num_rows: 7811\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['instruction', 'input', 'output'],\n",
              "        num_rows: 1953\n",
              "    })\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "dataset_split = dataset.train_test_split(test_size=0.2, seed=42)\n",
        "train_dataset = dataset_split['train']\n",
        "test_dataset = dataset_split['test']\n",
        "dataset_split"
      ],
      "execution_count": 9
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "69e109c0-3408-426b-b32b-e17265a47a38"
      },
      "outputs": [],
      "source": [
        "# Select a small set of data for the resource limitation\n",
        "# This dataset will be only used for evaluation parts, not for the training\n",
        "tiny_test_dataset=test_dataset.select(range(10))\n",
        "tiny_train_dataset=train_dataset.select(range(10))"
      ],
      "execution_count": 10
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8d855728-31b6-4e74-acab-0307816594dd"
      },
      "source": [
        "# Model and tokenizer\n",
        "\n",
        "In this exercise, let's fine-tune the [`opt-350m`](https://huggingface.co/facebook/opt-350m) model from Facebook. A description of this OpenSource model was published [here](https://arxiv.org/abs/2205.01068), and the model was originally made available on [metaseq's Github repository](https://github.com/facebookresearch/metaseq).\n",
        "\n",
        "The below lines load the base model from Hugging Face:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 113,
          "referenced_widgets": [
            "c8cdc13231094592bd81c28684e9fb2a",
            "e479e23815dd4abf95c197cd21a318ff",
            "2f5c0fe22f034699aa0ea96df619ab1a",
            "4bf87dc0b6604fe7a09a6c2ac4de8c58",
            "22ebcd4623b6436cbd8c110c921e8dc8",
            "52cc00e60acb481d87ced063d2076391",
            "4e0e5e1c24454dbc9cc797a952a715f2",
            "a31e5c6e54c248c0b3aa8c7aa14f21df",
            "1a94c7ead114410087fc2cdbb6336012",
            "639a510fab45448bb954858e01436423",
            "b424b6856eb740ea9a6eb1d26211e027",
            "38764cfaeb7f4515ac584bb305028a9d",
            "535b6645d97b47d7b21773e6bf4b5df9",
            "245e372f16ea41a1b4f04f02266bf58c",
            "f7c6ab1fe5ce4b6291e48f19b2f5db4d",
            "ad13890abdae4e2f9fc7871444eb97e2",
            "b8f8556fd8184858b684472212c06917",
            "050f25d8e2ae4dd5828b563eeb6623f5",
            "3632a3e1a9e94466a73cfb163a62e671",
            "f7a309b732a04cf697ed7fb38c733cc7",
            "c8d2f259b35041c5aaeebca1bc0c62eb",
            "8d34b10a6d4247ac87759d144d6bb3d5",
            "e6421a90e19c4d02885a319aaf9fe570",
            "c895258a12c846f499c23bcec90d73f2",
            "7db77992b91944c5908e67c87f1593f0",
            "1e05781a4f2c447bb4dc1cfeb7f88dbe",
            "9eac4d6f00f340d587f4d56a4765fe83",
            "5612b4edd760418b9b9681cd0ea22452",
            "2e386e462f384cc98320acd5e20b9c96",
            "fb7cd56fb9ff49d392fd490c7de64e80",
            "a4d41ea0eb37482a9b92b7eeb9d19244",
            "60e923ea4a874703ae87ededaeb305da",
            "de44cc2a5a14497c88bf3f88c1a67692"
          ]
        },
        "id": "67feeaf2-f800-44b1-9061-3424e3901d8b",
        "outputId": "f04bd15d-e313-4a2f-cbed-72606e1d362a"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "config.json:   0%|          | 0.00/644 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c8cdc13231094592bd81c28684e9fb2a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "pytorch_model.bin:   0%|          | 0.00/663M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "38764cfaeb7f4515ac584bb305028a9d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/137 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e6421a90e19c4d02885a319aaf9fe570"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "# Base model\n",
        "model = AutoModelForCausalLM.from_pretrained(\"facebook/opt-350m\").to(device)"
      ],
      "execution_count": 11
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "85a24a79-47c1-49a5-9425-466ab2ab569e"
      },
      "source": [
        "This model comes with its own tokenizer which you will be loading here:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 145,
          "referenced_widgets": [
            "ca6e35c97b9e4a9f855edd2f199566a8",
            "c00a08b06dbf4866aa1ffc060ff263d8",
            "3ed01a138c30476d8c0aa428d8d37593",
            "f1af1952c322494287e53c07dbfcb982",
            "40ea2df4d7d24e6bbb0e1e8303121448",
            "b8a5a1c73e3a48079f8ec078c23009f1",
            "1b941551b894444086f6b9a360ce9429",
            "16a2402b708a45c7b8a751498a2aa1d4",
            "9d50bad624f247d9b4135a61d7d3d42e",
            "5be0eeb25ade4322828f2b94cce4e227",
            "9eb87b87cc41412d95c78c727f70f04c",
            "108704c293f9413b9416456dc7a9c62f",
            "7cb5bb4b9721418090f8d15b728eaf8c",
            "ed583888de654eb5b89ef9b73298b5f8",
            "67db5326b44447a8bd06fd92254c8c01",
            "6a2c6235059540db9781fc9b15a8dfbd",
            "5a9cf5d9ceb9465b97b402b1898eeaae",
            "ecfee5b1ca3b42cf82becfb29fb12348",
            "e81db49931784e19b72a2bf6e2f903ef",
            "97be585f2c774a53b2c69f99331d4603",
            "5c2169f70aee4d589ea0c455aa7620e4",
            "26ae4b273edb41e89ff724a99fbb8ec1",
            "08412af0f9774c8489203f39d7a561d6",
            "5e7ee32b076547b08a38959803d7ff84",
            "68c0d5087c4c48b29b35fe143a9a426e",
            "b8f55486a7ab4a4e9aabbdd6b5426313",
            "d977c7dfa1794df797a2a2c2437c7557",
            "c1a56fea3e8f4a9e895ce5728b3056b2",
            "c199a33a39d441d496f1c35409fb7f7e",
            "4f9ff10654894b20868fd93b7d3fc5dc",
            "c0e02fe256a24314b45e9e1f2b791630",
            "390b46ec62fe427ab72d891257eec1d0",
            "27fbd933a5854cc89d80df953276fab0",
            "6f03a18f010c4bfdb965c021f3cfd6a8",
            "e940f97c4e2b4c228f9bd6f5553a5b60",
            "c6c7cd3f24804eed963072699946b5ec",
            "e689e6c4c6df48df89756f7123e2f51e",
            "67c221f61aa644c3afa940b0ea1a6605",
            "4cee318043684da38d0bf1fe90c2d2a9",
            "98c584da8b334ee99e5dd2c7c732292f",
            "c18028c92a7b4d4eb008414ae441ec72",
            "9cf6642dcd114e508122e0e4e7519fbc",
            "118eefe4b2ab43a69dc48f777392309b",
            "0af2704b3d924b958d9e42761e39f809"
          ]
        },
        "id": "f01a9320-8f0f-4aa2-ad53-4ae92654b9ff",
        "outputId": "33bdd2e7-0fa4-4618-8c8c-df19f3811b3d"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/685 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ca6e35c97b9e4a9f855edd2f199566a8"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "108704c293f9413b9416456dc7a9c62f"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "08412af0f9774c8489203f39d7a561d6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/441 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6f03a18f010c4bfdb965c021f3cfd6a8"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "tokenizer = AutoTokenizer.from_pretrained(\"facebook/opt-350m\", padding_side='left')"
      ],
      "execution_count": 12
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "76a7e17a-83ab-4068-8f23-cf6d12c19cf0"
      },
      "source": [
        "Let's find the end of sentence (EOS) token. This is a special tokenizer token. Once this token is encountered, the model will stop generating further tokens:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "595c4498-62ee-4f4d-ad3a-7e5a01d3a59f",
        "outputId": "9daaaed6-120f-4805-a776-24b5673fe2c5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'</s>'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "tokenizer.eos_token"
      ],
      "execution_count": 13
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8768edee-e3b7-45a8-966c-71cbc45bb755"
      },
      "source": [
        "# Preprocessing the data\n",
        "\n",
        "To perform the fine-tuning, first, preprocess the data by creating functions that generate the prompt.\n",
        "\n",
        "The `formatting_prompts_func` function takes a dataset as input. For every element in the dataset format, the instruction and the output into a template using the format:\n",
        "\n",
        "```\n",
        "### Instruction:\n",
        "Translate the following sentence to Spanish: \"Hello, how are you?\"\n",
        "\n",
        "### Response:\n",
        "\"Hola, ¿cómo estás?</s>\"\n",
        "```\n",
        "\n",
        "_**Note:**_\n",
        "1. The template provided in this section may differ from the **Instruction and output template** presented in the introduction of this lab. You can replace the  `### Response:` with `### Output:` to generate similar results.\n",
        "\n",
        "2. Introducing the `</s>` end of sentence token at the end of the text informs the model to stop generating text beyond this point.\n",
        "\n",
        "Finally, the `formatting_prompts_func_no_response` function behaves similarly to the `formatting_prompts_func` except the response is not included.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3e98dadd-2bd2-40b1-8393-a1fcbe23b2cc"
      },
      "outputs": [],
      "source": [
        "def formatting_prompts_func(mydataset):\n",
        "    output_texts = []\n",
        "    for i in range(len(mydataset['instruction'])):\n",
        "        text = (\n",
        "            f\"### Instruction:\\n{mydataset['instruction'][i]}\"\n",
        "            f\"\\n\\n### Response:\\n{mydataset['output'][i]}</s>\"\n",
        "        )\n",
        "        output_texts.append(text)\n",
        "    return output_texts\n",
        "\n",
        "def formatting_prompts_func_no_response(mydataset):\n",
        "    output_texts = []\n",
        "    for i in range(len(mydataset['instruction'])):\n",
        "        text = (\n",
        "            f\"### Instruction:\\n{mydataset['instruction'][i]}\"\n",
        "            f\"\\n\\n### Response:\\n\"\n",
        "        )\n",
        "        output_texts.append(text)\n",
        "    return output_texts"
      ],
      "execution_count": 14
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "adf9610c-255f-4277-8605-3e4f51880fbf"
      },
      "source": [
        "The following code block generates the `instructions` (the part of the prompt that does not include the response), the `instructions_with_responses` (the full prompt with the response and `eos` token), and the `expected_outputs`, which are the parts of the `instructions_with_responses` that are between the `instructions` and the `eos` token.\n",
        "\n",
        "To find the `expected_outputs`, tokenize `instructions` and the `instructions_with_responses`. Then, count the number of tokens in `instructions`, and discard the equivalent amount of tokens from the beginning of the tokenized `instructions_with_responses` vector. Finally, discard the final token in `instructions_with_responses`, corresponding to the `eos` token. Decode the resulting vector using the tokenizer, resulting in the `expected_output`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ec94c9dc-e692-4f4a-9f30-2393569d5eea",
        "outputId": "163d69e7-4492-4ca1-c9e3-573645c3c2d5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 1953/1953 [00:01<00:00, 1202.51it/s]\n"
          ]
        }
      ],
      "source": [
        "expected_outputs = []\n",
        "instructions_with_responses = formatting_prompts_func(test_dataset)\n",
        "instructions = formatting_prompts_func_no_response(test_dataset)\n",
        "for i in tqdm(range(len(instructions_with_responses))):\n",
        "    tokenized_instruction_with_response = tokenizer(instructions_with_responses[i], return_tensors=\"pt\", max_length=1024, truncation=True, padding=False)\n",
        "    tokenized_instruction = tokenizer(instructions[i], return_tensors=\"pt\")\n",
        "    expected_output = tokenizer.decode(tokenized_instruction_with_response['input_ids'][0][len(tokenized_instruction['input_ids'][0])-1:], skip_special_tokens=True)\n",
        "    expected_outputs.append(expected_output)\n"
      ],
      "execution_count": 15
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "90050fd3-3579-4798-ba37-a5c4a9bd1b47"
      },
      "source": [
        "Let's look at the example to view what `instructions` include, `instructions_with_responses`, and `expected_outputs`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "727dcffa-527b-475b-92e5-99811a311836",
        "outputId": "466ac222-e356-4a00-97f4-3882a7d0d753"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "############## instructions ##############\n",
            "### Instruction:\n",
            "What type of data structure would you use to store key-value pairs in a Python program? Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "############## instructions_with_responses ##############\n",
            "### Instruction:\n",
            "What type of data structure would you use to store key-value pairs in a Python program? Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "The data structure to use for key-value pairs in Python is a dictionary. A dictionary is a data type that consists of key-value pairs, and is denoted by {} in Python. Each key has a unique value associated with it that can be accessed using the key. For example, a dictionary called \"person\" could look like this: \n",
            "\n",
            "person = {'name':'John', 'age': 32} \n",
            "\n",
            "The value of the key \"name\" can be accessed using person['name'] which returns \"John\".</s>\n",
            "\n",
            "############## expected_outputs ##############\n",
            "The data structure to use for key-value pairs in Python is a dictionary. A dictionary is a data type that consists of key-value pairs, and is denoted by {} in Python. Each key has a unique value associated with it that can be accessed using the key. For example, a dictionary called \"person\" could look like this: \n",
            "\n",
            "person = {'name':'John', 'age': 32} \n",
            "\n",
            "The value of the key \"name\" can be accessed using person['name'] which returns \"John\".\n"
          ]
        }
      ],
      "source": [
        "print('############## instructions ##############\\n' + instructions[0])\n",
        "print('############## instructions_with_responses ##############\\n' + instructions_with_responses[0])\n",
        "print('\\n############## expected_outputs ##############' + expected_outputs[0])"
      ],
      "execution_count": 16
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b8272cf5-1779-4069-a669-df25cccb24c2"
      },
      "source": [
        "Instead of keeping the instructions as-is, it's beneficial to convert the `instructions` list into a `torch` `Dataset`. The following code defines a class called `ListDataset` that inherits from `Dataset` and creates a `torch` `Dataset` from a list. This class is then used to generate a `Dataset` object from `instructions`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "93311b0e-0839-4897-a3d8-735c5f1438c8"
      },
      "outputs": [],
      "source": [
        "class ListDataset(Dataset):\n",
        "    def __init__(self, original_list):\n",
        "        self.original_list = original_list\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.original_list)\n",
        "\n",
        "    def __getitem__(self, i):\n",
        "        return self.original_list[i]\n",
        "\n",
        "instructions_torch = ListDataset(instructions)"
      ],
      "execution_count": 17
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "49794a72-016d-4bfe-b7f7-8895ea3a212f",
        "outputId": "144efa0f-6677-4e2d-fb20-5b9bee01faf5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'### Instruction:\\nWhat type of data structure would you use to store key-value pairs in a Python program? Write corresponding code in Python.\\n\\n### Response:\\n'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "instructions_torch[0]"
      ],
      "execution_count": 18
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ac72ef3e-2374-4cb3-a3c4-ac595621b0cd"
      },
      "source": [
        "# Test the base model\n",
        "\n",
        "Let's understand how the base model performs without performing fine-tuning in the model. This may involve response generation from the base, that is from the non-fine-tuned mode.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4226274d-041c-40dc-aafe-fcabc08be23b"
      },
      "source": [
        "The below code defines a text generation pipeline using the `pipeline` class from `transformers`. This pipeline is useful to generate text given by a model and a tokenizer:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4315b46f-ac31-4c16-98a5-157a8b348e2a"
      },
      "outputs": [],
      "source": [
        "gen_pipeline = pipeline(\"text-generation\",\n",
        "                        model=model,\n",
        "                        tokenizer=tokenizer,\n",
        "                        device=device,\n",
        "                        batch_size=2,\n",
        "                        max_length=50,\n",
        "                        truncation=True,\n",
        "                        padding=False,\n",
        "                        return_full_text=False)"
      ],
      "execution_count": 19
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0d4fa112-ff9f-4dfd-aadf-63774b5d6a3b"
      },
      "source": [
        "**_Note:_** The generation pipeline can generate tokens or text. If```return_tensors=True```, the pipeline returns token IDs; otherwise, it returns words. Additionally, the generation pipeline generates both the instructions *and* the responses by default. However, to assess the model's performance, exclude the generated instructions and focus on the responses. To do this, set ```return_full_text=False```.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "621125da-93dc-4d7f-a6e5-7144faa49616"
      },
      "source": [
        "The below code leverages the pre-defined generation pipeline to generate outputs using the model.\n",
        "\n",
        "**_Note:_** The code is commented out because it may take a long time for CPU. Instead of generating the raw tokens here, you can load output from this model later.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5f006be9-5f4d-4118-8176-de4ca2ed1b4b"
      },
      "outputs": [],
      "source": [
        "tokenizer.padding_side = 'left'\n",
        "\n",
        "with torch.no_grad():\n",
        "    # Due to resource limitation, only apply the function on 3 records using \"instructions_torch[:10]\"\n",
        "    pipeline_iterator= gen_pipeline(instructions_torch[:3],\n",
        "                                    max_length=50, # this is set to 50 due to resource constraint, using a GPU, you can increase it to the length of your choice\n",
        "                                    num_beams=5,\n",
        "                                    early_stopping=True,)\n",
        "\n",
        "generated_outputs_base = []\n",
        "for text in pipeline_iterator:\n",
        "    generated_outputs_base.append(text[0][\"generated_text\"])"
      ],
      "execution_count": 20
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5d651ce4-a2c4-4261-bff8-29a6080a20dd"
      },
      "source": [
        "The below code loads the generated responses for the whole dataset using machine that has access to a fast CUDA-enabled GPU:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ce2bcd0c-23c6-4e34-8fd3-30504add3574"
      },
      "outputs": [],
      "source": [
        "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/VvQRrSqS1P0_GobqtL-SKA/instruction-tuning-generated-outputs-base.pkl')\n",
        "generated_outputs_base = pickle.load(io.BytesIO(urlopened.read()))"
      ],
      "execution_count": 21
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5738588a-0016-44af-b992-c8a23ab8adbe"
      },
      "source": [
        "Let's look at the sample responses generated by the base model and the expected responses from the dataset.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d3e05e91-4e03-4d7d-950e-62db51d6e50d",
        "outputId": "9f168f9c-8478-48e0-9104-90b2750802bf"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@ Instruction 1: \n",
            "### Instruction:\n",
            "What type of data structure would you use to store key-value pairs in a Python program? Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Expected response 1: \n",
            "\n",
            "The data structure to use for key-value pairs in Python is a dictionary. A dictionary is a data type that consists of key-value pairs, and is denoted by {} in Python. Each key has a unique value associated with it that can be accessed using the key. For example, a dictionary called \"person\" could look like this: \n",
            "\n",
            "person = {'name':'John', 'age': 32} \n",
            "\n",
            "The value of the key \"name\" can be accessed using person['name'] which returns \"John\".\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Generated response 1: \n",
            "Thank you for your question.\n",
            "\n",
            "### Response:\n",
            "Thank you for your question.\n",
            "\n",
            "### Response:\n",
            "Thank you for your question.\n",
            "\n",
            "### Response:\n",
            "Thank you for your question.\n",
            "\n",
            "### Response:\n",
            "Thank you for your question.\n",
            "\n",
            "### Response:\n",
            "Thank you for your question.\n",
            "\n",
            "\n",
            "\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@ Instruction 2: \n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Expected response 2: \n",
            "\n",
            "The equation ax + b = 0 can be solved by subtracting b from both sides and then dividing both sides by a. This will yield the solution x = -b/a.\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Generated response 2: \n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "\n",
            "\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@ Instruction 3: \n",
            "### Instruction:\n",
            "Write a CSS rule to set the text size of all elements with the class “big-header” to 24px.\n",
            "\n",
            "### Response:\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Expected response 3: \n",
            "\n",
            ".big-header {\n",
            "    font-size: 24px;\n",
            "}\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Generated response 3: \n",
            "The CSS rule is set to “big-header”.\n",
            "\n",
            "### Response:\n",
            "The CSS rule is set to “big-header”.\n",
            "\n",
            "### Response:\n",
            "The CSS rule is set to “big-header”.\n",
            "\n",
            "### Response:\n",
            "The CSS rule is set to “big-header”.\n",
            "\n",
            "### Response:\n",
            "The CSS rule is set to “big-header”.\n",
            "\n",
            "\n",
            "\n",
            "@@@@@@@@@@@@@@@@@@@@\n"
          ]
        }
      ],
      "source": [
        "for i in range(3):\n",
        "    print('@@@@@@@@@@@@@@@@@@@@')\n",
        "    print('@@@@@ Instruction '+ str(i+1) +': ')\n",
        "    print(instructions[i])\n",
        "    print('\\n\\n')\n",
        "    print('@@@@@ Expected response '+ str(i+1) +': ')\n",
        "    print(expected_outputs[i])\n",
        "    print('\\n\\n')\n",
        "    print('@@@@@ Generated response '+ str(i+1) +': ')\n",
        "    print(generated_outputs_base[i])\n",
        "    print('\\n\\n')\n",
        "    print('@@@@@@@@@@@@@@@@@@@@')\n",
        ""
      ],
      "execution_count": 22
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ff88dd86-85d6-4ce6-b39f-5cac04f5ee7e"
      },
      "source": [
        "You can see that the responses generated by the base model are not up to the mark. Also, the responses have the tendency to extend and repeat the answers until they generate the maximum number of tokens. Later on, you can see that the instruction-tuning can fix both of these issues. First, the instruction fine-tuned model will be able to provide more meaningful responses. Second, because, you appended the `eos` token `<\\s>` to the output, you will teach the model via instruction fine-tuning to not generate responses without bound.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b899d38f-ac0d-400c-817e-4748ad76447c"
      },
      "source": [
        "## BLEU score\n",
        "\n",
        "Let's set up a metric that compares the generated responses and the expected responses in the test environment. In this lab, let's use the [BLEU score](https://en.wikipedia.org/wiki/BLEU), a metric originally intended to check the quality of translations made by translation models. You can calculate the BLEU scores for individual generated segments by comparing them with a set of expected outputs and average the scores for the individual segments. Depending on the implementation, BLEU scores range from 0 to 1 or from 0 to 100 (as in the implementation used herein), with higher scores indicating a better match between the model generated output and the expected output.\n",
        "\n",
        "_**Note:**_\n",
        "1. The BLEU score was originally implemented for assessing the quality of translations. However, it may not necessarily be the best metric for instruction fine-tuning in general, but it is nonetheless a useful metric that gives a sense of the alignment between the model generated output and the expected output.\n",
        "2. BLEU scores are very challenging to compare from one study to the next because it is a parametrized metric. As a result, you can employ a variant of BLEU called [SacreBLEU](https://aclanthology.org/W18-6319/) invariant to the metric's parametrization.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84,
          "referenced_widgets": [
            "63ff4733f8c74402b9ffbd7bbc3ecdae",
            "c9c3f4bb99174c78955a2aba0445ae3e",
            "37f4a74d6f644f068d26e3e3db81ab87",
            "8c34e52649f34d4a9eb284ea7c4ceec1",
            "c239cdd94256492ba080c92c2536e671",
            "23402071a458494b9ad891369b52e3c8",
            "1c5607f96a2648c0ad831d181587a2bd",
            "919c94639eb74b3a9b70614580320e49",
            "9a7672a991ca4beaacf8f0cedcdb18b5",
            "1d46a982870a48619c07265f5605ffbe",
            "2036147505ab420a8303e93d4c007649"
          ]
        },
        "id": "7bd826da-75fb-4933-9af4-db660685e686",
        "outputId": "d3bb6453-3e0e-4fc0-c838-fe67a5bbfde1"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/8.15k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "63ff4733f8c74402b9ffbd7bbc3ecdae"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['score', 'counts', 'totals', 'precisions', 'bp', 'sys_len', 'ref_len']\n",
            "0.4\n"
          ]
        }
      ],
      "source": [
        "sacrebleu = evaluate.load(\"sacrebleu\")\n",
        "results_base = sacrebleu.compute(predictions=generated_outputs_base,\n",
        "                                 references=expected_outputs)\n",
        "\n",
        "print(list(results_base.keys()))\n",
        "print(round(results_base[\"score\"], 1))"
      ],
      "execution_count": 23
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "80e71f4e-b6c6-44df-ac53-b7e751de08c3"
      },
      "source": [
        "The SacreBLEU score of 0.4/100 indicates that there is very little alignment between the base model's generated responses and the expected responses for the examples in the test dataset.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "70c64d11-a483-4879-a2ed-759ce8d14b1a"
      },
      "source": [
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4b61833f-d2d7-481f-a6cb-f777f6358f44"
      },
      "source": [
        "## Perform instruction fine-tuning with LoRA\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c617b20e-7eb4-406c-992b-22eef42957cc"
      },
      "source": [
        "To save time, let's perform instruction fine-tuning using a parameter-efficient fine-tuning (PEFT) method called low-rank adaptation (LoRA).\n",
        "First, convert the model into a PEFT model suitable for LoRA fine-tuning by defining a `LoraConfig` object from the `peft` library that outlines LoRA parameters, such as the LoRA rank and the target modules. Next, apply LoRA configuration on the model using `get_peft_model()`, which effectively converts `model` into a LoRA `model`.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8e1404f0-c932-42a4-89e4-bfaacb093f3b"
      },
      "outputs": [],
      "source": [
        "lora_config = LoraConfig(\n",
        "    r=16,  # Low-rank dimension\n",
        "    lora_alpha=32,  # Scaling factor\n",
        "    target_modules=[\"q_proj\", \"v_proj\"],  # Modules to apply LoRA\n",
        "    lora_dropout=0.1,  # Dropout rate\n",
        "    task_type=TaskType.CAUSAL_LM  # Task type should be causal language model\n",
        ")\n",
        "\n",
        "model = get_peft_model(model, lora_config)"
      ],
      "execution_count": 24
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "25e77243-b432-4504-84c2-7231424a3fcb"
      },
      "source": [
        "Instruction fine-tuning using the `SFTTrainer` has the effect of generating the instructions *and* the responses. However, for the purposes of assessing the quality of the generated text, consider only the quality of the response and not the quality of the instruction. For the purposes of calculating the BLEU score, eliminate the length of tokens corresponding to the instruction from the beginning of the tokenized model output.\n",
        "\n",
        "For example, suppose the tokenized instruction had a length of ten, but the generated text had a length of fourteen. Then the tokenized response that was kept for the purposes of calculating the BLEU score was just the four tokens at the end of the tokenized generated text because the first ten tokens represent the model's generation of the tokenized instruction.\n",
        "\n",
        "Although eliminating the first few tokens of the tokenized output worked well for the purposes of calculating BLEU. However, during fine-tuning, the first few tokens won't have an impact on the loss function. You can mask those tokens using -100 by ignoring the value of PyTorch loss functions such as [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html). By masking the tokens corresponding to the instruction with -100, only the tokens associated with the response can bear the loss.\n",
        "\n",
        "You can create such a masking manually by defining your own function. However, it is easier to instead use the `DataCollatorForCompletionOnlyLM` class from `trl`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9764e255-33a1-4dad-9814-07a3fb475640"
      },
      "outputs": [],
      "source": [
        "response_template = \"### Response:\\n\"\n",
        "collator = DataCollatorForCompletionOnlyLM(response_template, tokenizer=tokenizer)"
      ],
      "execution_count": 25
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "57885a7b-96da-41cc-a570-0442b912276f"
      },
      "source": [
        "Now, pass the `collator`, `DataCollatorForCompletionOnlyLM` object to the data collator into `SFTTrainer`, resulting in the generated instructions without bearing on the loss.\n",
        "\n",
        "To perform the training, first configure our `SFTTrainer`, and create the `SFTTrainer` object by passing to the `collator`:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81,
          "referenced_widgets": [
            "6a18ea97fe4f46f08014acdb59e22266",
            "494facd5086b42c4aaa7e2a38b735821",
            "ab8a888a4fd64c32af30ca317a432a98",
            "95b354a5b265406b8349094140e12a14",
            "4b76d12ed713489286ed51b35916df35",
            "53a6e50880c94d2f9fdca1fcc2724855",
            "a6ed54e51fa14d94b1e7dbf57e33f4d3",
            "3d041e15ca0a4d0a9e11430adae3a354",
            "1aa7e94e50c0466b8b7c334ecfed31d9",
            "da6df5170eef450cabeaa2b62c507795",
            "4e09134cc8c94837a05ceb95868186b9",
            "a1982556021b4e05ad6f33c856c51d72",
            "4f2c44d451c1469891eafdda581bb59a",
            "3926f6ca257e4eb4a6406f5d0d2c8459",
            "35d3a3214ef541b3a4ca7470e4961130",
            "7698b0045ab14a15a076754ebc6e192f",
            "e811ba36d20e4e59b9d98c429d8b59ac",
            "ded47bacb3f242daa0c2ff6496450699",
            "2079a42b86544c6c8522f5982b9b0435",
            "17d309c86eb04c76b0cd0878db3c8af7",
            "45aca610bd2e4160812b59cf879b72b2",
            "aa6da4fb803340e4aecbaab9ccfe9c00"
          ]
        },
        "id": "44e705f1-21dc-438e-a93b-65104a172fb0",
        "outputId": "38147815-6f7c-4f9c-a38b-da61176c624b"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/7811 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6a18ea97fe4f46f08014acdb59e22266"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/1953 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a1982556021b4e05ad6f33c856c51d72"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "training_args = SFTConfig(\n",
        "    output_dir=\"/tmp\",\n",
        "    num_train_epochs=10,\n",
        "    save_strategy=\"epoch\",\n",
        "    fp16=True,\n",
        "    per_device_train_batch_size=2,  # Reduce batch size\n",
        "    per_device_eval_batch_size=2,  # Reduce batch size\n",
        "    max_seq_length=1024,\n",
        "    do_eval=True\n",
        ")\n",
        "\n",
        "trainer = SFTTrainer(\n",
        "    model,\n",
        "    train_dataset=train_dataset,\n",
        "    eval_dataset=test_dataset,\n",
        "    formatting_func=formatting_prompts_func,\n",
        "    args=training_args,\n",
        "    packing=False,\n",
        "    data_collator=collator,\n",
        ")"
      ],
      "execution_count": 26
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f24c1c9e-370d-4d2f-a601-092a89af91c0"
      },
      "source": [
        "Please ignore the above warning.\n",
        "The below comments, runs the trainer, because this would take a long time on the CPU. Therefore, let's not run the trainer here.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c590d56e-ec16-4f17-a1d4-15429357dab7"
      },
      "outputs": [],
      "source": [
        "# trainer.train()"
      ],
      "execution_count": 27
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9e305829-5e05-4a95-80a9-d3964c319d05"
      },
      "source": [
        "If you want to train the trainer, the `trainer` object would have a state history for every training step. You would be able to access this state history using the below commented out line:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c3e2e232-fdc0-4845-8618-065d49f4790d"
      },
      "outputs": [],
      "source": [
        "# log_history_lora = trainer.state.log_history"
      ],
      "execution_count": 28
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1150daf3-b9d9-49b8-bd1a-82e580643102"
      },
      "source": [
        "Instead of extracting the state history above, let's load the state history of a model that was instruction fine-tuned to the above specifications on a GPU.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "03741459-c9b5-4ae4-8f65-f68517f0c790"
      },
      "outputs": [],
      "source": [
        "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/49I70jQD0-RNRg2v-eOoxg/instruction-tuning-log-history-lora.json')\n",
        "log_history_lora = json.load(io.BytesIO(urlopened.read()))"
      ],
      "execution_count": 29
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "04f4f714-fa59-46f6-9cf3-d96b4a967739"
      },
      "source": [
        "You can plot the training loss for each training step.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 487
        },
        "id": "07ed2a47-9718-46c1-94f8-a1f29e3754b0",
        "outputId": "d6623cba-dfbf-458c-ec05-bd5eec23e2f5"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x500 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA08AAAHWCAYAAACvyLK4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAACBxklEQVR4nO3dd3hU1drG4d/MpPfeIBB6J/QqIIICKip67AXk2LHrUTnnE7uIvfeCBbuCiKCCSu8l9E4ggTQCpPeZ/f2RZCTSkjDJTMJzX9dch8zsveedTY7Jw1rrXSbDMAxERERERETkpMzOLkBERERERKQhUHgSERERERGpBoUnERERERGRalB4EhERERERqQaFJxERERERkWpQeBIREREREakGhScREREREZFqUHgSERERERGpBoUnERERERGRalB4EhERlzFu3Dji4uJqde7jjz+OyWRybEEiIiJHUXgSEZFTMplM1XrMnz/f2aU6xbhx4/Dz83N2GSIiUsdMhmEYzi5CRERc2xdffFHl688++4y5c+fy+eefV3n+3HPPJTIystbvU1pais1mw9PTs8bnlpWVUVZWhpeXV63fv7bGjRvH999/T15eXr2/t4iI1B83ZxcgIiKu77rrrqvy9fLly5k7d+4xz/9TQUEBPj4+1X4fd3f3WtUH4ObmhpubfqyJiEjd0bQ9ERFxiLPPPpvOnTuzZs0aBg8ejI+PD//9738B+Omnn7jggguIiYnB09OTVq1a8dRTT2G1Wqtc459rnvbu3YvJZOLFF1/k/fffp1WrVnh6etK7d29WrVpV5dzjrXkymUzceeedzJgxg86dO+Pp6UmnTp349ddfj6l//vz59OrVCy8vL1q1asV7773n8HVU3333HT179sTb25uwsDCuu+46Dhw4UOWYtLQ0brzxRpo2bYqnpyfR0dFcfPHF7N27137M6tWrGTFiBGFhYXh7e9OiRQvGjx/vsDpFROT49E90IiLiMIcOHWLUqFFcddVVXHfddfYpfFOnTsXPz4/7778fPz8//vzzTyZNmkROTg4vvPDCKa/75Zdfkpuby6233orJZOL555/n0ksvZc+ePaccrVq8eDE//vgjd9xxB/7+/rz++utcdtllJCUlERoaCsC6desYOXIk0dHRPPHEE1itVp588knCw8NP/6ZUmDp1KjfeeCO9e/dm8uTJpKen89prr7FkyRLWrVtHUFAQAJdddhmbN2/mrrvuIi4ujoyMDObOnUtSUpL96/POO4/w8HAeeeQRgoKC2Lt3Lz/++KPDahURkRMwREREamjChAnGP3+EDBkyxACMd99995jjCwoKjnnu1ltvNXx8fIyioiL7c2PHjjWaN29u/zoxMdEAjNDQUOPw4cP253/66ScDMH7++Wf7c4899tgxNQGGh4eHsWvXLvtz69evNwDjjTfesD83evRow8fHxzhw4ID9uZ07dxpubm7HXPN4xo4da/j6+p7w9ZKSEiMiIsLo3LmzUVhYaH9+1qxZBmBMmjTJMAzDOHLkiAEYL7zwwgmvNX36dAMwVq1adcq6RETEsTRtT0REHMbT05Mbb7zxmOe9vb3tf87NzSUzM5NBgwZRUFDAtm3bTnndK6+8kuDgYPvXgwYNAmDPnj2nPHf48OG0atXK/nXXrl0JCAiwn2u1Wpk3bx6XXHIJMTEx9uNat27NqFGjTnn96li9ejUZGRnccccdVRpaXHDBBbRv355ffvkFKL9PHh4ezJ8/nyNHjhz3WpUjVLNmzaK0tNQh9YmISPUoPImIiMM0adIEDw+PY57fvHkzY8aMITAwkICAAMLDw+3NJrKzs0953WbNmlX5ujJInShgnOzcyvMrz83IyKCwsJDWrVsfc9zxnquNffv2AdCuXbtjXmvfvr39dU9PT6ZMmcKcOXOIjIxk8ODBPP/886SlpdmPHzJkCJdddhlPPPEEYWFhXHzxxXzyyScUFxc7pFYRETkxhScREXGYo0eYKmVlZTFkyBDWr1/Pk08+yc8//8zcuXOZMmUKADab7ZTXtVgsx33eqMZuG6dzrjPce++97Nixg8mTJ+Pl5cWjjz5Khw4dWLduHVDeBOP7779n2bJl3HnnnRw4cIDx48fTs2dPtUoXEaljCk8iIlKn5s+fz6FDh5g6dSr33HMPF154IcOHD68yDc+ZIiIi8PLyYteuXce8drznaqN58+YAbN++/ZjXtm/fbn+9UqtWrXjggQf4/fff2bRpEyUlJbz00ktVjunXrx/PPPMMq1evZtq0aWzevJmvv/7aIfWKiMjxKTyJiEidqhz5OXqkp6SkhLfffttZJVVhsVgYPnw4M2bMICUlxf78rl27mDNnjkPeo1evXkRERPDuu+9WmV43Z84ctm7dygUXXACU74tVVFRU5dxWrVrh7+9vP+/IkSPHjJp169YNQFP3RETqmFqVi4hInRowYADBwcGMHTuWu+++G5PJxOeff+5S0+Yef/xxfv/9dwYOHMjtt9+O1WrlzTffpHPnziQkJFTrGqWlpTz99NPHPB8SEsIdd9zBlClTuPHGGxkyZAhXX321vVV5XFwc9913HwA7duxg2LBhXHHFFXTs2BE3NzemT59Oeno6V111FQCffvopb7/9NmPGjKFVq1bk5ubywQcfEBAQwPnnn++weyIiIsdSeBIRkToVGhrKrFmzeOCBB/i///s/goODue666xg2bBgjRoxwdnkA9OzZkzlz5vDggw/y6KOPEhsby5NPPsnWrVur1Q0QykfTHn300WOeb9WqFXfccQfjxo3Dx8eH5557jocffhhfX1/GjBnDlClT7B30YmNjufrqq/njjz/4/PPPcXNzo3379nz77bdcdtllQHnDiJUrV/L111+Tnp5OYGAgffr0Ydq0abRo0cJh90RERI5lMlzpn/5ERERcyCWXXMLmzZvZuXOns0sREREXoDVPIiIiQGFhYZWvd+7cyezZszn77LOdU5CIiLgcjTyJiIgA0dHRjBs3jpYtW7Jv3z7eeecdiouLWbduHW3atHF2eSIi4gK05klERAQYOXIkX331FWlpaXh6etK/f3+effZZBScREbHTyJOIiIiIiEg1aM2TiIiIiIhINSg8iYiIiIiIVMMZt+bJZrORkpKCv78/JpPJ2eWIiIiIiIiTGIZBbm4uMTExmM2nHlc648JTSkoKsbGxzi5DRERERERcRHJyMk2bNj3lcWdcePL39wfKb1BAQICTqxEREREREWfJyckhNjbWnhFO5YwLT5VT9QICAhSeRERERESk2st51DBCRERERESkGhSeREREREREqkHhSUREREREpBrOuDVPIiIiItJ4GIZBWVkZVqvV2aWIi3J3d8disTjkWgpPIiIiItIglZSUkJqaSkFBgbNLERdmMplo2rQpfn5+p30thScRERERaXBsNhuJiYlYLBZiYmLw8PCodsc0OXMYhsHBgwfZv38/bdq0Oe0RKIUnEREREWlwSkpKsNlsxMbG4uPj4+xyxIWFh4ezd+9eSktLTzs8qWGEiIiIiDRYZrN+nZWTc+SIpL7bREREREREqkHhSUREREREpBoUnkREREREGrC4uDheffXVah8/f/58TCYTWVlZdVZTY6XwJCIiIiJSD0wm00kfjz/+eK2uu2rVKm655ZZqHz9gwABSU1MJDAys1ftVV2MMaeq2JyIiIiJSD1JTU+1//uabb5g0aRLbt2+3P3f0PkSGYWC1WnFzO/Wv6+Hh4TWqw8PDg6ioqBqdI+U08uREPyUcYOSrC3nmly3OLkVERESkQTMMg4KSMqc8DMOoVo1RUVH2R2BgICaTyf71tm3b8Pf3Z86cOfTs2RNPT08WL17M7t27ufjii4mMjMTPz4/evXszb968Ktf957Q9k8nEhx9+yJgxY/Dx8aFNmzbMnDnT/vo/R4SmTp1KUFAQv/32Gx06dMDPz4+RI0dWCXtlZWXcfffdBAUFERoaysMPP8zYsWO55JJLav13duTIEW644QaCg4Px8fFh1KhR7Ny50/76vn37GD16NMHBwfj6+tKpUydmz55tP/faa68lPDwcb29v2rRpwyeffFLrWqpLI09OVFBiZVtaLk2DvZ1dioiIiEiDVlhqpeOk35zy3lueHIGPh2N+rX7kkUd48cUXadmyJcHBwSQnJ3P++efzzDPP4OnpyWeffcbo0aPZvn07zZo1O+F1nnjiCZ5//nleeOEF3njjDa699lr27dtHSEjIcY8vKCjgxRdf5PPPP8dsNnPdddfx4IMPMm3aNACmTJnCtGnT+OSTT+jQoQOvvfYaM2bMYOjQobX+rOPGjWPnzp3MnDmTgIAAHn74Yc4//3y2bNmCu7s7EyZMoKSkhIULF+Lr68uWLVvso3OPPvooW7ZsYc6cOYSFhbFr1y4KCwtrXUt1KTw5UbCPBwCH80ucXImIiIiIuIInn3ySc8891/51SEgI8fHx9q+feuoppk+fzsyZM7nzzjtPeJ1x48Zx9dVXA/Dss8/y+uuvs3LlSkaOHHnc40tLS3n33Xdp1aoVAHfeeSdPPvmk/fU33niDiRMnMmbMGADefPNN+yhQbVSGpiVLljBgwAAApk2bRmxsLDNmzODyyy8nKSmJyy67jC5dugDQsmVL+/lJSUl0796dXr16AeWjb/VB4cmJQnzLw9ORglInVyIiIiLSsHm7W9jy5AinvbejVIaBSnl5eTz++OP88ssvpKamUlZWRmFhIUlJSSe9TteuXe1/9vX1JSAggIyMjBMe7+PjYw9OANHR0fbjs7OzSU9Pp0+fPvbXLRYLPXv2xGaz1ejzVdq6dStubm707dvX/lxoaCjt2rVj69atANx9993cfvvt/P777wwfPpzLLrvM/rluv/12LrvsMtauXct5553HJZdcYg9hdUlrnpwoxNcd0MiTiIiIyOkymUz4eLg55WEymRz2OXx9fat8/eCDDzJ9+nSeffZZFi1aREJCAl26dKGk5OS/P7q7ux9zf04WdI53fHXXctWVm266iT179nD99dezceNGevXqxRtvvAHAqFGj2LdvH/fddx8pKSkMGzaMBx98sM5rUnhyosppe9mFpZRZa5faRURERKTxWrJkCePGjWPMmDF06dKFqKgo9u7dW681BAYGEhkZyapVq+zPWa1W1q5dW+trdujQgbKyMlasWGF/7tChQ2zfvp2OHTvan4uNjeW2227jxx9/5IEHHuCDDz6wvxYeHs7YsWP54osvePXVV3n//fdrXU91adqeEwX5eGAygWFAVmEpYX6ezi5JRERERFxImzZt+PHHHxk9ejQmk4lHH3201lPlTsddd93F5MmTad26Ne3bt+eNN97gyJEj1Rp127hxI/7+/vavTSYT8fHxXHzxxdx888289957+Pv788gjj9CkSRMuvvhiAO69915GjRpF27ZtOXLkCH/99RcdOnQAYNKkSfTs2ZNOnTpRXFzMrFmz7K/VJYUnJ7KYTQR5u3OkoJQj+SUKTyIiIiJSxcsvv8z48eMZMGAAYWFhPPzww+Tk5NR7HQ8//DBpaWnccMMNWCwWbrnlFkaMGIHFcur1XoMHD67ytcVioaysjE8++YR77rmHCy+8kJKSEgYPHszs2bPtUwitVisTJkxg//79BAQEMHLkSF555RWgfK+qiRMnsnfvXry9vRk0aBBff/214z/4P5gMZ09mrGc5OTkEBgaSnZ1NQECAs8vhnJfms+dgPt/c0o++LUOdXY6IiIhIg1BUVERiYiItWrTAy8vL2eWccWw2Gx06dOCKK67gqaeecnY5J3Wy75WaZgONPDlZiI8He8jnSIGaRoiIiIiIa9q3bx+///47Q4YMobi4mDfffJPExESuueYaZ5dWr9QwwsmCfSv3elK7chERERFxTWazmalTp9K7d28GDhzIxo0bmTdvXr2sM3IlGnlyshCfyr2eNPIkIiIiIq4pNjaWJUuWOLsMp9PIk5P9PfKk8CQiIiIi4soUnpyscqPcIwpPIiIiIjV2hvU+k1pw5PeIwpOTVW6Ue1jT9kRERESqrbKddUFBgZMrEVdXUlL+e3Z12qqfitY8OVlIxbQ9jTyJiIiIVJ/FYiEoKIiMjAwAfHx8qrVhq5xZbDYbBw8exMfHBze3048+Ck9OVrnm6ZDCk4iIiEiNREVFAdgDlMjxmM1mmjVr5pBwrfDkZPZuewpPIiIiIjViMpmIjo4mIiKC0lJt+yLH5+HhgdnsmNVKCk9OVjnylF9ipajUipf76c/FFBERETmTWCwWh6xnETkVNYxwsgAvNyzm8iHErAL9i4mIiIiIiKtSeHIyk8n0d8c9Td0TEREREXFZCk8uwL7Xk9qVi4iIiIi4LIUnF6CRJxERERER16fw5AJC/So67mnkSURERETEZSk8uQCNPImIiIiIuD6FJxcQ4qu9nkREREREXJ3CkwuwjzypVbmIiIiIiMtSeHIBGnkSEREREXF9Ck8uINhXa55ERERERFydwpMLCPFRtz0REREREVfn1PC0cOFCRo8eTUxMDCaTiRkzZpzynLfeeosOHTrg7e1Nu3bt+Oyzz+q+0DoWXLFJ7uH8EgzDcHI1IiIiIiJyPE4NT/n5+cTHx/PWW29V6/h33nmHiRMn8vjjj7N582aeeOIJJkyYwM8//1zHldatyjVPxWU2CkutTq5GRERERESOx82Zbz5q1ChGjRpV7eM///xzbr31Vq688koAWrZsyapVq5gyZQqjR4+uqzLrnLe7BU83M8VlNg7nl+Dj4dS/FhEREREROY4GteapuLgYLy+vKs95e3uzcuVKSkuP3+a7uLiYnJycKg9XYzKZ7KNPahohIiIiIuKaGlR4GjFiBB9++CFr1qzBMAxWr17Nhx9+SGlpKZmZmcc9Z/LkyQQGBtofsbGx9Vx19dj3elJ4EhERERFxSQ0qPD366KOMGjWKfv364e7uzsUXX8zYsWMBMJuP/1EmTpxIdna2/ZGcnFyfJVebfa8nddwTEREREXFJDSo8eXt78/HHH1NQUMDevXtJSkoiLi4Of39/wsPDj3uOp6cnAQEBVR6u6O+9no4//VBERERERJyrQXYmcHd3p2nTpgB8/fXXXHjhhScceWooQnzK25Uf0bQ9ERERERGX5NTwlJeXx65du+xfJyYmkpCQQEhICM2aNWPixIkcOHDAvpfTjh07WLlyJX379uXIkSO8/PLLbNq0iU8//dRZH8Fh7CNPmrYnIiIiIuKSnBqeVq9ezdChQ+1f33///QCMHTuWqVOnkpqaSlJSkv11q9XKSy+9xPbt23F3d2fo0KEsXbqUuLi4+i7d4exrnjTyJCIiIiLikpwans4++2wMwzjh61OnTq3ydYcOHVi3bl0dV+UcalUuIiIiIuLaGvZCoUYkxEfd9kREREREXJnCk4tQtz0REREREdem8OQijt7n6WRTGUVERERExDkUnlxEUEWrcqvNIKeozMnViIiIiIjIPyk8uQhPNwt+nuX9O9RxT0RERETE9Sg8uZBg3/LRJ+31JCIiIiLiehSeXIi9455GnkREREREXI7CkwsJ1l5PIiIiIiIuS+HJhVSOPCk8iYiIiIi4HoUnF2IfedKaJxERERERl6Pw5ELsez1p5ElERERExOUoPLmQYPu0vVInVyIiIiIiIv+k8ORCQipalR/RtD0REREREZej8ORCgtWqXERERETEZSk8uZAQNYwQEREREXFZCk8upLLbXnZhKWVWm5OrERERERGRoyk8uZAgb3dMJjCM8gAlIiIiIiKuQ+HJhbhZzAR6q2mEiIiIiIgrUnhyMSFqVy4iIiIi4pIUnlxM5bqnw+q4JyIiIiLiUhSeXIy9Xbmm7YmIiIiIuBSFJxdTuVGuRp5ERERERFyLwpOLqZy2p41yRURERERci8KTi7E3jNC0PRERERERl6Lw5GLUMEJERERExDUpPLmYypEnTdsTEREREXEtCk8uxj7ypGl7IiIiIiIuReHJxYTYG0Zok1wREREREVei8ORiKqft5RWXUVxmdXI1IiIiIiJSSeHJxfh7uWExmwDIKtDok4iIiIiIq1B4cjFms4lgH22UKyIiIiLiahSeXFCwOu6JiIiIiLgchScXpI57IiIiIiKuR+HJBYX6auRJRERERMTVKDy5IPvIk9qVi4iIiIi4DIUnF1TZrvyIpu2JiIiIiLgMhScX9PfIk8KTiIiIiIirUHhyQSG+5a3KNfIkIiIiIuI6FJ5cUGWrco08iYiIiIi4DqeGp4ULFzJ69GhiYmIwmUzMmDHjlOdMmzaN+Ph4fHx8iI6OZvz48Rw6dKjui61HIeq2JyIiIiLicpwanvLz84mPj+ett96q1vFLlizhhhtu4N///jebN2/mu+++Y+XKldx88811XGn9so88adqeiIiIiIjLcHPmm48aNYpRo0ZV+/hly5YRFxfH3XffDUCLFi249dZbmTJlSl2V6BSVI09FpTYKSsrw8XDqX5OIiIiIiNDA1jz179+f5ORkZs+ejWEYpKen8/3333P++eef8Jzi4mJycnKqPFydj4cFD7fyvxqtexIRERERcQ0NKjwNHDiQadOmceWVV+Lh4UFUVBSBgYEnnfY3efJkAgMD7Y/Y2Nh6rLh2TCbT33s9aaNcERERERGX0KDC05YtW7jnnnuYNGkSa9as4ddff2Xv3r3cdtttJzxn4sSJZGdn2x/Jycn1WHHt2fd60ronERERERGX0KAW00yePJmBAwfyn//8B4CuXbvi6+vLoEGDePrpp4mOjj7mHE9PTzw9Peu71NNm3+tJ0/ZERERERFxCgxp5KigowGyuWrLFYgHAMAxnlFRntNeTiIiIiIhrcWp4ysvLIyEhgYSEBAASExNJSEggKSkJKJ9yd8MNN9iPHz16ND/++CPvvPMOe/bsYcmSJdx999306dOHmJgYZ3yEOmPf60nT9kREREREXIJTp+2tXr2aoUOH2r++//77ARg7dixTp04lNTXVHqQAxo0bR25uLm+++SYPPPAAQUFBnHPOOY2uVTlo5ElERERExNWYjMY23+0UcnJyCAwMJDs7m4CAAGeXc0KfLt3LYzM3c36XKN6+tqezyxERERERaXRqmg0a1JqnM0nltD2NPImIiIiIuAaFJxdlX/OkfZ5ERERERFyCwpOLsq95UsMIERERERGXoPDkov4eeSppdG3YRUREREQaIoUnFxXkU75JbpnNILe4zMnViIiIiIiIwpOL8nK34OtRvgHwETWNEBERERFxOoUnFxasjnsiIiIiIi5D4cmFObpd+dbUHFYmHnbItUREREREzjQKTy7M3nHPAeHpcH4J/3pnKdd8sJy07KLTvp6IiIiIyJlG4cmF2TvuOaBd+dSle8kvsVJmM1ibdOS0ryciIiIicqZReHJhf488nd5GuXnFZXy6dK/96/XJWad1PRERERGRM5HCkwsL8S1vV3663fa+WpFEdmEpJlP51+v3Z51mZSIiIiIiZx6FJxdm77Z3GtP2isusfLh4DwDjB7YAYOP+bKw2bbwrIiIiIlITCk8uLKRi2t7pjDz9uPYA6TnFRAV48Z8R7fDxsJBfYmXPwTxHlSkiIiIickZQeHJhpzvyZLUZvLdgNwA3DWqBl7uFzk0CAUjQuicRERERkRpReHJh9m57tRx5mr0xlb2HCgjycefqPs0A6BYbBMCG/dkOqVFERERE5Eyh8OTCKrvtZRWW1niNkmEYvD2/fNRp3IA4fD3dAOjatHzkSU0jRERERERqRuHJhQX7lHfbMwzILqxZu/L5Ow6yNTUHHw8L4wbE2Z+PbxoEwNbUHIrLrI4qVURERESk0VN4cmFuFjOB3uUB6nANp+6981f5qNM1fZoRVDGCBdA02JsQXw9KrQZbU3MdV6yIiIiISCOn8OTi7OueatA0YvXew6zcexh3i4mbBrWs8prJZLJP3dugqXsiIiIiItWm8OTiKqfu1WTkqXKt02U9mhIV6HXM65VT99Ynq2mEiIiIiEh1KTy5uJp23NuamsOf2zIwm+DWIa2Oe0x8rJpGiIiIiIjUlMKTi6vsuFfdvZ7eqRh1GtUlmhZhvsc9pmvFyNPug3nkFtWsEYWIiIiIyJlK4cnFVY48Hc47dXjadyifWRtSALjj7OOPOgGE+XnSJMgbw4CNBzR1T0RERESkOhSeXFywb/VHnt5buAebAWe3C6dTTOBJj9VmuSIiIiIiNaPw5OJCfKq35ikjp4jvV+8H4I6zW5/yuvbNcpOzTq9AEREREZEzhJuzC5CTqxx5WrDjIAMm/0FkoBdRAV5EBngRVfHniABPZm9MpcRqo1fzYPq0CDnldeM18iQiIiIiUiMKTy4uPjaQMD9PMvOKSckuIiW76KTH3zH0xGudjta5SSAmExzIKuRgbjHh/p6OKFdEREREpNFSeHJxEf5eLJ94Dhm5xaTlFJGeXUR6ThFpOcXl/1vxdXpOEX1bhjK0XUS1ruvn6UabCD92pOexYX8WwzpE1vEnERERERFp2BSeGgA3i5mYIG9igrwdet2uTYPYkZ7H+mSFJxERERGRU1HDiDNY5bqn9Vr3JCIiIiJySgpPZ7D4yo57+7MwDMPJ1YiIiIiIuDaFpzNY+6gAPCxmsgpKST5c6OxyRERERERcmsLTGczDzUyHmAAAEvZnObcYEREREREXp/B0hqucurdBm+WKiIiIiJyUwtMZLr5pEFC+7klERERERE5M4ekMFx9bPvK06UAOZVabk6sREREREXFdCk9nuJZhfvh5ulFYamVnRp6zyxERERERcVkKT2c4s9lElyYV6540dU9ERERE5IQUnkSb5YqIiIiIVINTw9PChQsZPXo0MTExmEwmZsyYcdLjx40bh8lkOubRqVOn+im4kbJvlquOeyIiIiIiJ+TU8JSfn098fDxvvfVWtY5/7bXXSE1NtT+Sk5MJCQnh8ssvr+NKG7fKkaftabkUlVpPefxvm9O48ZOV7D9SUMeViYiIiIi4DjdnvvmoUaMYNWpUtY8PDAwkMDDQ/vWMGTM4cuQIN954Y12Ud8aIDvQizM+TzLxiNqfk0LN58AmPXbDjIBOmraXMZvDWX7uYfGnXeqxURERERMR5GvSap48++ojhw4fTvHnzEx5TXFxMTk5OlYdUZTKZ6BZ76qYRG/ZncfsXayizGQD8lJBCblFpfZQoIiIiIuJ0DTY8paSkMGfOHG666aaTHjd58mT7iFVgYCCxsbH1VGHD0rVys9wTrHtKzMznxk9WUVBi5azWYbSO8KOgxMqMhJT6K1JERERExIkabHj69NNPCQoK4pJLLjnpcRMnTiQ7O9v+SE5Orp8CG5jKdU8bjtNxLyO3iBs+XsGh/BI6Nwng3et7ck2fZgBMW74PwzDqs1QREREREadokOHJMAw+/vhjrr/+ejw8PE56rKenJwEBAVUecqyuFXs97cnMJ7vw76l4uUWl3PjJKpIPF9IsxIdPxvXBz9ONy3o0xdPNzLa0XNapS5+IiIiInAEaZHhasGABu3bt4t///rezS2k0gn09aB7qA8DGitGn4jIrt32xhs0pOYT5efDZ+D6E+3sCEOjjzoVdYwCYtjzJOUWLiIiIiNQjp4anvLw8EhISSEhIACAxMZGEhASSksp/GZ84cSI33HDDMed99NFH9O3bl86dO9dnuY2efd3T/ixsNoMHvl3Pkl2H8PGw8Mm4PsSF+VY5/tp+5VP3Zm1IIbtAjSNEREREpHFzanhavXo13bt3p3v37gDcf//9dO/enUmTJgGQmppqD1KVsrOz+eGHHzTqVAcqN8tNSM7iqV+2MGtDKm5mE+9e15MuTQOPOb57bBDto/wpLrPxw9r99V2uiIiIiEi9Mhln2Gr/nJwcAgMDyc7O1vqnf1i19zCXv7sMi9mEtaId+atXduOS7k1OeM7ny/fx6IxNtI7wY+59gzGZTPVVroiIiIjIaalpNmiQa56kbnSKCcBswh6c/u+CDicNTgCXdIvBx8PCrow8ViYero8yRUREREScQuFJ7Hw83OhSse7p5kEtuGlQy1Oe4+/lzkXx5Y0jvlypxhEiIiIi0ngpPEkVr1/VjXev68nEUR2qfc61fZsDMGdjGofzS+qqNBERERERp1J4kiqah/oysnMUZnP11y51aRpIlyaBlFhtfL9GmxCLiIiISOOk8CQOcW3f8rblX65IwmY7o3qQiIiIiMgZQuFJHGJ0fAx+nm7sPVTAsj2HnF2OiIiIiIjDKTyJQ/h6ujGmojPftBX7nFyNiIiIiIjjKTyJw1xTMXXv983pZOQWObkaERERERHHUngSh+kQHUCPZkGU2Qy+W73f2eWIiIiIiDiUwpM41DUVbcu/Wplk32xXRERERKQxUHgSh7qwazQBXm7sP1LIwp0HnV2OiIiIiIjDKDyJQ3m5W7isZ1OgvG25iIiIiEhjofAkDle559MfW9NJzS50cjUiIiIiIo6h8CQO1zrCnz4tQrAZ8MVytS0XERERkcZB4UnqxI0D4gCYumQvh/KKnVuMiIiIiIgDKDxJnRjRKYrOTQLIL7Hy9vzdzi5HREREROS0KTxJnTCbTTx4XjsAPl++j5QsrX0SERERkYZN4UnqzJC24fRpEUJJmY03/tzp7HJERERERE6LwpPUGZPJxEMjykefvl29nz0H85xckYiIiIhI7Sk8SZ3qFRfCOe0jsNoMXpmn0ScRERERabgUnqTOPXBeWwB+Xp/C5pRsJ1cjIiIiIlI7Ck9S5zrFBDI6PgaAl37f4eRqRERERERqR+FJ6sV9w9tgMZv4c1sGq/cednY5IiIiIiI1VqvwlJyczP79++1fr1y5knvvvZf333/fYYVJ49Iy3I/LezYF4PnftmMYhpMrEhERERGpmVqFp2uuuYa//voLgLS0NM4991xWrlzJ//73P5588kmHFiiNx93D2uDhZmZl4mEW7sx0djkiIiIiIjVSq/C0adMm+vTpA8C3335L586dWbp0KdOmTWPq1KmOrE8akZggb67v1xyAF37bptEnEREREWlQahWeSktL8fT0BGDevHlcdNFFALRv357U1FTHVSeNzh1nt8LXw8KmAznM2ZTm7HJERERERKqtVuGpU6dOvPvuuyxatIi5c+cycuRIAFJSUggNDXVogdK4hPp58u9BLQF46fftlFltTq5IRERERKR6ahWepkyZwnvvvcfZZ5/N1VdfTXx8PAAzZ860T+cTOZGbB7UgyMed3Qfz+XHdAWeXIyIiIiJSLSajlgtPrFYrOTk5BAcH25/bu3cvPj4+REREOKxAR8vJySEwMJDs7GwCAgKcXc4Z6/2Fu3l29jaaBHnz54ND8HSzOLskERERETnD1DQbuNXmTQoLCzEMwx6c9u3bx/Tp0+nQoQMjRoyozSXlDHND/zg+WpzIgaxCxk9dRdtIfyL8vQj39yx/+JX/b4ivBxazydnlioiIiIjUbuTpvPPO49JLL+W2224jKyuL9u3b4+7uTmZmJi+//DK33357XdTqEBp5ch1frUxi4o8bT3qM2VS+Tuq8jpH89/wO+HrWKu+LiIiIiByjXkae1q5dyyuvvALA999/T2RkJOvWreOHH35g0qRJLh2exHVc1TuWcD9P9h7K52BeMQdz/35k5hVzKL8EmwEHc4uZtiKJZbsP8cY13ekUE+js0kVERETkDFSr8FRQUIC/vz8Av//+O5deeilms5l+/fqxb98+hxYojZfJZGJ4x8gTvl5mtXE4v4RNKdn898dN7MnMZ8zbS/nf+R24oX9zTCZN5xMRERGR+lOrbnutW7dmxowZJCcn89tvv3HeeecBkJGRoalw4jBuFjMRAV6c0z6SOfcMYniHCErKbDw2czO3fL6GrIISZ5coIiIiImeQWoWnSZMm8eCDDxIXF0efPn3o378/UD4K1b17d4cWKAIQ7OvBBzf04rHRHfGwmJm7JZ3zX1vEqr2HnV2aiIiIiJwhat2qPC0tjdTUVOLj4zGbyzPYypUrCQgIoH379g4t0pHUMKLh23Qgm7u+WkdiZj5mE9w3vC13DG2trnwiIiIiUiM1zQa1Dk+V9u/fD0DTpk1P5zL1RuGpccgrLmPSjE32TXb7twzl1au6ERng5eTKRERERKShqGk2qNW0PZvNxpNPPklgYCDNmzenefPmBAUF8dRTT2Gz2WpzSZEa8fN04+Uru/HS5fH4eFhYtucQo15bxPztGc4uTUREREQaqVqFp//973+8+eabPPfcc6xbt45169bx7LPP8sYbb/Doo49W+zoLFy5k9OjRxMTEYDKZmDFjxinPKS4u5n//+x/NmzfH09OTuLg4Pv7449p8DGkELuvZlJ/vOouO0QEczi9h3CermDxnK6VWhXgRERERcaxatSr/9NNP+fDDD7nooovsz3Xt2pUmTZpwxx138Mwzz1TrOvn5+cTHxzN+/HguvfTSap1zxRVXkJ6ezkcffUTr1q1JTU3VaNcZrlW4Hz/eMYBnZ2/ls2X7eG/BHlYmHub1q7oTG+Lj7PJEREREpJGoVXg6fPjwcZtCtG/fnsOHq9/9bNSoUYwaNarax//6668sWLCAPXv2EBISAkBcXFy1z5fGy8vdwpMXd2ZAq1D+8/0G1iVlccHri3j+X/GM7Bzl7PJEREREpBGo1bS9+Ph43nzzzWOef/PNN+natetpF3UiM2fOpFevXjz//PM0adKEtm3b8uCDD1JYWHjCc4qLi8nJyanykMZrZOdoZt89iG6xQeQUlXHbF2t47KdNFJVanV2aiIiIiDRwtRp5ev7557nggguYN2+efY+nZcuWkZyczOzZsx1a4NH27NnD4sWL8fLyYvr06WRmZnLHHXdw6NAhPvnkk+OeM3nyZJ544ok6q0lcT2yID9/d1p8Xf9/Oewv28Omyfazae4Q3r+lOy3A/Z5cnIiIiIg1UrUaehgwZwo4dOxgzZgxZWVlkZWVx6aWXsnnzZj7//HNH12hns9kwmUxMmzaNPn36cP755/Pyyy/z6aefnnD0aeLEiWRnZ9sfycnJdVafuA53i5mJozrwyY29CfH1YEtqDqPfWMxXK5PIKSqt9XUzcor4amUSk2dvJaugxIEVi4iIiIirO+19no62fv16evTogdVa8ylSJpOJ6dOnc8kll5zwmLFjx7JkyRJ27dplf27r1q107NiRHTt20KZNm1O+j/Z5OvOk5xRxz9frWL6nfD2exWyiR7MgBrcJZ1DbcLo0CTzhBruGYbAtLZd5W9KZtzWd9fuz7a/dOqQlE0d1qJfPICIiIiKOV9NsUKtpe84ycOBAvvvuO/Ly8vDzK59+tWPHDsxmc4PZpFfqX2SAF9Nu6sf7C/fw7epkEjPzWbX3CKv2HuGluTsI8nHnrNZhDG4TzuC24YT4erAi8VBFYMrgQFbVUc3YEG+SDxeyYPtBhScRERGRM4hTw1NeXl6VUaTExEQSEhIICQmhWbNmTJw4kQMHDvDZZ58BcM011/DUU09x44038sQTT5CZmcl//vMfxo8fj7e3t7M+hjQAFrOJ289uxe1ntyL5cAELdx5k0Y5MluzOJKuglFkbUpm1IRUATzczxWV/t7/3dDMzqE0YwztEck77CNwsZno+PZdtabmk5xQRGeDlrI8lIiIiIvXIqeFp9erVDB061P71/fffD5RPz5s6dSqpqakkJSXZX/fz82Pu3Lncdddd9OrVi9DQUK644gqefvrpeq9dGq7YEB+u7duca/s2p8xqIyE5i4U7M1m44yAb9mdRXGYjzM+T4R0iGN4hkoGtw/D2sFS5Rtcmgazfn83CHQe5vFeskz6JiIiIiNSnGq15OtVGtllZWSxYsKBWa57qi9Y8yclkFZSQkVtM63A/zCdYBwXw0u/beePPXVzYNZo3r+lRjxWKiIiIiKPU6ZqnwMDAU75+ww031OSSIi4lyMeDIB+PUx43pG04b/y5i8W7MrHajBM2nBARERGRxqNG4elEeymJnGm6xQbh7+VGVkEpG/Zn0b1ZsLNLEhEREZE6Vqt9nkTOdG4WM2e1DgNg4Y5MJ1cjIiIiIvVB4Umkloa0DQdgwY4MJ1ciIiIiIvVB4UmklgZXhKeE5CyyC0qdXI2IiIiI1DWFJ5Faignypk2EHzYDFu/S1D0RERGRxk7hSeQ0aOqeiIiIyJlD4UnkNAy2h6eD1GDLNBERERFpgBSeRE5DnxYheLmbSc8pZkd6nrPLEREREZE6pPAkchq83C30axkKaOqeiIiISGOn8CRymga3+XvqnoiIiIg0XgpPIqdpSLvy8LQq8QgFJWVOrkZERERE6orCk8hpahnmS9Ngb0qsNpbvOeTsckRERESkjig8iZwmk8lk77q3cIf2exIRERFprBSeRBxgSFutexIRERFp7BSeRBxgQKtQ3MwmEjPzSTpU4OxyRERERKQOKDyJOIC/lzs9mgcDsGCnRp9EREREGiOFJxEHsU/d267wJCIiItIYKTyJOEhleFq2O5OSMpuTqxERERERR1N4EnGQjtEBhPl5kF9iZc2+I84uR0REREQcTOFJxEHMZhOD2jT+rnvzt2cw+Pm/WKE9rUREROQMo/Ak4kBD7Ps9Nd7w9PofO0k6XMDUpXudXYqIiIhIvVJ4EnGgQW3CMJlgS2oOGTlFzi7H4ZIPF7A2KQuAJbsysdoM5xYkIiIiUo8UnkQcKNTPk84xgQAs3Jnp5Goc7+cNKfY/5xSVsWF/lvOKEREREalnCk8iDtaYp+7NTCgPTz4eFgAWN8KAKCIiInIiCk8iDjakXXl4WrTzYKOa1rYzPZdtabm4W0xMGNoagEUKTyIiInIGUXgScbBusUH4e7pxpKCUjQeynV2Ow8xcXz7qNKRtOKO7xgCwNukIecVlzixLREREpN4oPIk4mLvFzMDWYQD8tS3DydU4hmEY9vA0Oj6GZqE+NA/1ocxmsHy3WpaLiIjImUHhSaQOnNcpEoDPl+8jt6jUydWcvg37s9l3qAAvdzPDO5R/trMqAuLiXZq6JyIiImcGhSeROnBRfAwtw305nF/CBwv3nNa1kg4VsGF/Fhk5RU5bQ1U56jS8QyS+nm4A9g2BF+5sfI0xRERERI7HzdkFiDRGbhYzD41ox21frOWDRYlc1785Ef5eNb7OpgPZXPr2UkqsNgAsZhMR/p5EBHgRFeBJVIAXkYFeRAV4ER8bRKtwP0d/FKw2g1kVLcovio+xP9+/VShmE+w5mM+BrEKaBHk7/L1FREREXInCk0gdGdEpim6xQSQkZ/HGH7t46pLONTq/zGrj4R82UGK14ethobDUitVmkJpdRGp2Eev/cbzFbOKuc1pz59DWuFkcN6i8MvEw6TnFBHi52TsJAgR6u9MtNoi1SVks3nmQK3s3c9h7ioiIiLgihSeROmIymXhkVHuuen85X61MYvxZLWgR5lvt8z9anMjmlBwCvd2Zd/8Qgn3cycwrIS2niLTsIjJyy/83LaeIvZn5rE3K4tV5O1m44yCvXNmN5qHVf6+TqZyyN7JzFJ5uliqvndUmnLVJWSzamanwJCIiIo2ewpNIHerXMpSh7cL5a/tBXvx9O29d06Na5yUdKuCVeTsA+N/5HQj39wQgKtCLqEAviD32nBnrDvDojE2sTcri/NcW8fhFnfhXz6aYTKZa119SZmPOplQALopvcszrg9uE8fofO1myKxObzcBsrv17iYiIiLg6NYwQqWMPjWyPyQS/bEhlw/6sUx5vGAb/nb6RolIbA1qFcnmvptV6n0u6N2HOvYPoExdCfomV/3y/gQlfriWroKTWtS/edZCsglLC/Dzp3yr0mNfjY4Pwq9jTanNKTq3fR0RERKQhUHgSqWMdogMY06181GbKr9tOefyPaw+weFcmnm5mnh3TpUYjR02Dffjqln78Z0Q73MwmZm9MY+Sri1hSy3biMxPKp+xd2DUay3FGldwtZnuoUtc9ERERaewUnkTqwX3ntsXDYmbJrkMsOknIyMwr5qlftgBwz/A2xNVgjVQli9nEhKGt+fGOAbQM8yUtp4hrP1zBM79sobjMWu3rFJZY+X1LOlC+Me6JDGpTsd/TTu33JCIiIo2bwpNIPYgN8eG6fs0BeG7ONmwn2K/pqVlbyCoopUN0ADcPanla79m1aRCz7j6La/qWN3L4YFEiY95aSkZOUbXO/2NbOgUlVpoGe9OjWdAJj6vc72n1vsMUlJSdVs0iIiIirkzhSaSe3HlOa/w83dicksOsjanHvP7X9gx+SkjBbILnLu2CuwPajft4uPHsmC58cEMvQnw92JKaw1UfLCcj99QBqnLK3uj4mJNOHYwL9aFJkDelVoMViYdPu2YRERERV6XwJFJPQnw9uHVw+WjSi79tp6TMZn8tv7iM/5u+CYAbB7YgPjbIoe99bsdIfpowkJhAL/YczOfq908eoLILS5m/vXx64UUnmbIH5S3ZB7ctn7q3aIem7omIiEjj5dTwtHDhQkaPHk1MTPm/bM+YMeOkx8+fPx+TyXTMIy0trX4KFjlN/x7UgjA/T5IOF/DVyiT78y/9voMDWYU0CfLm/nPb1sl7x4b48PUt/YkJ9GL3wXyu+WAFB3OLj3vsb5vTKLHaaBPhR/so/1Ne+6zW5VP3Fu9S0wgRERFpvJwanvLz84mPj+ett96q0Xnbt28nNTXV/oiIiKijCkUcy8fDjXuHtwHgjT93kldcxvrkLKYuTQTgmTGd8fWsu+3XmoWWd+OLDvRiV0Ye13yw/LgB6ueKjXEvOsWUvUoDW4diMsGO9DzSsqu3pkpERESkoXFqeBo1ahRPP/00Y8aMqdF5ERERREVF2R9ms2YfSsNxZe9YWoT5kplXwnsLdvPwDxuwGXBxtxjOblf3/xDQPNSXr2/pR1SAFzsrAlRm3t8B6mBusb21+cm67B0tyMeDrk0CAVhczbboSYcKGP3GYl6bt7OGn0BERETEORpk6ujWrRvR0dGce+65LFmy5KTHFhcXk5OTU+Uh4kzuFjMPntcOgDf+3MW2tFyCfNx59MKO9VbDyQLU7I2p2AyIbxpYo1bplV33TtaKvVJhiZVbv1jDxgPZvD1/F3nF6tInIiIirq9Bhafo6GjeffddfvjhB3744QdiY2M5++yzWbt27QnPmTx5MoGBgfZHbGxsPVYscnznd4mia9NA+9ePXtCRMD/Peq0hLsyXr27pR2SAJzvS87j2gxUcyitm5vq/u+zVxFkV+z0t2ZV5wlbsAIZh8N/pG9maWv4PGcVlNuZV7CclIiIi4soaVHhq164dt956Kz179mTAgAF8/PHHDBgwgFdeeeWE50ycOJHs7Gz7Izk5uR4rFjk+k8nE/87vgLvFxLD2EVzao4lT6mgR5svXt/Qnwt+T7em5XP7eMtbsO4LJVPPw1KNZMD4eFjLzStiaduIR3s+X72P6ugNYzCaGtC0frZq1IeW0PoeIiIhIfWhQ4el4+vTpw65du074uqenJwEBAVUeIq6gb8tQVvx3OO9e37NaTRnqSnmA6keEvyd7DuaX19YihMgArxpdx8PNTL+WoQAs3nn8dU9r9h3myZ+3ADBxVHv+d0EHABbsOEh2YWltP4KIiIhIvWjw4SkhIYHo6GhnlyFSKyG+Hg7ZDPd0tQz346uKAAUwpnvtRsIGVUzdW3Sc8JSRW8TtX6ylzGZwQddo/n1WC9pG+tM20o9Sq8Hvm7XlgIiIiLi2uuuJXA15eXlVRo0SExNJSEggJCSEZs2aMXHiRA4cOMBnn30GwKuvvkqLFi3o1KkTRUVFfPjhh/z555/8/vvvzvoIIo1Gq3A/frpzICv2HD7lxrgnUtk0YuXewxSVWvFytwBQarVx55fryMgtpk2EH89f1tU+2nZh1xhenruDnzekcnkvrUkUERER1+XUf/JevXo13bt3p3v37gDcf//9dO/enUmTJgGQmppKUtLfG4mWlJTwwAMP0KVLF4YMGcL69euZN28ew4YNc0r9Io1NdKA3l3Rvgtlcu2mErcJ9iQ70oqTMxsrEw/bnn5uzjZWJh/HzdOPd63tW2cvqwq7lI8dLdmVyOL/k9D6AiIiISB0yGYZx4rZYjVBOTg6BgYFkZ2dr/ZNIHXjo+/V8u3o/twxuyX/P78DM9Snc/dU6AN69ricjO0cdc84Fry9ic0oOz47pwjV9m9V3ySIiInKGqmk2cP5iCxFpVM6qmLq3cMdBtqfl8vD3GwC4/exWxw1OUD51D+Dn9eq6JyIiIq5L4UlEHOqs1mGYTLAtLZd/f7qKwlIrZ7UOs28MfDyVU/dWJB4iI7eovkoVERERqRGFJxFxqBBfDzrFlA977z9SSJMgb16/ujuWk6yjig3xoVtsEDYD5mxU1z0RERFxTQpPIuJwlV33PCxm3r62ByG+Hqc8p3L0SVP3RERExFUpPImIw13XrzlntQ7jtau6ER8bVK1zLqgIT6v3HSElq7AOqxMRERGpHYUnEXG4JkHefHFTX0Z1qf4G1tGB3vSJCwFg9sbUuipNREREpNYUnkTEZVwYr6l7IiIi4roUnkTEZYzqHI3ZBOv3Z5N0qMDZ5dS5vOIySq02Z5chIiIi1aTwJCIuI9zfk/6tQgGYtbFxjz6lZBXS95l5XP7uMopKrc4uR0RERKpB4UlEXErlhrmz1tfPuqeSMhur9h7mtXk7ueK9ZQx+/i827M+q8/f9bXMa+SVWEpKzmDx7a52/38/rU5gwbS3ZBaV1/l4iIiKNlZuzCxAROdrITlE8OmMTW1Jz2H0wj1bhfg69vtVmsDU1hyW7Mlm6+xCr9h6moKTqyM9jMzfz4+0DMJlOvDfV6Zq//aD9z58u28eA1mGM6BRVJ++VVVDCxB83kldcRseYACYMbV0n7yMiItLYKTyJiEsJ9vVgYOswFuw4yKz1qdwzvM0pzykssbInM4/CEiv5JVYKS8oo+MefC0us7DtUwLI9h8gurDr6EuLrQf9WofRqHszzv25nXVIWv29Jr7MwU1RqZfmeQwAM7xDJvK3pPPT9Bjo3CaRJkLfD3+/jxYnkFZcBMH3dAe44u1WdBkMREZHGSuFJRFzO6PgYFuw4yM8bUrh7WOuT/qL/y4ZUJv20iUP5JdW+vp+nG31bhDCgdRgDWoXSLtIfs7n8PQ7llfDmX7t4/tdtDGsfgZvF8bObl+85RHGZjehAL96+tgeXv7uU9fuzufurdXxzSz+Hvmd2QSmfLNlr/3pXRh6bU3Lo3CTQYe8hIiJyplB4EhGXc16nSDx+NLMrI4/t6bm0jwo45pjMvGIm/bSJ2RvTAAjwciPY1wMfDzd8PCxHPdzw9rDg424h1M+Tvi1D6Nok8IQB5ZYhLZm2Yh+7D+bzw9r9XNm7mcM/34Id5VP2zm4XjoebmTeu7sEFry9izb4jvDJvB/8Z0d5h7/XxkkRyi8toF+lPqwhfZm9MY/q6AwpPIiIitaDwJCIuJ8DLncFtw5m3NZ1Z61OrhCfDMJhVMdp0pKAUi9nEhLNbMeGc1ni6WRzy3hOGtubpX7byytydXBTfBG+P07/u0RZUrHca0jYcgGahPjx7aRfu+modb8/fTf+WYZzVJuy03ye7sJSPlyQCcPewNni6mZm9MY2Z61OYOKp9nYyqiYiINGb6ySkiLml0xYa5szakYBgGABm5Rdz2xRru+modRwpK6RAdwE8TBnL/ee0cEpwqXdevOU2CvEnLKWLq0r0Ouy5A0qEC9mTm42Y2MaD13wFpdHwMV/eJxTDgvm8TOJhbfNrvNXXJXnKLymgb6ceozlEMbhtOsI87B3OLWbr70GlfX0RE5Eyj8CQiLml4h0i83M3sPVTApgM5/JRwgPNeWchvm9NxM5u4d3gbfpowsE6mn3m5W7j/3LYAvDN/F1kF1V9PdSoLdmQA0KN5MAFe7lVem3RhJ9pG+nEwt5j7v03AZjNq/T45RaV8tHgPAHed0waz2YSHm9neCn7GugO1vraIiMiZSuFJRFySr6cb57SPAGD8p6u45+sEsgpK6RQTwMw7z+Le4W3xcKu7/4Rd0r0J7aP8ySkq4535ux123coW5We3Cz/mNW8PC29e0wMvdzOLdmby/qI9tX6fT5fsJaeojNYRfpzfJdr+/CXdmwDw6+Y0CkrKan39+nIkv4QjNWgGIiIiUpcUnkTEZVWOkhzMLcbdYuKBc9syY8JAOsYc20DC0SxmEw+NbAfAJ0v3kpJVeNrXLC6z2qfLnd024rjHtI305/HRnQB48bftrE06UuP3yS0q5cPF5Wud7jqnNRbz390KezQLonmoDwUlVuZuSa/xtevTkfwSznt1IX0n/8FPCRopExER51N4EhGXdU77CPq3DKV/y1B+vuss7hrWBvd6bHIwtF0EfVqEUFJm49V5O077eqsSj1BYaiXC35MO0f4nPO7K3rFc2DWaMpvBXV+uO2ZfqlP5bNk+sgtLaRXuaw+glUwmE5d0Kx99+nGtaweS1/7YycHcYkrKbNzzdQJTft12WlMZRURETpfCk4i4LC93C1/d0o+vbul33Hbldc1kMvHIqPK24d+v2c/O9NzTul7leqchbcNPuneVyWTi2Uu70CzEhwNZhTzw7XqKy6zVeo+84jI+WPT3WqejR50qVU7dW7TzoEMaU9SFPQfz+GL5PgBGdS7frPid+bu55fPV5BbVLEyKiIg4isKTiMhJ9GgWzIhOkdgMeP637ad1rcr1TkOOs97pnwK83Hnj6u64W0zM25rO1e8vJyOn6JTnfbZsL1kFpbQM82V0fMxxj2kR5ku32CBsBvy8PqVmH6KePDdnG2U2g6Htwnnnup68emU3PNzMzNuawWXvLCXpUIGzSxQRkTOQwpOIyCn8Z0Q7zCaYuyWd1XsP1+oaB7IK2ZmRh9kEg1qfOjwBxMcG8eHY3gR4ubE2KYvRby5m3UnWQOUXl/HBwvJRpzv/sdbpn8ZUjD7NcMG1RMv3HOL3LelYzCb+e34HoHy07Ltb+xPh78mO9DwuemsxS3dnOrlSERE50yg8iYicQusIf67oFQvAlF+32fedqonKjXG7Nwsm0Mf9FEf/bUjbcH668yzaRPiRnlPMle8t59vVycc99vPl+zhSUEpcqA8XnWDUqdKFXaNxM5vYsD+bXRl51f8gdcxmM3j6ly0AXN0nljaRf68Ni48N4ue7ziK+aSBZBaXc8NFKPq+Y2iciIlIfFJ5ERKrh3uFt8XQzs2rvEf7YmlHj8+dvLz/n7LbVG3U6WoswX6ZPGMh5HSMpsdp46PsNPPbTJkqtNvsxBSVHjzq1we0UjTVC/TwZUlGLK3Wym5FwgE0HcvDzdOPe4W2PeT0ywItvbu3Pxd1iKLMZPDpjE/+bvrHKvRAREakrCk8iItUQFejFjQNbAPD8b9uw1qDrW0mZzd6ivDrrnY7Hz9ONd6/ryb3D2wDw6bJ9XPfhCg7llTd8+GL5Pg7ll9A81IdLup181KlSZeOI6esOuEQXu8ISKy9UrCu7Y2grwvw8j3ucl7uFV6/sxsMj22MywbQVSYz9eGW1m2qIiIjUlsKTiEg13T6kFYHe7uxIz+P7NcefOnc8a/YdIa+4jDA/DzrHBNb6/c1mE/cOb8v71/fE18PCisTDXPTmElbtPcz7FaNOE4a2PuWoU6XhHSLx83Rj/5FC1tRiPylH+2jxHlKzi2gS5M34iqB6IiaTidvPbsWHN/TC18PC0t2H+HVTWj1VKiIiZyqFJxGRagr0cefOoa0BmDxnW7XbfC/YUb7eaXCbcMwnaeJQXed1imLGhIG0CPPlQFYhl7+7jMy8EmJDvO2NIKrD28PCyIo24NPXOXfqXkZuEe/M3w3AQyPb4eVuqdZ5wzpE8u9BLQHX37dKREQaPoUnEZEaGDcwjo7RAWQVlPLYzE3VOqdyvVNtp+wdT5tIf2ZMGMjZR13zzqGta7yJcGXY+mVDqlOnvb0ydyf5JVbimwYyumv1ph1WGnPUvlXVaecuIiJSWwpPIiI14G4x8/y/umIxm5i9MY1fN6We9Pj0nCK2peViMsGgNo4LTwCB3u58NLY3/zu/A7cNacWlPZrW+Br9WoYSGeBJdmGpfR+q+rY9LZdvViUB8H8Xdqzx6FyLMF96NCvft2qmi+5bJSIijYPCk4hIDXVuEshtQ8qnij3602ayC0pPeGxli/KuTYMI8fVweC0Ws4mbB7fkkVHtazzqVHn+Jd0q9nxy0tS9Z2dvxWbAyE5R9I4LqdU1xlQER03dExGRuqTwJCJSC3ed04ZW4b4czC3mqYp9iY6ncr1TbVqU15fKrnt/bM04aRCsCwt3HGTBjoO4W0w8Mqp9ra9zYZdo3C0mtqTmsC0tx4EVioiI/E3hSUSkFrzcLTz/r66YTPD9mv32kHS0MquNRTvLn3fkeidH6xAdQPsof0qsNmafYhqiI1ltBs/O3grA9f3iiAvzrfW1gn09OKd9BADTNfokIiJ1ROFJRKSWejYPYWz/OAD+++NG8orLqry+LjmLnKIygnzciW8aVP8F1sDRez7Vl+9WJ7MtLZdAb3fuHtb6tK83pnv51L0ZCQdqtA+XiIhIdSk8iYichv+MaEfTYG8OZBXywq/bqrxWud5pUJtwLA5oUV6XLu4Wg8kEKxMPs/9IQZ2/X2GJlZfm7gDg7mFtCPI5/fVgQ9uHE+jtTnpOMcsqNiUWERFxJIUnEZHT4OvpxuRLuwDw6bJ9rNp72P7a/B3lLcpdeb1TpehAb/q3DAXgmg9WHHcaoiPN3ZrOwdximgR5c32/5g65pqebhQu7RgPw49r9DrmmiIjI0RSeRERO06A24VzRq3zK2MPfb6Co1MrB3GI2HShvXDC4AYQngP+e34GoAC+SDhcw9uOVTPhyLel1tG/SrIqW4pd0j8HDzXE/iirbtf+6OY2CkrJTHC0iIlIzCk8iIg7wvws6EuHvyZ7MfF6dt5OFFSM3nZsEEO7v6eTqqqdzk0DmPTCEf5/VArOpfOPc4S8t4NOlex26hii3qJT5Fffngi412xD3VHo0CyIu1IeCEiu/bU5z6LVFREQUnkREHCDQ252nL+kMwAeL9vDJ0kQAzm4b4cyyaszP041HL+zIzDvPIj42iNziMh6buZlL3lrCxv3ZDnmPuVvSKSmz0TLclw7R/g65ZiWTyWRvfqE9n0RExNGcGp4WLlzI6NGjiYmJwWQyMWPGjGqfu2TJEtzc3OjWrVud1SciUhPndYrigq7RWG2GfcqeK7coP5nOTQL58fYBPHVJZ/y93Nh4IJuL31rM4zM3k1t0entBzdpQ3g79wq7l/+13tDEV4WnJrsw6m3YoIiJnJqeGp/z8fOLj43nrrbdqdF5WVhY33HADw4YNq6PKRERq54mLOhHs4w6Av5cb3WODnFvQabCYTVzfrzl/PDCEi7vFYDNg6tK9DHtpAQnJWbW6ZnZBqX3vq9EVzR0crXmoL72aB2Mz4KcEjT6JiIjjODU8jRo1iqeffpoxY8bU6LzbbruNa665hv79+9dRZSIitRPm58mTF3fGZCofWXGzNPzZ0RH+Xrx2VXc+/3cf4kJ9yMgtZtJPmzCMmq+D+m1zGqVWg3aR/rSJdOyUvaON6aGpeyIi4ngN7qf6J598wp49e3jssceqdXxxcTE5OTlVHiIidWl0fAxLHzmHJy7q5OxSHGpQm3B+uH0Anm5mNuzPZvW+IzW+xs8byrvsXVhHo06VLuwSg4fFzLa0XLak6L/7IiLiGA0qPO3cuZNHHnmEL774Ajc3t2qdM3nyZAIDA+2P2NjYOq5SRKR83yRHtuB2FaF+nvY1RR8vTqzRuYfyillasXnthfGO7bL3T4E+7pzTvrxZx/R1rrfnU0mZjc+W7WXOxlRnlyIiIjXQYH6yW61WrrnmGp544gnatm1b7fMmTpxIdna2/ZGcnFyHVYqINH7jz2oBlE/BSz5cUO3zft2chtVm0CkmgBZhvnVVnt2lFVP3fkpIcWir9dO1KyOPS99ZwqSfNnP7tLV8tTLJ2SWJiEg1NZjwlJuby+rVq7nzzjtxc3PDzc2NJ598kvXr1+Pm5saff/553PM8PT0JCAio8hARkdprG+nPoDZh9gYS1fXLUV326sPZ7SII9nEnI7eYJbsy6+U9T8YwDD5btpcL31jEpgM59pHJ/03fyGyNQImINAgNJjwFBASwceNGEhIS7I/bbruNdu3akZCQQN++fZ1doojIGaNy9OmbVcnVal1+MLeY5XsqpuzV8XqnSh5uZntQ+3Gtc6fuZeQWcePUVUz6aTNFpTYGtQlj4X+GcnWfWGwG3PP1OnsXQhERcV1ODU95eXn2IASQmJhIQkICSUnlUxgmTpzIDTfcAIDZbKZz585VHhEREXh5edG5c2d8fet+CoiIiJQb0iacVuG+5BWX8e3qUweTOZtSsRkQHxtEbIhPPVRYrrLr3m+b08kvLqu39z3ar5vSGPHKQuZvP4iHm5nHRnfk0xv7EBXoxdOXdOGCLtGUWg1u+WwNa2rRhENEROqPU8PT6tWr6d69O927dwfg/vvvp3v37kyaNAmA1NRUe5ASERHXYTab7KNPU5cmnnJN0az15dPS6mpvpxPpHhtEizBfCkut/LoprV7fO6+4jIe/38BtX6zhSEEpHaMDmHXXWdw4sAVmc/nmwBaziVeu7MagNmEUlloZP3UV29LUHVBExFWZjNps1NGA5eTkEBgYSHZ2ttY/iYichsISK/2f+4OsglLeva4nIztHHfe4tOwi+j/3B4YBSx85h5gg73qt8/U/dvLy3B2c1TqML26qnynea/Yd4b5vEkg6XIDJBLcObsV957bB081y3OMLSsq47sMVrE3KItzfkx9uG0Cz0PoboRMROVPVNBs0mDVPIiLiWrw9LFzTpxlw8rblv2xMxTCgV/Pgeg9OgL21+pLdmTw7eyvvLtjN1yuT+HVTKsv3HGJbWg7pOUUUlVod8n6zNqRwxXvLSDpcQJMgb766uR+PjGp/wuAE4OPhxifj+tA+yp+DucVc99EKMnKKHFKPiIg4TvU2SxIRETmOG/rH8f7CPazce5iN+7Pp0jTwmGNm1dPGuCcSG+JD3xYhrEg8zPsL95z02CAfdx4e2Z6rK0JhTc3emMo9XydgtRlc0CWayZd1IcDLvVrnBvq489n4Pvzr3fLgdf1HK/nm1n4E+XjUqhYREXE8jTyJiEitRQV62UPRR4uPDSb7jxSwLikLkwnO7+Kc8ATw0hXx/GdEO8YPbMGlPZowrH0EPZoF0TLclxBfDyqWIJFVUMrEHzfy/K/bsNVwb6hfN6Vy91frsNoMLuvRlDeu7l7t4FQpIsCLL/7dlwh/T7an53Lj1FUUlDin0YWIiBxLa55EROS0bNyfzeg3F+NmNrHkkXOIDPCyv/begt1MnrONfi1D+PqW/k6s8uRsNoO8kjI+XpzIq/N2AnBJtxie/1e8fT+mk/ltcxoTpq2lzGZwafcmvHB5PJbKRFYL29NyueK9ZWQXljKgVSivXNmtyn0VERHH0JonERGpV12aBtInLoQyW/kmsEebVc8b49aW2WwiwMude4e35YV/dcXNbGJGQgpjP15JduHJ97GatyWdO78sD04Xd4s57eAE0C7Kn09u7I23u4Wluw8x7KUFfLQ4kTKr7bSuKyIip0fhSURETtv4s+IAmLYiicKS8sYLezPz2XggG4vZxKgTdOJzRZf3iuXjcb3x83Rj2Z5DXP7uUg5kFR732D+2pnP7tDWUWg1Gx8fwkgOCU6UezYL54fYBdIsNIq+4jKdmbeHCNxazeu9hh1xfRERqTuFJRERO27kdo4gN8SaroJQf15VvmvvLxvJRpwGtQgn183RmeTU2uG04397an8gAT3ak5zHmrSVsTsmucsxf2zK4/Yu1lFoNLugazStXxONmceyP1Y4xAfx4+wAmX9qFIB93tqXl8q93l/Gf79ZzKK/Yoe9VZrWRkJxFqUa3RKSOrdl3mMTMfGeXUSsKTyIictosZhPjBpRvmvvx4kRsNoOf1zu3y97p6hgTwPQ7BtIu0p+M3GKueHcZC3YcBGDBjoPc+sUaSqw2zu8SxWtXdnN4cKpkNpu4uk8z/nzgbK7sFQvAd2v2c85LC5i2Yl+NG1scT0pWIVe8t4xL3lrCtR+sIKug5LSvKSJyIo/O2MzQF+fbu7E2JApPIiLiEFf0aoqfpxu7D+bzydK9bEvLxc1sYkSnhjNl759igrz59rb+DGgVSn6JlfFTV/HML1u4+bPVlJTZGNkpiteu6l5nweloIb4eTPlXV364vT8dogPILizlf9M3MeadpWzcn33qC5zAn9vSOf/1RaxNygJg5d7D/OvdZew/UuCgyuVMszczn9Ts4091Fdmbmc+W1BwsZhMDWoU5u5waU3gSERGH8Pdy54qKkZHJs7cCMKhNWIPfpyjQ252pN/ZhTPcmWG0GHyxKpKTMxnkdI3n96u6410NwOlrP5iH8fOdAHhvdET9PN9YnZ3HRW4u5+6t1NZoGU2q1MXnOVsZPXU1WQSldmwby0dheRAV4sSsjjzFvL2XTgdqHMjkzZeYVc8Hri7j07aWnPQV0zb7DvLdgt0NGV8V1zN7095TuEN+G9/NB4UlERBzmxoFxmE1QVvHLjqt32asuDzczL18Rz51DW2MywYhOkbx5TY9qtTGvC24WMzcObMGfDwzh4m4xGAbMXJ/C8JcX8PD3G07Y4KJSSlYhV72/nPcWlO/NNW5AHN/d1p9hHSKZPmEA7SL9OZhbzJXvLWNhxVRFabjmbknn2g+X10sYXpl4mPwSK6nZRSzbfajW17HZDCZMW8fkOdvs02WlcZizMQ2AUZ0b5pRuhScREXGY2BAfzutYPk3Pw2Lm3E6RTq7IcUwmEw+OaEfCo+fx7nU9nRacjhYR4MVrV3Vn1l1ncU77CKw2g29WJzP0hfk8PnMzGblFx5zz17YMLnh9EWv2HcHfy413r+vB4xd1wtPNAkB0YPlUxf4t/56q+P2a/fX90RympMzGzvRc1uw7gvUMG8EwDIMPF+3hls9Xs2TXIV76fXudv+eqo7pB/ro5rdbXWZd8hLSc8u/fDacxLVVcS/LhAjYeyMZsgvMa6M8HN2cXICIijcuEoa2ZvyODMd2bEODl7uxyHC7Qx/U+U+cmgXw8rjdr9h3mxd92sGzPIaYu3cvXq5IYOyCO2wa3ws/LjZd+38G7C3YD0KVJIG9d04NmoT7HXC/Q252p43vz0Pcb+CkhhQe/W09qViF3ntMak8kxrdgdLb+4jN0H89iVcdTjYB5JhwrsI6HxTQOZfGlXOsaceiPMhq7MauOJn7fw+fJ99ufm7zhI8uECYkOO/Tt3lDX7jtj//PvmNJ66uHOt2vf/suHv4PXPTpfScM2u6MLar2UoYQ2sC2slhScREXGoLk0DSZh0Hh71vBZIytdDfXVLP5bsyuSF37aTkJzFewv28OXyJJqG+LA1NQeAsf2b898LOthHm47H083CK1d0IybIm3fm7+aluTtIyS7kqYs710uDjOooKrXyxp87mbEu5aRTFX09LBjA+v3ZXPTmYm4d0pK7zmmDl/uJP39Dlldcxp1frmX+9oOYTPC/8zswf/tBFu/K5KuVSTw0sn2dvG9BSRmbU8q/x7zczWTmlbBm3xH6tAip0XVsNoM5FetiAPs1peGbvaliyl6XhjllDxSeRESkDjTWX0obioGtwxjQKpQ/tmbw0twdbE3NYWtqDv6ebkz5V1fOr+YvLmaziYdHticm0IvHZm7mq5XJpOcU8+pV3U57VLGo1Iqb2VTrILZ0dyb//XEjew/93RUwzM+DVuF+tI6o+ogK8OJgbjGPzdzMnE1pvPXXbuZsTOPZS7vQr2XoaX0OV5OaXcj4qavZmpqDl7uZV6/sxsjO0TQN9mbxrky+XZ3MvcPb1sm004SkLKw2g+hAL/q3CuXHtQeYsym1xuEpYX8WqdlFeLtbKCy1ciCrkKyCkgbffOZMt/9IAeuTs+zrRhsqhScREZFGyGQyMbxjJOe0j2D2plRW7DnMTYNa0DzUt8bXur5/HJEBXtz99Tr+3JZBz6fm0qNZMIPbhjO4TTidYgIwn2JqVqnVxob9WSzeeYgluzJZl3wEb3cL1/ZrzrgB5devjuyCUp6dvZVvVicDEBngyaMXdmRgqzCCT9K5KyLAi3eu68mvm9KY9NMm9mTmc9X7y7m6TyyPjOpAoLfrTcesqc0p2Yyfuor0nGLC/Dz4cGxvusUGATCsQyQR/p5k5Bbz+5a0Omnmsrpiyl6vuBBGdorix7UH+G1TGpMu7Fij6Z6zN5SPOp3XKZK1SUdIPlzIlpQcBrRueG2t5W+/Vow69Y4LIcK/ev9/d0UKTyIiIo2Y2Wziwq4xp/3L8nmdovjy5n48+N169hzMZ0XiYVYkHuaF37YT6uvBWW3CGNwmnEFtwogI8MIwDHZm5LF4ZyZLd2eyfM9h8orLqlyz1FrGO/N38+GiPVzcrQk3D2pJuyj/476/YRjM2ZTGpJ82k5lXDMB1/Zrx0Mj2NRoFG9k5iv6tQpny6za+XJHEVyuT+WNrBk9e3ImRDbT7F5Q3Apnw5VoKSqy0ifDj43G9q6xtcreYuap3LK//uYtpy5PqNjw1Lw/WPh4WUrKL2LA/m/iKEHcqlX/PUN6NrbjURvLhQjYrPDV4leudLmjAU/ZA4UlERESqqUezYP584Gz2ZuazaOdBFuzIZNnuTA7ll/BTQgo/JaQA0C7Sn8MFJRzMLa5yfrCPOwNahzGwVfm0wh3puXywaA+r9h7h+zX7+X7Nfoa0DeeWwS0Z0CrUPlqRll3Eoz9tYu6WdABahfvy3GVd6R1Xs+lglQK93Xl2TBcujo9h4o8b2ZOZz21frOW8jpEMaBWKxWzCZDJhMZuwmEyYzSYsZjCbTLhbzPRr6Vr703y+bC+PzdyMzYCBrUN5+9qexx1Ju6pPM978axfL9hxiV0YerSP8HFaD1WawtiI89WwejJe7haHtIvhlYyq/bk6rdnhavz+bA1mF+HpYOLtdODvTc/l1c5qaRjRwqdmF9o24R3ZuuBung8KTiIiI1FBcmC9xYb5c3z+OkjIb65KOsHDnQRbtzGTjgWy2p+cC4Olmpk+LEM5qHcbA1mF0jK46vS8uzJfzOkWxNukIHy7aw6+b0liw4yALdhykY3QAtwxuSV5xGVPmbCO3uAw3s4k7zm7FHUNbO2RdXd+Wocy+ZxBv/bWLd+bv5vct6fxeEdBOpmvTQGbcMfCUUxVPR1Gplblb0tl0IJv8kjIKiq0UlFgpKLVSWFJGfrGVwlIr+cVlZFSE1Mt7NuWZMV1OuJ4pJsibc9pHMm9rOl+uSGLS6I4Oq3d7Wi55xWX4ebrRvmL0cGTnqPLwtCmNh0a0q9bUvcrRiXM6ROLlbqFTk/LOiKfTNKKwxMqsDSmc3yUaX0/96usMlVP2ejUPrvYUXVel7yARERGpNQ83M31bhtK3ZSj/GQGH80tYmXiIAG93ejQLrlbI6dEsmLev7cm+Q/l8vDiRb1fvZ0tqDvd+k2A/pltsEM9d1oX2UY5tM+7lbuGB89pxQddopi7ZS25RGTbDwGoz7P9rNcqnk1ltBgnJWWzYn82P6w7wr55NHVqLYRis3neEH9bs55cNqeT+Y5rjiZhM8MC5bZkw9NSt5K/t14x5W9P5fk0yD41s57DmLqv3le/v1L1ZkL0JyND2EXi4mUnMzGdHet4Jp2RWMgzDHp7Orxid6BQTCMDug3kUlljx9qh5ve8s2M3rf+xkReJhXrw8vsbny+mz/7028Cl7oPAkIiIiDhTi61HrtUPNQ3154uLO3Du8LdNW7GPq0n0UlJTxnxHtuKF/XK32C6qu9lEBPHdZ11Me996C3Uyes43nf93GqM5RDhnJSD5cwA9r9/Pj2gMkHf67e2CTIG/O7RhJoLc7Ph4WfDwseHu44ethwdvDgo+HGz4eFiICPKu9AH9wm3CaBnuz/0ghszakOiwArt7795S9Sn6ebgxuE8a8rRnM2ZR6yvC08UA2+48U4u1u4ex2EQBE+HsS5udBZl4J29Jy6N4s+KTXOJ4luzIBmLk+hYmj2hPaQPcXaqjSc4rs6+Ea+pQ9UHgSERERFxPs68Gd57ThtiGtKLUatRptqCvjBsYxbUUSSYcLeG/Bbu4/r12trpNXXMYvG1L4Yc0BVu49bH/e18PC+V2iuaxnU/rEhTh8aqDFbOLqPs144bftTFuxz2HhqXJz3H+uQxvRKYp5WzP4dVMa9w5ve9JrzN5YPrXrnPYR9r9zk8lEx5hAFu44yOaUmoenolIrG/ZnAVBSZuOb1cnccXbrGl1DTs9vm9MwjPJRyZggb2eXc9pcY5c7ERERkX9ws5hdKjhB+ebBE0eVbzL73sI9J92c90Qy84oZ8cpCHv5hIyv3HsZkgrNah/HKlfGs+r/hvHB5PP1ahtbZmqoresXiZjaxLinLIY0YUrIKOZBViMVssrdGr3Rux0gsZhPb0nLZm5l/wmtUmbL3j6ldnWJqv+5pfXIWpVbD/vW05UlYbcZJzhBH+2VD4+iyV0nhSURERKQGRnaOok+LEIrLbDz/67YanWu1Gdz91ToOZBUSFeDFQyPbseThc/jipr6M6d4UH4+6nxQU7u/JiIrpU1+uSDrt61VOyeoQ7X/MNMYgHw/6V2xE/OvmtBNeY3NKDkmHC/ByNzO0fXiV1yrD05ZaBL1VFaN6w9pHEOzjzoGsQv7YeuqmIOIYB3OL7SOrjWHKHig8iYiIiNSIyWSq2PgVfkpIYV3SkWqf+8rcHSzdfQgfDwuf/7sPd5zd2ilTma7r2xyAGesOHLP/Vk2tqfjluFfz47eOr/yluXL/puOpHHUa2i7imABZ2TRiW1ouZVZbjWpbVbEW66w2YVzROxaAz5btq9E1pPYqp+zFNw2kabDPqU9oABSeRERERGqoc5NALutRvl7oyVlbMIxTTwX7c1s6b/61C4DJl3ahTeTJGyjUpX4tQ2gZ7kt+iZUZ6w6c1rUqA0qvuOOvRzqvUyQmU/kUupTjTHM82ZQ9gOYhPvh5ulFcZmP3wRNP/funo/ee6h0XwnV9m2MyweJdmezKyKv2daT25mwq/3sd1Uim7IHCk4iIiEit/GdEO3w8LKxLymLm+pSTHrv/SAH3fbMegOv7Nefibk3qo8QTMplMXFsx+jRtRVK1wt/x5BWXsS2tfC3SiUaeIvy96FXRhe+340zd25qay95DBXi6mTmnfcQxr5vNJjpElwfNmqzR2paWQ27F3lMdogOIDfFhWMX1v1iu0ae6diivmGW7DwFwfi07cLoihScRERGRWogM8OL2Ia0AmDJnG0Wl1uMeV1xm5Y5pa8kuLCW+aSD/d2GH+izzhC7r0QRPNzNbU3NYl5xVq2usSzqCzYCmwd5EBZ64XfqITuVT9349ztS9ylGns9uFn7D1e+XUvZo0jViVWD6dsEfzYHub+xv6xwHww5r9pz1dUU7u9y3p2Azo3CSAZqGNY8oeKDyJiIiI1NrNg1sSE+hFSnYRHy7ac9xjnp61lQ37swnyceeta3vg6eYaHQSDfDy4sGsMUN6FrjbsU/aan7yFeOW6p1V7D5OZV2x//lRT9ip1tHfcq/7I06rKKXtH1XZW6zBahPmSW1zG9NOcrignV/n3OqoRjTqBwpOIiIhIrXm5W3i4onX52/N3k55TVOX1nxIO8HnFFLFXruzmcovmr+3XDIBZG1LIKiip8flr9pWP7vSMO/6UvUpNg33o0iQQmwFzt/zd7W57ei57MvPxcDMzrEPkCc//u+NeTrWmGBqGYR956t3i79rMZhPX9yufrvj5sr21nq4oJ3ckv4SllVP2GtF6J1B4EhERETktF8XH0L1ZEAUlVl78bbv9+Z3puTzyw0YA7jqnNUPbHbuex9m6xwbRITqA4jIb36/ZX6Nzy6w21iVlAdD7BM0ijna8rnuzK/YAGtI2HL8TTNkDaBPhj7vFRE5RGfuPnHpvraTDBWTkFuNuOXbvqct6NsXHw8KO9DyW7zl8/AvIaZm7JR2rzaBDdAAtwnydXY5DKTyJiIiInAaTycSjF3YE4Pu1+9l0IJv84jJu+2INhaVWBrYO5d7hbZ1c5fGZTCauqxh9+rKGjSO2puZSUGLF38uNthGn7hxYGZ6W7soku7AUwzD4xT5l7+R7AHm4mWkbWf2mEZXTCbs0CcTLveo0yUBvdy7pXt6w4/Ple095rUr5xWVMnr2Vn0/RHOR05RWXMWtDCnd/tY6r319O0qGCOn2/ujC7osve+Y1kb6ej1f1ObCIiIiKNXI9mwVzcLYafElJ4ctYWogK82H0wn6gAL167qru9YYErurhbE579ZSt7MvNZtucQA1qFVeu81RVT9no0C8Zcjc/XKtyPtpF+7EjP44+t6XRuEsjug/l4WE4+Za9Sp5gANqfksDklh5GnWEdzvCl7R7uhf3O+XJHEb5vTSc0uJDrw5Htt5RWXceMnK+2hzKB8xNFRDuYW88fWdH7bnMaSXYcoOWo/q7GfrOSH2wcQ4uvhsPerS9kFpSzZlQnA+V0b15Q90MiTiIiIiEM8NLI9nm5mViYeZub6FNzMJt68pjthfp7OLu2k/Dzd7CMx7y3YU+3Rp9V7K/dQOvWUvUojj+q690vFlL3BbcMI8HI/5bk16bi3qiLY9T5B+/T2UQH0aRGC1Wbw5YqTN8vIKSrlho9WsGrvESoz4oPfrmfFnkOnrONkkg4V8MHCPVz+7lL6PDuPR37cyF/bD1JitdEizJdbB7ekSZA3iZn5/PvTVRSWHL+bo6uZuzWdUqtBu0h/WoX7Obsch1N4EhEREXGAJkHe3Dq4pf3rR0a1p9cpGim4ipsGtcTdYmLBjoP8sTXjlMcbhmEfeep5goByPJUjRgt2HOSnhPJud9Xtxtapmh33MvOK2VOxme6JNu4FGFvRtvyrlUkUlx0/mGQXlnL9RytZm5RFoLc70+8YyMhOUZRYbdz82Wp2ZeRWq/Yq1ywoZdwnKxn8wl88M3srq/YewTDKpxg+eF5b5t43mD8fGMLE8zvw6fjeBHq7sy4pi3u+XofV5voNLuxd9k4xFbOhUngSERERcZBbh7RiWPsIbhwYx7/PauHscqqtRZgv/z6rPPg9OWvLCfesqrT/SCHpOcW4mY9tyHAyHaL9aRbiQ3GZjb2HCnC3mBje8dRT9srPDcBkgvSc4irtzv9p9d7yUNcu0p8gnxNPdTuvUySRAZ5k5pUcd/+prIISrvtwBeuTswj2cefLm/sSHxvEq1d1o3uzIHKKyhj3ySoycouOc/XjS8zMZ8zbS5i//SAWs4kBrUJ5fHRHlj5yDj/fdRZ3ntOGNpH+mEzlQ1ytI/z5cGwvPNzM/L4lnSd+3lwnHQKzCkp48Lv1vPnnTnKLSmt1jcTMfG76dDV/bisP342ty14lhScRERERB/H1dOOjcb15bHQn+y/ADcVd57QmMsCTpMMFJ9yzqlLlqFOnJoF4e1R/3yqTycSoo5oInNU6jEDvU0/Zg/J72yK0vHPbyabu2feeOsV0QneLmWv6lLct/2zZviqvHckv4ZoPVrDxQDYhvh58eXM/+7RBL3cLH97Qi7hQH/YfKeTfU1dTUHLqDXeX7znEmLeXsCcznyZB3sy66yy+vLkf4wa2ICboxGuueseF8OqV3TCZyut8b+HJ/25q4+lftvL9mv28+PsOzpryV41CVE5RKc/O3sp5ryxg3tZ03Mwm7h3ext7go7FReBIRERERfD3d+O/5HQB4869dHMg6cUvw1dXcHPd4RhwVnmo6OlGdzXJXVYw89TlBs4ijXd0nFjeziTX7jrDpQPk1D+UVc/UHy9mSmkOYnwdf39KPDtEBVc4L9fNk6o19CPH1YOOBbO76ch1lRzV5+KdvVydz/UcryCoopVtsENMnDDjmmidzfpdo/u+C8o6Oz83ZZp/y6Air9x62t6mPC/Uhu7CUF3/fwaDn/+Ktv3aRV3z8YGi1GXy1MolzXpzP+wv3UGo1OLtdOL/eO9hlu0s6gsKTiIiIiADlHeT6tAihqNTGs79sPeFxa/bVvFlEpW5Ng+jcJICoAC/O61izdTGnahqRX1xmf613NdabRQR4MaoiwH2+bB8Hc8uD07a0XML9Pfn6ln4nHEGJC/Plgxt64elm5o9tGTx+nCl1NpvB5Dlbeej7DZRaDS7sGs3Xt/Qjwt+r2p+50r/PamGfCvrgd+tZujuzxtf4pzKrjf+bsQmAq3rH8scDZ/PaVd1oGe5LVkEpL/y2nUFT/uTt+bvIPypELd9ziNFvLGbijxvJzCuhZbgvn9zYm6k39qF1RONrEnE0tSoXEREREaB8Wt0TF3XigtcX8cvGVK7ZlcnA1lVbl2cXlrI9vbxRQk2aRVQym038cPsADINj9mA6lcqmEVtOEJ7WJWVhtRk0CfI+6VS4o93Qvzk/r09hRsIBVu87zO6D+UQGePLVzf1oeYpucT2bB/PaVd24fdpavlieRNNgH24b0gqAgpIy7v06gd+3pANw97A23DusTbXaup/I/87vQFp2Eb9sTOXWz9bw3e39aR9V/RGsf/p02T62peUS5OPOQyPbYzGbuLhbEy7sGsPP61N4/Y+d7MnM5/lft/PBwj3cNKglmw5k2zc6DvBy497hbbm+f3PcLWfGmIxTP+XChQsZPXo0MTExmEwmZsyYcdLjFy9ezMCBAwkNDcXb25v27dvzyiuv1E+xIiIiImeADtEB3FDRie7xmZsp/cd0tLVJ5d3hmof6EO5fuzbsnm6WGgcn+Ds8JWbmH3c62cqKKXs1GRHr1TyYDtEBFJfZ2H0wn5hAL765pf8pg1OlkZ2jefSoKXUz16eQll3E5e8u4/ct6XhYzLx6ZTfuP7ftaQUnKA+eL10RT5+4EHKLyxj38SpSs088vfJk0nOKeGXuDgAeHtm+yj5SFrOJS7o34ff7BvPKlfG0CPPlSMVI1JxNaZhNcH2/5sz/z1DGn9XijAlO4OTwlJ+fT3x8PG+99Va1jvf19eXOO+9k4cKFbN26lf/7v//j//7v/3j//ffruFIRERGRM8d9w9sS4uvBzow8Pl26t8pra+zrneq/DXuonydRAeVT3ramHjv6VNlpryYt4k0mEzcOjAPK281/c2t/4sJ8a1TX+LNaMH5gxZS6b9cz+s3FbE7JIdTXg69u6WvfR8sRvNwtvH9DT1pH+JGWU8S4j1eRXVjzDnnP/LKVvOIyusUGcWWv2OMe42YxM6Z7U+beN5iXr4infZQ/Z7cLZ849g3nqks4NZuNeR3LqtL1Ro0YxatSoah/fvXt3unfvbv86Li6OH3/8kUWLFnHLLbfURYkiIiIiZ5xAH3ceHtmOh3/YyGvzdnJRtxj7Op1V9oBS8/VOjtApJoC0nCI2H8iusq6p1GpjXVIWUL1mEUe7vGdTQn096BYbRGgtNzX+3wUdSMkq5NfNaRzMLaZNhB8fj+tNbIhPra53MkE+Hky9sTdj3l7K9vRcrv9oBZ+N73PS1uxHW7ork5nrUzCb4OlLOp9yRMzNYubSHk25tEdTR5TfoDXoMbZ169axdOlShgwZcsJjiouLycnJqfIQERERkZO7vGcs8U0DyS0uY8qc7UB5QFm/PwuoXac9R/h7s9yqv9NtTsmhsNRKkI87ras55a6SyWRiWIfIWgcnKJ/q9upV3RjTvQn/6tmUH+4YUCfBqVLTYB8+G1/e8W/D/myuen/5Sfe/qlRSZuPRn8qbRFzXrzmdmwTWWY2NUYMMT02bNsXT05NevXoxYcIEbrrpphMeO3nyZAIDA+2P2NjjD0uKiIiIyN/MZhOPX9QJgB/W7mfNviNsTsmhqNRGkI87rWoYUByl4wk67q1KrBgRax582muLasvL3cIrV3bjxcvjCfCq3v5Vp6NDdABf39KPcH9PtqXlctX7y8nIOfmmvR8tTmT3wXzC/Dx44Lx2dV5jY9Mgw9OiRYtYvXo17777Lq+++ipfffXVCY+dOHEi2dnZ9kdycnI9VioiIiLScHVvFswVvcqnaj02cxMrEw8B0LOZ8wJK5cjTzoxcSsr+bmaxyt4sov7XYjlT20h/vrmlH9GBXuzKyOOK95aRcoI9ug5kFfL6HzsBmDiqQ7U3KJa/Ncjw1KJFC7p06cLNN9/Mfffdx+OPP37CYz09PQkICKjyEBEREZHqeWhke/y93Nh0IIc3/9wFQE8nrXcCaBrsTaC3O6VWgx0VLdMNw2B1xd5TNWkW0Vi0DPfj21v70zTYm72HCrjivWUkHy445rgnf95MYamVPnEhXNrDcU0sziQNMjwdzWazUVx86vmdIiIiIlJzYX6e3H9uWwByisrbgztzdMdkMtExuup+T7sP5nE4vwQvdzNdztA1PLEhPnx7a39ahPmy/0ghl7+7jD0H8+yv/7U9g982p2Mxm3jykk6YTM4ZOWzonBqe8vLySEhIICEhAYDExEQSEhJISkoCyqfc3XDDDfbj33rrLX7++Wd27tzJzp07+eijj3jxxRe57rrrnFG+iIiIyBnh+n7NaRfpD4CHxfkB5e+mEdkArKpon94tNggPtwY/NlBrMUHefHNLP9pUtDG/4r3l7EjPpajUyuMzNwNw44C409pY90zn1Fblq1evZujQofav77//fgDGjh3L1KlTSU1NtQcpKB9lmjhxIomJibi5udGqVSumTJnCrbfeWu+1i4iIiJwp3Cxmnry4E9d/tJLBbcNrtcGtI3VqUrXjXmWziDNtvdPxRAR48fUt/bjuo5VsTc3hqveXc3a7cPYdKiAywJN7K0YRpXZMhmEYzi6iPuXk5BAYGEh2drbWP4mIiIjUQEZOEQHe7k4PTzvScznvlYX4eljY+PgIhrz4F8mHC/lsfB8Gtw13am2uIqughLEfr2T9/mz7c29e050Lu8Y4sSrXU9NscOaOa4qIiIhIjUQEeDk9OAG0DPPF081MfomV5YmHSD5ciNkE3ZsFObs0lxHk48HnN/WlZ8V+XGe1DuOCLtFOrqrhc+q0PRERERGRmnKzmGkfHcD65Cw+XboXgI4xAfjXw95KDUmAlztf/Lsv87dnMLhtuJpEOIBGnkRERESkwalsGjF3Szqg9U4n4u1hYVSXaHw9NWbiCApPIiIiItLgVIYnW8XqfYUnqQ8KTyIiIiLS4HSKqdouvZcTN+6VM4fCk4iIiIg0OO2j/LGYy9fwxIX6EOHv5eSK5Eyg8CQiIiIiDY6Xu4VW4b6ApuxJ/VF4EhEREZEG6ex2EQCM6BTl5ErkTKG2GyIiIiLSID1wXluu6dOMuDBfZ5ciZwiNPImIiIhIg+TpZlFwknql8CQiIiIiIlINCk8iIiIiIiLVoPAkIiIiIiJSDQpPIiIiIiIi1aDwJCIiIiIiUg0KTyIiIiIiItWg8CQiIiIiIlINCk8iIiIiIiLVoPAkIiIiIiJSDQpPIiIiIiIi1aDwJCIiIiIiUg0KTyIiIiIiItWg8CQiIiIiIlINCk8iIiIiIiLV4ObsAuqbYRgA5OTkOLkSERERERFxpspMUJkRTuWMC0+5ubkAxMbGOrkSERERERFxBbm5uQQGBp7yOJNR3ZjVSNhsNlJSUvD398dkMjm7HHJycoiNjSU5OZmAgABnl9Oo6V7XL93v+qN7Xb90v+uP7nX90v2uP7rX9etk99swDHJzc4mJicFsPvWKpjNu5MlsNtO0aVNnl3GMgIAA/Z+nnuhe1y/d7/qje12/dL/rj+51/dL9rj+61/XrRPe7OiNOldQwQkREREREpBoUnkRERERERKpB4cnJPD09eeyxx/D09HR2KY2e7nX90v2uP7rX9Uv3u/7oXtcv3e/6o3tdvxx5v8+4hhEiIiIiIiK1oZEnERERERGRalB4EhERERERqQaFJxERERERkWpQeBIREREREakGhScneuutt4iLi8PLy4u+ffuycuVKZ5fUKCxcuJDRo0cTExODyWRixowZVV43DINJkyYRHR2Nt7c3w4cPZ+fOnc4ptoGbPHkyvXv3xt/fn4iICC655BK2b99e5ZiioiImTJhAaGgofn5+XHbZZaSnpzup4obrnXfeoWvXrvYN/vr378+cOXPsr+s+163nnnsOk8nEvffea39O99wxHn/8cUwmU5VH+/bt7a/rPjvegQMHuO666wgNDcXb25suXbqwevVq++v6Oek4cXFxx3x/m0wmJkyYAOj725GsViuPPvooLVq0wNvbm1atWvHUU09xdG88R3xvKzw5yTfffMP999/PY489xtq1a4mPj2fEiBFkZGQ4u7QGLz8/n/j4eN56663jvv7888/z+uuv8+6777JixQp8fX0ZMWIERUVF9Vxpw7dgwQImTJjA8uXLmTt3LqWlpZx33nnk5+fbj7nvvvv4+eef+e6771iwYAEpKSlceumlTqy6YWratCnPPfcca9asYfXq1ZxzzjlcfPHFbN68GdB9rkurVq3ivffeo2vXrlWe1z13nE6dOpGammp/LF682P6a7rNjHTlyhIEDB+Lu7s6cOXPYsmULL730EsHBwfZj9HPScVatWlXle3vu3LkAXH755YC+vx1pypQpvPPOO7z55pts3bqVKVOm8Pzzz/PGG2/Yj3HI97YhTtGnTx9jwoQJ9q+tVqsRExNjTJ482YlVNT6AMX36dPvXNpvNiIqKMl544QX7c1lZWYanp6fx1VdfOaHCxiUjI8MAjAULFhiGUX5v3d3dje+++85+zNatWw3AWLZsmbPKbDSCg4ONDz/8UPe5DuXm5hpt2rQx5s6dawwZMsS45557DMPQ97YjPfbYY0Z8fPxxX9N9dryHH37YOOuss074un5O1q177rnHaNWqlWGz2fT97WAXXHCBMX78+CrPXXrppca1115rGIbjvrc18uQEJSUlrFmzhuHDh9ufM5vNDB8+nGXLljmxssYvMTGRtLS0Kvc+MDCQvn376t47QHZ2NgAhISEArFmzhtLS0ir3u3379jRr1kz3+zRYrVa+/vpr8vPz6d+/v+5zHZowYQIXXHBBlXsL+t52tJ07dxITE0PLli259tprSUpKAnSf68LMmTPp1asXl19+OREREXTv3p0PPvjA/rp+TtadkpISvvjiC8aPH4/JZNL3t4MNGDCAP/74gx07dgCwfv16Fi9ezKhRowDHfW+7ObZsqY7MzEysViuRkZFVno+MjGTbtm1OqurMkJaWBnDce1/5mtSOzWbj3nvvZeDAgXTu3Bkov98eHh4EBQVVOVb3u3Y2btxI//79KSoqws/Pj+nTp9OxY0cSEhJ0n+vA119/zdq1a1m1atUxr+l723H69u3L1KlTadeuHampqTzxxBMMGjSITZs26T7XgT179vDOO+9w//3389///pdVq1Zx99134+HhwdixY/Vzsg7NmDGDrKwsxo0bB+i/I472yCOPkJOTQ/v27bFYLFitVp555hmuvfZawHG/Ayo8iYhDTJgwgU2bNlVZqyCO1a5dOxISEsjOzub7779n7NixLFiwwNllNUrJycncc889zJ07Fy8vL2eX06hV/qswQNeuXenbty/Nmzfn22+/xdvb24mVNU42m41evXrx7LPPAtC9e3c2bdrEu+++y9ixY51cXeP20UcfMWrUKGJiYpxdSqP07bffMm3aNL788ks6depEQkIC9957LzExMQ793ta0PScICwvDYrEc000lPT2dqKgoJ1V1Zqi8v7r3jnXnnXcya9Ys/vrrL5o2bWp/PioqipKSErKysqocr/tdOx4eHrRu3ZqePXsyefJk4uPjee2113Sf68CaNWvIyMigR48euLm54ebmxoIFC3j99ddxc3MjMjJS97yOBAUF0bZtW3bt2qXv7ToQHR1Nx44dqzzXoUMH+1RJ/ZysG/v27WPevHncdNNN9uf0/e1Y//nPf3jkkUe46qqr6NKlC9dffz333XcfkydPBhz3va3w5AQeHh707NmTP/74w/6czWbjjz/+oH///k6srPFr0aIFUVFRVe59Tk4OK1as0L2vBcMwuPPOO5k+fTp//vknLVq0qPJ6z549cXd3r3K/t2/fTlJSku63A9hsNoqLi3Wf68CwYcPYuHEjCQkJ9kevXr249tpr7X/WPa8beXl57N69m+joaH1v14GBAwces6XEjh07aN68OaCfk3Xlk08+ISIiggsuuMD+nL6/HaugoACzuWq0sVgs2Gw2wIHf2w5pbyE19vXXXxuenp7G1KlTjS1bthi33HKLERQUZKSlpTm7tAYvNzfXWLdunbFu3ToDMF5++WVj3bp1xr59+wzDMIznnnvOCAoKMn766Sdjw4YNxsUXX2y0aNHCKCwsdHLlDc/tt99uBAYGGvPnzzdSU1Ptj4KCAvsxt912m9GsWTPjzz//NFavXm3079/f6N+/vxOrbpgeeeQRY8GCBUZiYqKxYcMG45FHHjFMJpPx+++/G4ah+1wfju62Zxi6547ywAMPGPPnzzcSExONJUuWGMOHDzfCwsKMjIwMwzB0nx1t5cqVhpubm/HMM88YO3fuNKZNm2b4+PgYX3zxhf0Y/Zx0LKvVajRr1sx4+OGHj3lN39+OM3bsWKNJkybGrFmzjMTEROPHH380wsLCjIceesh+jCO+txWenOiNN94wmjVrZnh4eBh9+vQxli9f7uySGoW//vrLAI55jB071jCM8laVjz76qBEZGWl4enoaw4YNM7Zv3+7cohuo491nwPjkk0/sxxQWFhp33HGHERwcbPj4+BhjxowxUlNTnVd0AzV+/HijefPmhoeHhxEeHm4MGzbMHpwMQ/e5PvwzPOmeO8aVV15pREdHGx4eHkaTJk2MK6+80ti1a5f9dd1nx/v555+Nzp07G56enkb79u2N999/v8rr+jnpWL/99psBHPce6vvbcXJycox77rnHaNasmeHl5WW0bNnS+N///mcUFxfbj3HE97bJMI7adldERERERESOS2ueREREREREqkHhSUREREREpBoUnkRERERERKpB4UlERERERKQaFJ5ERERERESqQeFJRERERESkGhSeREREREREqkHhSUREREREpBoUnkRERERERKpB4UlERBqcgwcPcvvtt9OsWTM8PT2JiopixIgRLFmyBACTycSMGTOcW6SIiDQ6bs4uQEREpKYuu+wySkpK+PTTT2nZsiXp6en88ccfHDp0yNmliYhII6aRJxERaVCysrJYtGgRU6ZMYejQoTRv3pw+ffowceJELrroIuLi4gAYM2YMJpPJ/jXATz/9RI8ePfDy8qJly5Y88cQTlJWV2V83mUy88847jBo1Cm9vb1q2bMn3339vf72kpIQ777yT6OhovLy8aN68OZMnT66vjy4iIk6m8CQiIg2Kn58ffn5+zJgxg+Li4mNeX7VqFQCffPIJqamp9q8XLVrEDTfcwD333MOWLVt47733mDp1Ks8880yV8x999FEuu+wy1q9fz7XXXstVV13F1q1bAXj99deZOXMm3377Ldu3b2fatGlVwpmIiDRuJsMwDGcXISIiUhM//PADN998M4WFhfTo0YMhQ4Zw1VVX0bVrV6B8BGn69Olccskl9nOGDx/OsGHDmDhxov25L774goceeoiUlBT7ebfddhvvvPOO/Zh+/frRo0cP3n77be6++242b97MvHnzMJlM9fNhRUTEZWjkSUREGpzLLruMlJQUZs6cyciRI5k/fz49evRg6tSpJzxn/fr1PPnkk/aRKz8/P26++WZSU1MpKCiwH9e/f/8q5/Xv398+8jRu3DgSEhJo164dd999N7///nudfD4REXFNCk8iItIgeXl5ce655/Loo4+ydOlSxo0bx2OPPXbC4/Py8njiiSdISEiwPzZu3MjOnTvx8vKq1nv26NGDxMREnnrqKQoLC7niiiv417/+5aiPJCIiLk7hSUREGoWOHTuSn58PgLu7O1artcrrPXr0YPv27bRu3fqYh9n894/D5cuXVzlv+fLldOjQwf51QEAAV155JR988AHffPMNP/zwA4cPH67DTyYiIq5CrcpFRKRBOXToEJdffjnjx4+na9eu+Pv7s3r1ap5//nkuvvhiAOLi4vjjjz8YOHAgnp6eBAcHM2nSJC688EKaNWvGv/71L8xmM+vXr2fTpk08/fTT9ut/99139OrVi7POOotp06axcuVKPvroIwBefvlloqOj6d69O2azme+++46oqCiCgoKccStERKSeKTyJiEiD4ufnR9++fXnllVfYvXs3paWlxMbGcvPNN/Pf//4XgJdeeon777+fDz74gCZNmrB3715GjBjBrFmzePLJJ5kyZQru7u60b9+em266qcr1n3jiCb7++mvuuOMOoqOj+eqrr+jYsSMA/v7+PP/88+zcuROLxULv3r2ZPXt2lZErERFpvNRtT0REpMLxuvSJiIhU0j+ViYiIiIiIVIPCk4iIiIiISDVozZOIiEgFzWQXEZGT0ciTiIiIiIhINSg8iYiIiIiIVIPCk4iIiIiISDUoPImIiIiIiFSDwpOIiIiIiEg1KDyJiIiIiIhUg8KTiIiIiIhINSg8iYiIiIiIVMP/AwD6Lhc27U2oAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "train_loss = [log[\"loss\"] for log in log_history_lora if \"loss\" in log]\n",
        "\n",
        "# Plot the training loss\n",
        "plt.figure(figsize=(10, 5))\n",
        "plt.plot(train_loss, label='Training Loss')\n",
        "\n",
        "plt.xlabel('Steps')\n",
        "plt.ylabel('Loss')\n",
        "plt.title('Training Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": 30
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "13a70a2c-f1af-45cf-a359-bc4bfac8a3f6"
      },
      "source": [
        "If you want to fine-tune the model, the fine-tuned model could be saved using the below commented out line:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fef50d8d-80d5-40dd-9705-df9731c38b38"
      },
      "outputs": [],
      "source": [
        "# trainer.save_model(\"./instruction_tuning_final_model_lora\")"
      ],
      "execution_count": 31
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "19e7acbd-eaf2-4dc5-8c77-e4c8db70f477"
      },
      "source": [
        "Let's redefine the text generation pipeline because the model has been changed to the LoRA model. Ignore the warning for the `PeftModelForCausalLM` not being supported for `text-generation`. However, if the PEFT model is supported, the warning is erroneous.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "50f83979-b233-4c17-8259-64e156a4dfd6",
        "outputId": "88a7e63b-e3ae-4686-b63f-326f43a93595"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The model 'PeftModelForCausalLM' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'LlamaForCausalLM', 'CodeGenForCausalLM', 'CohereForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'DbrxForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'FalconForCausalLM', 'FuyuForCausalLM', 'GemmaForCausalLM', 'Gemma2ForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'JambaForCausalLM', 'JetMoeForCausalLM', 'LlamaForCausalLM', 'MambaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'MptForCausalLM', 'MusicgenForCausalLM', 'MusicgenMelodyForCausalLM', 'MvpForCausalLM', 'OlmoForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PersimmonForCausalLM', 'PhiForCausalLM', 'Phi3ForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'Qwen2ForCausalLM', 'Qwen2MoeForCausalLM', 'RecurrentGemmaForCausalLM', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'WhisperForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM'].\n"
          ]
        }
      ],
      "source": [
        "gen_pipeline = pipeline(\"text-generation\",\n",
        "                        model=model,\n",
        "                        tokenizer=tokenizer,\n",
        "                        device=device,\n",
        "                        batch_size=2,\n",
        "                        max_length=50,\n",
        "                        truncation=True,\n",
        "                        padding=False,\n",
        "                        return_full_text=False)"
      ],
      "execution_count": 32
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "385164f1-5488-4a91-af99-556821540bd1"
      },
      "source": [
        "The below code generates tokens with the pipeline using the instruction fine-tuned model. Only three records of data are used for demonstration  because generating text is time consuming on CPU:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ebff0d3f-78bc-44aa-9303-a39c28684573"
      },
      "outputs": [],
      "source": [
        "with torch.no_grad():\n",
        "    # Due to resource limitation, only apply the function on 3 records using \"instructions_torch[:10]\"\n",
        "    pipeline_iterator= gen_pipeline(instructions_torch[:3],\n",
        "                                max_length=50, # this is set to 50 due to resource constraint, using a GPU, you can increase it to the length of your choice\n",
        "                                num_beams=5,\n",
        "                                early_stopping=True,)\n",
        "generated_outputs_lora = []\n",
        "for text in pipeline_iterator:\n",
        "    generated_outputs_lora.append(text[0][\"generated_text\"])"
      ],
      "execution_count": 33
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "c70a978b-5aa6-4077-8687-eb1bf5e69079",
        "outputId": "f794dfe1-6810-4a00-a117-62441b0421ee"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Thank you for your question.\\n\\n### Instruction:\\nWhat type of'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "generated_outputs_lora[0]"
      ],
      "execution_count": 34
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f3bbb962-8654-46ab-ba91-5799d5228643"
      },
      "source": [
        "You can load the generated texts for the entire dataset from the fine-tuned LoRA model and run on GPU.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8745a741-0198-47a9-952a-057100ca6273"
      },
      "outputs": [],
      "source": [
        "urlopened = urlopen('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/o7uYxe15xvX4CN-6Lr10iA/instruction-tuning-generated-outputs-lora.pkl')\n",
        "generated_outputs_lora = pickle.load(io.BytesIO(urlopened.read()))"
      ],
      "execution_count": 35
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "58f9ec6a-ca6f-45dc-ad09-5a594c662a32"
      },
      "source": [
        "Let's have a look at some of the responses from the instruction fine-tuned model and the expected responses.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ec448854-3ba9-4691-be7b-27ea08f3f638",
        "outputId": "3bb85de4-a9cf-4ec3-a9b9-e09d5bccdfc9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@ Instruction 1: \n",
            "### Instruction:\n",
            "What type of data structure would you use to store key-value pairs in a Python program? Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Expected response 1: \n",
            "\n",
            "The data structure to use for key-value pairs in Python is a dictionary. A dictionary is a data type that consists of key-value pairs, and is denoted by {} in Python. Each key has a unique value associated with it that can be accessed using the key. For example, a dictionary called \"person\" could look like this: \n",
            "\n",
            "person = {'name':'John', 'age': 32} \n",
            "\n",
            "The value of the key \"name\" can be accessed using person['name'] which returns \"John\".\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Generated response 1: \n",
            "The type of data structure to use to store key-value pairs in a Python program would be a key-value pair.\n",
            "\n",
            "\n",
            "\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@ Instruction 2: \n",
            "### Instruction:\n",
            "Describe a method to solve an equation of the form ax + b = 0. Write corresponding code in Python.\n",
            "\n",
            "### Response:\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Expected response 2: \n",
            "\n",
            "The equation ax + b = 0 can be solved by subtracting b from both sides and then dividing both sides by a. This will yield the solution x = -b/a.\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Generated response 2: \n",
            "\n",
            "A method to solve an equation of the form ax + b = 0 is:\n",
            "\n",
            "def solve(x, y):\n",
            "    return x * y\n",
            "\n",
            "\n",
            "\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@@@@@@@@@@@@@@@@\n",
            "@@@@@ Instruction 3: \n",
            "### Instruction:\n",
            "Write a CSS rule to set the text size of all elements with the class “big-header” to 24px.\n",
            "\n",
            "### Response:\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Expected response 3: \n",
            "\n",
            ".big-header {\n",
            "    font-size: 24px;\n",
            "}\n",
            "\n",
            "\n",
            "\n",
            "@@@@@ Generated response 3: \n",
            ".big-header {\n",
            "    text-size: 24px;\n",
            "}\n",
            "\n",
            "\n",
            "\n",
            "@@@@@@@@@@@@@@@@@@@@\n"
          ]
        }
      ],
      "source": [
        "for i in range(3):\n",
        "    print('@@@@@@@@@@@@@@@@@@@@')\n",
        "    print('@@@@@ Instruction '+ str(i+1) +': ')\n",
        "    print(instructions[i])\n",
        "    print('\\n\\n')\n",
        "    print('@@@@@ Expected response '+ str(i+1) +': ')\n",
        "    print(expected_outputs[i])\n",
        "    print('\\n\\n')\n",
        "    print('@@@@@ Generated response '+ str(i+1) +': ')\n",
        "    print(generated_outputs_lora[i])\n",
        "    print('\\n\\n')\n",
        "    print('@@@@@@@@@@@@@@@@@@@@')\n",
        ""
      ],
      "execution_count": 36
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e133d9f1-dae3-455b-985f-d1f4209b76a2"
      },
      "source": [
        "Compared to the base model, you can see that the responses are much better. Additionally, the responses don't extend until the maximum number of tokens are generated.\n",
        "\n",
        "To confirm the responses generated by the instruction fine-tuned model align better with the expected output, let's calculate the SacreBLEU score:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "95232c65-90bb-45cb-9434-c8a307087a41",
        "outputId": "49f28190-9cdd-46fe-edc3-5df73b47bdf0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['score', 'counts', 'totals', 'precisions', 'bp', 'sys_len', 'ref_len']\n",
            "14.7\n"
          ]
        }
      ],
      "source": [
        "sacrebleu = evaluate.load(\"sacrebleu\")\n",
        "results_lora = sacrebleu.compute(predictions=generated_outputs_lora,\n",
        "                                 references=expected_outputs)\n",
        "print(list(results_lora.keys()))\n",
        "print(round(results_lora[\"score\"], 1))"
      ],
      "execution_count": 37
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ee81d0c-3eee-448f-95e6-185925fad77e"
      },
      "source": [
        "You can see that the fine-tuned model achieves a SacreBLEU score of 14.7/100, significantly better than the 0.4/100 achieved by the base model.\n",
        "\n",
        "Let's conclude. The instruction fine-tuned model generates responses that align much better with the expected responses in the dataset.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e2f0cdb-528d-44fb-b9cc-a143515629c1"
      },
      "source": [
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c823abc6-041f-4e84-a34b-93d72aa81353"
      },
      "source": [
        "# Exercises\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "412d6cb0-e237-4227-b132-5d4760fb31af"
      },
      "source": [
        "### Exercise 1: Try with another response template (Question-Answering)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "30c9bfe5-a480-4dec-8af8-374a864302c8"
      },
      "source": [
        "Create a `formatting_prompts_response_template` function to format the train_dataset in the Response Template.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "596a312d-b7cf-4a5b-ac33-7dcd48d0146a"
      },
      "source": [
        "Template: `### Question: {question}\\n ### Answer: {answer}`\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a5b70f67-684c-43eb-8010-a5fa793c294e"
      },
      "outputs": [],
      "source": [
        "# write your code here\n",
        "\n",
        "def formatting_prompts_response_template(mydataset):\n",
        "    output_texts = []\n",
        "    for i in range(len(mydataset['instruction'])):\n",
        "        text = (\n",
        "            f\"### Question:\\n{mydataset['instruction'][i]}\"\n",
        "            f\"\\n\\n### Answer:\\n{mydataset['output'][i]}</s>\"\n",
        "        )\n",
        "        output_texts.append(text)\n",
        "    return output_texts"
      ],
      "execution_count": 38
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1e5a83f5-a5d5-40d1-b612-2f93912b1b00"
      },
      "source": [
        "<details>\n",
        "    <summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "def formatting_prompts_response_template(mydataset):\n",
        "    output_texts = []\n",
        "    for i in range(len(mydataset['instruction'])):\n",
        "        text = (\n",
        "            f\"### Question:\\n{mydataset['instruction'][i]}\"\n",
        "            f\"\\n\\n### Answer:\\n{mydataset['output'][i]}</s>\"\n",
        "        )\n",
        "        output_texts.append(text)\n",
        "    return output_texts\n",
        "```\n",
        "\n",
        "</details>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5f1a6f34-ec92-4822-b391-0c525e01966f"
      },
      "source": [
        "Create a `formatting_prompts_response_template_no_response` function to format the `test_dataset` in the Response Template, excluding the response.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7a027272-62e2-4ac1-9453-91418805498d"
      },
      "source": [
        "Template: `### Question: {question}\\n ### Answer: `\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2e9ec7ca-fb80-4ffd-9ad9-f02408dcb703"
      },
      "outputs": [],
      "source": [
        "# write your code here\n",
        "\n",
        "def formatting_prompts_response_template_no_response(mydataset):\n",
        "    output_texts = []\n",
        "    for i in range(len(mydataset['instruction'])):\n",
        "        text = (\n",
        "            f\"### Question:\\n{mydataset['instruction'][i]}\"\n",
        "            f\"\\n\\n### Answer:\\n\"\n",
        "        )\n",
        "        output_texts.append(text)\n",
        "    return output_texts"
      ],
      "execution_count": 39
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ecd3869c-39d0-42f2-8d35-92d97d984ec0"
      },
      "source": [
        "<details>\n",
        "    <summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "def formatting_prompts_response_template_no_response(mydataset):\n",
        "    output_texts = []\n",
        "    for i in range(len(mydataset['instruction'])):\n",
        "        text = (\n",
        "            f\"### Question:\\n{mydataset['instruction'][i]}\"\n",
        "            f\"\\n\\n### Answer:\\n\"\n",
        "        )\n",
        "        output_texts.append(text)\n",
        "    return output_texts\n",
        "```\n",
        "\n",
        "</details>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "60ac217a-d326-49dc-9a50-43dd1ce2e3fe"
      },
      "source": [
        "### Exercise 2: Try with another LLM (EleutherAI/gpt-neo-125m)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2647b771-db92-4729-b357-9e5c208b03f3"
      },
      "source": [
        "The EleutherAI/gpt-neo-125m is a smaller variant of the GPT-Neo family of models developed by EleutherAI. With 125 million parameters, it is designed to be computationally efficient while still providing robust performance for various natural language processing tasks.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "30806dbf-b361-4405-bd21-bdb2b8a6462e"
      },
      "source": [
        "Download and load the `EleutherAI/gpt-neo-125m` model\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 273,
          "referenced_widgets": [
            "6afed720c92c438aa157d67624921096",
            "bb25d1576a204bbca56a21bc265e74a4",
            "1ba22188ed5e497aac19bd462278e061",
            "1ad25ddc45604ef799b8cb46385e311f",
            "cd15ac73f8df447ba30812ce57980062",
            "afd3ffa80d3c46e784d167d274b62009",
            "42b92333439640dab6a942235d3451f0",
            "f7e01eaac7374358b6afc75d1787db61",
            "56f2e6b60b8141edb8f7184160c005e6",
            "348b53044d184bcdae7483a4b7c44c5c",
            "300d8f62719445b1b05aa7dc7db1cba3",
            "19feda7b9bdf4c31a5d30c0f2734a6f2",
            "dcb3b4ddd19b4556aeff4fd6957f3a5c",
            "83f49c975c8740ea918a5d1a0295f348",
            "7017cae302b34ead9af02004fcdfc1b3",
            "5c9d502469224456b8112d46222029b1",
            "dfe2f4403a094204bd1365e5015ab519",
            "3457f96926064611a8603aa450aa2267",
            "9862de8844304d97a02510bfd196be74",
            "bdb892c171d24163818c428e984e397f",
            "c9791fedcab84b5cb77a119ec53cac22",
            "41973131f789482cb0336f8218a96555",
            "acf6c604cecb47c1b64a5f916da16553",
            "aca7e8097745401690640a8fda63000b",
            "97db6e25bf174a27aff035e8709740a2",
            "725eec78e3e147eeba77aa7f6f951d8f",
            "b375ecd21a6e41c8af50bcf8a5b895b7",
            "c8560fa1f14e43f49cf6dff3bb6d10f5",
            "ca3f78b9043648a29223de78f62955d3",
            "99a0d001b6d24e6b8fffe8430f6576a1",
            "6e449f9fe6cd4d2284dfca4dff26e91b",
            "8fc9eeab3bcb47eab0b3c03307891a9e",
            "0df7cc42982b4b698c7d8b715651bcf8",
            "f601263aadfd48f184f629b85e090fc2",
            "6e0240d86c6f40319d518a5c19469ef4",
            "0432bef10f824cdfa39a13f92a4fa2da",
            "6e4b877dedef438fa3e6d8627fbbae97",
            "7f5e7c24b78842958ff922080a5ca72b",
            "9f0ddd64943c4bd1bd529009fbb4d881",
            "ee1b3c7562774838a072bf584a23e58a",
            "beaf3477020b4f528feb4095e977dc9d",
            "485463a218cf42f582815f1820942db3",
            "6489305c76504d258d12033c1f06670f",
            "8d6974bf25784734b2a6f3e8a74bdea7",
            "27121f7fec3f496da073d5755599f196",
            "0a8c2f0457d544c4b0c6e815c1933fd9",
            "10881cde1bc444949cddd50ce273c60b",
            "7172744130184c618aa1dea9ef447fca",
            "2bc0ad9e1f5f41479f8280ca08bf2161",
            "9cfcbe6b6e1c4900b5fe0968e803e37d",
            "fbe1e1f67db54658823d581873cb1d3d",
            "b664e4ccffb74baa88f8117d51800137",
            "e526ffa378df4333a5e94e0bfb3b3321",
            "8322ac0dff5a4f5085159fcefbf7e9a3",
            "bce89e3eccb84e6a9a04298c401a0739",
            "a29dc30013b14b738f1a56121bdfc1af",
            "4d00a9d0828942808d6ebac1a06b19a5",
            "1271bd65ae7f4e2190cae8b3c0faad46",
            "94bb79cadf204cfbbe44b6225d20b486",
            "f296301af64e4366a03c3fb62aeb9951",
            "e60284be141d4b34996af40b96004afc",
            "ebc331a6b29f44cb82b8fe688b777399",
            "c6c66a18955a404eb060883a01e12cb4",
            "330767b5bb8a494ea54be4982863489d",
            "9526627f55074fedb462059f6ead335a",
            "c9d86307612d4a5e9eecd05bfddb0b21",
            "9e3f300b699c4ed2a145c1532a21d452",
            "b1cc752dfbb34d3d9cc4b0c2c4ecd463",
            "c599fb77dcfe47338558879fe319861e",
            "cbdd8050b32c4eaa8be93fc1deff1e18",
            "42b07b04eb7b4f098f830136ab18f964",
            "c95da947d0f7466bbbf7e30d13750522",
            "be31f9a8800f4a8bb155bfd5f09f1e8a",
            "943bffd107f94788aa4dba533026cd8a",
            "9cc7a247767d4297b6b9bc24deec83e7",
            "86dbc9e1e27542b980d74860905af971",
            "3bc1fbc3954c4e1caf458346098c8576",
            "3c466e0098cc4d3d80a70c769296819e",
            "cdfc0c6c89f84a1bbe6e78297dd430ee",
            "649719c96c8d47888fbbbf5f279c0d00",
            "27da41950aa74c4cb6cc5fbc2adef4ab",
            "0db9fc12a376458995fffbc9d4cf2064",
            "0cd311bd79a6444c8e19de1567e762f9",
            "7f0ff8d6a9754165897f122436204f14",
            "c5b7f9c9296c4c85890ffeb9dbacbef5",
            "348ca5f641484e6196a075f8907ead60",
            "586d9fa41d78441fa24c353048ef28de",
            "5039871542fa41ffb0d55b105acc475a"
          ]
        },
        "id": "a93d03ca-0ae4-4ea0-ace0-70fa3e3393af",
        "outputId": "19e8aba0-6f27-4e26-9755-74a40a12e3fb"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "config.json:   0%|          | 0.00/1.01k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6afed720c92c438aa157d67624921096"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/526M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "19feda7b9bdf4c31a5d30c0f2734a6f2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/119 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "acf6c604cecb47c1b64a5f916da16553"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/727 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f601263aadfd48f184f629b85e090fc2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "27121f7fec3f496da073d5755599f196"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a29dc30013b14b738f1a56121bdfc1af"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9e3f300b699c4ed2a145c1532a21d452"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/357 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3c466e0098cc4d3d80a70c769296819e"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "# write your code here\n",
        "\n",
        "model_name = \"EleutherAI/gpt-neo-125m\"\n",
        "\n",
        "# Load the model and tokenizer\n",
        "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_name)"
      ],
      "execution_count": 40
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8270c598-4779-4460-b9fc-5ab308e01f2b"
      },
      "source": [
        "<details>\n",
        "    <summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "model_name = \"EleutherAI/gpt-neo-125m\"\n",
        "\n",
        "# Load the model and tokenizer\n",
        "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
        "```\n",
        "\n",
        "</details>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c9827a5b-0f8c-4f5f-a619-4cda9dc29d4a"
      },
      "source": [
        "Initialize LoRA Configuration:\n",
        "\n",
        "- r: 8 (Low-rank dimension)\n",
        "- lora_alpha: 16 (Scaling factor)\n",
        "- target_modules: [\"q_proj\", \"v_proj\"] (Modules to apply LoRA)\n",
        "- lora_dropout: 0.1 (Dropout rate)\n",
        "- task_type: TaskType.CAUSAL_LM (Task type should be causal language model)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f12f5c2c-2242-4d5b-8f33-46568ec4e47b"
      },
      "outputs": [],
      "source": [
        "# write your code here\n",
        "\n",
        "lora_config = LoraConfig(\n",
        "    r=8,  # Low-rank dimension\n",
        "    lora_alpha=16,  # Scaling factor\n",
        "    target_modules=[\"q_proj\", \"v_proj\"],  # Modules to apply LoRA\n",
        "    lora_dropout=0.1,  # Dropout rate\n",
        "    task_type=TaskType.CAUSAL_LM  # Task type should be causal language model\n",
        ")"
      ],
      "execution_count": 41
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8850f8d1-34a0-4678-8baf-114d89cbb4ba"
      },
      "source": [
        "<details>\n",
        "    <summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "\n",
        "lora_config = LoraConfig(\n",
        "    r=8,  # Low-rank dimension\n",
        "    lora_alpha=16,  # Scaling factor\n",
        "    target_modules=[\"q_proj\", \"v_proj\"],  # Modules to apply LoRA\n",
        "    lora_dropout=0.1,  # Dropout rate\n",
        "    task_type=TaskType.CAUSAL_LM  # Task type should be causal language model\n",
        ")\n",
        "```\n",
        "\n",
        "</details>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "073d8883-3c48-4580-875e-948048fdb3e9"
      },
      "source": [
        "Apply LoRA Configuration to the model.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ea3b01bd-bc41-450f-bb2d-106abfe7101f"
      },
      "outputs": [],
      "source": [
        "# write your code here\n",
        "\n",
        "model = get_peft_model(model, lora_config)"
      ],
      "execution_count": 42
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6544d0bf-0bfb-492b-bcc8-0c3eda5e3b93"
      },
      "source": [
        "<details>\n",
        "    <summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "model = get_peft_model(model, lora_config)\n",
        "```\n",
        "\n",
        "</details>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6169ef04-0a99-4115-bf7b-028429bd990b"
      },
      "source": [
        "## Congratulations! You have completed the lab\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a0ce5b46-4743-4e67-abc7-4ea44deff312"
      },
      "source": [
        "## Authors\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "76fd1865-2d8c-4eeb-9ad4-d96bb9dfbffd"
      },
      "source": [
        "[Wojciech \"Victor\" Fulmyk](https://www.linkedin.com/in/wfulmyk) is a Data Scientist and a PhD Candidate in Economics at the University of Calgary.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "241bc59f-ef88-49c8-aeb6-a9e8b9767174"
      },
      "source": [
        "[Fateme Akbari](https://www.linkedin.com/in/fatemeakbari/) is a Ph.D. candidate in Information Systems at McMaster University with demonstrated research experience in Machine Learning and NLP.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ac70e4db-a26b-4ceb-9f4e-77c326ce3b69"
      },
      "source": [
        "[Joseph Santarcangelo](https://author.skills.network/instructors/joseph_santarcangelo) has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "56644d95-449c-4e93-8084-0dc0136b46a7"
      },
      "source": [
        "## References\n",
        "\n",
        "[Supervised Fine-tuning Trainer](https://huggingface.co/docs/trl/main/en/sft_trainer)\n",
        "\n",
        "[Finetuning To Follow Instructions](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/ch07.ipynb)\n",
        "\n",
        "[Finetuning with LoRA -- A Hands-On Example](https://lightning.ai/lightning-ai/studios/code-lora-from-scratch)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "819c30a0-791e-414f-afba-f093af1a36d7"
      },
      "source": [
        "```{## Change Log|Date (YYYY-MM-DD)|Version|Changed By|Change Description||-|-|-|-||2024-07-18|1.0|Wojciech \"Victor\" Fulmyk|Lab Written||2024-07-25|2.0|Fateme Akbari|Bugs Fixed||2024-07-31|3.0|Bhavika Chhatbar|ID reviewed|}\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "66e33959-f500-43ba-a871-742d8b489ab9"
      },
      "source": [
        "© Copyright IBM Corporation. All rights reserved.\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.9"
    },
    "prev_pub_hash": "280a2cf79e2287085899526a711a657e3abe91f52fd641be6356c1ef9f2bafbd",
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "cb71b09a16774e72aeec01c0b42f7d66": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_96b5a5169e3741e2820d75cac5190494",
              "IPY_MODEL_0d17230fabdc49c2991673954cffbaac",
              "IPY_MODEL_d2157dd40a694e7981a45b3dcdc1ff38"
            ],
            "layout": "IPY_MODEL_ac4a9b64af5847c6be5e40721f09a8b8"
          }
        },
        "96b5a5169e3741e2820d75cac5190494": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_469908a524f4474da3b481d7c63d34e6",
            "placeholder": "​",
            "style": "IPY_MODEL_dd11fc531a0e4df5b8d67f2affbdb97f",
            "value": "Downloading readme: 100%"
          }
        },
        "0d17230fabdc49c2991673954cffbaac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_89790b7d33de4edf81a42725598ccfdc",
            "max": 677,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_612150bc29284f29bc6c855ce0f56f83",
            "value": 677
          }
        },
        "d2157dd40a694e7981a45b3dcdc1ff38": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_52813aed63804e14b08ee718e1e749e3",
            "placeholder": "​",
            "style": "IPY_MODEL_50efc449012b4896a75fc88863329284",
            "value": " 677/677 [00:00&lt;00:00, 11.3kB/s]"
          }
        },
        "ac4a9b64af5847c6be5e40721f09a8b8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "469908a524f4474da3b481d7c63d34e6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd11fc531a0e4df5b8d67f2affbdb97f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "89790b7d33de4edf81a42725598ccfdc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "612150bc29284f29bc6c855ce0f56f83": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "52813aed63804e14b08ee718e1e749e3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50efc449012b4896a75fc88863329284": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6ef8949a8ea7469bbcaea6d519e85c27": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_094f4fc7fc6b4944874ec4b8edbf857e",
              "IPY_MODEL_a3a7f97cae8246f4afada61d26c62183",
              "IPY_MODEL_d979a2231053453ca94f13a9e4b7131e"
            ],
            "layout": "IPY_MODEL_99f29bb6eb464adc9f76a3278ee53464"
          }
        },
        "094f4fc7fc6b4944874ec4b8edbf857e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_129dc7c02f714b0792b56489e8d2ce10",
            "placeholder": "​",
            "style": "IPY_MODEL_f3c0c6a080c74770a9a1d0e6a4da108c",
            "value": "Downloading data: 100%"
          }
        },
        "a3a7f97cae8246f4afada61d26c62183": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_751abdc766d4495ca14d169b667127b8",
            "max": 3450938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c2dd774f5aa7461f95b645bc08f0d38e",
            "value": 3450938
          }
        },
        "d979a2231053453ca94f13a9e4b7131e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0bf7354018534fbf925cf52a92470558",
            "placeholder": "​",
            "style": "IPY_MODEL_6cf18909c3bb409fb4fb7266f8d86351",
            "value": " 3.45M/3.45M [00:00&lt;00:00, 9.35MB/s]"
          }
        },
        "99f29bb6eb464adc9f76a3278ee53464": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "129dc7c02f714b0792b56489e8d2ce10": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f3c0c6a080c74770a9a1d0e6a4da108c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "751abdc766d4495ca14d169b667127b8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c2dd774f5aa7461f95b645bc08f0d38e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0bf7354018534fbf925cf52a92470558": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6cf18909c3bb409fb4fb7266f8d86351": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ed1ed0052c8045f8b23dca0169dca90a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e6c5dec324e54e2d8d887d8a8cb224e4",
              "IPY_MODEL_899a8a9246974ac9953e7768b6afc2a0",
              "IPY_MODEL_e394ca8bdcc74b58aff34c1bdba7eeae"
            ],
            "layout": "IPY_MODEL_848dd9f9a081460ba7097bf11ae8d5b2"
          }
        },
        "e6c5dec324e54e2d8d887d8a8cb224e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_48314243197c4d48872d53c8ee5882ca",
            "placeholder": "​",
            "style": "IPY_MODEL_be2b16450065479ea4347261c85a97ce",
            "value": "Generating train split: 100%"
          }
        },
        "899a8a9246974ac9953e7768b6afc2a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bfa52b79af9c41fa967506dcf9dbbc19",
            "max": 20022,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4d8a2d293aa242b5b12cb1d8a90b7114",
            "value": 20022
          }
        },
        "e394ca8bdcc74b58aff34c1bdba7eeae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_795a6bbe483a4cd9b65644c91d48dd42",
            "placeholder": "​",
            "style": "IPY_MODEL_66b7b785275c4ea7bd13d325a462efaf",
            "value": " 20022/20022 [00:00&lt;00:00, 9888.98 examples/s]"
          }
        },
        "848dd9f9a081460ba7097bf11ae8d5b2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "48314243197c4d48872d53c8ee5882ca": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "be2b16450065479ea4347261c85a97ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bfa52b79af9c41fa967506dcf9dbbc19": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4d8a2d293aa242b5b12cb1d8a90b7114": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "795a6bbe483a4cd9b65644c91d48dd42": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "66b7b785275c4ea7bd13d325a462efaf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d27d92400e834384bab9a0b1a2a56576": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3dbdc0ec32ba473ca50c7fe836df924b",
              "IPY_MODEL_7eb2ef695c4b4dd48f4cc82bdf80b7e8",
              "IPY_MODEL_ff1501322ad349b4984eeaf83c145628"
            ],
            "layout": "IPY_MODEL_d4cf71ae37f64f8c8a9f978141f77933"
          }
        },
        "3dbdc0ec32ba473ca50c7fe836df924b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_884388e5313147dd885dac5d57ec0dd2",
            "placeholder": "​",
            "style": "IPY_MODEL_4e07ff7572aa48cd86f5ea25866e6d66",
            "value": "Filter: 100%"
          }
        },
        "7eb2ef695c4b4dd48f4cc82bdf80b7e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1997613e0a2a400a865690b3052c14b9",
            "max": 20022,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2154d9b562cf49258e18370200ca9884",
            "value": 20022
          }
        },
        "ff1501322ad349b4984eeaf83c145628": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2d0461f4983f489ca4164b9ebc666bfe",
            "placeholder": "​",
            "style": "IPY_MODEL_e51dc15c3ac94fda8701d46a707db362",
            "value": " 20022/20022 [00:00&lt;00:00, 40633.89 examples/s]"
          }
        },
        "d4cf71ae37f64f8c8a9f978141f77933": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "884388e5313147dd885dac5d57ec0dd2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4e07ff7572aa48cd86f5ea25866e6d66": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1997613e0a2a400a865690b3052c14b9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2154d9b562cf49258e18370200ca9884": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2d0461f4983f489ca4164b9ebc666bfe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e51dc15c3ac94fda8701d46a707db362": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c8cdc13231094592bd81c28684e9fb2a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e479e23815dd4abf95c197cd21a318ff",
              "IPY_MODEL_2f5c0fe22f034699aa0ea96df619ab1a",
              "IPY_MODEL_4bf87dc0b6604fe7a09a6c2ac4de8c58"
            ],
            "layout": "IPY_MODEL_22ebcd4623b6436cbd8c110c921e8dc8"
          }
        },
        "e479e23815dd4abf95c197cd21a318ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_52cc00e60acb481d87ced063d2076391",
            "placeholder": "​",
            "style": "IPY_MODEL_4e0e5e1c24454dbc9cc797a952a715f2",
            "value": "config.json: 100%"
          }
        },
        "2f5c0fe22f034699aa0ea96df619ab1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a31e5c6e54c248c0b3aa8c7aa14f21df",
            "max": 644,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1a94c7ead114410087fc2cdbb6336012",
            "value": 644
          }
        },
        "4bf87dc0b6604fe7a09a6c2ac4de8c58": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_639a510fab45448bb954858e01436423",
            "placeholder": "​",
            "style": "IPY_MODEL_b424b6856eb740ea9a6eb1d26211e027",
            "value": " 644/644 [00:00&lt;00:00, 12.6kB/s]"
          }
        },
        "22ebcd4623b6436cbd8c110c921e8dc8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "52cc00e60acb481d87ced063d2076391": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4e0e5e1c24454dbc9cc797a952a715f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a31e5c6e54c248c0b3aa8c7aa14f21df": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1a94c7ead114410087fc2cdbb6336012": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "639a510fab45448bb954858e01436423": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b424b6856eb740ea9a6eb1d26211e027": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "38764cfaeb7f4515ac584bb305028a9d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_535b6645d97b47d7b21773e6bf4b5df9",
              "IPY_MODEL_245e372f16ea41a1b4f04f02266bf58c",
              "IPY_MODEL_f7c6ab1fe5ce4b6291e48f19b2f5db4d"
            ],
            "layout": "IPY_MODEL_ad13890abdae4e2f9fc7871444eb97e2"
          }
        },
        "535b6645d97b47d7b21773e6bf4b5df9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b8f8556fd8184858b684472212c06917",
            "placeholder": "​",
            "style": "IPY_MODEL_050f25d8e2ae4dd5828b563eeb6623f5",
            "value": "pytorch_model.bin: 100%"
          }
        },
        "245e372f16ea41a1b4f04f02266bf58c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3632a3e1a9e94466a73cfb163a62e671",
            "max": 662513657,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f7a309b732a04cf697ed7fb38c733cc7",
            "value": 662513657
          }
        },
        "f7c6ab1fe5ce4b6291e48f19b2f5db4d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c8d2f259b35041c5aaeebca1bc0c62eb",
            "placeholder": "​",
            "style": "IPY_MODEL_8d34b10a6d4247ac87759d144d6bb3d5",
            "value": " 663M/663M [00:05&lt;00:00, 161MB/s]"
          }
        },
        "ad13890abdae4e2f9fc7871444eb97e2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b8f8556fd8184858b684472212c06917": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "050f25d8e2ae4dd5828b563eeb6623f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3632a3e1a9e94466a73cfb163a62e671": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f7a309b732a04cf697ed7fb38c733cc7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c8d2f259b35041c5aaeebca1bc0c62eb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8d34b10a6d4247ac87759d144d6bb3d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e6421a90e19c4d02885a319aaf9fe570": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c895258a12c846f499c23bcec90d73f2",
              "IPY_MODEL_7db77992b91944c5908e67c87f1593f0",
              "IPY_MODEL_1e05781a4f2c447bb4dc1cfeb7f88dbe"
            ],
            "layout": "IPY_MODEL_9eac4d6f00f340d587f4d56a4765fe83"
          }
        },
        "c895258a12c846f499c23bcec90d73f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5612b4edd760418b9b9681cd0ea22452",
            "placeholder": "​",
            "style": "IPY_MODEL_2e386e462f384cc98320acd5e20b9c96",
            "value": "generation_config.json: 100%"
          }
        },
        "7db77992b91944c5908e67c87f1593f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fb7cd56fb9ff49d392fd490c7de64e80",
            "max": 137,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a4d41ea0eb37482a9b92b7eeb9d19244",
            "value": 137
          }
        },
        "1e05781a4f2c447bb4dc1cfeb7f88dbe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_60e923ea4a874703ae87ededaeb305da",
            "placeholder": "​",
            "style": "IPY_MODEL_de44cc2a5a14497c88bf3f88c1a67692",
            "value": " 137/137 [00:00&lt;00:00, 2.60kB/s]"
          }
        },
        "9eac4d6f00f340d587f4d56a4765fe83": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5612b4edd760418b9b9681cd0ea22452": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2e386e462f384cc98320acd5e20b9c96": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fb7cd56fb9ff49d392fd490c7de64e80": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a4d41ea0eb37482a9b92b7eeb9d19244": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "60e923ea4a874703ae87ededaeb305da": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "de44cc2a5a14497c88bf3f88c1a67692": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ca6e35c97b9e4a9f855edd2f199566a8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c00a08b06dbf4866aa1ffc060ff263d8",
              "IPY_MODEL_3ed01a138c30476d8c0aa428d8d37593",
              "IPY_MODEL_f1af1952c322494287e53c07dbfcb982"
            ],
            "layout": "IPY_MODEL_40ea2df4d7d24e6bbb0e1e8303121448"
          }
        },
        "c00a08b06dbf4866aa1ffc060ff263d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b8a5a1c73e3a48079f8ec078c23009f1",
            "placeholder": "​",
            "style": "IPY_MODEL_1b941551b894444086f6b9a360ce9429",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "3ed01a138c30476d8c0aa428d8d37593": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_16a2402b708a45c7b8a751498a2aa1d4",
            "max": 685,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9d50bad624f247d9b4135a61d7d3d42e",
            "value": 685
          }
        },
        "f1af1952c322494287e53c07dbfcb982": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5be0eeb25ade4322828f2b94cce4e227",
            "placeholder": "​",
            "style": "IPY_MODEL_9eb87b87cc41412d95c78c727f70f04c",
            "value": " 685/685 [00:00&lt;00:00, 17.5kB/s]"
          }
        },
        "40ea2df4d7d24e6bbb0e1e8303121448": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b8a5a1c73e3a48079f8ec078c23009f1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b941551b894444086f6b9a360ce9429": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "16a2402b708a45c7b8a751498a2aa1d4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9d50bad624f247d9b4135a61d7d3d42e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5be0eeb25ade4322828f2b94cce4e227": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9eb87b87cc41412d95c78c727f70f04c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "108704c293f9413b9416456dc7a9c62f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7cb5bb4b9721418090f8d15b728eaf8c",
              "IPY_MODEL_ed583888de654eb5b89ef9b73298b5f8",
              "IPY_MODEL_67db5326b44447a8bd06fd92254c8c01"
            ],
            "layout": "IPY_MODEL_6a2c6235059540db9781fc9b15a8dfbd"
          }
        },
        "7cb5bb4b9721418090f8d15b728eaf8c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5a9cf5d9ceb9465b97b402b1898eeaae",
            "placeholder": "​",
            "style": "IPY_MODEL_ecfee5b1ca3b42cf82becfb29fb12348",
            "value": "vocab.json: 100%"
          }
        },
        "ed583888de654eb5b89ef9b73298b5f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e81db49931784e19b72a2bf6e2f903ef",
            "max": 898822,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_97be585f2c774a53b2c69f99331d4603",
            "value": 898822
          }
        },
        "67db5326b44447a8bd06fd92254c8c01": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5c2169f70aee4d589ea0c455aa7620e4",
            "placeholder": "​",
            "style": "IPY_MODEL_26ae4b273edb41e89ff724a99fbb8ec1",
            "value": " 899k/899k [00:00&lt;00:00, 3.05MB/s]"
          }
        },
        "6a2c6235059540db9781fc9b15a8dfbd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5a9cf5d9ceb9465b97b402b1898eeaae": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ecfee5b1ca3b42cf82becfb29fb12348": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e81db49931784e19b72a2bf6e2f903ef": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "97be585f2c774a53b2c69f99331d4603": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5c2169f70aee4d589ea0c455aa7620e4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "26ae4b273edb41e89ff724a99fbb8ec1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "08412af0f9774c8489203f39d7a561d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5e7ee32b076547b08a38959803d7ff84",
              "IPY_MODEL_68c0d5087c4c48b29b35fe143a9a426e",
              "IPY_MODEL_b8f55486a7ab4a4e9aabbdd6b5426313"
            ],
            "layout": "IPY_MODEL_d977c7dfa1794df797a2a2c2437c7557"
          }
        },
        "5e7ee32b076547b08a38959803d7ff84": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c1a56fea3e8f4a9e895ce5728b3056b2",
            "placeholder": "​",
            "style": "IPY_MODEL_c199a33a39d441d496f1c35409fb7f7e",
            "value": "merges.txt: 100%"
          }
        },
        "68c0d5087c4c48b29b35fe143a9a426e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4f9ff10654894b20868fd93b7d3fc5dc",
            "max": 456318,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c0e02fe256a24314b45e9e1f2b791630",
            "value": 456318
          }
        },
        "b8f55486a7ab4a4e9aabbdd6b5426313": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_390b46ec62fe427ab72d891257eec1d0",
            "placeholder": "​",
            "style": "IPY_MODEL_27fbd933a5854cc89d80df953276fab0",
            "value": " 456k/456k [00:00&lt;00:00, 7.64MB/s]"
          }
        },
        "d977c7dfa1794df797a2a2c2437c7557": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c1a56fea3e8f4a9e895ce5728b3056b2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c199a33a39d441d496f1c35409fb7f7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4f9ff10654894b20868fd93b7d3fc5dc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c0e02fe256a24314b45e9e1f2b791630": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "390b46ec62fe427ab72d891257eec1d0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "27fbd933a5854cc89d80df953276fab0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6f03a18f010c4bfdb965c021f3cfd6a8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e940f97c4e2b4c228f9bd6f5553a5b60",
              "IPY_MODEL_c6c7cd3f24804eed963072699946b5ec",
              "IPY_MODEL_e689e6c4c6df48df89756f7123e2f51e"
            ],
            "layout": "IPY_MODEL_67c221f61aa644c3afa940b0ea1a6605"
          }
        },
        "e940f97c4e2b4c228f9bd6f5553a5b60": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4cee318043684da38d0bf1fe90c2d2a9",
            "placeholder": "​",
            "style": "IPY_MODEL_98c584da8b334ee99e5dd2c7c732292f",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "c6c7cd3f24804eed963072699946b5ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c18028c92a7b4d4eb008414ae441ec72",
            "max": 441,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9cf6642dcd114e508122e0e4e7519fbc",
            "value": 441
          }
        },
        "e689e6c4c6df48df89756f7123e2f51e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_118eefe4b2ab43a69dc48f777392309b",
            "placeholder": "​",
            "style": "IPY_MODEL_0af2704b3d924b958d9e42761e39f809",
            "value": " 441/441 [00:00&lt;00:00, 7.57kB/s]"
          }
        },
        "67c221f61aa644c3afa940b0ea1a6605": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4cee318043684da38d0bf1fe90c2d2a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "98c584da8b334ee99e5dd2c7c732292f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c18028c92a7b4d4eb008414ae441ec72": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9cf6642dcd114e508122e0e4e7519fbc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "118eefe4b2ab43a69dc48f777392309b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0af2704b3d924b958d9e42761e39f809": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "63ff4733f8c74402b9ffbd7bbc3ecdae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c9c3f4bb99174c78955a2aba0445ae3e",
              "IPY_MODEL_37f4a74d6f644f068d26e3e3db81ab87",
              "IPY_MODEL_8c34e52649f34d4a9eb284ea7c4ceec1"
            ],
            "layout": "IPY_MODEL_c239cdd94256492ba080c92c2536e671"
          }
        },
        "c9c3f4bb99174c78955a2aba0445ae3e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_23402071a458494b9ad891369b52e3c8",
            "placeholder": "​",
            "style": "IPY_MODEL_1c5607f96a2648c0ad831d181587a2bd",
            "value": "Downloading builder script: 100%"
          }
        },
        "37f4a74d6f644f068d26e3e3db81ab87": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_919c94639eb74b3a9b70614580320e49",
            "max": 8146,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9a7672a991ca4beaacf8f0cedcdb18b5",
            "value": 8146
          }
        },
        "8c34e52649f34d4a9eb284ea7c4ceec1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1d46a982870a48619c07265f5605ffbe",
            "placeholder": "​",
            "style": "IPY_MODEL_2036147505ab420a8303e93d4c007649",
            "value": " 8.15k/8.15k [00:00&lt;00:00, 268kB/s]"
          }
        },
        "c239cdd94256492ba080c92c2536e671": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "23402071a458494b9ad891369b52e3c8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1c5607f96a2648c0ad831d181587a2bd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "919c94639eb74b3a9b70614580320e49": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9a7672a991ca4beaacf8f0cedcdb18b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1d46a982870a48619c07265f5605ffbe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2036147505ab420a8303e93d4c007649": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6a18ea97fe4f46f08014acdb59e22266": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_494facd5086b42c4aaa7e2a38b735821",
              "IPY_MODEL_ab8a888a4fd64c32af30ca317a432a98",
              "IPY_MODEL_95b354a5b265406b8349094140e12a14"
            ],
            "layout": "IPY_MODEL_4b76d12ed713489286ed51b35916df35"
          }
        },
        "494facd5086b42c4aaa7e2a38b735821": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53a6e50880c94d2f9fdca1fcc2724855",
            "placeholder": "​",
            "style": "IPY_MODEL_a6ed54e51fa14d94b1e7dbf57e33f4d3",
            "value": "Map: 100%"
          }
        },
        "ab8a888a4fd64c32af30ca317a432a98": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3d041e15ca0a4d0a9e11430adae3a354",
            "max": 7811,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1aa7e94e50c0466b8b7c334ecfed31d9",
            "value": 7811
          }
        },
        "95b354a5b265406b8349094140e12a14": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_da6df5170eef450cabeaa2b62c507795",
            "placeholder": "​",
            "style": "IPY_MODEL_4e09134cc8c94837a05ceb95868186b9",
            "value": " 7811/7811 [00:02&lt;00:00, 2800.65 examples/s]"
          }
        },
        "4b76d12ed713489286ed51b35916df35": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53a6e50880c94d2f9fdca1fcc2724855": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a6ed54e51fa14d94b1e7dbf57e33f4d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3d041e15ca0a4d0a9e11430adae3a354": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1aa7e94e50c0466b8b7c334ecfed31d9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "da6df5170eef450cabeaa2b62c507795": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4e09134cc8c94837a05ceb95868186b9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a1982556021b4e05ad6f33c856c51d72": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4f2c44d451c1469891eafdda581bb59a",
              "IPY_MODEL_3926f6ca257e4eb4a6406f5d0d2c8459",
              "IPY_MODEL_35d3a3214ef541b3a4ca7470e4961130"
            ],
            "layout": "IPY_MODEL_7698b0045ab14a15a076754ebc6e192f"
          }
        },
        "4f2c44d451c1469891eafdda581bb59a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e811ba36d20e4e59b9d98c429d8b59ac",
            "placeholder": "​",
            "style": "IPY_MODEL_ded47bacb3f242daa0c2ff6496450699",
            "value": "Map: 100%"
          }
        },
        "3926f6ca257e4eb4a6406f5d0d2c8459": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2079a42b86544c6c8522f5982b9b0435",
            "max": 1953,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_17d309c86eb04c76b0cd0878db3c8af7",
            "value": 1953
          }
        },
        "35d3a3214ef541b3a4ca7470e4961130": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_45aca610bd2e4160812b59cf879b72b2",
            "placeholder": "​",
            "style": "IPY_MODEL_aa6da4fb803340e4aecbaab9ccfe9c00",
            "value": " 1953/1953 [00:00&lt;00:00, 2557.30 examples/s]"
          }
        },
        "7698b0045ab14a15a076754ebc6e192f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e811ba36d20e4e59b9d98c429d8b59ac": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ded47bacb3f242daa0c2ff6496450699": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2079a42b86544c6c8522f5982b9b0435": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "17d309c86eb04c76b0cd0878db3c8af7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "45aca610bd2e4160812b59cf879b72b2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa6da4fb803340e4aecbaab9ccfe9c00": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6afed720c92c438aa157d67624921096": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bb25d1576a204bbca56a21bc265e74a4",
              "IPY_MODEL_1ba22188ed5e497aac19bd462278e061",
              "IPY_MODEL_1ad25ddc45604ef799b8cb46385e311f"
            ],
            "layout": "IPY_MODEL_cd15ac73f8df447ba30812ce57980062"
          }
        },
        "bb25d1576a204bbca56a21bc265e74a4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_afd3ffa80d3c46e784d167d274b62009",
            "placeholder": "​",
            "style": "IPY_MODEL_42b92333439640dab6a942235d3451f0",
            "value": "config.json: 100%"
          }
        },
        "1ba22188ed5e497aac19bd462278e061": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f7e01eaac7374358b6afc75d1787db61",
            "max": 1007,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_56f2e6b60b8141edb8f7184160c005e6",
            "value": 1007
          }
        },
        "1ad25ddc45604ef799b8cb46385e311f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_348b53044d184bcdae7483a4b7c44c5c",
            "placeholder": "​",
            "style": "IPY_MODEL_300d8f62719445b1b05aa7dc7db1cba3",
            "value": " 1.01k/1.01k [00:00&lt;00:00, 36.6kB/s]"
          }
        },
        "cd15ac73f8df447ba30812ce57980062": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "afd3ffa80d3c46e784d167d274b62009": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "42b92333439640dab6a942235d3451f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f7e01eaac7374358b6afc75d1787db61": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "56f2e6b60b8141edb8f7184160c005e6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "348b53044d184bcdae7483a4b7c44c5c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "300d8f62719445b1b05aa7dc7db1cba3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "19feda7b9bdf4c31a5d30c0f2734a6f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_dcb3b4ddd19b4556aeff4fd6957f3a5c",
              "IPY_MODEL_83f49c975c8740ea918a5d1a0295f348",
              "IPY_MODEL_7017cae302b34ead9af02004fcdfc1b3"
            ],
            "layout": "IPY_MODEL_5c9d502469224456b8112d46222029b1"
          }
        },
        "dcb3b4ddd19b4556aeff4fd6957f3a5c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dfe2f4403a094204bd1365e5015ab519",
            "placeholder": "​",
            "style": "IPY_MODEL_3457f96926064611a8603aa450aa2267",
            "value": "model.safetensors: 100%"
          }
        },
        "83f49c975c8740ea918a5d1a0295f348": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9862de8844304d97a02510bfd196be74",
            "max": 525979192,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_bdb892c171d24163818c428e984e397f",
            "value": 525979192
          }
        },
        "7017cae302b34ead9af02004fcdfc1b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c9791fedcab84b5cb77a119ec53cac22",
            "placeholder": "​",
            "style": "IPY_MODEL_41973131f789482cb0336f8218a96555",
            "value": " 526M/526M [00:07&lt;00:00, 39.2MB/s]"
          }
        },
        "5c9d502469224456b8112d46222029b1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dfe2f4403a094204bd1365e5015ab519": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3457f96926064611a8603aa450aa2267": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9862de8844304d97a02510bfd196be74": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bdb892c171d24163818c428e984e397f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c9791fedcab84b5cb77a119ec53cac22": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "41973131f789482cb0336f8218a96555": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "acf6c604cecb47c1b64a5f916da16553": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_aca7e8097745401690640a8fda63000b",
              "IPY_MODEL_97db6e25bf174a27aff035e8709740a2",
              "IPY_MODEL_725eec78e3e147eeba77aa7f6f951d8f"
            ],
            "layout": "IPY_MODEL_b375ecd21a6e41c8af50bcf8a5b895b7"
          }
        },
        "aca7e8097745401690640a8fda63000b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c8560fa1f14e43f49cf6dff3bb6d10f5",
            "placeholder": "​",
            "style": "IPY_MODEL_ca3f78b9043648a29223de78f62955d3",
            "value": "generation_config.json: 100%"
          }
        },
        "97db6e25bf174a27aff035e8709740a2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_99a0d001b6d24e6b8fffe8430f6576a1",
            "max": 119,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6e449f9fe6cd4d2284dfca4dff26e91b",
            "value": 119
          }
        },
        "725eec78e3e147eeba77aa7f6f951d8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8fc9eeab3bcb47eab0b3c03307891a9e",
            "placeholder": "​",
            "style": "IPY_MODEL_0df7cc42982b4b698c7d8b715651bcf8",
            "value": " 119/119 [00:00&lt;00:00, 3.56kB/s]"
          }
        },
        "b375ecd21a6e41c8af50bcf8a5b895b7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c8560fa1f14e43f49cf6dff3bb6d10f5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ca3f78b9043648a29223de78f62955d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "99a0d001b6d24e6b8fffe8430f6576a1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e449f9fe6cd4d2284dfca4dff26e91b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8fc9eeab3bcb47eab0b3c03307891a9e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0df7cc42982b4b698c7d8b715651bcf8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f601263aadfd48f184f629b85e090fc2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6e0240d86c6f40319d518a5c19469ef4",
              "IPY_MODEL_0432bef10f824cdfa39a13f92a4fa2da",
              "IPY_MODEL_6e4b877dedef438fa3e6d8627fbbae97"
            ],
            "layout": "IPY_MODEL_7f5e7c24b78842958ff922080a5ca72b"
          }
        },
        "6e0240d86c6f40319d518a5c19469ef4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9f0ddd64943c4bd1bd529009fbb4d881",
            "placeholder": "​",
            "style": "IPY_MODEL_ee1b3c7562774838a072bf584a23e58a",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "0432bef10f824cdfa39a13f92a4fa2da": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_beaf3477020b4f528feb4095e977dc9d",
            "max": 727,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_485463a218cf42f582815f1820942db3",
            "value": 727
          }
        },
        "6e4b877dedef438fa3e6d8627fbbae97": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6489305c76504d258d12033c1f06670f",
            "placeholder": "​",
            "style": "IPY_MODEL_8d6974bf25784734b2a6f3e8a74bdea7",
            "value": " 727/727 [00:00&lt;00:00, 31.8kB/s]"
          }
        },
        "7f5e7c24b78842958ff922080a5ca72b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f0ddd64943c4bd1bd529009fbb4d881": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ee1b3c7562774838a072bf584a23e58a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "beaf3477020b4f528feb4095e977dc9d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "485463a218cf42f582815f1820942db3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6489305c76504d258d12033c1f06670f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8d6974bf25784734b2a6f3e8a74bdea7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "27121f7fec3f496da073d5755599f196": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0a8c2f0457d544c4b0c6e815c1933fd9",
              "IPY_MODEL_10881cde1bc444949cddd50ce273c60b",
              "IPY_MODEL_7172744130184c618aa1dea9ef447fca"
            ],
            "layout": "IPY_MODEL_2bc0ad9e1f5f41479f8280ca08bf2161"
          }
        },
        "0a8c2f0457d544c4b0c6e815c1933fd9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9cfcbe6b6e1c4900b5fe0968e803e37d",
            "placeholder": "​",
            "style": "IPY_MODEL_fbe1e1f67db54658823d581873cb1d3d",
            "value": "vocab.json: 100%"
          }
        },
        "10881cde1bc444949cddd50ce273c60b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b664e4ccffb74baa88f8117d51800137",
            "max": 898669,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e526ffa378df4333a5e94e0bfb3b3321",
            "value": 898669
          }
        },
        "7172744130184c618aa1dea9ef447fca": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8322ac0dff5a4f5085159fcefbf7e9a3",
            "placeholder": "​",
            "style": "IPY_MODEL_bce89e3eccb84e6a9a04298c401a0739",
            "value": " 899k/899k [00:00&lt;00:00, 10.3MB/s]"
          }
        },
        "2bc0ad9e1f5f41479f8280ca08bf2161": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9cfcbe6b6e1c4900b5fe0968e803e37d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fbe1e1f67db54658823d581873cb1d3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b664e4ccffb74baa88f8117d51800137": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e526ffa378df4333a5e94e0bfb3b3321": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8322ac0dff5a4f5085159fcefbf7e9a3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bce89e3eccb84e6a9a04298c401a0739": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a29dc30013b14b738f1a56121bdfc1af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4d00a9d0828942808d6ebac1a06b19a5",
              "IPY_MODEL_1271bd65ae7f4e2190cae8b3c0faad46",
              "IPY_MODEL_94bb79cadf204cfbbe44b6225d20b486"
            ],
            "layout": "IPY_MODEL_f296301af64e4366a03c3fb62aeb9951"
          }
        },
        "4d00a9d0828942808d6ebac1a06b19a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e60284be141d4b34996af40b96004afc",
            "placeholder": "​",
            "style": "IPY_MODEL_ebc331a6b29f44cb82b8fe688b777399",
            "value": "merges.txt: 100%"
          }
        },
        "1271bd65ae7f4e2190cae8b3c0faad46": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c6c66a18955a404eb060883a01e12cb4",
            "max": 456318,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_330767b5bb8a494ea54be4982863489d",
            "value": 456318
          }
        },
        "94bb79cadf204cfbbe44b6225d20b486": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9526627f55074fedb462059f6ead335a",
            "placeholder": "​",
            "style": "IPY_MODEL_c9d86307612d4a5e9eecd05bfddb0b21",
            "value": " 456k/456k [00:00&lt;00:00, 11.9MB/s]"
          }
        },
        "f296301af64e4366a03c3fb62aeb9951": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e60284be141d4b34996af40b96004afc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ebc331a6b29f44cb82b8fe688b777399": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c6c66a18955a404eb060883a01e12cb4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "330767b5bb8a494ea54be4982863489d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9526627f55074fedb462059f6ead335a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c9d86307612d4a5e9eecd05bfddb0b21": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9e3f300b699c4ed2a145c1532a21d452": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b1cc752dfbb34d3d9cc4b0c2c4ecd463",
              "IPY_MODEL_c599fb77dcfe47338558879fe319861e",
              "IPY_MODEL_cbdd8050b32c4eaa8be93fc1deff1e18"
            ],
            "layout": "IPY_MODEL_42b07b04eb7b4f098f830136ab18f964"
          }
        },
        "b1cc752dfbb34d3d9cc4b0c2c4ecd463": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c95da947d0f7466bbbf7e30d13750522",
            "placeholder": "​",
            "style": "IPY_MODEL_be31f9a8800f4a8bb155bfd5f09f1e8a",
            "value": "tokenizer.json: 100%"
          }
        },
        "c599fb77dcfe47338558879fe319861e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_943bffd107f94788aa4dba533026cd8a",
            "max": 2107652,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9cc7a247767d4297b6b9bc24deec83e7",
            "value": 2107652
          }
        },
        "cbdd8050b32c4eaa8be93fc1deff1e18": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_86dbc9e1e27542b980d74860905af971",
            "placeholder": "​",
            "style": "IPY_MODEL_3bc1fbc3954c4e1caf458346098c8576",
            "value": " 2.11M/2.11M [00:00&lt;00:00, 10.3MB/s]"
          }
        },
        "42b07b04eb7b4f098f830136ab18f964": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c95da947d0f7466bbbf7e30d13750522": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "be31f9a8800f4a8bb155bfd5f09f1e8a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "943bffd107f94788aa4dba533026cd8a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9cc7a247767d4297b6b9bc24deec83e7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "86dbc9e1e27542b980d74860905af971": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3bc1fbc3954c4e1caf458346098c8576": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3c466e0098cc4d3d80a70c769296819e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cdfc0c6c89f84a1bbe6e78297dd430ee",
              "IPY_MODEL_649719c96c8d47888fbbbf5f279c0d00",
              "IPY_MODEL_27da41950aa74c4cb6cc5fbc2adef4ab"
            ],
            "layout": "IPY_MODEL_0db9fc12a376458995fffbc9d4cf2064"
          }
        },
        "cdfc0c6c89f84a1bbe6e78297dd430ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0cd311bd79a6444c8e19de1567e762f9",
            "placeholder": "​",
            "style": "IPY_MODEL_7f0ff8d6a9754165897f122436204f14",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "649719c96c8d47888fbbbf5f279c0d00": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c5b7f9c9296c4c85890ffeb9dbacbef5",
            "max": 357,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_348ca5f641484e6196a075f8907ead60",
            "value": 357
          }
        },
        "27da41950aa74c4cb6cc5fbc2adef4ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_586d9fa41d78441fa24c353048ef28de",
            "placeholder": "​",
            "style": "IPY_MODEL_5039871542fa41ffb0d55b105acc475a",
            "value": " 357/357 [00:00&lt;00:00, 22.6kB/s]"
          }
        },
        "0db9fc12a376458995fffbc9d4cf2064": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0cd311bd79a6444c8e19de1567e762f9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f0ff8d6a9754165897f122436204f14": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c5b7f9c9296c4c85890ffeb9dbacbef5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "348ca5f641484e6196a075f8907ead60": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "586d9fa41d78441fa24c353048ef28de": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5039871542fa41ffb0d55b105acc475a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}