Transformers_Part-4_distilgpt2_finetune.ipynb

{"cells":[{"cell_type":"markdown","id":"a09e8965","metadata":{"id":"a09e8965"},"source":["# Transformers Part 2:\n","\n","Text generation is the task of producing new text from a given input prompt. An example of text generation using machine learning is `GitHub’s Copilot`, which can generate code. Apart from code generation, text generation models can:\n","\n","1. Generate stories, for example, try passing “Once upon a time \" as input to a GPT-2 model.\n","2. Generate poetry.\n","3. Complete a paragraph with missing sentences. \n","4. Summarize an article.\n","5. Answer questions given some context.\n","\n","And many more...\n","\n","In this notebook, we focus on training a causal language model from scratch. A causal language model tries to predict the next token in a given input sequence. In this way, it builds an understanding of the data it is trained on. \n","\n","So, the model in this notebook is __not finetuned__ but rather it is trained from scratch with randomly initialized parameters. \n","\n","__Agenda__:\n","\n","1. Load the dataset from csv files\n","2. Tokenize and preprocess text\n","3. Create Batches\n","4. Train GPT-2 model \n","5. Evaluate trained model"]},{"cell_type":"code","source":["!pip install datasets \n","!pip install transformers\n","!pip install evaluate\n","!pip install bert_score"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"PcICtGI6wwyR","executionInfo":{"status":"ok","timestamp":1681608467973,"user_tz":240,"elapsed":16099,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"32121f11-4cc5-44fa-dbf8-adf937cb82a9"},"id":"PcICtGI6wwyR","execution_count":1,"outputs":[{"output_type":"stream","name":"stdout","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Requirement already satisfied: datasets in /usr/local/lib/python3.9/dist-packages (2.11.0)\n","Requirement already satisfied: multiprocess in /usr/local/lib/python3.9/dist-packages (from datasets) (0.70.14)\n","Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.9/dist-packages (from datasets) (9.0.0)\n","Requirement already satisfied: dill<0.3.7,>=0.3.0 in /usr/local/lib/python3.9/dist-packages (from datasets) (0.3.6)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from datasets) (6.0)\n","Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.9/dist-packages (from datasets) (2023.4.0)\n","Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.9/dist-packages (from datasets) (0.18.0)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.9/dist-packages (from datasets) (3.8.4)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from datasets) (1.5.3)\n","Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.9/dist-packages (from datasets) (2.27.1)\n","Requirement already satisfied: huggingface-hub<1.0.0,>=0.11.0 in /usr/local/lib/python3.9/dist-packages (from datasets) (0.13.4)\n","Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.9/dist-packages (from datasets) (4.65.0)\n","Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from datasets) (23.0)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from datasets) (1.22.4)\n","Requirement already satisfied: xxhash in /usr/local/lib/python3.9/dist-packages (from datasets) (3.2.0)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (1.8.2)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (22.2.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (1.3.3)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (4.0.2)\n","Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (2.0.12)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (3.11.0)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (4.5.0)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets) (3.4)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets) (2022.12.7)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets) (1.26.15)\n","Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets) (2022.7.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)\n","Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Requirement already satisfied: transformers in /usr/local/lib/python3.9/dist-packages (4.28.1)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.9/dist-packages (from transformers) (0.13.4)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers) (3.11.0)\n","Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.9/dist-packages (from transformers) (4.65.0)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from transformers) (1.22.4)\n","Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from transformers) (2.27.1)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from transformers) (6.0)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers) (2022.10.31)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from transformers) (23.0)\n","Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.9/dist-packages (from transformers) (0.13.3)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.5.0)\n","Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (2.0.12)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (2022.12.7)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (1.26.15)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (3.4)\n","Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Requirement already satisfied: evaluate in /usr/local/lib/python3.9/dist-packages (0.4.0)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from evaluate) (1.22.4)\n","Requirement already satisfied: dill in /usr/local/lib/python3.9/dist-packages (from evaluate) (0.3.6)\n","Requirement already satisfied: xxhash in /usr/local/lib/python3.9/dist-packages (from evaluate) (3.2.0)\n","Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.9/dist-packages (from evaluate) (4.65.0)\n","Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.9/dist-packages (from evaluate) (0.18.0)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from evaluate) (1.5.3)\n","Requirement already satisfied: datasets>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from evaluate) (2.11.0)\n","Requirement already satisfied: multiprocess in /usr/local/lib/python3.9/dist-packages (from evaluate) (0.70.14)\n","Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.9/dist-packages (from evaluate) (2023.4.0)\n","Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from evaluate) (23.0)\n","Requirement already satisfied: huggingface-hub>=0.7.0 in /usr/local/lib/python3.9/dist-packages (from evaluate) (0.13.4)\n","Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.9/dist-packages (from evaluate) (2.27.1)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.9/dist-packages (from datasets>=2.0.0->evaluate) (3.8.4)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from datasets>=2.0.0->evaluate) (6.0)\n","Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.9/dist-packages (from datasets>=2.0.0->evaluate) (9.0.0)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.7.0->evaluate) (4.5.0)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.7.0->evaluate) (3.11.0)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->evaluate) (3.4)\n","Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->evaluate) (2.0.12)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->evaluate) (1.26.15)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->evaluate) (2022.12.7)\n","Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->evaluate) (2022.7.1)\n","Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->evaluate) (2.8.2)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.8.2)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (6.0.4)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (22.2.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.3.3)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (4.0.2)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.3.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->evaluate) (1.16.0)\n","Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Requirement already satisfied: bert_score in /usr/local/lib/python3.9/dist-packages (0.3.13)\n","Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from bert_score) (2.27.1)\n","Requirement already satisfied: transformers>=3.0.0 in /usr/local/lib/python3.9/dist-packages (from bert_score) (4.28.1)\n","Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from bert_score) (1.22.4)\n","Requirement already satisfied: matplotlib in /usr/local/lib/python3.9/dist-packages (from bert_score) (3.7.1)\n","Requirement already satisfied: pandas>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from bert_score) (1.5.3)\n","Requirement already satisfied: torch>=1.0.0 in /usr/local/lib/python3.9/dist-packages (from bert_score) (2.0.0+cu118)\n","Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.9/dist-packages (from bert_score) (23.0)\n","Requirement already satisfied: tqdm>=4.31.1 in /usr/local/lib/python3.9/dist-packages (from bert_score) (4.65.0)\n","Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.0.1->bert_score) (2.8.2)\n","Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.0.1->bert_score) (2022.7.1)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.9/dist-packages (from torch>=1.0.0->bert_score) (3.1)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from torch>=1.0.0->bert_score) (3.11.0)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from torch>=1.0.0->bert_score) (3.1.2)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from torch>=1.0.0->bert_score) (4.5.0)\n","Requirement already satisfied: sympy in /usr/local/lib/python3.9/dist-packages (from torch>=1.0.0->bert_score) (1.11.1)\n","Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.9/dist-packages (from torch>=1.0.0->bert_score) (2.0.0)\n","Requirement already satisfied: lit in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.0.0->bert_score) (16.0.1)\n","Requirement already satisfied: cmake in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.0.0->bert_score) (3.25.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from transformers>=3.0.0->bert_score) (6.0)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers>=3.0.0->bert_score) (2022.10.31)\n","Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.9/dist-packages (from transformers>=3.0.0->bert_score) (0.13.3)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.9/dist-packages (from transformers>=3.0.0->bert_score) (0.13.4)\n","Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (1.0.7)\n","Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (8.4.0)\n","Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (1.4.4)\n","Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (4.39.3)\n","Requirement already satisfied: importlib-resources>=3.2.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (5.12.0)\n","Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (3.0.9)\n","Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/dist-packages (from matplotlib->bert_score) (0.11.0)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->bert_score) (3.4)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->bert_score) (1.26.15)\n","Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->bert_score) (2.0.12)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->bert_score) (2022.12.7)\n","Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.9/dist-packages (from importlib-resources>=3.2.0->matplotlib->bert_score) (3.15.0)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas>=1.0.1->bert_score) (1.16.0)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from jinja2->torch>=1.0.0->bert_score) (2.1.2)\n","Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.9/dist-packages (from sympy->torch>=1.0.0->bert_score) (1.3.0)\n"]}]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7jJS65t70AJH","outputId":"f57fe506-178f-4864-d19e-c9d8f66a1225","executionInfo":{"status":"ok","timestamp":1681608469395,"user_tz":240,"elapsed":1430,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/gdrive')"],"id":"7jJS65t70AJH"},{"cell_type":"code","execution_count":3,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"r3j3gtfT0A7s","outputId":"7245f862-591b-494b-b1d1-6089db691d53","executionInfo":{"status":"ok","timestamp":1681608469396,"user_tz":240,"elapsed":18,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["/content/gdrive/My Drive/nlp\n"]}],"source":["%cd /content/gdrive/My Drive/nlp"],"id":"r3j3gtfT0A7s"},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"BbzrNgYe0DgT","outputId":"9d2fdafe-b84e-4998-da8d-cffb5a11792f","executionInfo":{"status":"ok","timestamp":1681608469396,"user_tz":240,"elapsed":12,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["ai-story_gen  data  story_bert\tstory_gpt2  story_gpt2_finetune\n"]}],"source":["!ls"],"id":"BbzrNgYe0DgT"},{"cell_type":"markdown","id":"05cc8f81","metadata":{"id":"05cc8f81"},"source":["## Parameters for notebook execution:\n","\n","We are going to execute this notebook from the terminal with the help of Papermill. So, it's better to store all the parameters need for a successful execution in one place. This way it's easy to manage the parameters. "]},{"cell_type":"code","execution_count":5,"id":"b1b65a58","metadata":{"id":"b1b65a58","executionInfo":{"status":"ok","timestamp":1681608469396,"user_tz":240,"elapsed":7,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["import os \n","\n","# current directory path\n","DIR_PATH=os.getcwd()"]},{"cell_type":"code","execution_count":6,"id":"5f38c189","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"id":"5f38c189","executionInfo":{"status":"ok","timestamp":1681608469397,"user_tz":240,"elapsed":7,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"61498cb2-f79b-48d7-a76b-93c99da8e69a"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["'/content/gdrive/My Drive/nlp'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":6}],"source":["DIR_PATH"]},{"cell_type":"code","execution_count":78,"id":"b309ff88","metadata":{"id":"b309ff88","executionInfo":{"status":"ok","timestamp":1681610349279,"user_tz":240,"elapsed":413,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["# parameters\n","# MODEL CHECKPOINT\n","MODEL_CHECKPOINT='distilgpt2'\n","\n","\n","# SAMPLE\n","TRAIN_ROWS=50000\n","TEST_ROWS=5000\n","\n","# PATH OF CSV FILES\n","TRAIN_PATH= DIR_PATH+\"/data/train_df.csv\"\n","VALID_PATH=DIR_PATH+\"/data/valid_df.csv\"\n","TEST_PATH= DIR_PATH+\"/data/test_df.csv\"\n","\n","# DATA PROCESSING\n","CONTEXT_LEN=256\n","\n","# HYPERPARAMETERS\n","TRAIN_BS= 64\n","TEST_BS= 64 \n","EPOCHS=5"]},{"cell_type":"markdown","id":"85957855","metadata":{"id":"85957855"},"source":["## Load the data:\n","\n","The first step is to load the dataset from `CSV` files.We have to create the huggingface `Dataset` object after loading the data. "]},{"cell_type":"code","execution_count":79,"id":"02e8d9c7","metadata":{"id":"02e8d9c7","executionInfo":{"status":"ok","timestamp":1681610352241,"user_tz":240,"elapsed":1016,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["# import libraries\n","from datasets import load_dataset"]},{"cell_type":"code","execution_count":80,"id":"dc45abe4","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":256,"referenced_widgets":["262a1ac528e24a50aa13290e63f14f2d","32381736e8344b199a14d2517368f5d4","a3ea813b71404513afef43fa9848f339","80cf4aaaa6934d1ca0f2a792f4fcdd2f","cd8dd9c9590a4b05969e07961c706b08","e88822279b7e463ebe2dbb6d49ec1a91","87dc74f744e247819654112d4830b597","343ad42a91d44bd49f4e16b65543fe4b","c43145c74b9b4d6090a4e7d8914d60c4","a458fcef3e1446b791736a9ae84065ee","48ca8c7e15514f998068f05c74780ba5"]},"id":"dc45abe4","executionInfo":{"status":"ok","timestamp":1681610354018,"user_tz":240,"elapsed":1424,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"1e338ad0-3e9b-4afa-f972-895d6b8e1f98"},"outputs":[{"output_type":"stream","name":"stderr","text":["WARNING:datasets.builder:Found cached dataset csv (/root/.cache/huggingface/datasets/csv/default-50b4a11504202c45/0.0.0/6954658bab30a358235fa864b05cf819af0e179325c740e4bc853bcc7ec513e1)\n"]},{"output_type":"display_data","data":{"text/plain":["  0%|          | 0/2 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"262a1ac528e24a50aa13290e63f14f2d"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["DatasetDict({\n","    train: Dataset({\n","        features: ['Unnamed: 0', 'stories', 'prompts'],\n","        num_rows: 50000\n","    })\n","    test: Dataset({\n","        features: ['Unnamed: 0', 'stories', 'prompts'],\n","        num_rows: 5000\n","    })\n","})"]},"metadata":{},"execution_count":80}],"source":["dataset = load_dataset(\"csv\",\n","                       data_files={\"train\":TRAIN_PATH, \"test\": VALID_PATH})\n","\n","#select a sample of dataset\n","dataset['train']= dataset['train'].select(range(TRAIN_ROWS))\n","dataset['test']=dataset['train'].select(range(TEST_ROWS))\n","\n","dataset"]},{"cell_type":"code","execution_count":10,"id":"f6c36130","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"f6c36130","executionInfo":{"status":"ok","timestamp":1681608473089,"user_tz":240,"elapsed":22,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"4d04b103-04d4-4510-963d-7c58434e87d8"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["DatasetDict({\n","    train: Dataset({\n","        features: ['stories', 'prompts'],\n","        num_rows: 50000\n","    })\n","    test: Dataset({\n","        features: ['stories', 'prompts'],\n","        num_rows: 5000\n","    })\n","})"]},"metadata":{},"execution_count":10}],"source":["# remove unnamed column\n","dataset = dataset.remove_columns(['Unnamed: 0'])\n","\n","dataset"]},{"cell_type":"markdown","id":"a5001c82","metadata":{"id":"a5001c82"},"source":["Let's see an example from the dataset"]},{"cell_type":"code","execution_count":11,"id":"7909f7f3","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7909f7f3","executionInfo":{"status":"ok","timestamp":1681608473090,"user_tz":240,"elapsed":12,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"457e1448-26ee-4535-fdb5-643ba92b0918"},"outputs":[{"output_type":"stream","name":"stdout","text":["STORIES: So many times have I walked on ruins , the remainings of places that I loved and got used to.. At first I was scared , each time I could feel my city , my current generation collapse , break into the black hole that thrives within it , I could feel humanity , the way I 'm able to feel my body.. After a few hundred years , the pattern became obvious , no longer the war and damage that would devastate me over and over again in the far past was effecting me so dominantly . <newline> It 's funny , b \n"," \n","PROMPTS:  You 've finally managed to discover the secret to immortality . Suddenly , Death appears before you , hands you a business card , and says , `` When you realize living forever sucks , call this number , I 've got a job offer for you . '' \n"," \n"]}],"source":["for key in dataset[\"train\"][0]:\n","    print(f\"{key.upper()}: {dataset['train'][0][key][:500]} \\n \")"]},{"cell_type":"markdown","id":"a1ad4c2f","metadata":{"id":"a1ad4c2f"},"source":["We have two columns in our dataset:\n","\n","1. stories\n","2. prompts\n","\n","Since we are training a casual language model to generate stories, we want to train our model on the `stories` column in the dataset. Given the starting of a story, our model should complete the rest of the story."]},{"cell_type":"markdown","id":"bbe05f4c","metadata":{"id":"bbe05f4c"},"source":["## Data Processing:\n","\n","The first step in the data processing phase is to tokenize the text. Since the input is going to small prompts, we can keep the context window small. This has the benefit that we can train the model much faster and it requires significantly less memory.  \n","\n","Many documents in the dataset will have a size greater than the context window set by us. In this case, we break the documents into chunks with size equal to the context window. Sometimes, we also get rid of the last chunk if it is too small."]},{"cell_type":"code","execution_count":12,"id":"3ad537d7","metadata":{"id":"3ad537d7","executionInfo":{"status":"ok","timestamp":1681608473090,"user_tz":240,"elapsed":5,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["from transformers import AutoTokenizer"]},{"cell_type":"code","execution_count":81,"id":"9741ee48","metadata":{"id":"9741ee48","executionInfo":{"status":"ok","timestamp":1681610364453,"user_tz":240,"elapsed":5838,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"colab":{"base_uri":"https://localhost:8080/","height":145,"referenced_widgets":["9d7a005553f94fd6b0605e357a4143f9","2823774d73224acea8963415f2fff76f","e22aa581ce574a118d7709aea7eb3b0f","50066ba6f04c4833bb442161751e6930","9962a57fbcb64fa3a49bd8d10f6a9550","595881a534c14e239bedd213378b3afb","b3fd81a8735a42e497cfeeea80e6123d","4abe774f17064dcfb67d147038756fa9","246e0bb7935940958cf7f1391d7aa89d","68b25b7a0e9b4518b786b60b10343112","67f103fb77f24854aea59ef7ce0ffe0d","c0ffced3fafa4f37bbe49551245b9dc9","e6f05b7d167f4486b2a2300f65f6fd0d","42bc3ffdd7a747d99669ba64a5bb23bb","81edff21f6a94a0892e1d2d372f813d4","76e42c9f914c4fb7acf2f6984434c4db","a90856e5d16c45e8bfa97c513fa6e27c","4400f5a1d7764e6e9a68a5d5a98894ee","44d33d53a035408d9cbf523edeae2faa","a9318ef2d9a243a2a713ec11f69d09eb","61aa343f4f334f37906d5f2f75f28124","bc9f97edc3d140bcb8b86d6297f4ee6f","6e3618fc4e8f4d668f02d692763cdf05","adc84f515b8043a9b0563e761199de2a","265bf1a539a643daa08fc9462111df8c","3a244693b85246c4b66d486b2f7f67ff","fa0d475578994db98dcb3d4079a91cf2","605022eadc644542ad41d67218fcfe65","9c228ed055a34d0e81a5c70fa2a1a457","694d775fc20b43bc85b35956abf84dc5","0c2f15c2df544e2091f7ada43222974f","4a9baf89ea974f5c8266876f8b1898a8","fd9016fe4ba6465aad5db936051d1b8e","3d22b167e7484ea19f8c0f7a0d0ceaad","432359b82b4947198db9cb501f8af536","82da4a43a28046179cc25a3793072f02","69ad993775dd4c8a955686a79850a01b","a81a8a9c226e4fbcbb345117c8ed9e0c","3e660b6091eb4c01af97f214417c0388","f247e37179224505a4c6a5bd016c1ac7","f24eccbe78904af4a1462ee8a43cb33a","60ccac1953a8428a8cdbcb5bad4e8d3d","f473b48eab9944738c3974a27b671fdf","2c80ff85923941ca8e5153f35d150e83"]},"outputId":"8e024b1a-d812-4759-8a6a-00da8c65af4e"},"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/762 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"9d7a005553f94fd6b0605e357a4143f9"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)olve/main/vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c0ffced3fafa4f37bbe49551245b9dc9"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)olve/main/merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6e3618fc4e8f4d668f02d692763cdf05"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3d22b167e7484ea19f8c0f7a0d0ceaad"}},"metadata":{}}],"source":["# loading a pretrained tokenizer for the selected model \n","\n","tokenizer = AutoTokenizer.from_pretrained(MODEL_CHECKPOINT)"]},{"cell_type":"markdown","id":"8a0e196f","metadata":{"id":"8a0e196f"},"source":["Let's see what the tokenizer does."]},{"cell_type":"code","execution_count":82,"id":"388902c7","metadata":{"id":"388902c7","executionInfo":{"status":"ok","timestamp":1681610364453,"user_tz":240,"elapsed":5,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["outputs = tokenizer(\n","    dataset[\"train\"][:2][\"stories\"],\n","    truncation=True,\n","    max_length=CONTEXT_LEN,\n","    return_overflowing_tokens=True,\n","    return_length=True,\n",")"]},{"cell_type":"code","execution_count":83,"id":"4182d301","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4182d301","executionInfo":{"status":"ok","timestamp":1681610364454,"user_tz":240,"elapsed":4,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"a3b26ec1-67d4-4ce4-870f-e64ae9da1415"},"outputs":[{"output_type":"stream","name":"stdout","text":["Input IDs length: 7\n","Input chunk lengths: [256, 256, 240, 256, 256, 256, 129]\n","Chunk mapping: [0, 0, 0, 1, 1, 1, 1]\n"]}],"source":["print(f\"Input IDs length: {len(outputs['input_ids'])}\")\n","print(f\"Input chunk lengths: {(outputs['length'])}\")\n","print(f\"Chunk mapping: {outputs['overflow_to_sample_mapping']}\")"]},{"cell_type":"markdown","id":"8416aea7","metadata":{"id":"8416aea7"},"source":["Now, let's create a tokenize function"]},{"cell_type":"code","execution_count":84,"id":"3792421d","metadata":{"id":"3792421d","executionInfo":{"status":"ok","timestamp":1681610364454,"user_tz":240,"elapsed":3,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["def tokenize(element):\n","    \n","    outputs = tokenizer(\n","        element[\"stories\"],\n","        truncation=True,\n","        max_length=CONTEXT_LEN,\n","        return_overflowing_tokens=True,\n","        return_length=True,\n","    )\n","    \n","    input_batch = []\n","    \n","    for length, input_ids in zip(outputs[\"length\"], outputs[\"input_ids\"]):\n","        if length == CONTEXT_LEN:\n","            input_batch.append(input_ids)\n","    return {\"input_ids\": input_batch}"]},{"cell_type":"code","execution_count":85,"id":"431be550","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":187,"referenced_widgets":["9087eab66ccb4072be571e86c3954a52","f163d1b03b2e45109e168c5cbb65f68c","94127139880d4ed29b0476e2fa498bc8","bf2f144cba73434793e3cd78a0546957","658aedb8b59f4a1984ac23eb4ee4776a","78c206636e7241d1ba19c3edcb6c1bb2","81c1e795a392497c91e1f7b8b708082a","9c17bb9158e7428cb9ac1fb4435e0204","50cad776220540fda270638433349d86","027aab807b4a41dea21b2b7f6b597a8b","90b31034567844d4974f382ce9bfff77","1db0901ec4b84299b854d4a2ff5d22bf","d6f8baceeb594331b4347c87525d3a6d","4931f99d1695457890abdec7f36af816","5887db7406b3458b8fab6481fed18b49","5f8f5a0d610344ef8a130948f4f29abc","b0ced110dd2e4c51af1a64856b62bea0","b00ee63a0e2a47718d0bd33dfd11568e","070afa9ba50e4f7daf8bedcd579166d3","a218fde2ce414bc2a26a9cec52270593","3cca48c3fe2f490ab746990159971cb7","367c979f535d4ebfa639726251b60945"]},"id":"431be550","executionInfo":{"status":"ok","timestamp":1681610397440,"user_tz":240,"elapsed":32451,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"ead8c64d-1cd0-4fbf-b510-2dfc49776287"},"outputs":[{"output_type":"display_data","data":{"text/plain":["Map:   0%|          | 0/50000 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"9087eab66ccb4072be571e86c3954a52"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Map:   0%|          | 0/5000 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1db0901ec4b84299b854d4a2ff5d22bf"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["DatasetDict({\n","    train: Dataset({\n","        features: ['input_ids'],\n","        num_rows: 114239\n","    })\n","    test: Dataset({\n","        features: ['input_ids'],\n","        num_rows: 11363\n","    })\n","})"]},"metadata":{},"execution_count":85}],"source":["tokenized_datasets = dataset.map(\n","    tokenize, batched=True, remove_columns=dataset[\"train\"].column_names\n",")\n","tokenized_datasets"]},{"cell_type":"markdown","id":"d0dad696","metadata":{"id":"d0dad696"},"source":["Ok!!! That was straightforward. We have completed the tokenization. Now, it is time to train the model. "]},{"cell_type":"code","execution_count":86,"id":"a510a5db","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":137},"id":"a510a5db","executionInfo":{"status":"ok","timestamp":1681610397441,"user_tz":240,"elapsed":13,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"855471cd-13aa-4c5d-e386-f2f1390e420c"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["\"So many times have I walked on ruins, the remainings of places that I loved and got used to.. At first I was scared, each time I could feel my city, my current generation collapse, break into the black hole that thrives within it, I could feel humanity, the way I'm able to feel my body.. After a few hundred years, the pattern became obvious, no longer the war and damage that would devastate me over and over again in the far past was effecting me so dominantly. <newline> It's funny, but I felt as if after gaining what I desired so long, what I have lived for my entire life, only then, when I achieved immortality I started truly aging. <newline> <newline> 5 world wars have passed, and now they feel like a simple sickeness that would pass by every so often, I could no longer evaluate the individual human as a being of its own, the importance of mortals is merely the same as the importance of my skin cells ; They are a part of a mechanism so much more advanced, a mechanism that is so dear to my fallen heart a mechanism that I have seen fall and rise so many times, a mechanism that when lost all of which it\""],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":86}],"source":["# we can get our tokenized text back using the decode function \n","tokenizer.decode(tokenized_datasets['train'][0]['input_ids'])"]},{"cell_type":"markdown","id":"d1c5b2b9","metadata":{"id":"d1c5b2b9"},"source":["## Training the model\n","\n","There are different transformer architectures that we can use:\n","\n","1. GPT-2\n","2. BART\n","3. BERT\n","\n","And maybe, a encoder-decoder architecture like `BART`, `T5`\n","\n","In this section, we are going to train a casual language model using the `BERT` transformer architecture. Later, we will focus on fine-tuning the hyperparameters for training and trying other architectures as well. "]},{"cell_type":"code","execution_count":87,"id":"eda32b8b","metadata":{"id":"eda32b8b","executionInfo":{"status":"ok","timestamp":1681610397442,"user_tz":240,"elapsed":10,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["from transformers import AutoModelForCausalLM, AutoConfig"]},{"cell_type":"markdown","id":"58200488","metadata":{"id":"58200488"},"source":["Let's initialize the model"]},{"cell_type":"code","execution_count":88,"id":"ae9550fa","metadata":{"id":"ae9550fa","executionInfo":{"status":"ok","timestamp":1681610397442,"user_tz":240,"elapsed":9,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["config = AutoConfig.from_pretrained(\n","    MODEL_CHECKPOINT,\n","    vocab_size=len(tokenizer),\n","    n_ctx=CONTEXT_LEN,\n","    bos_token_id=tokenizer.bos_token_id,\n","    eos_token_id=tokenizer.eos_token_id,\n",")"]},{"cell_type":"code","execution_count":92,"id":"d6c8ed9e","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"d6c8ed9e","executionInfo":{"status":"ok","timestamp":1681610463252,"user_tz":240,"elapsed":2256,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"fea42cfa-65a0-4784-d254-f436244101df"},"outputs":[{"output_type":"stream","name":"stdout","text":["Distilgpt2 size: 81.9M parameters\n"]}],"source":["# initialize the model\n","model = AutoModelForCausalLM.from_pretrained(MODEL_CHECKPOINT)\n","model_size = sum(t.numel() for t in model.parameters())\n","\n","# how many parameters?\n","print(f\"Distilgpt2 size: {model_size/1000**2:.1f}M parameters\")"]},{"cell_type":"markdown","id":"2b31fe20","metadata":{"id":"2b31fe20"},"source":["### Creating batches:\n","\n","HuggingFace provides us with the `DataCollatorForLanguageModeling` collator, which is designed specifically for language modeling. Besides stacking and padding batches, it also takes care of creating the language model labels — in causal language modeling the inputs serve as labels too (just shifted by one element), and this data collator creates them on the fly during training so we don’t need to duplicate the `input_ids`."]},{"cell_type":"code","execution_count":93,"id":"78b1b7f0","metadata":{"id":"78b1b7f0","executionInfo":{"status":"ok","timestamp":1681610466113,"user_tz":240,"elapsed":3,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["from transformers import DataCollatorForLanguageModeling"]},{"cell_type":"code","execution_count":94,"id":"9ec4b17f","metadata":{"id":"9ec4b17f","executionInfo":{"status":"ok","timestamp":1681610466113,"user_tz":240,"elapsed":2,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["tokenizer.pad_token = tokenizer.eos_token\n","data_collator = DataCollatorForLanguageModeling(tokenizer, mlm=False)"]},{"cell_type":"markdown","id":"a01b4ccf","metadata":{"id":"a01b4ccf"},"source":["Inspect the data collator with an example"]},{"cell_type":"code","execution_count":95,"id":"5121a81b","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"5121a81b","executionInfo":{"status":"ok","timestamp":1681610468650,"user_tz":240,"elapsed":6,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"5b44b71b-0443-4bed-839f-dd537546aa0d"},"outputs":[{"output_type":"stream","name":"stderr","text":["You're using a GPT2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"]},{"output_type":"stream","name":"stdout","text":["Collating data:\n","input_ids shape: torch.Size([5, 256])\n","attention_mask shape: torch.Size([5, 256])\n","labels shape: torch.Size([5, 256])\n"]}],"source":["out = data_collator([tokenized_datasets[\"train\"][i] for i in range(5)])\n","\n","print(\"Collating data:\")\n","for key in out:\n","    print(f\"{key} shape: {out[key].shape}\")"]},{"cell_type":"markdown","id":"00f1e0e3","metadata":{"id":"00f1e0e3"},"source":["### Set training parameters:"]},{"cell_type":"code","execution_count":96,"id":"68b5028d","metadata":{"id":"68b5028d","executionInfo":{"status":"ok","timestamp":1681610470341,"user_tz":240,"elapsed":11,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["from transformers import Trainer, TrainingArguments\n","import math "]},{"cell_type":"markdown","id":"d38eaee4","metadata":{"id":"d38eaee4"},"source":["Let's set the hyperparameters for training."]},{"cell_type":"code","execution_count":97,"id":"5cdd946f","metadata":{"id":"5cdd946f","executionInfo":{"status":"ok","timestamp":1681610473570,"user_tz":240,"elapsed":2,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["args = TrainingArguments(\n","    output_dir=\"ai-story_gen\",\n","    per_device_train_batch_size=TRAIN_BS,\n","    per_device_eval_batch_size=TEST_BS,\n","    evaluation_strategy=\"steps\",\n","    eval_steps=1_00,\n","    logging_steps=1_00,\n","    gradient_accumulation_steps=8,\n","    num_train_epochs=EPOCHS,\n","    weight_decay=0.1,\n","    warmup_steps=1_00,\n","    lr_scheduler_type=\"cosine\",\n","    learning_rate=5e-3,\n","    save_steps=5_00,\n","    fp16=True,\n","    push_to_hub=False,\n",")"]},{"cell_type":"markdown","id":"4496a997","metadata":{"id":"4496a997"},"source":["Initialize the `Trainer` API instance. "]},{"cell_type":"code","source":["def compute_metrics(pred):\n","    # compute perplexity from the model's output\n","    perplexity = math.exp(-pred['eval_loss'])\n","    return {\"perplexity\": perplexity}"],"metadata":{"id":"5rfxaKl2cFte","executionInfo":{"status":"ok","timestamp":1681610477565,"user_tz":240,"elapsed":592,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"id":"5rfxaKl2cFte","execution_count":98,"outputs":[]},{"cell_type":"code","execution_count":99,"id":"c2cf2749","metadata":{"id":"c2cf2749","executionInfo":{"status":"ok","timestamp":1681610483882,"user_tz":240,"elapsed":7,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["trainer = Trainer(\n","    model=model,\n","    tokenizer=tokenizer,\n","    args=args,\n","    data_collator=data_collator,\n","    train_dataset=tokenized_datasets[\"train\"],\n","    eval_dataset=tokenized_datasets[\"test\"],\n","    # compute_metrics=compute_metrics\n",")"]},{"cell_type":"code","execution_count":100,"id":"0c430e3a","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":504},"id":"0c430e3a","outputId":"00614b10-da8e-443e-970c-f526d1a8c3a0","executionInfo":{"status":"ok","timestamp":1681612449933,"user_tz":240,"elapsed":1961694,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[{"output_type":"stream","name":"stderr","text":["/usr/local/lib/python3.9/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n","  warnings.warn(\n"]},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","    <div>\n","      \n","      <progress value='1115' max='1115' style='width:300px; height:20px; vertical-align: middle;'></progress>\n","      [1115/1115 32:39, Epoch 4/5]\n","    </div>\n","    <table border=\"1\" class=\"dataframe\">\n","  <thead>\n"," <tr style=\"text-align: left;\">\n","      <th>Step</th>\n","      <th>Training Loss</th>\n","      <th>Validation Loss</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <td>100</td>\n","      <td>3.400200</td>\n","      <td>3.222552</td>\n","    </tr>\n","    <tr>\n","      <td>200</td>\n","      <td>3.322300</td>\n","      <td>3.091470</td>\n","    </tr>\n","    <tr>\n","      <td>300</td>\n","      <td>3.193900</td>\n","      <td>3.019011</td>\n","    </tr>\n","    <tr>\n","      <td>400</td>\n","      <td>3.176000</td>\n","      <td>2.952123</td>\n","    </tr>\n","    <tr>\n","      <td>500</td>\n","      <td>3.076600</td>\n","      <td>2.887738</td>\n","    </tr>\n","    <tr>\n","      <td>600</td>\n","      <td>3.035200</td>\n","      <td>2.818650</td>\n","    </tr>\n","    <tr>\n","      <td>700</td>\n","      <td>2.976700</td>\n","      <td>2.744796</td>\n","    </tr>\n","    <tr>\n","      <td>800</td>\n","      <td>2.876700</td>\n","      <td>2.687469</td>\n","    </tr>\n","    <tr>\n","      <td>900</td>\n","      <td>2.859700</td>\n","      <td>2.633229</td>\n","    </tr>\n","    <tr>\n","      <td>1000</td>\n","      <td>2.739500</td>\n","      <td>2.606083</td>\n","    </tr>\n","    <tr>\n","      <td>1100</td>\n","      <td>2.738600</td>\n","      <td>2.598103</td>\n","    </tr>\n","  </tbody>\n","</table><p>"]},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["TrainOutput(global_step=1115, training_loss=3.0318873486711304, metrics={'train_runtime': 1961.3507, 'train_samples_per_second': 291.225, 'train_steps_per_second': 0.568, 'total_flos': 3.729201094773965e+16, 'train_loss': 3.0318873486711304, 'epoch': 5.0})"]},"metadata":{},"execution_count":100}],"source":["trainer.train()"]},{"cell_type":"code","execution_count":101,"id":"a38bf026","metadata":{"id":"a38bf026","executionInfo":{"status":"ok","timestamp":1681612450964,"user_tz":240,"elapsed":1038,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["trainer.save_model(\"./story_distilgpt2_finetune\")"]},{"cell_type":"markdown","id":"63d914bb","metadata":{"id":"63d914bb"},"source":["## Story generator pipeline:"]},{"cell_type":"code","execution_count":102,"id":"a33363da","metadata":{"id":"a33363da","executionInfo":{"status":"ok","timestamp":1681612450965,"user_tz":240,"elapsed":11,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["import torch\n","from transformers import pipeline"]},{"cell_type":"code","execution_count":103,"id":"58d73ec9","metadata":{"id":"58d73ec9","executionInfo":{"status":"ok","timestamp":1681612450966,"user_tz":240,"elapsed":9,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"outputs":[],"source":["device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n","pipe = pipeline(\n","    \"text-generation\", model=model,tokenizer=tokenizer, device=device\n",")"]},{"cell_type":"markdown","id":"28c88777","metadata":{"id":"28c88777"},"source":["Let's see an example of the text generated by our trained model"]},{"cell_type":"code","execution_count":104,"id":"d3c27b1f","metadata":{"id":"d3c27b1f","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612451737,"user_tz":240,"elapsed":779,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"7de2abc9-1416-47af-d59c-3b7c5fe2475f"},"outputs":[{"output_type":"stream","name":"stderr","text":["/usr/local/lib/python3.9/dist-packages/transformers/generation/utils.py:1219: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)\n","  warnings.warn(\n"]},{"output_type":"stream","name":"stdout","text":["Once upon a time, there was a little girl. <newline> <newline> She was the most beautiful thing in the world. <newline> <newline> She was a beautiful girl. <newline> <newline> She had a lot of freckles on her face. <newline> <newline> She had a lot of freckles. <newline> <newline> She had a lot of freckles. <newline> <newline\n"]}],"source":["txt = \"Once upon a time\"\n","print(pipe(txt, num_return_sequences=1,num_beams=3,\n","           do_sample=True, max_new_tokens=100,\n","          pad_token_id=tokenizer.eos_token_id)[0][\"generated_text\"])"]},{"cell_type":"markdown","id":"7b416c1e","metadata":{"id":"7b416c1e"},"source":["## Finally let's evaluate our language model:\n","\n","The evaluation of a text generation task can be subjective and depend on the specific goals of the task. However, there are several common methods that can be used to evaluate the quality of the generated text.\n","\n","1. __Human evaluation:__ One of the most reliable methods for evaluating text generation is through human evaluation. This involves having human evaluators read the generated text and rate its quality based on various metrics such as coherence, fluency, and relevance to the prompt or topic. This method provides a subjective evaluation of the text and can give insights into how well the generated text matches human writing.\n","\n","2. __Automated metrics:__ Automated metrics can be used to evaluate text generation based on various quantitative measures. Examples of automated metrics include perplexity, BLEU score, and ROUGE score. These metrics compare the generated text to a reference text or corpus, and provide a measure of the similarity between the two texts.\n","\n","3. __Domain-specific metrics:__ For text generation tasks that are focused on a specific domain, such as medical or legal writing, domain-specific metrics can be used to evaluate the quality of the generated text. These metrics are designed to measure the accuracy and completeness of the generated text within the specific domain.\n","\n","4. __User testing:__ In some cases, user testing can be used to evaluate the quality of the generated text. This involves having users interact with the generated text and providing feedback on its quality, usefulness, and relevance to the task or topic.\n","\n","\n","In this notebook, we focus on evaluating the model based on automated metrics. We will check the model's performance based on three metrics:\n","- Perplexity: Here's how we can calculate perplexity\n","- BLEU score\n","- BERT score"]},{"cell_type":"code","source":["import torch\n","from tqdm import tqdm\n","\n","from datasets import load_metric\n","from evaluate import load"],"metadata":{"id":"PsIraYbuZ39r","executionInfo":{"status":"ok","timestamp":1681612451738,"user_tz":240,"elapsed":7,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"id":"PsIraYbuZ39r","execution_count":105,"outputs":[]},{"cell_type":"markdown","id":"c011e669","metadata":{"id":"c011e669"},"source":["__Perplexity__"]},{"cell_type":"code","source":["encodings = tokenizer(\"\\n\\n\".join(dataset['test']['stories']),return_tensors=\"pt\")"],"metadata":{"id":"NZEcwyqCDghv","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612464989,"user_tz":240,"elapsed":13256,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"cc6041f1-75f2-4f63-960f-df8f8f23c302"},"id":"NZEcwyqCDghv","execution_count":106,"outputs":[{"output_type":"stream","name":"stderr","text":["Token indices sequence length is longer than the specified maximum sequence length for this model (3576967 > 1024). Running this sequence through the model will result in indexing errors\n"]}]},{"cell_type":"code","source":["encodings['input_ids'].shape"],"metadata":{"id":"29VhBco_Z6z_","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612464990,"user_tz":240,"elapsed":21,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"08eb93e1-91c8-4381-be01-32fa87b79410"},"id":"29VhBco_Z6z_","execution_count":107,"outputs":[{"output_type":"execute_result","data":{"text/plain":["torch.Size([1, 3576967])"]},"metadata":{},"execution_count":107}]},{"cell_type":"code","source":["max_length = model.config.n_positions\n","stride = 512\n","seq_len = encodings.input_ids.size(1)\n","\n","nlls = []\n","prev_end_loc = 0\n","\n","for begin_loc in tqdm(range(0, seq_len, stride)):\n","    end_loc = min(begin_loc + max_length, seq_len)\n","    trg_len = end_loc - prev_end_loc  # may be different from stride on last loop\n","    input_ids = encodings.input_ids[:, begin_loc:end_loc].to(device)\n","    target_ids = input_ids.clone()\n","    target_ids[:, :-trg_len] = -100\n","\n","    with torch.no_grad():\n","        outputs = model(input_ids, labels=target_ids)\n","\n","        # loss is calculated using CrossEntropyLoss which averages over valid labels\n","        # N.B. the model only calculates loss over trg_len - 1 labels, because it internally shifts the labels\n","        # to the left by 1.\n","        neg_log_likelihood = outputs.loss\n","\n","    nlls.append(neg_log_likelihood)\n","\n","    prev_end_loc = end_loc\n","    if end_loc == seq_len:\n","        break\n","\n","ppl = torch.exp(torch.stack(nlls).mean())"],"metadata":{"id":"vEb01TXZZ8oI","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612575799,"user_tz":240,"elapsed":110825,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"de3b970b-f2e4-4f4c-b70f-fb2e6a4282ae"},"id":"vEb01TXZZ8oI","execution_count":108,"outputs":[{"output_type":"stream","name":"stderr","text":["100%|█████████▉| 6985/6987 [01:50<00:00, 63.43it/s]\n"]}]},{"cell_type":"code","source":["ppl"],"metadata":{"id":"0-Fg9PzpZ_gy","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612575800,"user_tz":240,"elapsed":21,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"2f0efcbf-089d-4a1d-987f-96e6d4f91201"},"id":"0-Fg9PzpZ_gy","execution_count":109,"outputs":[{"output_type":"execute_result","data":{"text/plain":["tensor(125.9746, device='cuda:0')"]},"metadata":{},"execution_count":109}]},{"cell_type":"markdown","source":["__BLEU score__"],"metadata":{"id":"YQTR4Dc3Dg-0"},"id":"YQTR4Dc3Dg-0"},{"cell_type":"code","source":["bleu_metric = load_metric('bleu')"],"metadata":{"id":"8XarWdGiDhO1","colab":{"base_uri":"https://localhost:8080/","height":135,"referenced_widgets":["3912e531c23e4d26ba814c9265a6bc23","b15696348f82461397bfae6fc0d4a7b5","96ebbdd16d0f48928cba02cc7de8d648","6b7fb890fce3447e92852ae7e0af3a2d","85a378ad363941d7ba45d89ed481e331","468d48a4e7704e7787f17a3ba3903ad1","a91a3a2b7887450f87701b508cac6d4d","2cffa85e9ce04944b583b5fc07eed37f","804f834ef79745ca9c592406aa428660","b529034a85e048bebdec20f1efcc496e","c9846363c2144a35851c47a9bf3986f8","d4f1ed128fa04e44b0baaee7f4dd2e68","c1d8ea4bae584158b69f5f0156985934","f44417f4c956462d8df5deb127e610c1","b902c7bafe6c4105820ab30ae9afc902","e89b6b5cdd8945138cac5806e96a432a","7b1fef83458c456c960e46dc9ae9891c","f1cba44ed55e4a4eb376a5f85c01a9dc","a88dc89e0d924f27a7a3f3684234bee4","ded7bee83ad24b168d09c8e4577fdab6","95c4a7e2041d4380b89cf9a791dfdab8","de42d4914e194932ae203e96b3879f50"]},"executionInfo":{"status":"ok","timestamp":1681612577122,"user_tz":240,"elapsed":1334,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"9100f11b-184e-4b46-b481-a9d25b339125"},"id":"8XarWdGiDhO1","execution_count":110,"outputs":[{"output_type":"stream","name":"stderr","text":["<ipython-input-110-37a5b62391a6>:1: FutureWarning: load_metric is deprecated and will be removed in the next major version of datasets. Use 'evaluate.load' instead, from the new library 🤗 Evaluate: https://huggingface.co/docs/evaluate\n","  bleu_metric = load_metric('bleu')\n"]},{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/2.48k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3912e531c23e4d26ba814c9265a6bc23"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading extra modules:   0%|          | 0.00/1.55k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d4f1ed128fa04e44b0baaee7f4dd2e68"}},"metadata":{}}]},{"cell_type":"code","source":["def data(n=10):\n","    for i in range(n):\n","        \n","        # pass first 10 words\n","        yield \" \".join(dataset['test'][i]['stories'].split(\" \")[:10]) "],"metadata":{"id":"T4oW9pnw63mZ","executionInfo":{"status":"ok","timestamp":1681612577122,"user_tz":240,"elapsed":3,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"id":"T4oW9pnw63mZ","execution_count":111,"outputs":[]},{"cell_type":"code","source":["predictions=[]\n","references=[]\n","\n","i=0\n","for out in pipe(data(),num_return_sequences=1,num_beams=3,\n","           do_sample=True, max_new_tokens=100,\n","          pad_token_id=tokenizer.eos_token_id):\n","\n","    references.append(\" \".join(dataset['test'][i]['stories'].split(\" \")[10:100]))\n","    predictions.append(\" \".join(out[0]['generated_text'].split(\" \")[10:100]))\n","    \n","    i+=1\n"],"metadata":{"id":"3jyeCwXQ64N9","executionInfo":{"status":"ok","timestamp":1681612585874,"user_tz":240,"elapsed":8754,"user":{"displayName":"mango jain","userId":"08538486604454803686"}}},"id":"3jyeCwXQ64N9","execution_count":112,"outputs":[]},{"cell_type":"code","source":["preds=[i.split(\" \") for i in predictions]\n","cands=[[i.split(\" \")] for i in references]\n","\n","bleu_scores = bleu_metric.compute(predictions=preds,\n","                                  references=cands)\n","print(bleu_scores)"],"metadata":{"id":"a2MAcvD967cN","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612585874,"user_tz":240,"elapsed":6,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"786f0f13-7037-41f3-8c47-167a7085e8c7"},"id":"a2MAcvD967cN","execution_count":113,"outputs":[{"output_type":"stream","name":"stdout","text":["{'bleu': 0.0, 'precisions': [0.2108843537414966, 0.02768166089965398, 0.0035211267605633804, 0.0], 'brevity_penalty': 0.5882447095837121, 'length_ratio': 0.6533333333333333, 'translation_length': 588, 'reference_length': 900}\n"]}]},{"cell_type":"markdown","source":["__BERT score__"],"metadata":{"id":"aeo-q40tDhf2"},"id":"aeo-q40tDhf2"},{"cell_type":"code","source":["bertscore = load(\"bertscore\")\n","results = bertscore.compute(predictions=predictions, references=references, lang=\"en\")"],"metadata":{"id":"efjuPUxgDhqD","colab":{"base_uri":"https://localhost:8080/","height":177,"referenced_widgets":["1302f4fe69b547a4943b7ee84b8c214e","f610104b6e3c4b1bab99589a2ec3a7e8","6ee4789894664b0fa173a19f9014a6cf","23e48c40acc54ed5a0633328c053cdf3","261b92c265d545959e52e0dd3161ceb7","dce1a34a62f54321ab718c0ee4cb0fad","5d7b7724adf94edeb7e2d439e77a4722","e792c569f1ac4900b8090a3ae922c14c","275c9ce2c3694a1dbaa729ac1b43cf62","6eba23a5957648efb54861826029fd84","19eb9700bc44419991189806f5419030","2fd41b9a33534f4bb65feae6a9d073fa","3495026dae6b4d6bb01434aee6083157","83d446588f604229a68757b429b996c2","500347736d2f48ce885e00b45b17c971","95cb6c0fb8b84f76b276736076bd9686","2e025b3a0ef04f3c9fa9a56189240c28","a8502ee80b9c4587a5243075f0e7facd","fcf72f32773b4decacd270689fc0d8c7","8840c7b2fd9949e2b25df89fdee9a878","5b6fdfb32c5340dbb55ddce9f0cebf50","2504d819e86e4847a5684423bb1220e2","222890d0de47416fb2a4437abef3b5c6","f6f254ecfd1b401eb8796114808df5d0","832bbc5676594cac9388fc5026d12eff","729b79e9cba546ceb89a2c083a592f9e","aa8626c3d29142a19065c1d02492fa54","086ae2b064b143a9bd907c475ce4f285","7dd2c889103049558d3485b19abc8f86","3d93c6a0dbb5463bb2bc9f19db9a3df4","1d81cae6e20940e6ad1c021e279e37c3","f853caea75f6442f8ffa08c4fa2ba093","97b7b0af5cb944fc88dd272130c3e8b3","a577c412ae044b4c914458ef472388de","b7a05ca680794df5b1a2182117642e8a","9868ee56a64b4639b82ebe5a52e577aa","5c7a65551d8046c3aeae7dab5e250a04","8614ded9212940b7ba3ae5ff7e907296","621566cd751d4572a0ba8857748994ac","0900c9f95c264fefb39d3371df3e6a33","7500766e5f8748b7ba45a8c84bf29830","1e2a188edbad42688268eea41eebd8c9","0d14b7f3c49543f49d3e0b88fb3fba1e","5697e83dfb364f9daf91baae943afe0f","106db60c93614397817a727006ead150","acd086b127b94740988e7eb7841e178e","88244efc70aa46abb5c94fe95f1933f9","13a6cb2f30174d0cb03e4be3d622d011","512c20bc029044ac93ac72d6b7033712","3c73b5fa75b348e684788db984198653","e238513935de4ecbaeaaa91903487bdb","7504689ea8104dbaa2f0e19ef72d73a2","16a2453580eb40b6ba544ec7895268f5","a22abb39a71e479b8adc3448eb754045","18713e0549cc4072832d3375fbb13bf0"]},"executionInfo":{"status":"ok","timestamp":1681612612392,"user_tz":240,"elapsed":26521,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"1e8d00f0-c562-4dc6-be9f-599e7b94565c"},"id":"efjuPUxgDhqD","execution_count":114,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/7.95k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1302f4fe69b547a4943b7ee84b8c214e"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/482 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2fd41b9a33534f4bb65feae6a9d073fa"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)olve/main/vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"222890d0de47416fb2a4437abef3b5c6"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)olve/main/merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a577c412ae044b4c914458ef472388de"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading pytorch_model.bin:   0%|          | 0.00/1.43G [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"106db60c93614397817a727006ead150"}},"metadata":{}}]},{"cell_type":"code","source":["results['f1']"],"metadata":{"id":"N2hM43aU7Edj","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1681612612393,"user_tz":240,"elapsed":30,"user":{"displayName":"mango jain","userId":"08538486604454803686"}},"outputId":"490064c1-5c40-4d86-9564-67c5efc896ff"},"id":"N2hM43aU7Edj","execution_count":115,"outputs":[{"output_type":"execute_result","data":{"text/plain":["[0.7954513430595398,\n"," 0.7691985368728638,\n"," 0.8038576245307922,\n"," 0.788618266582489,\n"," 0.8339035511016846,\n"," 0.7960880994796753,\n"," 0.8031459450721741,\n"," 0.8546534180641174,\n"," 0.8405726552009583,\n"," 0.8452156782150269]"]},"metadata":{},"execution_count":115}]}],"metadata":{"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"},"colab":{"provenance":[],"machine_shape":"hm"},"accelerator":"GPU","gpuClass":"premium","widgets":{"application/vnd.jupyter.widget-state+json":{"262a1ac528e24a50aa13290e63f14f2d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_32381736e8344b199a14d2517368f5d4","IPY_MODEL_a3ea813b71404513afef43fa9848f339","IPY_MODEL_80cf4aaaa6934d1ca0f2a792f4fcdd2f"],"layout":"IPY_MODEL_cd8dd9c9590a4b05969e07961c706b08"}},"32381736e8344b199a14d2517368f5d4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e88822279b7e463ebe2dbb6d49ec1a91","placeholder":"​","style":"IPY_MODEL_87dc74f744e247819654112d4830b597","value":"100%"}},"a3ea813b71404513afef43fa9848f339":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_343ad42a91d44bd49f4e16b65543fe4b","max":2,"min":0,"orientation":"horizontal","style":"IPY_MODEL_c43145c74b9b4d6090a4e7d8914d60c4","value":2}},"80cf4aaaa6934d1ca0f2a792f4fcdd2f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a458fcef3e1446b791736a9ae84065ee","placeholder":"​","style":"IPY_MODEL_48ca8c7e15514f998068f05c74780ba5","value":" 2/2 [00:00&lt;00:00, 83.30it/s]"}},"cd8dd9c9590a4b05969e07961c706b08":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e88822279b7e463ebe2dbb6d49ec1a91":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"87dc74f744e247819654112d4830b597":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"343ad42a91d44bd49f4e16b65543fe4b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c43145c74b9b4d6090a4e7d8914d60c4":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a458fcef3e1446b791736a9ae84065ee":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"48ca8c7e15514f998068f05c74780ba5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9d7a005553f94fd6b0605e357a4143f9":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_2823774d73224acea8963415f2fff76f","IPY_MODEL_e22aa581ce574a118d7709aea7eb3b0f","IPY_MODEL_50066ba6f04c4833bb442161751e6930"],"layout":"IPY_MODEL_9962a57fbcb64fa3a49bd8d10f6a9550"}},"2823774d73224acea8963415f2fff76f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_595881a534c14e239bedd213378b3afb","placeholder":"​","style":"IPY_MODEL_b3fd81a8735a42e497cfeeea80e6123d","value":"Downloading (…)lve/main/config.json: 100%"}},"e22aa581ce574a118d7709aea7eb3b0f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4abe774f17064dcfb67d147038756fa9","max":762,"min":0,"orientation":"horizontal","style":"IPY_MODEL_246e0bb7935940958cf7f1391d7aa89d","value":762}},"50066ba6f04c4833bb442161751e6930":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_68b25b7a0e9b4518b786b60b10343112","placeholder":"​","style":"IPY_MODEL_67f103fb77f24854aea59ef7ce0ffe0d","value":" 762/762 [00:00&lt;00:00, 55.8kB/s]"}},"9962a57fbcb64fa3a49bd8d10f6a9550":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"595881a534c14e239bedd213378b3afb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b3fd81a8735a42e497cfeeea80e6123d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4abe774f17064dcfb67d147038756fa9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"246e0bb7935940958cf7f1391d7aa89d":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"68b25b7a0e9b4518b786b60b10343112":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"67f103fb77f24854aea59ef7ce0ffe0d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c0ffced3fafa4f37bbe49551245b9dc9":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_e6f05b7d167f4486b2a2300f65f6fd0d","IPY_MODEL_42bc3ffdd7a747d99669ba64a5bb23bb","IPY_MODEL_81edff21f6a94a0892e1d2d372f813d4"],"layout":"IPY_MODEL_76e42c9f914c4fb7acf2f6984434c4db"}},"e6f05b7d167f4486b2a2300f65f6fd0d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a90856e5d16c45e8bfa97c513fa6e27c","placeholder":"​","style":"IPY_MODEL_4400f5a1d7764e6e9a68a5d5a98894ee","value":"Downloading (…)olve/main/vocab.json: 100%"}},"42bc3ffdd7a747d99669ba64a5bb23bb":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_44d33d53a035408d9cbf523edeae2faa","max":1042301,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a9318ef2d9a243a2a713ec11f69d09eb","value":1042301}},"81edff21f6a94a0892e1d2d372f813d4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_61aa343f4f334f37906d5f2f75f28124","placeholder":"​","style":"IPY_MODEL_bc9f97edc3d140bcb8b86d6297f4ee6f","value":" 1.04M/1.04M [00:00&lt;00:00, 1.23MB/s]"}},"76e42c9f914c4fb7acf2f6984434c4db":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a90856e5d16c45e8bfa97c513fa6e27c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4400f5a1d7764e6e9a68a5d5a98894ee":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"44d33d53a035408d9cbf523edeae2faa":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a9318ef2d9a243a2a713ec11f69d09eb":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"61aa343f4f334f37906d5f2f75f28124":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bc9f97edc3d140bcb8b86d6297f4ee6f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6e3618fc4e8f4d668f02d692763cdf05":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_adc84f515b8043a9b0563e761199de2a","IPY_MODEL_265bf1a539a643daa08fc9462111df8c","IPY_MODEL_3a244693b85246c4b66d486b2f7f67ff"],"layout":"IPY_MODEL_fa0d475578994db98dcb3d4079a91cf2"}},"adc84f515b8043a9b0563e761199de2a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_605022eadc644542ad41d67218fcfe65","placeholder":"​","style":"IPY_MODEL_9c228ed055a34d0e81a5c70fa2a1a457","value":"Downloading (…)olve/main/merges.txt: 100%"}},"265bf1a539a643daa08fc9462111df8c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_694d775fc20b43bc85b35956abf84dc5","max":456318,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0c2f15c2df544e2091f7ada43222974f","value":456318}},"3a244693b85246c4b66d486b2f7f67ff":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4a9baf89ea974f5c8266876f8b1898a8","placeholder":"​","style":"IPY_MODEL_fd9016fe4ba6465aad5db936051d1b8e","value":" 456k/456k [00:00&lt;00:00, 2.09MB/s]"}},"fa0d475578994db98dcb3d4079a91cf2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"605022eadc644542ad41d67218fcfe65":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9c228ed055a34d0e81a5c70fa2a1a457":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"694d775fc20b43bc85b35956abf84dc5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0c2f15c2df544e2091f7ada43222974f":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"4a9baf89ea974f5c8266876f8b1898a8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fd9016fe4ba6465aad5db936051d1b8e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3d22b167e7484ea19f8c0f7a0d0ceaad":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_432359b82b4947198db9cb501f8af536","IPY_MODEL_82da4a43a28046179cc25a3793072f02","IPY_MODEL_69ad993775dd4c8a955686a79850a01b"],"layout":"IPY_MODEL_a81a8a9c226e4fbcbb345117c8ed9e0c"}},"432359b82b4947198db9cb501f8af536":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3e660b6091eb4c01af97f214417c0388","placeholder":"​","style":"IPY_MODEL_f247e37179224505a4c6a5bd016c1ac7","value":"Downloading (…)/main/tokenizer.json: 100%"}},"82da4a43a28046179cc25a3793072f02":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f24eccbe78904af4a1462ee8a43cb33a","max":1355256,"min":0,"orientation":"horizontal","style":"IPY_MODEL_60ccac1953a8428a8cdbcb5bad4e8d3d","value":1355256}},"69ad993775dd4c8a955686a79850a01b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f473b48eab9944738c3974a27b671fdf","placeholder":"​","style":"IPY_MODEL_2c80ff85923941ca8e5153f35d150e83","value":" 1.36M/1.36M [00:00&lt;00:00, 2.14MB/s]"}},"a81a8a9c226e4fbcbb345117c8ed9e0c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3e660b6091eb4c01af97f214417c0388":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f247e37179224505a4c6a5bd016c1ac7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f24eccbe78904af4a1462ee8a43cb33a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"60ccac1953a8428a8cdbcb5bad4e8d3d":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f473b48eab9944738c3974a27b671fdf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2c80ff85923941ca8e5153f35d150e83":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9087eab66ccb4072be571e86c3954a52":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f163d1b03b2e45109e168c5cbb65f68c","IPY_MODEL_94127139880d4ed29b0476e2fa498bc8","IPY_MODEL_bf2f144cba73434793e3cd78a0546957"],"layout":"IPY_MODEL_658aedb8b59f4a1984ac23eb4ee4776a"}},"f163d1b03b2e45109e168c5cbb65f68c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_78c206636e7241d1ba19c3edcb6c1bb2","placeholder":"​","style":"IPY_MODEL_81c1e795a392497c91e1f7b8b708082a","value":"Map: 100%"}},"94127139880d4ed29b0476e2fa498bc8":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_9c17bb9158e7428cb9ac1fb4435e0204","max":50000,"min":0,"orientation":"horizontal","style":"IPY_MODEL_50cad776220540fda270638433349d86","value":50000}},"bf2f144cba73434793e3cd78a0546957":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_027aab807b4a41dea21b2b7f6b597a8b","placeholder":"​","style":"IPY_MODEL_90b31034567844d4974f382ce9bfff77","value":" 50000/50000 [00:28&lt;00:00, 1634.43 examples/s]"}},"658aedb8b59f4a1984ac23eb4ee4776a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"78c206636e7241d1ba19c3edcb6c1bb2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"81c1e795a392497c91e1f7b8b708082a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9c17bb9158e7428cb9ac1fb4435e0204":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"50cad776220540fda270638433349d86":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"027aab807b4a41dea21b2b7f6b597a8b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"90b31034567844d4974f382ce9bfff77":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1db0901ec4b84299b854d4a2ff5d22bf":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d6f8baceeb594331b4347c87525d3a6d","IPY_MODEL_4931f99d1695457890abdec7f36af816","IPY_MODEL_5887db7406b3458b8fab6481fed18b49"],"layout":"IPY_MODEL_5f8f5a0d610344ef8a130948f4f29abc"}},"d6f8baceeb594331b4347c87525d3a6d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b0ced110dd2e4c51af1a64856b62bea0","placeholder":"​","style":"IPY_MODEL_b00ee63a0e2a47718d0bd33dfd11568e","value":"Map: 100%"}},"4931f99d1695457890abdec7f36af816":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_070afa9ba50e4f7daf8bedcd579166d3","max":5000,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a218fde2ce414bc2a26a9cec52270593","value":5000}},"5887db7406b3458b8fab6481fed18b49":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3cca48c3fe2f490ab746990159971cb7","placeholder":"​","style":"IPY_MODEL_367c979f535d4ebfa639726251b60945","value":" 5000/5000 [00:02&lt;00:00, 1715.90 examples/s]"}},"5f8f5a0d610344ef8a130948f4f29abc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"b0ced110dd2e4c51af1a64856b62bea0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b00ee63a0e2a47718d0bd33dfd11568e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"070afa9ba50e4f7daf8bedcd579166d3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a218fde2ce414bc2a26a9cec52270593":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3cca48c3fe2f490ab746990159971cb7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"367c979f535d4ebfa639726251b60945":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3912e531c23e4d26ba814c9265a6bc23":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b15696348f82461397bfae6fc0d4a7b5","IPY_MODEL_96ebbdd16d0f48928cba02cc7de8d648","IPY_MODEL_6b7fb890fce3447e92852ae7e0af3a2d"],"layout":"IPY_MODEL_85a378ad363941d7ba45d89ed481e331"}},"b15696348f82461397bfae6fc0d4a7b5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_468d48a4e7704e7787f17a3ba3903ad1","placeholder":"​","style":"IPY_MODEL_a91a3a2b7887450f87701b508cac6d4d","value":"Downloading builder script: "}},"96ebbdd16d0f48928cba02cc7de8d648":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_2cffa85e9ce04944b583b5fc07eed37f","max":2478,"min":0,"orientation":"horizontal","style":"IPY_MODEL_804f834ef79745ca9c592406aa428660","value":2478}},"6b7fb890fce3447e92852ae7e0af3a2d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b529034a85e048bebdec20f1efcc496e","placeholder":"​","style":"IPY_MODEL_c9846363c2144a35851c47a9bf3986f8","value":" 6.06k/? [00:00&lt;00:00, 413kB/s]"}},"85a378ad363941d7ba45d89ed481e331":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"468d48a4e7704e7787f17a3ba3903ad1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a91a3a2b7887450f87701b508cac6d4d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2cffa85e9ce04944b583b5fc07eed37f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"804f834ef79745ca9c592406aa428660":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b529034a85e048bebdec20f1efcc496e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c9846363c2144a35851c47a9bf3986f8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d4f1ed128fa04e44b0baaee7f4dd2e68":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c1d8ea4bae584158b69f5f0156985934","IPY_MODEL_f44417f4c956462d8df5deb127e610c1","IPY_MODEL_b902c7bafe6c4105820ab30ae9afc902"],"layout":"IPY_MODEL_e89b6b5cdd8945138cac5806e96a432a"}},"c1d8ea4bae584158b69f5f0156985934":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7b1fef83458c456c960e46dc9ae9891c","placeholder":"​","style":"IPY_MODEL_f1cba44ed55e4a4eb376a5f85c01a9dc","value":"Downloading extra modules: "}},"f44417f4c956462d8df5deb127e610c1":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a88dc89e0d924f27a7a3f3684234bee4","max":1554,"min":0,"orientation":"horizontal","style":"IPY_MODEL_ded7bee83ad24b168d09c8e4577fdab6","value":1554}},"b902c7bafe6c4105820ab30ae9afc902":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_95c4a7e2041d4380b89cf9a791dfdab8","placeholder":"​","style":"IPY_MODEL_de42d4914e194932ae203e96b3879f50","value":" 4.07k/? [00:00&lt;00:00, 264kB/s]"}},"e89b6b5cdd8945138cac5806e96a432a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7b1fef83458c456c960e46dc9ae9891c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f1cba44ed55e4a4eb376a5f85c01a9dc":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a88dc89e0d924f27a7a3f3684234bee4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ded7bee83ad24b168d09c8e4577fdab6":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"95c4a7e2041d4380b89cf9a791dfdab8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"de42d4914e194932ae203e96b3879f50":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1302f4fe69b547a4943b7ee84b8c214e":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f610104b6e3c4b1bab99589a2ec3a7e8","IPY_MODEL_6ee4789894664b0fa173a19f9014a6cf","IPY_MODEL_23e48c40acc54ed5a0633328c053cdf3"],"layout":"IPY_MODEL_261b92c265d545959e52e0dd3161ceb7"}},"f610104b6e3c4b1bab99589a2ec3a7e8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dce1a34a62f54321ab718c0ee4cb0fad","placeholder":"​","style":"IPY_MODEL_5d7b7724adf94edeb7e2d439e77a4722","value":"Downloading builder script: 100%"}},"6ee4789894664b0fa173a19f9014a6cf":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e792c569f1ac4900b8090a3ae922c14c","max":7950,"min":0,"orientation":"horizontal","style":"IPY_MODEL_275c9ce2c3694a1dbaa729ac1b43cf62","value":7950}},"23e48c40acc54ed5a0633328c053cdf3":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6eba23a5957648efb54861826029fd84","placeholder":"​","style":"IPY_MODEL_19eb9700bc44419991189806f5419030","value":" 7.95k/7.95k [00:00&lt;00:00, 574kB/s]"}},"261b92c265d545959e52e0dd3161ceb7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dce1a34a62f54321ab718c0ee4cb0fad":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5d7b7724adf94edeb7e2d439e77a4722":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e792c569f1ac4900b8090a3ae922c14c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"275c9ce2c3694a1dbaa729ac1b43cf62":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6eba23a5957648efb54861826029fd84":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"19eb9700bc44419991189806f5419030":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2fd41b9a33534f4bb65feae6a9d073fa":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3495026dae6b4d6bb01434aee6083157","IPY_MODEL_83d446588f604229a68757b429b996c2","IPY_MODEL_500347736d2f48ce885e00b45b17c971"],"layout":"IPY_MODEL_95cb6c0fb8b84f76b276736076bd9686"}},"3495026dae6b4d6bb01434aee6083157":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2e025b3a0ef04f3c9fa9a56189240c28","placeholder":"​","style":"IPY_MODEL_a8502ee80b9c4587a5243075f0e7facd","value":"Downloading (…)lve/main/config.json: 100%"}},"83d446588f604229a68757b429b996c2":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_fcf72f32773b4decacd270689fc0d8c7","max":482,"min":0,"orientation":"horizontal","style":"IPY_MODEL_8840c7b2fd9949e2b25df89fdee9a878","value":482}},"500347736d2f48ce885e00b45b17c971":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5b6fdfb32c5340dbb55ddce9f0cebf50","placeholder":"​","style":"IPY_MODEL_2504d819e86e4847a5684423bb1220e2","value":" 482/482 [00:00&lt;00:00, 32.1kB/s]"}},"95cb6c0fb8b84f76b276736076bd9686":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e025b3a0ef04f3c9fa9a56189240c28":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a8502ee80b9c4587a5243075f0e7facd":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"fcf72f32773b4decacd270689fc0d8c7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8840c7b2fd9949e2b25df89fdee9a878":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"5b6fdfb32c5340dbb55ddce9f0cebf50":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2504d819e86e4847a5684423bb1220e2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"222890d0de47416fb2a4437abef3b5c6":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f6f254ecfd1b401eb8796114808df5d0","IPY_MODEL_832bbc5676594cac9388fc5026d12eff","IPY_MODEL_729b79e9cba546ceb89a2c083a592f9e"],"layout":"IPY_MODEL_aa8626c3d29142a19065c1d02492fa54"}},"f6f254ecfd1b401eb8796114808df5d0":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_086ae2b064b143a9bd907c475ce4f285","placeholder":"​","style":"IPY_MODEL_7dd2c889103049558d3485b19abc8f86","value":"Downloading (…)olve/main/vocab.json: 100%"}},"832bbc5676594cac9388fc5026d12eff":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3d93c6a0dbb5463bb2bc9f19db9a3df4","max":898823,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1d81cae6e20940e6ad1c021e279e37c3","value":898823}},"729b79e9cba546ceb89a2c083a592f9e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f853caea75f6442f8ffa08c4fa2ba093","placeholder":"​","style":"IPY_MODEL_97b7b0af5cb944fc88dd272130c3e8b3","value":" 899k/899k [00:00&lt;00:00, 4.18MB/s]"}},"aa8626c3d29142a19065c1d02492fa54":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"086ae2b064b143a9bd907c475ce4f285":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7dd2c889103049558d3485b19abc8f86":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3d93c6a0dbb5463bb2bc9f19db9a3df4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1d81cae6e20940e6ad1c021e279e37c3":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f853caea75f6442f8ffa08c4fa2ba093":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"97b7b0af5cb944fc88dd272130c3e8b3":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a577c412ae044b4c914458ef472388de":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b7a05ca680794df5b1a2182117642e8a","IPY_MODEL_9868ee56a64b4639b82ebe5a52e577aa","IPY_MODEL_5c7a65551d8046c3aeae7dab5e250a04"],"layout":"IPY_MODEL_8614ded9212940b7ba3ae5ff7e907296"}},"b7a05ca680794df5b1a2182117642e8a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_621566cd751d4572a0ba8857748994ac","placeholder":"​","style":"IPY_MODEL_0900c9f95c264fefb39d3371df3e6a33","value":"Downloading (…)olve/main/merges.txt: 100%"}},"9868ee56a64b4639b82ebe5a52e577aa":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_7500766e5f8748b7ba45a8c84bf29830","max":456318,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1e2a188edbad42688268eea41eebd8c9","value":456318}},"5c7a65551d8046c3aeae7dab5e250a04":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0d14b7f3c49543f49d3e0b88fb3fba1e","placeholder":"​","style":"IPY_MODEL_5697e83dfb364f9daf91baae943afe0f","value":" 456k/456k [00:00&lt;00:00, 13.2MB/s]"}},"8614ded9212940b7ba3ae5ff7e907296":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"621566cd751d4572a0ba8857748994ac":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0900c9f95c264fefb39d3371df3e6a33":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7500766e5f8748b7ba45a8c84bf29830":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1e2a188edbad42688268eea41eebd8c9":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0d14b7f3c49543f49d3e0b88fb3fba1e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5697e83dfb364f9daf91baae943afe0f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"106db60c93614397817a727006ead150":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_acd086b127b94740988e7eb7841e178e","IPY_MODEL_88244efc70aa46abb5c94fe95f1933f9","IPY_MODEL_13a6cb2f30174d0cb03e4be3d622d011"],"layout":"IPY_MODEL_512c20bc029044ac93ac72d6b7033712"}},"acd086b127b94740988e7eb7841e178e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3c73b5fa75b348e684788db984198653","placeholder":"​","style":"IPY_MODEL_e238513935de4ecbaeaaa91903487bdb","value":"Downloading pytorch_model.bin: 100%"}},"88244efc70aa46abb5c94fe95f1933f9":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_7504689ea8104dbaa2f0e19ef72d73a2","max":1425941629,"min":0,"orientation":"horizontal","style":"IPY_MODEL_16a2453580eb40b6ba544ec7895268f5","value":1425941629}},"13a6cb2f30174d0cb03e4be3d622d011":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a22abb39a71e479b8adc3448eb754045","placeholder":"​","style":"IPY_MODEL_18713e0549cc4072832d3375fbb13bf0","value":" 1.43G/1.43G [00:17&lt;00:00, 78.0MB/s]"}},"512c20bc029044ac93ac72d6b7033712":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3c73b5fa75b348e684788db984198653":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e238513935de4ecbaeaaa91903487bdb":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7504689ea8104dbaa2f0e19ef72d73a2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"16a2453580eb40b6ba544ec7895268f5":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a22abb39a71e479b8adc3448eb754045":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"18713e0549cc4072832d3375fbb13bf0":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":5}