02_Multilingual_Chat_with_Ray_Serve_GPU.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6db26119-1723-43d4-9853-d791f142f4fd",
   "metadata": {},
   "source": [
    "# Multilingual Chat with Ray Serve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4980aa61-246b-4863-9881-a41912c29737",
   "metadata": {
    "id": "4980aa61-246b-4863-9881-a41912c29737",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import ray\n",
    "import requests, json\n",
    "from starlette.requests import Request\n",
    "from typing import Dict\n",
    "\n",
    "from ray import serve"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "16e6fa3f-87b0-4dca-ac3e-1bdfb7e08821",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Ray Serve is a microservices framework for serving ML – the model serving component of Ray\n",
    "\n",
    "Ray Serve provides resource management, scaling, a straightforward component framework, FastAPI compatibility ... and direct integration to the entire Ray ecosystem for scale-out compute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfbb23a6-4696-4531-9aa0-5f8fcc476ec1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 101
    },
    "id": "bfbb23a6-4696-4531-9aa0-5f8fcc476ec1",
    "outputId": "ff217c11-2a16-4ca9-9c46-c2d535b88261",
    "tags": []
   },
   "outputs": [],
   "source": [
    "ray.init()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b152dd42-24cb-40ab-8de4-4de6ff96463c",
   "metadata": {
    "id": "b152dd42-24cb-40ab-8de4-4de6ff96463c",
    "tags": []
   },
   "source": [
    "### Chatbot using Huggingface LLM\n",
    "\n",
    "To get started with Ray Serve, our main task is to take a standard Python class representing a service and\n",
    "* add the `@serve.deployment` decorator\n",
    "* optionally specify properties like resource requirements, environment, scaling, and more\n",
    "\n",
    "Here, we take a \"chatbot hello world\" using a model from the Huggingface hub and a FastAPI-style HTTP wrapper, and create our first Ray Serve deployment.\n",
    "\n",
    "Note that reserving a resource -- even a fractional GPU -- is done declaratively in the same line as the deployment decorator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73497dc6-b78a-4edb-ac4f-96c495d59fd3",
   "metadata": {
    "id": "73497dc6-b78a-4edb-ac4f-96c495d59fd3",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f648424f-17dd-414b-9d27-39781881f15b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "c77294be-ca64-4f8b-b198-8f16032894df",
    "outputId": "7bbce78c-c094-48a7-d862-0378cd30b92c",
    "tags": []
   },
   "outputs": [],
   "source": [
    "@serve.deployment(ray_actor_options={\"num_gpus\": 0.5})\n",
    "class Chat:\n",
    "    def __init__(self, model: str):\n",
    "        # configure stateful elements of our service such as loading a model\n",
    "        self._tokenizer = AutoTokenizer.from_pretrained(model)\n",
    "        self._model = AutoModelForSeq2SeqLM.from_pretrained(model).to(0)\n",
    "\n",
    "    async def __call__(self, request: Request) -> Dict:\n",
    "        # path to handle HTTP requests\n",
    "        data = await request.json()\n",
    "        data = json.loads(data)\n",
    "        # after decoding the payload, we delegate to get_response for logic\n",
    "        return {\"response\": self.get_response(data[\"user_input\"], data[\"history\"])}\n",
    "\n",
    "    def get_response(self, user_input: str, history: list[str]) -> str:\n",
    "        # this method receives calls directly (from Python) or from __call__ (from HTTP)\n",
    "        history.append(user_input)\n",
    "        # the history is client-side state and will be a list of raw strings;\n",
    "        # for the default config of the model and tokenizer, history should be joined with '</s><s>'\n",
    "        inputs = self._tokenizer(\"</s><s>\".join(history), return_tensors=\"pt\").to(0)\n",
    "        reply_ids = self._model.generate(**inputs, max_new_tokens=500)\n",
    "        response = self._tokenizer.batch_decode(\n",
    "            reply_ids.cpu(), skip_special_tokens=True\n",
    "        )[0]\n",
    "        return response"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9093e4de-0837-4b92-9e58-a04fd3ec6e90",
   "metadata": {},
   "source": [
    "We've defined a service using the `@serve.deployment` decorator.\n",
    "* Deployments can be service endpoints\n",
    "* Or they can be individual components accessed from other services\n",
    "* Each deployment can have its own resource management and autoscaling configuration\n",
    "\n",
    "Here is example of extended configuration -- see https://docs.ray.io/en/latest/serve/scaling-and-resource-allocation.html#scaling-and-resource-allocation for more details\n",
    "\n",
    "```python\n",
    "@serve.deployment(\n",
    "    autoscaling_config={\n",
    "        'min_replicas': 1,\n",
    "        'initial_replicas': 2,\n",
    "        'max_replicas': 5,\n",
    "        'target_num_ongoing_requests_per_replica': 10,\n",
    "    }\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "979e8ac0-906a-4c37-8bdc-4d5c5c0a0029",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "c77294be-ca64-4f8b-b198-8f16032894df",
    "outputId": "7bbce78c-c094-48a7-d862-0378cd30b92c",
    "tags": []
   },
   "outputs": [],
   "source": [
    "chat = Chat.bind(model=\"facebook/blenderbot-400M-distill\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5da93367-7c4d-4ddf-88b9-3a1bbde820ac",
   "metadata": {},
   "source": [
    "We created the object that represents this service. We do that by using the `.bind()` class method, which captures the \"recipe\" for instantiating replicas of our service. The params in `.bind()` -- in this case, the name of the Huggingface model we want to use -- will be provided to our deployment class constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a84d11fc-d54e-48d2-9a97-f20aa23fcd4e",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "c77294be-ca64-4f8b-b198-8f16032894df",
    "outputId": "7bbce78c-c094-48a7-d862-0378cd30b92c",
    "tags": []
   },
   "outputs": [],
   "source": [
    "handle = serve.run(chat, name=\"basic_chat\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d6713b7a-1a0f-40d9-bfe2-df0acdac96d6",
   "metadata": {
    "tags": []
   },
   "source": [
    "<img src='https://technical-training-assets.s3.us-west-2.amazonaws.com/Ray_Serve/deployment.png' width=600/>\n",
    "\n",
    "`serve.run(...)` exposes out service endpoint for call. We can run multiple simultaneous applications with their own collections of deployments, resource management, scaling, etc. The `name` parameter in `serve.run(...)` becomes an identifier not only for the application but for each deployment that is launched to support that application.\n",
    "\n",
    "Let's test out the service using Ray's Python API first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6265cf13-1363-42fb-8d4c-629b71c306af",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 109
    },
    "id": "6265cf13-1363-42fb-8d4c-629b71c306af",
    "outputId": "573423af-4b55-4f1c-e72b-29d7dcccf6d5",
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = \"I can't wait to see more XFL football.\"\n",
    "history = []\n",
    "response_handle = handle.get_response.remote(message, history)\n",
    "response = ray.get(response_handle)\n",
    "response"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c8b6b6ff-5030-4182-9bf7-27dd61d180f9",
   "metadata": {},
   "source": [
    "We prepare a message and a chat history list and call our chat service via Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34549288-e4a2-4fbb-b41f-e7ea70f51d13",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "34549288-e4a2-4fbb-b41f-e7ea70f51d13",
    "outputId": "8c682533-9ebd-4965-dc75-b0661cf1f3d6",
    "tags": []
   },
   "outputs": [],
   "source": [
    "history += [message, response]\n",
    "history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1b448ae-3662-4211-bde6-72734f80ad30",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 127
    },
    "id": "e1b448ae-3662-4211-bde6-72734f80ad30",
    "outputId": "28f4ea8b-2ce7-4aee-d1df-5546a74a5f9b",
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = \"That is accurate... but there is much more to it.\"\n",
    "response_handle = handle.get_response.remote(message, history)\n",
    "response = ray.get(response_handle)\n",
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fabf2482-0689-4bf9-b088-b2b369184560",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fabf2482-0689-4bf9-b088-b2b369184560",
    "outputId": "23103c62-4b6f-43cb-b515-ef3d5f06767b",
    "tags": []
   },
   "outputs": [],
   "source": [
    "history += [message, response]\n",
    "history"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "aa28fca4-46a1-4367-98ce-cbbbacd8d668",
   "metadata": {},
   "source": [
    "And we'll check that our HTTP endpoint is alive as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7b1c235-95d5-4ddd-9357-4339b81ea758",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "id": "f7b1c235-95d5-4ddd-9357-4339b81ea758",
    "outputId": "0388c217-69cc-4b7e-eeac-a437f39d0613",
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = (\n",
    "    \"Even between XFL and NLF American football leagues, there are big differences.\"\n",
    ")\n",
    "\n",
    "json_doc = json.dumps({\"user_input\": message, \"history\": history})\n",
    "\n",
    "json_doc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da29de38-1ead-49c1-ba68-f61227fe337f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "da29de38-1ead-49c1-ba68-f61227fe337f",
    "outputId": "d30576d4-80fe-4cf2-b71f-5eee3d62ee70",
    "tags": []
   },
   "outputs": [],
   "source": [
    "requests.post(\"http://localhost:8000/\", json=json_doc).json()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6f637329-ab6a-4510-9e0e-e6889aea1943",
   "metadata": {},
   "source": [
    "Since we'll be creating a new application example, we can delete the old one -- that allows Ray to remove the replicas of our Chat deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d1fb523-12ff-458f-8c56-2a37bd36756a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "serve.delete(\"basic_chat\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d96b79c3-5423-4786-a4b6-047be1fcec78",
   "metadata": {},
   "source": [
    "## Goals for Model Serving in Production\n",
    "\n",
    "As we assemble more complex applications, Ray Serve delivers increasingly valuable features\n",
    "* The same platform -- so zero integration -- for preprocessing, training, batch prediction and other parts of the ML workflow\n",
    "* Fine-grained resource allocation and scaling at the level of critical functions and classes\n",
    "    * Instead of or in addition to scaling at the level of containers, like with Kubernetes, or of nodes\n",
    "* Automatic batching\n",
    "* Minimal data movement and data-locality-aware scheduling\n",
    "\n",
    "<img src='https://technical-training-assets.s3.us-west-2.amazonaws.com/Ray_Serve/serve_architecture.png' width=700/>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b54a5b65-7b86-4db0-821c-f3f9ce143316",
   "metadata": {
    "id": "b54a5b65-7b86-4db0-821c-f3f9ce143316",
    "tags": []
   },
   "source": [
    "## Composing Services with Ray for Chatbot en Français\n",
    "\n",
    "### Roadmap for Additional Services\n",
    "\n",
    "The underlying chatbot model we’ve used only supports English interaction. But we can use the following recipe to add French language support:\n",
    "\n",
    "1. Implement a translation service between French and English\n",
    "1. Implement a language detection service\n",
    "1. Implement a routing (dispatch) service:\n",
    "    1. If the incoming prompt is French, then\n",
    "        1. Route the inbound prompt through the FR-EN translator\n",
    "        1. Pass the EN prompt to the chat model\n",
    "        1. Pass the EN output from the chat model through the EN-FR translator\n",
    "        1. Return the French response\n",
    "    1. Otherwise (if the prompt is in English), pass it straight to the chatbot as we did earlier and return the (English) response\n",
    "\n",
    "Let’s look using Ray Serve to implement model inference with these composed and conditional-flow elements using Python method calls (https://docs.ray.io/en/latest/serve/key-concepts.html#servehandle-composing-deployments).\n",
    "We’ll implement parts 1 and 2 first…\n",
    "\n",
    "### Translation service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d55d5f93-b22c-4a76-9261-290862c62882",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 358,
     "referenced_widgets": [
      "ed182dd854d1439ab6d44e39210d7074",
      "4ed3f579be5d450aacfa0f970b8d81da",
      "5f0dc2de13284933ae2439af1fe88753",
      "3d98bfe346ed40a8bd5d46de450bca24",
      "1737ac3e202a41688c4adb0b498ec58f",
      "3f127b730e024ffa91ab921ac5291c73",
      "bbd6c25a5bef464fa74837811f1e3a39",
      "442b5bbd50f145e9844495dcda8c09d5",
      "13efed5b0c204ce58120470ae37e6af2",
      "5f28f3c322484ff5a88e988790417d28",
      "d5ff21c53b644abbaa8c2a06c38d2d1d",
      "9860b04b1772493e819f80ceb1000aa0",
      "506f934b75c84618a49f92188d1520b3",
      "4de12f5510974cd79848308c361db070",
      "5b28b54f06f7445aaeab92194bc5951f",
      "eb16d41e87a14b18a33ce16aab9c9c8c",
      "ecd2111b9e974b3eaf712610ac19bb9c",
      "cc0c31d2e0f3494ca2d0440cf5d030d5",
      "7635b8cd4b0748e784b264301b94e535",
      "2b2393d742b344f5b1f79424dec2c4ca",
      "5f8fc5621284480bb1ca39ec6ddf7ff9",
      "13a316e0e13c492b8c9a7b7b3ffdad99",
      "9e8c49cfece042298d111c9dcd5d8831",
      "dd2b9a3f127a445995d25db6058fa425",
      "c2a59a1b016e4e4eabe47c41641d3322",
      "71d0e0924bba466f8cabc03b0a050b88",
      "ef16c90488864b088aa2f402cb3dbdf7",
      "ba44f27002c34b248ed4f70e1ac6f74d",
      "ea43a14dd52e45038a3746743ab3faac",
      "40b0e1c0bc2a40b08e4f46ab95fd5806",
      "465b13292f2b40839211874c8c7c0aee",
      "b707f3785b2d407db18f937c53d3cd65",
      "384044f6a71b46c8bcf6495c6797c74c",
      "38b584bc5a984778868f19a690a18529",
      "bd8e3a9d758140ca88d08f95fc3d0e58",
      "36b7159684c24fa18f5b250d57ef370e",
      "6431501c8c424881855cd372d2b37e77",
      "81c37bcc6d684b47841805ed43c0d8aa",
      "a710d60f9cfa405b8de2f9658c65a24f",
      "a448c57bdb41434295c745f66c0639e6",
      "6ae7d8496ac747e8989a8047775ba099",
      "0fa581a4f548419a9d7ed301a89c9712",
      "49cbb21b99d94a37b290943c926b1410",
      "c479d3b81e4e4abcae5351bb1737155d",
      "6a04740303ed4265907db573a01dd5dc",
      "90b52d08441e4366a03ce5d943d4b3c3",
      "12b42905d56947b2949ebfe2b202fe18",
      "c723559e7e9b420bbf4c417625631647",
      "49fe982496a64136a5a858a6191f1254",
      "0bb83c7a35754c46b704c68b95cf13d9",
      "053a38fae8664d7a9f509fd6f9b13b35",
      "771cd557b6ce42a586878d2d05568e89",
      "12b77bb22ca943bb9fd842278b6b135f",
      "1373256b833b402b8c2e28ab825537a5",
      "1970ae7144244e3eb650bc94f7b1c209"
     ]
    },
    "id": "d55d5f93-b22c-4a76-9261-290862c62882",
    "outputId": "ff6a42ad-fe0a-4b65-f289-e610536baa1d",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from transformers import pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d1b79e65-6a72-4046-a551-dea32116b83d",
   "metadata": {
    "id": "d1b79e65-6a72-4046-a551-dea32116b83d",
    "tags": []
   },
   "outputs": [],
   "source": [
    "@serve.deployment(ray_actor_options={\"num_gpus\": 0.25})\n",
    "class Translate:\n",
    "    def __init__(self, task: str, model: str):\n",
    "        self._pipeline = pipeline(task=task, model=model, device=0)\n",
    "\n",
    "    def get_response(self, user_input: str) -> str:\n",
    "        outputs = self._pipeline(user_input)\n",
    "        response = outputs[0][\"translation_text\"]\n",
    "        return response\n",
    "\n",
    "\n",
    "translate_en_fr = Translate.bind(task=\"translation_en_to_fr\", model=\"t5-small\")\n",
    "translate_fr_en = Translate.bind(\n",
    "    task=\"translation_fr_to_en\", model=\"Helsinki-NLP/opus-mt-fr-en\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6c01fb59-96bd-4bd1-a422-e9d1980b4310",
   "metadata": {},
   "source": [
    "Notice how we have two different services but they are built on the same reusable code by calling `.bind()` with different initialization parameters.\n",
    "\n",
    "*We don’t need to define new deployments for every service we use.*\n",
    "\n",
    "This time we’re haven't published an application (via `serve.run()`) because these components will be invoked only by our main service deployment."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1e6f7e12-cfa0-41c2-abd5-4d300b14195e",
   "metadata": {
    "id": "1e6f7e12-cfa0-41c2-abd5-4d300b14195e"
   },
   "source": [
    "### Language detection\n",
    "\n",
    "We can create the language detection service in a similar way. \n",
    "\n",
    "> This service is lighter weight because we’re using https://github.com/pemistahl/lingua-py … which leverages traditional NLP and n-grams for detection instead of a deep learning model. It can handle more traffic than, e.g., the chat model -- and it won't require a GPU. So we can benefit from Ray Serve's fine-grained resource allocation.\n",
    "    \n",
    "Lingua is optimized for strong detection on very short text snippets, like tweets, so it should be useful for our chat exchanges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d365afa-f56d-4d11-9b0c-ceb58d206f0e",
   "metadata": {
    "id": "8d365afa-f56d-4d11-9b0c-ceb58d206f0e",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from lingua import Language, LanguageDetectorBuilder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f6bd63d-4958-4580-b72a-caeba1a1f578",
   "metadata": {
    "id": "5f6bd63d-4958-4580-b72a-caeba1a1f578",
    "tags": []
   },
   "outputs": [],
   "source": [
    "@serve.deployment\n",
    "class LangDetect:\n",
    "    def __init__(self):\n",
    "        languages = [Language.ENGLISH, Language.FRENCH]\n",
    "        self._detector = LanguageDetectorBuilder.from_languages(*languages).build()\n",
    "\n",
    "    def get_response(self, user_input: str) -> str:\n",
    "        output = self._detector.detect_language_of(user_input)\n",
    "        if output == Language.ENGLISH:\n",
    "            return \"en\"\n",
    "        else:\n",
    "            return \"fr\"\n",
    "\n",
    "\n",
    "lang_detect = LangDetect.bind()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "778ede38-d221-4f45-8e2d-71ec0702aae4",
   "metadata": {
    "id": "778ede38-d221-4f45-8e2d-71ec0702aae4"
   },
   "source": [
    "### Combine services\n",
    "\n",
    "Let's bring the whole system together. We'll implement a service which represents our external endpoint for HTTP or Python invocations.\n",
    "* This service will have references to the deployments we've built so far, and will implement some conditional logic to ensure the correct language is used\n",
    "* Note that even if the user is interacting in French, we need to return the English response as well so that client can use that to build the chat history\n",
    "\n",
    "<img src='https://technical-training-assets.s3.us-west-2.amazonaws.com/Ray_Serve/deployment_endpoint.png' width=700/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22f85caa-133e-4df7-8aab-4e3278ffc54b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "22f85caa-133e-4df7-8aab-4e3278ffc54b",
    "outputId": "2086ccc1-0d59-419f-d47f-d6144980f536",
    "tags": []
   },
   "outputs": [],
   "source": [
    "@serve.deployment\n",
    "class Endpoint:\n",
    "    def __init__(self, chat, lang_detect, translate_en_fr, translate_fr_en):\n",
    "        # assign dependent service handles to instance variables\n",
    "        self._chat = chat\n",
    "        self._lang_detect = lang_detect\n",
    "        self._translate_en_fr = translate_en_fr\n",
    "        self._translate_fr_en = translate_fr_en\n",
    "\n",
    "    async def __call__(self, request: Request) -> Dict:\n",
    "        data = await request.json()\n",
    "        data = json.loads(data)\n",
    "        return {\n",
    "            \"response\": await self.get_response(data[\"user_input\"], data[\"history\"])\n",
    "        }\n",
    "\n",
    "    async def get_response(self, user_input: str, history: list[str]):\n",
    "        lang_obj_ref = await self._lang_detect.get_response.remote(user_input)\n",
    "\n",
    "        # if we didn't need the literal value of the language yet, we could pass that (future) object reference to other services\n",
    "        # here, though, we need the value in order to decide whether to call the translation services\n",
    "        # we get the Python value by awaiting the object reference\n",
    "        lang = await lang_obj_ref\n",
    "\n",
    "        if lang == \"fr\":\n",
    "            user_input = await self._translate_fr_en.get_response.remote(user_input)\n",
    "\n",
    "        response = response_en = await self._chat.get_response.remote(\n",
    "            user_input, history\n",
    "        )\n",
    "\n",
    "        if lang == \"fr\":\n",
    "            response = await self._translate_en_fr.get_response.remote(response_en)\n",
    "            user_input = await user_input\n",
    "\n",
    "        response = await response\n",
    "        response_en = await response_en\n",
    "\n",
    "        return response + \"|\" + user_input + \"|\" + response_en\n",
    "\n",
    "\n",
    "endpoint = Endpoint.bind(chat, lang_detect, translate_en_fr, translate_fr_en)\n",
    "\n",
    "endpoint_handle = serve.run(endpoint, name=\"multilingual_chat\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6433e5c4-687a-4405-a790-4b6544f897e9",
   "metadata": {},
   "source": [
    "We've implemented control flow through our services and used the async/await pattern in several places so that we don't unnecessarily block.\n",
    "\n",
    "Then we construct the service endpoint and start a new application serving that endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbe6a39a-1b55-49c6-8a7e-632256588bf3",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 127
    },
    "id": "fbe6a39a-1b55-49c6-8a7e-632256588bf3",
    "outputId": "fd81afe6-5f31-4777-ecd5-c4d138ca901e",
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = \"I can't wait to see more XFL football.\"\n",
    "history = []\n",
    "response = ray.get(endpoint_handle.get_response.remote(message, history))\n",
    "response.split(\"|\")[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b852c27-20d3-4cc3-b300-5fd15e0ba124",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0b852c27-20d3-4cc3-b300-5fd15e0ba124",
    "outputId": "386cf1b6-daeb-4a19-d0cd-248131bd9f52",
    "tags": []
   },
   "outputs": [],
   "source": [
    "history += response.split(\"|\")[1:]\n",
    "history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73855166-c226-40f9-a365-babb36f751b3",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 234
    },
    "id": "73855166-c226-40f9-a365-babb36f751b3",
    "outputId": "17498373-04e4-4595-80af-88cb24c568f2",
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = \"C'est exact... mais il y a bien plus que cela.\"\n",
    "response = ray.get(endpoint_handle.get_response.remote(message, history))\n",
    "response.split(\"|\")[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25952d4a-43fe-49fa-a437-eb91813cde5c",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "25952d4a-43fe-49fa-a437-eb91813cde5c",
    "outputId": "29e847c0-4819-46c6-d15c-681bab01e028",
    "tags": []
   },
   "outputs": [],
   "source": [
    "history += response.split(\"|\")[1:]\n",
    "history"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9d0cd70a-6b6e-4023-a10d-97ef097ef3df",
   "metadata": {
    "tags": []
   },
   "source": [
    "> We haven’t looked under the hood at how all of this works on Ray. The details aren't covered in this session, but there is more information at https://docs.ray.io/en/latest/serve/architecture.html\n",
    "\n",
    "At this point we have a service which can support the many functional and operational properties we expect to need in production, including scalability, separation of concerns, and composability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29b367ed-e1c6-41aa-ba97-d77161b14c95",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "serve.delete(\"multilingual_chat\")\n",
    "serve.shutdown()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3d0d1933-dd99-49bd-ba2e-80e375522c5e",
   "metadata": {},
   "source": [
    "## Ray Serve: Summary\n",
    "\n",
    "__Ray Serve is a microservices framework for serving ML – the model serving component of Ray.__\n",
    "\n",
    "Key features include\n",
    "* High-performance operation at large scale with heterogeneous resources\n",
    "* Fine-grained resource allocation and scheduling\n",
    "* Autoscaling\n",
    "* Simple interaction with preprocessing, training, tuning, and other parts of the ML workflow on Ray\n",
    "* Out-of-the box integration with third-party tools like PyTorch, Huggingface, Gradio, Weights and Biases, Delta Lake, and more\n",
    "* Straightforward composition and orchestration of services in a declarative or imperative style, all using Python\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d070e23-ab64-4324-bafd-08039e559cae",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "053a38fae8664d7a9f509fd6f9b13b35": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0bb83c7a35754c46b704c68b95cf13d9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0fa581a4f548419a9d7ed301a89c9712": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "12b42905d56947b2949ebfe2b202fe18": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_771cd557b6ce42a586878d2d05568e89",
      "max": 1389353,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_12b77bb22ca943bb9fd842278b6b135f",
      "value": 1389353
     }
    },
    "12b77bb22ca943bb9fd842278b6b135f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1373256b833b402b8c2e28ab825537a5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "13a316e0e13c492b8c9a7b7b3ffdad99": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "13efed5b0c204ce58120470ae37e6af2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1737ac3e202a41688c4adb0b498ec58f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1970ae7144244e3eb650bc94f7b1c209": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2b2393d742b344f5b1f79424dec2c4ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "36b7159684c24fa18f5b250d57ef370e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6ae7d8496ac747e8989a8047775ba099",
      "max": 791656,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0fa581a4f548419a9d7ed301a89c9712",
      "value": 791656
     }
    },
    "384044f6a71b46c8bcf6495c6797c74c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "38b584bc5a984778868f19a690a18529": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_bd8e3a9d758140ca88d08f95fc3d0e58",
       "IPY_MODEL_36b7159684c24fa18f5b250d57ef370e",
       "IPY_MODEL_6431501c8c424881855cd372d2b37e77"
      ],
      "layout": "IPY_MODEL_81c37bcc6d684b47841805ed43c0d8aa"
     }
    },
    "3d98bfe346ed40a8bd5d46de450bca24": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f28f3c322484ff5a88e988790417d28",
      "placeholder": "​",
      "style": "IPY_MODEL_d5ff21c53b644abbaa8c2a06c38d2d1d",
      "value": " 1.21k/1.21k [00:00&lt;00:00, 20.4kB/s]"
     }
    },
    "3f127b730e024ffa91ab921ac5291c73": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "40b0e1c0bc2a40b08e4f46ab95fd5806": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "442b5bbd50f145e9844495dcda8c09d5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "465b13292f2b40839211874c8c7c0aee": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "49cbb21b99d94a37b290943c926b1410": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "49fe982496a64136a5a858a6191f1254": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4de12f5510974cd79848308c361db070": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7635b8cd4b0748e784b264301b94e535",
      "max": 242065649,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2b2393d742b344f5b1f79424dec2c4ca",
      "value": 242065649
     }
    },
    "4ed3f579be5d450aacfa0f970b8d81da": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3f127b730e024ffa91ab921ac5291c73",
      "placeholder": "​",
      "style": "IPY_MODEL_bbd6c25a5bef464fa74837811f1e3a39",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "506f934b75c84618a49f92188d1520b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ecd2111b9e974b3eaf712610ac19bb9c",
      "placeholder": "​",
      "style": "IPY_MODEL_cc0c31d2e0f3494ca2d0440cf5d030d5",
      "value": "Downloading pytorch_model.bin: 100%"
     }
    },
    "5b28b54f06f7445aaeab92194bc5951f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f8fc5621284480bb1ca39ec6ddf7ff9",
      "placeholder": "​",
      "style": "IPY_MODEL_13a316e0e13c492b8c9a7b7b3ffdad99",
      "value": " 242M/242M [00:01&lt;00:00, 180MB/s]"
     }
    },
    "5f0dc2de13284933ae2439af1fe88753": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_442b5bbd50f145e9844495dcda8c09d5",
      "max": 1206,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_13efed5b0c204ce58120470ae37e6af2",
      "value": 1206
     }
    },
    "5f28f3c322484ff5a88e988790417d28": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5f8fc5621284480bb1ca39ec6ddf7ff9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6431501c8c424881855cd372d2b37e77": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_49cbb21b99d94a37b290943c926b1410",
      "placeholder": "​",
      "style": "IPY_MODEL_c479d3b81e4e4abcae5351bb1737155d",
      "value": " 792k/792k [00:00&lt;00:00, 12.4MB/s]"
     }
    },
    "6a04740303ed4265907db573a01dd5dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_90b52d08441e4366a03ce5d943d4b3c3",
       "IPY_MODEL_12b42905d56947b2949ebfe2b202fe18",
       "IPY_MODEL_c723559e7e9b420bbf4c417625631647"
      ],
      "layout": "IPY_MODEL_49fe982496a64136a5a858a6191f1254"
     }
    },
    "6ae7d8496ac747e8989a8047775ba099": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "71d0e0924bba466f8cabc03b0a050b88": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b707f3785b2d407db18f937c53d3cd65",
      "placeholder": "​",
      "style": "IPY_MODEL_384044f6a71b46c8bcf6495c6797c74c",
      "value": " 147/147 [00:00&lt;00:00, 4.17kB/s]"
     }
    },
    "7635b8cd4b0748e784b264301b94e535": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "771cd557b6ce42a586878d2d05568e89": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "81c37bcc6d684b47841805ed43c0d8aa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "90b52d08441e4366a03ce5d943d4b3c3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0bb83c7a35754c46b704c68b95cf13d9",
      "placeholder": "​",
      "style": "IPY_MODEL_053a38fae8664d7a9f509fd6f9b13b35",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "9860b04b1772493e819f80ceb1000aa0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_506f934b75c84618a49f92188d1520b3",
       "IPY_MODEL_4de12f5510974cd79848308c361db070",
       "IPY_MODEL_5b28b54f06f7445aaeab92194bc5951f"
      ],
      "layout": "IPY_MODEL_eb16d41e87a14b18a33ce16aab9c9c8c"
     }
    },
    "9e8c49cfece042298d111c9dcd5d8831": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_dd2b9a3f127a445995d25db6058fa425",
       "IPY_MODEL_c2a59a1b016e4e4eabe47c41641d3322",
       "IPY_MODEL_71d0e0924bba466f8cabc03b0a050b88"
      ],
      "layout": "IPY_MODEL_ef16c90488864b088aa2f402cb3dbdf7"
     }
    },
    "a448c57bdb41434295c745f66c0639e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a710d60f9cfa405b8de2f9658c65a24f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b707f3785b2d407db18f937c53d3cd65": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ba44f27002c34b248ed4f70e1ac6f74d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bbd6c25a5bef464fa74837811f1e3a39": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bd8e3a9d758140ca88d08f95fc3d0e58": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a710d60f9cfa405b8de2f9658c65a24f",
      "placeholder": "​",
      "style": "IPY_MODEL_a448c57bdb41434295c745f66c0639e6",
      "value": "Downloading (…)ve/main/spiece.model: 100%"
     }
    },
    "c2a59a1b016e4e4eabe47c41641d3322": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_40b0e1c0bc2a40b08e4f46ab95fd5806",
      "max": 147,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_465b13292f2b40839211874c8c7c0aee",
      "value": 147
     }
    },
    "c479d3b81e4e4abcae5351bb1737155d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c723559e7e9b420bbf4c417625631647": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1373256b833b402b8c2e28ab825537a5",
      "placeholder": "​",
      "style": "IPY_MODEL_1970ae7144244e3eb650bc94f7b1c209",
      "value": " 1.39M/1.39M [00:00&lt;00:00, 8.00MB/s]"
     }
    },
    "cc0c31d2e0f3494ca2d0440cf5d030d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d5ff21c53b644abbaa8c2a06c38d2d1d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dd2b9a3f127a445995d25db6058fa425": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ba44f27002c34b248ed4f70e1ac6f74d",
      "placeholder": "​",
      "style": "IPY_MODEL_ea43a14dd52e45038a3746743ab3faac",
      "value": "Downloading (…)neration_config.json: 100%"
     }
    },
    "ea43a14dd52e45038a3746743ab3faac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "eb16d41e87a14b18a33ce16aab9c9c8c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ecd2111b9e974b3eaf712610ac19bb9c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ed182dd854d1439ab6d44e39210d7074": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4ed3f579be5d450aacfa0f970b8d81da",
       "IPY_MODEL_5f0dc2de13284933ae2439af1fe88753",
       "IPY_MODEL_3d98bfe346ed40a8bd5d46de450bca24"
      ],
      "layout": "IPY_MODEL_1737ac3e202a41688c4adb0b498ec58f"
     }
    },
    "ef16c90488864b088aa2f402cb3dbdf7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}