W13_02_RNN_Text_Generation_Solutions.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "br_Yx7iVQVNf"
      },
      "source": [
        "# Text generation with an RNN\n",
        "\n",
        "In this notebook we will learn how to generate text with a Recurrent Neural Network.\n",
        "\n",
        "We will work with a dataset of Shakespeare's writing from Andrej Karpathy's article [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/). \n",
        "\n",
        "Given a sequence of characters from this data (for example \"Shakespear\"), we will train a model to predict the next character in the sequence (\"e\"). Longer sequences of text can be generated by calling the model repeatedly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ISUjH1WOSAAL"
      },
      "source": [
        "### Import the necessary modules"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h4Z7-Wm7_5K6",
        "outputId": "d393c235-5fbb-4b9c-bfe7-8b37d9789eea"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "PyTorch Version:  1.11.0+cu113\n",
            "Using device: cuda:0\n"
          ]
        }
      ],
      "source": [
        "from __future__ import print_function \n",
        "from __future__ import division\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import numpy as np\n",
        "from torch.utils.data import TensorDataset, DataLoader\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "from collections import Counter\n",
        "import time\n",
        "import os\n",
        "import copy\n",
        "print(\"PyTorch Version: \",torch.__version__)\n",
        "\n",
        "%matplotlib inline\n",
        "\n",
        "\n",
        "# Detect if we have a GPU available\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "print(\"Using device:\", device)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Thoa-XIQR50H"
      },
      "source": [
        "### Download the Shakespeare dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "50MQzxMJYgOb",
        "outputId": "df04c471-64d2-4e1d-bf09-d403c282633d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-05-09 21:15:59--  https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt\n",
            "Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.197.128, 74.125.135.128, 74.125.142.128, ...\n",
            "Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.197.128|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1115394 (1.1M) [text/plain]\n",
            "Saving to: ‘shakespeare.txt.1’\n",
            "\n",
            "\rshakespeare.txt.1     0%[                    ]       0  --.-KB/s               \rshakespeare.txt.1   100%[===================>]   1.06M  --.-KB/s    in 0.01s   \n",
            "\n",
            "2022-05-09 21:15:59 (106 MB/s) - ‘shakespeare.txt.1’ saved [1115394/1115394]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "! wget https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WtnrLVpySa2G"
      },
      "source": [
        "###  Create a dataset class\n",
        "\n",
        "For training we need a dataset of (input, label) pairs. Where input and label are sequences of characters. At each time step the input is the current character and the label is the next character. For example:\n",
        "\n",
        "```\n",
        "input = ['S', 'h', 'a', 'k', 'e', 's', 'p', 'e', 'a', 'r']\n",
        "label = ['h', 'a', 'k', 'e', 's', 'p', 'e', 'a', 'r', 'e']\n",
        " ```\n",
        "\n",
        "Actually we will vectorize the characters to convert them into numerical values. \n",
        "\n",
        "```\n",
        "input = [44, 7, 0, 10, 4, 18, 15, 4, 0, 17]\n",
        "label = [7, 0, 10, 4, 18, 15, 4, 0, 17, 4]\n",
        "```\n",
        "\n",
        "where each integer value is a class label, and the number of classes is the number of unique characters in our training data (our vocabulary size)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "kducKEs4Vfm8"
      },
      "outputs": [],
      "source": [
        "class ShakespeareDataset():\n",
        "    def __init__(self, sequence_length,):\n",
        "        self.sequence_length = sequence_length\n",
        "        # The whole sequence of characters in the text file (including chracters like the end of line character '\\n', punctuation marks, etc.)\n",
        "        self.chars = self.load_chars()\n",
        "        # The list of unique characters (our vocabulary)\n",
        "        self.uniq_chars = self.get_uniq_chars()\n",
        "\n",
        "        # A dictionary with indexes as keys and characters as values\n",
        "        self.index_to_char = {index: char for index, char in enumerate(self.uniq_chars)}\n",
        "        # A dictionary with characters as keys and indexes as values\n",
        "        self.char_to_index = {char: index for index, char in enumerate(self.uniq_chars)}\n",
        "\n",
        "        # The whole sequence of character indexes\n",
        "        # We want to vectorize our text using one-hot vectors at character level\n",
        "        # But instead of using the one-hot vector we just keep the index of the 1 value\n",
        "        self.chars_indexes = [self.char_to_index[w] for w in self.chars]\n",
        "\n",
        "        # Split the sequence into chunks of `sequence_length` characters\n",
        "        # `x` is the input to the model and `y` is the output of the model\n",
        "        # `y` is the same seuqence as `x` but one character shifted to the right.\n",
        "        self.x, self.y = self.get_data()\n",
        "\n",
        "    def load_chars(self):\n",
        "        with open('shakespeare.txt') as fp:\n",
        "          text = fp.read()\n",
        "        print(f'Length of text: {len(text)} characters')\n",
        "        return text\n",
        "\n",
        "    def get_uniq_chars(self):\n",
        "        char_counts = Counter(self.chars)\n",
        "        uniq_chars = sorted(char_counts, key=char_counts.get, reverse=True)\n",
        "        print(f'Unique characters (vocabulary size): {len(uniq_chars)}')\n",
        "        return uniq_chars\n",
        "\n",
        "    def get_data(self):\n",
        "        x,y = [], []\n",
        "        for index in range(0, len(self.chars_indexes)-self.sequence_length,self.sequence_length):\n",
        "            x.append(np.array(self.chars_indexes[index:index+self.sequence_length]))\n",
        "            y.append(np.array(self.chars_indexes[index+1:index+self.sequence_length+1]))\n",
        "        \n",
        "        return  np.array(x), np.array(y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "049N0Z8oyt-F"
      },
      "source": [
        "Now we can use the [`torch.utils.data.TensorDataset`](https://pytorch.org/docs/stable/data.html) and [`torch.utils.data.Dataloader`](https://pytorch.org/docs/stable/data.html) classes to create the PyTorch dataloader. Notice that the `TensorDataset` class allows us to easily create a PyTorch `Dataset` from numpy arrays `(inputs, labels)`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y08ha1OKVYjM",
        "outputId": "f9762733-f8cb-45e5-9aaf-11ab30b0e894"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Length of text: 1115394 characters\n",
            "Unique characters (vocabulary size): 65\n",
            "We have 11153 input sequence samples of length 100 and 11153 label sequences of length 100\n"
          ]
        }
      ],
      "source": [
        "# Create the dataset\n",
        "sequence_length = 100\n",
        "dataset = ShakespeareDataset(sequence_length)\n",
        "\n",
        "print(f'We have {dataset.x.shape[0]} input sequence samples of length {dataset.x.shape[1]} \\\n",
        "and {dataset.y.shape[0]} label sequences of length {dataset.y.shape[1]}')\n",
        "\n",
        "# Create the Torch DataLoader\n",
        "batch_size = 64\n",
        "train_data = TensorDataset(torch.from_numpy(dataset.x), torch.from_numpy(dataset.y))\n",
        "train_loader = DataLoader(train_data, shuffle=True, batch_size=batch_size, drop_last=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D1rfa97KRc3s"
      },
      "source": [
        "## Build the model\n",
        "\n",
        "Our model has three layers:\n",
        "\n",
        "* `torch.nn.Embedding`: The input layer. A trainable lookup table that will map each character-ID (class ID) to a dense vector with `embedding_dim` dimensions;\n",
        "* `torch.nn.GRU`: A type of RNN with `hidden_dim` units\n",
        "* `torch.nn.Linear`: The output layer, with number of outputs as our vocabulary size (`len(dataset.uniq_chars)`). It outputs one logit for each character in the vocabulary. These are the log-likelihood of each character according to the model.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YVULa-WP7ox6"
      },
      "source": [
        "For each character the model looks up the embedding, runs the GRU one timestep with the embedding as input, and applies the dense layer to generate logits predicting the log-likelihood of the next character:\n",
        "\n",
        "![text_generation_training.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA0oAAALaCAYAAAAPy6MaAACAAElEQVR42uy9b5BcV3n/eTdWYsUWRiYTmJiRPDhyaogVmGBhjY0Mg00sgwwC5NgYGYYggohtECBLYyOw5JLBSkSQQQQbZCwHYWz/hEsUgihgBy0o4BDBT96IjZLV1k5tqTaqiqpWVasXeqEXd/vbuk/P02fO7b490zPTfz6fqqekuX3/9e1zzvN8z3nOuUkCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0DjrSnaqZCnWsXayZLdTprA2LlNA/cOofwAAM85ZnE7XOFbKFNauZQqofxj1DwBgxsHhdI9RprB2LVNA/cOofwAAs+dU0//vV1iH2WwLJX4DyhTQpmPUPwAAnCqGU6VMUaaANh2j/gEA4FQxnCplijIFtOkY9Q8AAKeK4VQpU5QpAjXadIz6BwCAU8VwqpQpjECNNh2j/gEA4FQxnCplCiNQo03HqH8AAIBTxalSpjACNdp0jPoHAAA4VZwqZQojUKNNx6h/AACAU8WpIpQwAjXadIz6BwAAOFUMoYQRqNGmU16pfwAAgFPFEEoYgRpQ/6h/AACAU8UQShiBGlD/qH8AAIBTxRBKGIEaUP+of9Q/AACcKoZQwihTQP3DqH8AADhVDKGEUaaA+ofNSv1bVrLdmW3Ptm1322S7SrapZItzzjGS7bemxrljNuD2Hc6uc7Bke0u2OjiX3VOP29aTbRtt4PuuKtme4Dpz3Od2b2JljXtf6a6fZ/W4O9vvxmD7XPfc7d7mlWxdyfZl967Pluec1z/Lp7PvHGNxdp1Hg2eQZL9N+H12lGxtdn9F0TPaWrID2f3sDMrRMvc8Y2Vvd4O/Saw8A3SOU/3Q6neW7fGvfm7Wr3fi2D/gVNu8TB360eOV3/jU2Auzfr2x33yfMgVd2aZ/7+m/nfXrUf9yRY7Oeapkh7JtY8H1zM6WbCg4XgH2SXeOuZFz59lwtt8mt+2E+//T7lx2T/1uW3+27WDB77rJfY+x7N80ExVhPRKba9z7Znf9PCv67PcF21dk25/P/u4t2bFs27ngGe2ICMrYs9wVuf5O9/mKiNjK+15Fn/dcd9+nXDk5U7IlwTPYXKfsFf1NVmXnOJf9C9BZTtXuSY5utq53+sTB9N5PfCBddMUCnGqbl6knH9tSuaeZCJLyrvfiP+1Oh974J+mW+z9KmYKubNNnquzHrqc2/WNrVtGm1w7WRyKixIuh0Wzbo8HxK13wq39vz7nO5iAg9iMKaXbNATeaYUH+jU0USgrWT7tRqT533z0RoRSyNvvsZHZsv7v3yTAvE2tns/8bu7Lz2qja3uxv/TvfjQYdD/Zbnv2t7Yvcfqey7UOBiDntvv/+HKG0OxgdOpptX1Lg+90eEWmbAxE8klMu7PkcCcrd5kh5jTGGUAKEUhNs+Pqry7btwY9Xtt38tuvK99C/8DKcKkKpIfvHfTsrZerk8R9VRiZnOlhEKNGmd6tQsvqnum/b3vbWa2jTpyaUkiwY17aHg+373fH698UGhdLuINj3gfpyJwyaIZRspGF9NkojBjNhNqeOUBrOjj3nBMdUhVKSCQb//edkwsbEU2/2+elkYsrbiuD727nCVLsbs2c5L/J7bs1Gfc5l4q+WUEqyUa7UCbFamIg+mt3D3OwelrnnX0so7XHfbw5CCXCqBYTSmZM/Kwe8RdKoFKhOJn1OTraWUz37378o34NM/yeobW+hpDJS5Lc89//+sryfeqcbuQcdUy9YVFnVfq2c7olQok2fDqGk+jTbbbr5lS5t02sJpYOZHc4C6SNJ9RyhPrc9cb3/ixsQSi/WOCaJ3NMJFwSfaFAorQme47FMKPRE6lESCDJLG1sTEWrn3D2ZFZ0fY2LieSdq0kwkeMES+47z3QhXkoynufUXuO7BbF+JxE2R32bYndvKgf0Gmwp+N4mbQ0l16ubzyfh8pFpCadT93r2RcnQq8syHEErQtULp8E+/VekVNBt83R9F89C1rz6z/ZRuobkjy64dLDvKR/56fWVf/S1Tqp3+1j5z5/5O+bg5cy6o+kxO+t23DJe327n1f22z0QOEUnsIJYmeL37+k2nfq19Z+Vy/u8pd7Ld86HN3pfMuvqjym7/v1pvK57byY/t9d8/fVLYpoFPKnb/G/Je/rPyZtmv/Z3Z/oVw+/bPr+b355XtDKEEnCyXVAbW3/r6X/Okfpy98/2sTzqF9B/6ov7Kf/q9tVtdU78I23dr5Wm262gT5Fd+ma99b331jN7XpRYSSzec5GwgFC7DXZ3+vy/7e2YBQOl5whGLMjVjZfb3YoFBKsmB6txM+Jpjm5wilWPpXKJTOunsyK7rAhI0gncsEm6XdLS8glOa46+eNusUYyPY77ASviZI5OULpsBOmR4PRp1poFOnu7Bxnk+r5RHlCaXn2PM5kI36xcnQs8swHEUrQlUJJDtEcnQWb/v4fe+QzVb33FtCamNL+2mbbvcMOrycnGjikymdy4nZ99VDKAZuD1WcEte0jlO68/R1Vz8iXmd5X/V5VkCSR5PdTmdLv7o/Ju97BH349OulU21Wurfws/uM/LJcpL5pmakEThBJt+kwLJaWoenHi65K2f/vxrZV9j/3qu7ltum3zaXbh9Wq16ap31jmh+qd5hHZfElAIpSp6k4nzacZcMD3mxEcsTSxPKB1I4osJLMlGV+bUEAH9DQgl3c+yYORqsRNbt+cIpVj6V3j9qQbkj2bnWZuJplPuWoPJ+LyjJPKMTLgkbvRmWbDfcGZ2zoeT8XllJijOBb+DCaUw9W5TjmiM0Z+dZ577DVa7a8eE0qKs/MTSMX05IvUOEEq2zYJHOUeNDGnb8Zf2VXoYJaIssNWEXTuH9TIqleKWm6+POuzwehoFkLPUNo0EWEqIn2fiey/loHVu9VAqfQOh1PpCyYsXjQZaKp3fVyNGlpJjIl1Ble2rUUsv3vOup7KnMmvbPnnX+yvbVQ5NmPmRLl1bptEmhBJ0mlBSGbc2XWVfdcnadNsuEWSpeL5Tw0abVC8tna6eUMpr03W98LwydVBYm95KaXgtIJTmugC2NxlPETsR9OqfyAlk84TS3cn4QgV+pORwIGCmKpQGnajwgmdHtv3uiFDa5IRQb44QaIZQsgUtbNGFcCW7E8nEhTL0HfZl27cH97sn2M8WYDDheTITRv53s+e9v45QylupL4aNjq0LRuhOZ9cPhdK8ZDx98OGccyKUAKfqhYucqG3zCy7I5NzsM0uzMPEUjvD489QSSnn57AqQradRTlyBrEayjv7Ls8wnaTOhpN/aeq7D+UYSTvaZgjH1fMeCMdlHRt5dVyjVmqOk9LrEpYdKRCmVtNE5UAglaCeh5DsqvvrFjVX7qvyHmQIa7dHfapf9vv48tYRSXpsuseRHtKxN1wgWc5QqosS/o8YCWFtC/OmcXn9bYODFgkJprgvkD2WfvxgRNc1YzOGwu7fNWSB/Lhsl648IpXPuvsJ39oy6659J4u/1USrdUFJszpJfFnso8kxtIQkJSs2rOuKEqs2xmh/8Tn4/+z1W5giduU7A9DmhdNx9n73J+Cp5IwWe9zJ3349mz/xwIIy9UDKhdy4Te7H3bm2u85sglKC7hJIPPMPcdQWy9pmCVm0zMaNeyHABhqkIJdl9n/5QNI1K4oze//YRSrUmdtsoj0wBkz9egVnevpMRSgrU/LyLJJj31moLOyCUaNObIZRq1alYXUly5q36fScjlGQaNcpr02fqvU8tLpS8jWXBbW8WmJ/NguYwxc6/V2lxAaGUZOfbk4zPYTmXBfK9kXuailDqTcZX6fPzk1ZE6lGS1H5H0sGk/nuU+p3g2Fnn3rYm+Sl2YrkTPTYvaV8yca5Qb/Y7nXPPcq97lvb9Yy+h3el+o+HI9zmTPa9GXvC7Oql+n5Pdz/yIUNqc1H/v1uak2PudEErQHUJJAsS2qXc/XC3JPrvno7eVt9n8JaVOhKslTVUo2QtFJcqUMhJWUksLRCh1hlDS/porYX//YO8jucJ5MkLJxL561TUfwqfyJVmqH0IJukko+XQ42986vywd1s9dmqpQsswEtf9hm67rWlpgFwql2WJuJi7mTfN15mXX6Zmh76Xgfn2TzjU/u/e5BZ/l3BZpn3pn+H4QStAdQkmpbbbNVioy8yLKHKXloiuVwqcwKRidilDSiNSRf/5OeX6SzUWSU1c6YNKC78hBKOULF4lq26bf0O9vq3BJcGsuhX7zUIzbPAu/suJkhJLEu8S1jZSqjClw9HMvWmlUCaFEm94MoaRFTGzbZzeuqdpX84OSYC6o1TMJGT8P1KeuTkYo6VzWpttcJIkv31mihVy6SChppODpNi7/SvPaXcMGZum+NDfq8AyKsplktMbz3j5L97Q8GV9hD6EEnS+U/Gpz6uGTc1RwK5FkPYCaoGuOTjnmdg6JJjleOT6/wlI9oWSThyW25NTVq+hz5zWXRIFyOE8qzLdHKLWmUPLz1ZRiI3EiQe4XAtFokZ3DAjWVIW3XOcOl6mtdzy8EomWHVaYkkuwcKmfWc22LOSTZIiVdMpkcukgo+bmkqlNqN2301layk6Cxsq/5p3YOe5GsBJZv0+sJJatr6gCxNt2PFvt9NXKc5MxL7ND6t6wFgttOFkqdTCsKpU4pz4BTLS6U1MsXS3WzINOnvCnQvPlt103Yz/fS1xNKcsL+WJ1P2xXkJm6Jcn9Pcvysetc+y4P7Jb9jZcULFAVVfiliK3e+nNW7nk1IT1wan3qz/Xkl+P3frdSbjVCiTW+mUFKdCl/z4NtWn/KmNj1835J1OhQVSuFcJHVG6LzeL4RtujpI6KgAAMCptpRTleOShSvcaeK7nJ3mbdhLAzVXyAej3rGql1LzlGRyoj79yQegsevpWppMr6BVAa6lXOm86t3USJWcqnr8dT+6r1ZbqQyhNP6+FvuNwxdIajRQv7OCI/3WGrnU72ujheHEcY0k6jwS1RqBii3mkHc9CSMruzaKZedVOdaqd/pM96JjW21xEAI12vSptunh6IxGWlWnVB9U9lUHNKoba9MlWFQ389p0f+7Y9VQX1bGheq46Zmnc9uJp1X17N5O16a3U8UX9AwDAqTZtBEFCSGkV9h6OcOKvf5lhpxpCqXmmoElBWriYiM11UnDV6eWJQI02fbZM7bW16b5DyqfQttoKddQ/AACcakuaT5tTD6J68uVgbZEH9Vy22rLLCKXWNp82p4BNI1HqhbY0OY1KIZSANn36Oipqteka2fedYtQ/AADAqeZY3rtpkhad94FQan3Tqlh+4rg3pfGEK+dRpoA2vXmmtLlabXorLaJD/QMAwKm2vCmfXT2OSo3SfBKZRpo0eb4bHCpCqfkmMaRRJCtPmlekFRZbbW4agRpteqe26Uqr9m265ghqriD1DwAAcKoYQgkjUAPqH/WP+gcAgFPFEEoYZaouQ8n4G9jNirw741BwzKECx6yLXGuINh1DKAEAIJQwnCplijLVaoHacDJx3sruAseNBccUeYP75si1hmnTMYQSAABCCcOpUqYoUwglhBJG/QMAQChhOFXKFIZQQihh1D8AAIQShlOlTGHMUcqDOUoY9Q8AAKGE4VQpUxiBGm06Rv0DAMCpYjhVyhRGoEabjlH/AAAAp4pTpUxhBGq06Rj1DwAAcKo4VcoURqBGm45R/wAAoLhTxTreKFNYu5YpoP5h1D8AgBnnHM6mK+w0ZQpr4zIF1D+M+gcAMONsx7F2vJ1Nzi9pTJnC2rVMAfUPo/4BAMA0M69kR3kMQJkCoP4BAADAOCPJ+R65YR4FUKYAqH8AAABwnkOZU93DowDKFAD1DwAAAJJkIKnO8Z7PIwHKFAD1DwAAoNt5OKmeELuWRwKUKQDqHwAAQDczp2QnA6d6mMcClCkA6h8AAEA3syKJL7O6mEcDlCkA6h8AAEC3si/Hqe7g0QBlCoD6BwAA0I30JvkvYDxVsrk8IqBMAVD/AAAAuo31Se23nK/iEQFlCoD6BwAA0G0cq+NUD/CIgDIFQP0DAADoJobqOFSZUjj6eFRAmQKg/gEAAHQLuwo4VdkmHhVQpgCofwAAAN3AvJKdKehUx3hcQJkCoP4BAAB0AyMFHarZMI8MKFMA1D8AAIBO51CDTnUPjwwoUwDUPwAAgE5moEGHKjtbsvk8OqBMAVD/AAAAOpWHJ+FUZWt5dECZAqD+AQAAdBvegQJQpgCofwAAAIBTBcoUAPUPAAAAcKpAmQKg/gEAAABOFShTANQ/AAAAwKkCZQqA+kf9AwAAwKkCZQoAqH8AAAA4VaBMAQD1DwAAAKcKlCkAoP4BAADgVIEyBQDUPwAAAJwqUKYAgPoHAACAUwXKFABQ/wAAAHCqQJkCAOofAAAAThUoUwBA/QMAAMCpAlCmAKh/AAAAOFUAyhQA9Q8AAACnCkCZAqD+AQAA4FShmfSXbFPJDpTseMlOB88faw/T73asZLtKNkSxBtp0AAAAnCpMjp6S7S7ZOURGR9qhkg1QzIE2HQAAAKcKxVEAPYaY6Hg7W7IRijvQpgMAQDcwLwt89pfsaEKaFGlSjdNbshP+ft5982D6zKN/mY79yxfS08d2pOn/83WszUy/26F9G9KPfeDN6dwLfzssb+toOmnTMVJfAQA6GTnTUzgk0qSmyF677ryLL0z/8alPIDQ6zI799MF08KoFYRlbTRNKm46R+goA0InswPGQJtUEBv01NYqEsOhMO/N/fCVdds2isMd7Pk0pbTpG6isAQCex1Te+/Qt+L33kwdvSo/+0mTQp0qQmHZzd8rbX8Vt0uJ06+rdp7ysv8WVrE80pbTpG6isAQKcw7BvcW1e8odxTjGPq6DQprUJ34zSVp6N2ne89cRfPvwvsyUc+5MvWQZpU2nSso9p0AICu5ogfATj3fz+KI+qONCm9z2jONJSns3aN4z/fyrPvAtPv7MrVGE0qbTrWUW06AEDXstwa2fmXXFROo8EBdVWa1IppKFOV87fK91a6yoduu26C3XfPzekXH7i1nI5E+ZhaSlBSPU8JaNOxzmnTAQC6lketgVXQiOPpfPvsund4p7qzG4RSkBoWNQknet4nbwnvsKFNxzq1TQcA6Foq80le3D+K0+kCe+HZT3mnerjbhNLAot50+No/KtvQG66omhT9kfcvo4wglGjTMdp0AAAoc84a2FbqTT/43fVRO/Ljz6Zn/6+v4hynYHrhazK980laWijp//6zE7/aVhZP9vnhf/hM9Bwnj2xvaLUw7Vsv7UlzDPR7FJlor3PpHhBK0GltOiO5Ld+mAwB0LS0X1EaCripT3v2W9e/EubbufJK2EkoyvRDXPv/kX95Y9ZnmMPX9waVVI1Lf/cbaqn207LKWX9bEas138hOstb/mSPn9dfzigcuqyrX2+/bOD0+4N72HalH/Kyv76V6++vk7EErQUW36ts+8l/aZOYIAADjVqTpVsztXLcVJtmZQ23ZCSaOU9rlEjm3XHA/b3vOKeWWLnUfC3QI+2byLL6yaYK1tNmok0WTbJYCUAuj3lWiz8z7+xQ9WncPvp3kJCCXopDb9nr94K+0z9Q8AAKdatNHXBHsfzCqI9L3rP/jWPThJhNKUhZLMRIhGhiydZc6c3ypv0wseNYIpk0A34WRpoCaUZO++ebCyXeXXttuoks6lvzWi5FPwJJh0rEaQbJvEkfZ92/WvrZzTJm7r3pQ2SKAG7dymqw33HQBKx6ONpv4BAOBUG3SqZppDkrh3hITzODQKYJP1dbxy4MMX9dny0NpfgfPNb72qsr8+D+eRKC3E9rH9wnQqv68CW+33vpVvLE+0RSi1vlCSQPJCSel0dowvE0qtC4M6L5T8UuM+pc/S6u792E2VbVpMQsfqPOHcO72sNzbK5Ee/Hvvr1QRq0PZtusp/kpMpoM4A1Rlre7XgSricf602Xfur0yNMVQvbdHVghL7C9n1odGWlTdf9taKYo/4BAOBUK2aT75Xi5F96GbxPotLzbr30oVPWG+uTSL68TZrXCELwQr+q8/oRLTlovyiANwXDCKXWFkr2ud5270d+kmxekASUzM9XsnN5oeSDMl/WbF99HiunKssK6mwBCM1Dss+0v13fBF3SYss/E6jRpk+lTbc6oX+9AMpr09WREKtnsfZa57CFWNTREM4P9Of1nRLyA3ltujpSEEoAADjVlnSq6tmzfSywtG1KiZKAUa+jUpksCDXx452qtmseiPVIhk5Q50lc6pSEk/Zd8vrLy05VQsvuyY7XctMKirWfD7ZjvZUIpdYQSn6U0sqc/+2s1zk0C6q8UMrrKffXVVlUL7XKURiA6bzhiJb2i12/lSbAE6jRpjerTbdt6rQwoaO6pjZdo0DWoWXiJ2zT1TGm9td3hKmdt8VRTBRZm67zKqVb25QFYNe34/05rV3Qvuqco/4BAOBUW86pmgCyHnw5LPtbq5T5tAl7T46lKXmnqmDVp3iEI0BKl7JtcpCWIqXz+lQp/d+uo/38vdqcKqWPIJRaUyjZvCOZ9VTnpd5JmCuw8qsuNiKUVHYkzGwZcv2ta1pQaOfwqXd+5FJlrRWXySdQo01vllBSCrPKeChyrP6FdcrXM9/+q96Gbbofqa3Vputvm6Po227VexvlaqUFVah/AAA41dwRJS23bH8rl9zy1WXqDfQCxjvVcJnn0Kmq598m1CcuPUPpHXK4Fix7p66A11/fnKruC6E0+0JJIlu/r0yBjk/V0Zwh/24SC5RsgQa/mIOEsS2m0IhQslEklQs7Xue13muVNxNEVvZ0jI2I2mIOiVsggkAN2r1N9/UwrLMaRfJtqnVKmYDx9czPCfWiytp01TnzCWY6n3yKrmltul+dUvcWa9PDObLUPwAAnGpLOFWbp2FBpXeqeWbn8k41nJSbROYUSQTJIVrQnARpUnKs/px5ZilVCKXZFUp5pnkL4QtdvShRMOVFs58f1IhQ0jYrS/pXZdmf18998COa2tcvT+5ThAjUoN3bdCvbmgM43W26RnMlvmJtunVo0aYDAEBbOtUX949OWCHJry7mJ/nmvSG+EafqV7TTudWL6SfU63582p5P/WB58Nb4viofefOMNJLje5JDk1ixOWkmqMLV5nS8nc9vl8gO5zNZoKZRKgsOJcLUa+0XHfH3rs+sF12pnK34wmUCNdr0ybbpGgWyz7Wgib2UOYms+tisNl31x9Je9ZJpv0iL6q1f3bKVVpek/gEA4FSrVqVTCpRMwaVy1S3tQYGrLZAgEWOBpAJQO4/SJxSMKlC1lIxGnKrmMEkU+cUg/IRgE0p+LpICact1l4DS/Sqly5aHRihhvMcFaNPPt+kSIb5Nt6W/lTZnnRN+9FSiSftrRMhSTxtp0zVKLFHkF4MIX+5sfsXuSx0V1jGh+bA6Vm16mLpN/QMAwKm2zFvcw5EbiZrErRKm0SZLaZLDs9XxGnGqcsTmrOVc1fOoHk8bCdDSseZAvXiSaFLPqfVSSsSF7/5AKGEIJaBNHzfN+/THhO8cU5tu84vUtsZWvavXpvvRK7XT1qabr7DXA4TiSW29n5+k+wjfz0T9AwDAqc6qU7UUJb8KmDc5QxtZStziCn7VskbTNHQtn5aRuAm+4fKwcqzh4g86tl7qCEIJQyhBt7bpmhcUayPVCSWxFLbp6gibSpuuTq3Y+5mUeRDOU1SHXNimS2C10kIq1D8AgC5zqpaa4S10YHmmNDw5MTnM2LuLlBZn5wyXWLbtPiXDzzfROWW1ehJ1TqXjaT+llrTaXBKEEkagRpveCm26jfLXM7XH1vZ6gTSVNl3t8mTbdOofAABOFSOopUxhBGq06Rj1DwAAp4qzwalSpjACNdp0jPoHAAA4VZwqQgkjUKNNx6h/AACAU8UQShiBGm06Rv0DAACcKoZQqvOiSq2GJYst4IERqNGmY9Q/AACcKs4Gp9p1ZUoCye61ld6JQpkC2nSM+gcAgFPFEEoIJcoU0KZj1D8AAJzqZM2/K0PvSsI5IpRmSyip/DXyHpii74o58atthfbVO2CK7kugRpveqmb1iDad+gcAgFNtwnwSu9cnH/kQzhGhNONC6R+f+kQ6eNWCqufX+8pL0i8+cGtln6E3XJH2L/i99N03D044fsnrLy9/9qHbrqts0wssl12zqHK+eRdfmN77sZuqAke97FLHyV549lNp3x9cWt53Uf8rCdSgbeuf2nHadOofAABOFaGEUGpzoXT0nzancy/87YqYkWixv2XPPPqX5f0kcmybH/U5tG/DhPIrkaRzaZvOpXPaPsPX/lG07JtIkr1v5RsJ1AChhFH/AAAQSgglhNLsCaXHv/jBdGBRb3kE6eSR7eVt+tfE0if/8sbytmM/fbByzm2feW/leH1ugshGi952/WvL2ySQTFT94Fv3VI7/7jfWTij7Elbf3vnh8n4aaSJQA4QSRv0DAEAoNeRUNY/jq5+/I73lba8r987L1AOvANP2UcqU0qA+9oE3RwNjffaR9y8rn8s7dzun/tXf/nOZjpNpFEHBsva9dcUbWmZeCUJp6os5aE6RhMz8Sy4qH+/T6ZR+p21K07NtNhJ056ql5b81v8mufd89N09I0fPn9GX/nr94K4EadKVQ0jxVtdk3v/WqqjZdKbG2z0OjK8v1JlZP5A/C9l5t92N/vbpyTrXp6ogIj/Vtuq6hfVWXmzlPkfoHAIBTnTGhJIcXOI8Jx8tx2jY5QH+8BbZyoLZNYid2Pm33Ysm2W8BsIwFy9Ail9hVKKoMKlnyKnJkXSgq8bPvxn2+tSrvTHCObd2TbJLZsDpJP6bP0O1/2VWYRStCNQkn1Ia9Nt9RXdUzZtiM//myVIOp5xbyqlFVtq9Wmx8q3n6Oo84WdZNQ/AACcassLJQWjtu9n172jHKzqeKVNaZtSnqxX34JSjRzF5pOYA/ZO3c6poHXOnN+aEMD656pracRAx5B6175CSb3Mtu/igcvKc5G+98RdFUHthZJGm6xcqfdZv32SzS+ywMoLJaX0WQ+5N0vna4e0UwI12vTpFEo+JVViSPVV7byN6KpjTPtp1N7aZD9S632CZRU88uBtlW1qC9Sm+23+nvxz1bW6rE0HAMCpdtqIkuaP+JQMBagaHUqyOSHhKJHvHbT5JH4UKJZOJVPvpI0ehU7LB8bMUWpvoSRxpP20Qp3fbr3UXijJlJZj5UWr04WBm8SUF97+WC3yoM/baX4egRpt+nSPKEkE2YispeJZmqpf/MTaebW/tk0dYWE7b6ND/lifjeC3233Kd3Rhmw4AgFPtxMUc5FQVnGpUx1YXC4WSxFQSpEXZKIEfZbLjFfRavrrMnK1GEEKnFaZvIJRaVyhJHOvv0NTDrH1N7EgwmXj2gV4olHwPtplWzvP72GIOCt7sM18e7doIpYYYKNnuwNYUOG57cMz2AsesjFxrgDZ9eoSSBIrqlUZzwzbdixrNHUxcSrVPu7NRWl9mwzbdOkV0TLivzTGk/gEAIJTaViipN956Gq1nUSM/Jmq8UJIT9cLIp935eUtJTm587BnmBc8IpdYVSnlmZcUv+63yonQ5G3VMgrlsZn4uk8pj+LnmUPhgz1JDkywdz0aVWl0oaRU/98zOzHKbNhz5HXcXOG4sOGaswDGbI9capk1vvlBSmrQJGKuXatOtjnmhpI4ML4x8p4WftzSZNt0LLYQSAABCqS2Fks0JUa66jgvT7LxQkmnUyXoQLe0u3MeC2CKjRAil9gnUYnODvPmJ3xJLNidCQZvmr9k5JJTCxTq8ELPRodC0nLjKlJ1XZVCC3a+mpeDO7senk7aKKW2xQYGBUKJNb0goWUeFRu6Vmhou8BCmz2nVO+vUsLQ71Vm/j3VSFHkfWeLmMiGUAAAQSi0tlLRErIKzmCmglUBJgnx09Xpb72MogjSJ185tk/BDh2hzTuRcbZlvnVujVDI/z6TLhdKxWO9tN5qVGQn2VllGeDpMi1q48nQAoUSb3qhQqtWmh2mqvk23DqxQKElM2bltcQddw+/z7psHK6tO2rvR1NkhQaURYC3EglACAEAotZ1QqmVyrH6JWDlQjRjZHJMkS3EKz69J+v48Ek9hz7/1QMpZKwj2S8Xa6nitLpRmIE1qt53fBxrdYionCgB9qp4Csk7+ztZjn9lmhBJteqNCKamT/uZHZ20lUZ/aGnZ++cUaTCyF77FTR46JKPkEtek+vc9e+NzqQqnFUl8BABBKrS6U1CtoKx8lbo6IehTtbwW04Qtm7bNwZTO/bLgXXNYbqffmtEvq3QykSa3yglROvJuEkgSzLx8S12FZ6yRT8GmjsJktmeU2bW7J+gPrKXBcX3BMX4Fj5keuNZc2vflCSe2IjSr5TjB1xtjfoRDyS33bKyFCUypr+F40tenyB+2Setdiqa8AAB1F26RJSfzkpWaEqXd2jFYSk8CyFcX0me0XBvB+tbFQ+MQm4eu8Ek4xIWDXaMV0qxlIk5pTsuN2Db9yYDeYRiI150FCXaNK4chkJ5lfcj+zgzSptOmNjIQUadP9MY206b7TQu9Bq1WOlaqn8+r9ZrEXg9s1/LL9pL4CAHQ+XZ0mJYco56pg1lIu1DveyfNJZihNapXvoVVKS7eNLHVD3bHFUTI7V7IhmlTa9NkulzKN4trov0aIOrn9abHUVwCAjqKr06T8Ig5mGgUgTaq5AVuSvXhVI3WdnIbWDaYedfXO2/LoBGi06a1kGkUL2/RWXYChQ1NfAQA6iq5Ok1LPoy3LrKBCIqkV37repmlSKlt7kwJzELC2tx00pbTprWDKBrA2XUuDazVS2nQAAGhKD2RCmhRpUs0P2jTScBYx0ZF2umR304TSpmOkvgIAdDKkSZEmNZ30Z9c8mpxfvhaR0d7i6HDJ1pesl6aTNh0j9RUAoNMhTYo0KZg6d2fPeAWPAmjTMdp0AIDOcqykSZEmBZPnUPas9/AogDYdo00HAOg8+hPSpEiTgkZZ5J67AtN5PBKgTcdo0wEAoB0hTQqayaYgoFnNIwGgTQcAAGhHSJOCZnI8EErP80gAaNMBAADaDdKkoJksSSamyGi5XlJjAGjTAQAA2grSpKCZ7Eji8wnW8WgAaNMBAADaCdKkoFloZbGTOULpMI8HgDYdAACgXSBNCprJjUntVaoW8YgAaNMBAADaAdKkoJnsqSOUtvKIAGjTAQAAWh3SpKCZzE3qv5fmOI8JgDYdAACg1SFNCprJ6qTYSyKX8agAaNMBAABaGdKkoJnsLyiUdvKoAGjTAQAAWhXSpKCZ9CTnJ4wXEUqnkvMpQgBAmw4AANBykCYFzeTuguXJbAWPDIA2HQAAoBUhTQqayaEGhdIeHhkAbToAAECrQZoUNJNFDYok2dmSzePRAdCmAwAAtBKkSUEz2TQJoSRbzaMDoE0HAABoJUiTgpnAlyEAoE0HAABoaUiTAoQSAG06bToAAEAAaVKAUAKgTadNBwAAIKgFyhQA9Q8AAABwqkCZAqD+Uf8AAABwqkCZAgDqHwAAAE4VKFMAQP0DAADAqQJlCgCofwAAADhVoEwBAPUPAAAApwqUKQCg/gEAAOBUgTIFANQ/AAAAnCpQpgCA+gcAAIBTBcoUZQqA+gcAAIBTBaBMAVD/AAAAcKoAlCkA6h8AAABOFYAyBUD9AwAAwKkCUKYAqH8AAAA4VR4FUKYAqH8AAACAUwXKFAD1DwAAAHCqQJkCoP4BAAAAThUoUwDUPwAAAMCpAmUKgPpH/QMAgMmyrmSnAoeCYWYnS3Y7ZQqbxfIE1D+MOggAMCucxXFgBRwrZQqbrfIE1D+MOggAMCvgNLAiRpnCZqs8AfUPow4CAMyuU/3P/z6KYRVrhlDiOWJNKE9A/cOogwAAOFUMoYRRnoA2HaMOAgDgVDGEEkZ5Atp0jDoIAIBTxRBKGOUJaNMx6iAAAE4VQyhhGEEabTpGHQQAwKliCCXKFEaQRpuOUQcBAHCqLWL/duJX6QPbPpNe9+ah9NULLivb5a9ZmL7t7TekX/vWVybs/72f7E2vedMbo6Zj7hi5Ld3xje3pv//XS1XHrXjP28v76N/YfdjnI2s/gFDqgODiX/7jZ+lXnvhSOrrl3vTjG+5Kv/i1bemPf/nD6L4/+fWP0j37nojasweeKp8rVm5tnwM//37DnxOkQSfXP7Xdy995U7ktt3Zdbfymh0bLdSPcP69Nv/6GZel737ey7CMOH/9F1TH3fu5Tlf1+/puDE87pP0coAQDgVNvOFEBeObCo5nsjJGC86FHgmRR434SctL+WHLW269/YvdjnXexUO6JMKQiT2L3wwguj5UK/709feqHqGAmpeuVp8OrXVQkeiSv7TIFcTHzV+pwgDTqx/h0Z+2W5w6pWXVKbHwqbIm26RJfvtFC9ss9U38J78Z8jlAAAcKptZeodNHFigah6G7ftfChdc9eH0osuvqjy2Sfv/3hUKMkhK8iV3fXptem7br2l6jiNLCGUuqdMSVBfvfQNVc+g97JXlX/bC+ZcUNmmv33vdBGhJLviytdURDtCCWjTJ5oXSapnqltq0zW649t7jS7FytlrFw9U2nTZn9+5quo4ZQwglAAAoOOd6kc/sabyHe5cc8eEz3/ws32VUQEFuzGhJAccHvf1p/4u6lQRSp1fpiSW7TtIMPlUO/VgexHlxbcXSmHApb8XD15V+fybzz6GUALa9IipbviOL40uhaNN6mywfXz9rFVXNIpkvkDHI5QAAKDjneqlr7i0IoLC+UQ+z13O16db1BNKElgIpe4TSgrCbDRRv2UYpFnAZfPfHtz+QCGhJPv8jgcnlDmEEtCmV5vSne075M3L05w/tevhfMF6dcX8BUIJAAA63ql6MdNoEBkTSpqXIkcp5/zWm95C6l0XCiUFX3b/Ej6NHFtPKGnEkxElgjTa9GKdX3ltbJFy5uuK6pDMjxSTegcAAB3vVL3YCYNaBbw+R93MHGHRxRwkmPxIFUKps8uU5kDY/cdWSywqlNQrrgDLTOUhcZPQmaMEtOm1y0rYhmql0librra8kcUc1Eb7RSAQShPoLVl/ZvNLNs/9bdZb5xy235wa5w6tL9h3bsmGSjYcud7c7JieYHtPtn1eA3VG33FZznXmu+eQ1Lj3/mD/mM0vcC9zCzwX/z3nBtv73L0kNZ5r7NnV+y1rnWNezjnnB/fZ68pFI2VhMPt9BuuUVQCcap5QUm9hnnPzZk61nlDSPBQtBR2m8yGUOrtMebHjAzD/24bWyGIOWqbYB2kIJaBNj5cVtcF+u0b+Y3XKd5IldVa7U0dIuDw4QmkCB91+m0s2kvNMT5VsZeT4QbfPqhrnDm3M7bemZKeDz3c7ATTstnl2Z9tHCtaXrSU7G7mOCbzN7jkkddp3v3/MNhe4n+GcY3WPj0a+557g+LHg97Xjb4xcY3eDbUpS4xxWRk4EItWex3Dw+/cXLAsqS8eDz45nwjZWVgFwqnlBZLiMt3/3hQ9wY0JJ78jRufS+HJufovSP7+z/+wnXtGXIEUqdWaa0YqLdv4TyZIWSlhaX+WNUzmJzoooKJa3chVCCThdKVmfUBoeLPFibrlXtagkl+QPVHc1lsoUftGKl3qNUKyUWoVQVfI5kgaoFwUeyYFS2o2TnsgA+7Mnf6a5zIPhsxJ3jtAtwZeuCoPtkydaX7G4XLO9oolAayvY9VrK1mTg7lm0bzRFKmwM7FgT2tv++yL7DDQilMXfc9kyUavvq4HuGIihPKB13ozrTKZRkDxcUSrGysCMoC4ezcrY123+n+z5ieXY9hBLgVPNMizjo/iVwwp7CWqMEeYs5+Dkql7z8ZRMcp6VQhU7czIQWQqn9RylDYaKgy17+6gO1WnOUwhW6YoGafaZl6WsJpUbnTCGUaNPb0VQP7Ds8+dzjDaVdJzlzlNSWJzkptb7eat5reC0/X7XLhFISBMFhYL0/274kSMk6nY0sHHFBcYyxnHs5mQXHi922vmz/3U0USiORwF7iaa8L1DfXCMJXZPd5JhlPCdvc4IhWnlA6GGxfn23fHhFKx9wIWJ5Qkm2aIaHkf7taQilWFvoj1z4VjFLtyq7bE9wLQglwqjHT8sy+FzFMlVOg6h1dPaEU9iCGgsd/ZhPyzTQClUQmCyOU2usdSj2v7KmI71gPsxfM9YSSza2w9y/pXwkufy4L4jRaWWupZI2SIpSg09t0346qkyF8qWxY1+oJpTBtT51cfgVU/5leNxG+p88Wl1DqXpcLpQNZULoiExISCEeT6nlIt7uAfl1EiNQTSpa2d7igoDiU3Z/ZoQaEyuIsqNf+L2aB9rJgnzyhNJB9/zT7zuH+u4L7GkmKzZsKhZKOWZQ9+zQb+fJC6XgggmJC6VQmXs9m55pOoXTU/S7NEErPu9HFXdmI2vyce0EoAU41ZlqpztLhzJnJ2SmolFjxPYkyC1JrCSU5RxupqjXipEBaaRu6ltKsavVaIpTax7Tkd+Imfuu3VDnTZ8/9+JkJ89+KrHrnP/MvnJXppZk+ULMlybX6oh+5yutdRyhBJ9U/mUZz/ci+tbN6ibjaeP89fQdCUiON1XeY+c8lxOz9SrIV73l7+Zya9+qvpTa+y4VSzNbmHCvB05sJkZPJxEUd8oRS3ohKnqDIs6IjOiuz+0uDEZqBGkJpXjKecrc1R1jFrL8BoRSz/cl4+pwJpVVOBPXnCCVtWxOI3ekSSpuduFnTBKHU60Sin691N0IJcKoNmBydXkxYq+HUKIGfc9LIC2fDHkif0x6zWAoVQqm9LG8xkFq/dS2hJGHkRY9/Ua3Koo045ZnKN0sTQ7fUP9UXdXTVqhMSN+pY8J0OtYTST196oaozyy/WoldA1KqD6ozLS+3uwhGl4Wzb0WBEZZEbwbBRlONJfFGHPKG0LBiRqCco/D0Nu6C6kdS3Odl1FWgfcSNMMaE0x11jf0QAbnajaMOBzW1AKNkcJXvGjwb77XYCxIugPKGUuNG2R6dZKC3KxIzKwY4pCiWjL7vG024UcDFCCXCqk3gHjgJXOTWNBCgw1d9ygjYiYKbeelu6ObZogy0IYfuEI0T6W6l+di2NEugFpO0+koRQqn5BbGwBBwVUWr1OYrqR9yj5FDwFef5lmUqxC3vKE5dS6oU6Qgm6pU1XnVHH1OLBq6raWS26EqsT1l6r7ub5CNsnTGXVtTSSZdeSqZ7rWqH/YI5SdPvDNYTtgYJCaX4yPu9nbrD9RNLcOUqbM/Hgl5yem4yvghcTSlvdqNO8nHM2c45ST/a9tW19jlBKMmGXOhERE0qWaniuQaFkI27+97AUy10RoSQ2udGfyQqlJdn+4ajdjuAZI5QAp4ohlGbTJKoVYGnkUaI6r2dZ2+3lluFcOT/6afvEzqNgza4l8RSbn4FQAtp0bBaE0t5k/J01S9zIysPZyIotwrA8GEk5VSc4TnLEzr5k/J1Ne7NtTzdRKI260au+7DvYdz0eEUornBhZnkx8989ct//6JP/dU8uT/DlLoVDy2yQ6BnKEkp9vlSeUQjFbVChZediejL8XybbdnSOU5ibj6YmTFUo92Xc+mz37JPv+Nkp5I0IJcKoYQglDKAH1D2vVOUpjWUC7osbI0fYkvqhDnlCal8TfsXPMiY1mCCVd52jkOudcYO6FUq35RyYGau1zsI5YyBNKYmewPRRK/jnXEkrz3HMvKpSGkvGFK8LfY36OUEoyITMVoZRkQiz2LPdGnhlCCXCqGEIJQygB9Q+bUaE0mFS/D+juLDBe5dKxVmSfDUXOtyj7LFz4YV2N4FajOyuzNCvNqVkTpH71Z8eGL7xdmW0fLFhf5ibj7+fZnaWMLQqC8M1udGxzDeuvs8+IExWnnehLIt9rJCLq7Dx97nv25+yTOKG3LjjXspxnV4v+bJRsV87vMZjE3xW1JrhPe3fS/JyyMD9ybY1ebs1+nx3Zfc9BKAFOFUMoYZQnoE3HZksoDSfFVmprVXqTielvZn2zeF+a3/Niizyj+TWeUX8SX62wlRhMxpehRygBThVDKGEIJaD+YTMilNo9+Iyl7/mUwdlCo289LfKM6qUT9rfRb4xQApwqhlDCEEpA/cOmtQ76kZj5jCh1NO0+otQpZRVwqhhCCaM8IZRo0zHqIAAAThVDKPEcMYI02nSMOggAgFOdLtOL/ka33Jte86Y3Vl4CqJcO6oWBeudMs67z3I+fmfHvNhvXRCidN72s8itPfKn80kmZ/t/uL3UlSIN2qX+qb3pxrLXpsma/sFvvIPvpSy/M6PeajWtSBwEAutSp6kWfl79mYa2JhulHP7Fmyo5Nb2OfyWcwG9dEKJ23I2O/TO9cc0d6wZwLJpQlbdPvon2meh29LPaOkdtm9LvNxjUJ0mjTG61/19+wrGabft2bh6ZUB9W5ps6Piy6+KN2z74kZ69Cb6WtSBwEAutipHj7+i3Ivo93faxcPpJseGk237XwoXXPXh9JLXv6yymefvP/jk76ORqZm+hnMxjURSkfTf/+vl9Krl76h6rv1XvaqqnIm04jlVAK1d916S/k8GgWdqe82G9ckSKNNb9Q0amT31/PKnvTjG+4qt+kSGb5TTGJpstfQ+ew8MyVaZuOa1EEAgC52qnKgdm8KAhXkhqNNl77i0vLnF154YXmUZjpFiwLn7+z/+6ak+yGUZr9MLR68qlyG/CjfW296S+VzjSxN9jomvGZStMzGNQnSaNMbMQkIu7crBxZNSHVVGzt49esq+0w2Da+oaJFPUfrzsweeKo8IIZQAAKBtnKoFfkpl0OhSbJ8Htn2mcv8abQqdsZxXzFFIqOhvBZWB86gEmiZm9PeD2x8oizE/uuWD7GZdE6E0vakxNgqpf2PCWvtYr7bS8CyQk0DW7yfzwZ3+b9u1j5UblVnrMdffFvBpJNT+VvnR//Xbr3jP2yvHmzXrmgRptOmtYr6D6MnnHo/u84Of7avso9GnvHbUzNpT+YuwMyR8Fl7MqO5oNNmPboV1phnXpA4CAOBUp7VRVz57rTlMtt/yd940LULJgmsF0OoFTVzKlgk4hFLrlyn1GhcZLVIKkO2nCedhgPWTX/+osq/+H/7usYBJgZQX/xq5MmGTuPlRn9/xYLSHeirXJEijTW8Vs/ZTZT/MEPAm0aL9lDEwXUJJ7brOrxEsm6+of9VOIJQAAKBthFLopPL2M7HRiGiplQbnt2vyfyyQ1v+beU2E0szMIdAIYa0FEUKx0Yho0TEmrhVI6W9LxfFzoRSg6Vo6zgJDn0LarGsSpNGmt1qWgAmMPPOdSY2KllppcH676p/NQ9Tokokl3zHXjGtSBwEAcKqzJpTUKzkTQsmn/umaFpSaU0UotZdQCn+jvHkUkxFKteYL5aWTauQqFHHNuiZBGm16uwklv+DKdAmlMM3O5ieqswKhBAAAbSOUagV+Pp9d8zymQygpxS7PkSuVBKHUfkLJ5rPFTAFUEqym2GyhFG6XaLLzjKz9AEKpuQwnE9Oidhc4biw4ZqzAMZsj1xqmTa9OvZMYqbV4gi3So5HW6RJKP/7lD6vOc9en11Y+s1FdhBIAALR872OtFe28c9PCDtMhlJSSkefwlb6BUGqPMqXVrZJgPlvMtOBCUnCOUt7v3qhQipWTvGv6uVYIJYRSO7Xpvt2zulVrRNfX02YLpXDxFKVX22cm4hBKAADQsk5VvfmJW9AhfK+NHJ1NiPdiygeSJp7CnvhGU+/8Ck3q/bd8djuPD8Knck2E0vS+Q8nmAun302hk7EXAllapMmVlLq8nWuViMkJJC4P47TpnrVGsqVyTIA2h1CqmNjtxi+H4DgCrf3qHWRJJj7O2PuzkiKXzFRFK6mTz59HrAsLz2DX96nuNXpM6CACAUJq2N7h7p6ng8qOfWFMOJPVeJRMrstEt91Ytn5y4d+XovUcyn/fuRYtWQPOO+etP/d0EMSOHqO0SYf48tq+/ppYOn+w1EUrTa345eYkm9Wqr91giSgLElzeVtVgQ9MWvbYsK3pho0e8fC7BkO76xvbL9jpHbJgRZzbomQRpCqZXMXoycZCvPaSRHbbr+tZS7JFsZMjaKr2NUV9V54OuNFy2qL4mbZ2j1x9cpdYSoPVA77UeTNKIcW6Wv0WvWmgeJUAIAQCg1xX760gtl4REJPiojA2HPYPj2dzMFweb4vGjxwXPi0ul8QOrft5G4l+DWu6YcaSPXRChN/6iSeqTzypOZxIZfvtgvQ6+gSfPh9Jv5Jb59YOQFsn5/GyXyQsl+d7/kvBc5fpSpyDX9SmH+mgRpyWDJDgY2WuC4p4Njni5wzEjkWoO06dV10IulmEkkhRkEmlcY7qe6oGyDULT4uauJS6fzQinWpstH+OvGrimBdd2bhwpfkzoIAIBQmnbHqt46CRE5MjknjRRp0rt6A/NGo9QzqIBRpv9r1EfLeUuw+PfVyJnpcxNSNpneCyWlhGgUSKNaClYldML3gOiaGoWYyjURSjNTnvT7WRqeN/VWqzc49o4XCQ8/iqnjfSqRFy36v9/XRLIf9fHlK8lS5sK5eM24JkEabXormkZOVUbVEWYdShJQtUbXVW/V/qod1r4SJxqR13n8SJBM7a3OrX3lO5Qy7YWSRohU1317HXuxuVahLHpNnT+8JnUQAACn2pHWDvOIEEpTE0wS2kq/kxBXWmW9HmCNcGp/BXm2r+ZZyMIecG1Tep32tc/CeUQKunS+PME/2WsqCAw/I0ijTe92a4d5RNRBAACcKkIJodSV1uoLLhCk0aYjlKiDAACAU0UoIZQQSgRptOkIJeogAADgVCezlK0cayuuXoRQak/TggwSSa04J40gjTa9000LpFibnvduPuogAADgVDGEEkaQRpuOUQcBAKCIU8WwGkaghhGk0aZj3dmmAwB0LedwGFgdO02ghhGk0aZjXdumAwB0LdtxrFgNO1uydQRqGEEabTrWtW06AAB0O0P39Q0v3bhgH0+CQA0jSIP2Z3jL788b2rhgbHhLMoenAQAAMAWu3dD3cMmpnrv2vr5FPA1oBktHF9xdKlPpNaOXr+BpAMwsQxsXjlD/AAAAmuJUFxyVUy3ZozwNaFKZOlQuUxsW7uFpAFD/AAAA2o6lG1/dl4kk2dk3bujv5anAVNDIpC9TSgPiqQDMUP27v3+A+gcAANAMp7qxb41zqunQ6IKtPBWYCqVytKmqTG3sW81TAZihNv18KjX1DwAAoAlCaa93qks3LjhNDyRMUSgdD8T38zwVgOlHizeU6txJ6h8AAEBznOqZ6t7HkljasJDVuWBSXLNxwZKwPGmhEFI6AWag/o1evoL6BwAA0AS0LHjEqcpOsKwsTIalGxfsiJUpxDfAjNS/fdQ/AACAJjAxl93bwhGeEDRCNO0ns2s3LjjMEwKYPjRqpNEj6h8AAEATcMuCT+yB3LjwCE8IGhLeGxfemC+8S8Ea7+kCmDaWblywnvoHAADQFKdatSx41HhZITQkvDcs3FOzTLGiIsA01r++Y9Q/AACAJjBhWfCYbVhwkCcFRRje0j83tjBIYMd5UgDNZ+nGVw/Vbc+pfwAAAIWF0t4CjjXVKmY8LaiH3tVSpDwtHV24jKcF0Oz6t3AX9Q8AAKAJ5C0LHp+rtGAfTwwKCKX9RcpTyXbytACa2Z7//ryi7Tn1DwAAoF5Qm78seMzOMQkYanH1+st68lbbitgplp4HaGYnxcKRBtpz6h8AAEAtai8LHrVHeWqQx9LRBXc3Up5YJASgmUJpwSHqHwAAQPMc69EGhdJZ3uwOzQrUtDoeTw1g6lx7f/9Ag2059Q8AACCPIsuCx4SSRg14ejAhULuvb9FkypPmVfD0AKbYnm9YuK6BtFfqHwAAQM3AtnpZ8LMlG9My4NWOdOGI5jGpt3L4k/3zeWqQR6m8bJqEUEq1Sh5PD6A5aJ7gdfcuWHz+pc8T5iwdKrfzVYKK+gcAADBRKN3XtygmgLxj5SlBEwQU5QmgxepfRVCVfABPCQAAgMAWKE8A1D8AAADAsQLlCYD6R/0DAADAsQLlCQCofwAAADhWoDwBAPUPAAAAxwqUJwCg/gEAAOBYgfIEANQ/AAAAHCtQngCA+gcAAIBjBcoTAFD/AAAAcKxAeQIA6h8AAACOFShPlCcA6h8AAACOFYDyBED9AwAAwLECUJ4AqH8AAAA4VgDKEwD1DwAAAMcKQHkCoP4BAADgWHGsQHkCoP4BAAAAjhUoTwDUPwAAgNZmsGSbSvZ8yY6V7HTJ0pmwwLHOpOk7jpXsQMlGS9ZPMSBQA6Atpz0HAAAQS0p2cIYdWpX1vemSis3mfWS2DweLUAKgLac9BwCA7mZzCziyVrQzJbud4jFpurFHm95soC2nPQcAgA7hae9MLrjggvSGd12fbvryp9Kv//BL6d5//Wb6D8ee6WjTd3z0+9vTdQ+tTa++/vUxB7uOYtIQ9GjTmw205bTnAADQ1mz1DkRO5fEfPdIVzrSWbf/2lnTBFZeFzpWeyGLQo01vNtCWt3J7fq5kyygyAABQi2HvWN+5enm6/zdP4Vgze+7XT6ZXXT0QBrp9FJuaMDpJbzbQlrdDe36cYgMAALU4Yk7jmuE34FhznGvQE7mbYpMLPdqMTgJtecvaM7/YlV5y6ct8PbyRogMA04F6RE8lpNJMxU7OcrB0o93L71z42+lThx7DkdYIdIPfjlElerQZnYRWgba8AVv14Vt8PdxJ8SGe6+A4D2aRs1SAplWi2WKn3ce7P/gOHGgdC1KoSJ2iR5vRSWgVaMsbsK277vd18HmKD/FcB8d5MItQ+Jtns8UxuweNmOBAa5vmm7jf7ABNAD3ajE5Ci0Bb3oApHdjVvzGKD3FYB8d50AoV6z//+yjWoLVIBTpj96CebhxobdPkfPebHaUJoEeb0UloEWjLG7DdL+xEKBHPdUucB1QsKlAzfkOcZ7GVzJLqF4oCPdqMTkJL+WPqF2058RxxHlCxqEA411kxGj56tBmdBIQSbTnxHIZQglmpWD/59Y/Sr33rK+m2nQ+l33z2sfTnvzlIBcK54lwJ1OjRBqD+0Za3qVD69/96KX32wFPpF7+2rWz6/7+d+BVCCahYRe3Hv/xhes2b3hidBLf8nTdFBZNE1cjaD0zpuju+sT198rnHEUo4TpwrZYmyBNQ/6h/xXJPt8zseTC99xaUTYrtLXv6y9IFtn5mW2Ozw8V+kd665A6EEnVGx1KvQe9mrKue76OKL0lcvuCy9YM4FlW2vXTxQ7pGwYzY9NJpeeOGF5f0mc80jY79Mr176hvK5NXqFUMJx4lwpS5QloP5R/4jnmmcSPP78Pa/smSCaJKR8bGad5pONzb7yxJfKIiyZ5VRCyhg0rWKpMti51Ltgguhf/uNn6eDVr6t89vWn/q5yjFWkyQoljUbZeRFKOFecK2WJsgTUP+of8VxzYx2L4dQZrswh266pFdYZrs+aGZu9930rW2LOFWUMmlYQP77hrsq5vveTvVWffWf/35dFkcwqjUaTJJBs6FbHa16TT+PTNlUWmYZf1cNgn+saStmza77t7TeU99dQrT/HRz+xpnz8HSO3VZ0foYRzpeGjLFGWgPpH/SOeq20WqymGCz9bc9eHKvGd4q8isZk6zBXTWXynz70AU5yoDCQ7hz4PBZdGuRTXxY5HKEFLViwVfDuXhmTv+vTa8kQ/n2oXq3jeVOD1mXJalZKXROY62T5+BMubejJs2Nan/Zld9+ahpk0+RCjhXAnUKEuUJaD+Uf86WShdf8Oyyrk13UHxl8VatbKLYrHZive8Pfq5Yr49+56oyjbyZiJN8ZviuPBzxXu+Mx2hBC1XsSSIYoVbc5XUoxCO5qgXQnmuto8EkOW4Xv6aheXtV1z5mnKvgY5dPHhV5ZxK59MolRaI8JVX59Bn6lkwkaQhYx1/7+c+VdmmUSaEEs6Vho+yRFkC6h/1j3iutj3342fKcVoY3ylWUyznR3NqxWZK1fOd3uoUt7nq2iYRpXMoXrM40PbVNosdbbtiOXXSv/Wmt1TEUrNHlihj0NSKpQl8KsSxCiVTr4QfYYrNUdLnGrpVr4RP4fO9FAd+/v2aebBWkVT5/HCvKpWNeCGUcK40fJQlyhJQ/6h/xHPFxFLeqsaKtfxoTl5spnhM4ujB7Q9UndsW5fKpfbE5SooPLb581623VC0mZotLKJsJoQQtX7EkmKTylaeqUSF/nU/e//FCiznoHBoJUqGXwPKpeKqwtSqjndfmPplpZCscBkYo4VyBskRZAuof9Y94rthrYCR0NGpkq9KZWPrpSy8UWszBzvHnd66qihHrCaUf/Gxf1b4+vrPpHIoXEUrQkhVLozzKL9Wwa/iZtpvQ0QS9ekJJFchXQK2m4if2mcjJq4w+TS/PLBcWoYRzBcoSZQmof9Q/4rl4x7c6p9X5Hb4LUyM5EjtJsER4Xmymc1maXJKlyml6hE3DqCeUFLfVi+1iC04glKAlKpapeYma2AIO1mvgRVFMKKmC2VwiDa2aKPKpd/WEkk3003m1T8yasaBDpwql5379ZHrfl9alt/3lyvTP3vOWdPmtN6Qf2Xhn+tV923KP2brr/vTTD/9Vrm15bDTd/cLO6LFPHXqsvI/OUeu+iu6HcyVQI1AD6t95U7ut9lvtuNrzVR++pdy+q52v1c7mmY790jNb0++9tKemL9B5at3Xpi9/qtB+1L/ZFUp+oS6fEWSmVLrw87zYzFZHVoyn+ecSTj4WrCeU1CGfuJXwYrFdKOYQStAyFUvLNNq5tOyjFyLqZbDPNFxbSygpz9X21cQ/264UPNtuk/V8ZfQ5r35fDdX6OUpKv2tWDutUKtC1owvXDn+yf34rOdf9v3kqveOv3pv+zoW/ndtb87pr/jgqeLS9Xk+PbOD1V6aP/+iRqmO3/f0DlXPXur+i++FcW08o7f3Xb5bLjUz/RyhBpzC0sW/18Jbfn9dq9e/Le7+QXjHQn9sWX/yyi9N7tnwkt52tZ7970dz0rs99ONcX6Dy17u9Vr/79Qvt1ev2bjvLT7BEly/DRv1rN2M858ivQSTTVis1iMZ9iRXWwh0LJx5S+891Gn3Rdfx+2aIR/VydCCVpKKKli+HQ5TbhTZfBvb1Yvgk/N83OGVOglYHwO6pUDi8oiS3Od/FLfljannoPEpeepkul4bbd70fUlkDQ87EeqZrsCDW1ckJbs7NCGhXuG7usbnm3nKpF09fWvrzhQCabt395SDmy//sMvpRv+5p6K05WDC4Ndc47qtYz1Qv7F+vdXHOPlVy4oXw+h1NmBmnqs9bsvuGLiqwCuXHxFOUjz5cBs7f0j5bJSy1RWFaSFx3/r4Ncq+9S6N9vvqqsH2rosNbHDBSZd/8bb8mtGL18xvCWZM9v1T6M61uGl9lKjQGrH1Z6rXX/3B99R+Vz1LdbOXnLpy6Jt+bqH1qZvecd1lXvVyBBCqbXKT7PnJY1uubfqWSveUnzn545ryoMJmrzYTJ3ovkNd8Z29zDYJpmZodCpxK+dZ3ObvRXGjRpb8dIvY9A+EErRMD4TyWMPFG8zUCxAuEa6h1yRYFS9c/tEElsSOVUpbJjK23r71JqiyWC+FN+XH2nBvCwglb2Ml27R046v7ZsO5rr7n1vI5FNQqiMwTU9cMn1+dRo425hzlSGulzl3aM7+8n1LxEEqdG6gpKLNASAGZhI3SfmT6/S644ILKbxmm8FhZ1PH6PDQrQ7I/vfZPqo5VIFjkO9h+9QRVq5elJna4wNR+A2+nSrZz6ejCZbNR/9TOqrNL51HKXd5+Ekyqh6qfPv3N2tl6dcPqqTo9EEqtVX6mYxEHCZe891tKsIQpb7HYTIs9hO/Q1N82euRXKtaqx76DXPGczw4K35OpY8PV9BBK0HJCyUxDs+opkNJXwdVwbN6cIFUGraOv/W3ZbzvHA9s+U95uc5IkxDSa5NPpdF6l6+kc+teLIH2mymnnt9XyWqECRRpHs3MlO7B0Q9+q4S39c2fCuT7zi11lZymnWWsekjlh7afexkaFkkyBsvaTk0UodWagpvJkYkY9z7E5CBJSNtKkuXCxAMyXkdC+sPuzlWBQveVdLpSa0eECzfsNvB0fGl2w9dr7+hbNVP3THCSdY9nypXX3fcf7/uz8ksouha6oULI6JH+AUGqt8jNdK97ZasSKqSScvvi1bbnvLcqLzfx2xWf6W+JIsZ1M71vymUqKAxVHhiNF+kzX1wiTOt39cQglaHmh1C02TUKpKuBdunHBjuvuXbB4Op2rUinKw9vXv77Q/gpS81Lv6gklG5HyPZ0Ipc4K1EwM67euF2hJoMv8xPIiQkn2wXW3l/e74V3XI5Sm3uEC01P/KnbtxgWHl25YuO6NG/p7p7P+WSeF5ijV21f1INyvqFDScUmWqo1Qaq3yQ3yGUAKEUrcIpapGMmc+wpSdqwLNJJKr3ogVEUoPfPXecu+jTCMKCKXOC9QkoG10Mly0I6/3+52rl1eNOhUVSibwffodQmnSHS4wzfWvSsSOLng+MrdwyvXv0e9vr8wvmuw5iggl1XNr89UxglBqrfJDfIZQAoRSy1WgSTR2k7VwPsKUnas5tqksu23n0OR4m4tiJiGmBRzsPjXBf7ZXveu9+mXpn679A0tVqrYNCw7GrBRs7is5qN1R29i3OWYSJkMbF45ErfQbxuy6Da8eHLqvvz9mrR6oaWL3VH+jokLJRid96h5CadIdLl0ialQvF6QtZJW5hc2of2rDrR2eqlDSqnZhWy5TvdNnSTanNUytnWmh1PemS9IW+01npfz8L79FPIdQAoQSQim0o81wrjZXJOaw/CpioeldGkWXB1cPp3LmY9eYDaF0+Q3zu9m5TlugptUSdbxGiqYqlLRgiC0pbqY5dBqZNJGktB+/XP1sCKU2DdS6cgGIFhRKFWtG/dPqpLXaSNXLWFuuuYRFlwfXiLFWQFXqa2ypf4TS7NjVd1+Wzv/D3yWeQygBQgmhVLJDGpHIRgOaNqLkJ8XXEkrWm+gdnJ1Dc48sqFV6nYJepWFdtrC3vMrSVASQVsprllD6w3e8Auc6DYFabLEOb3kvsfSLiJhQqmcK1sL5FQilyXW4FJzDhlCapvZ86eiCu5tR/6wt1Qh+EaHU0/uKCW2qnUOf+U4KjVZphTu150qXjS3tj1CavfIz53d/i3gOoQQIpe6do1Syk9du6Hv42vv7B5o9r0ROL4msPlYvGI4JpdgcJevllMCKTTAuKpR0bu137Y1vnHJwK6dy4cvnpGFqmwLGvJQ4TYyPpdBdu7FvTV7qnVafi6bqqTc/J8VPgWs0JXDjghOtHqjZKlp5QilP9Pj9TShJCPmUH0vf1GT1PNGNUJp0hwvM3ByT84usbFywKRCoTVnBNMlWovMLpNSrBzGhFKsbOqfVQz8KNRWhpHlVzFFqbvkhPkMoAUKpW4TSOc2LqfNunCk7V61iZ44xr5dwKkLJH6NrhA7cJiDXC1ptlTMJOxZzaM1ATSOKtYKocCRJojdPKIViS2VT6Xh5cyNs+XrmKE2qwwWm/zc4qUU1rtm4YMl0teUyjfroHFrspNlCydpre1mtX1bcTCnWSZ2FfVSX7RwSdwil5pYf4jOEEsyiUNLa9Xlmb2Weqmk9fK2zrzXy9e6jZp23bYTShr5jpQZxfY3VyZruXNV7X3RUyRxhI0JJueyW5hEKHTlNS+fzq+GFpgnKRQMAhNLsBGqat2ajPkVEd0wU1VvMQcvYJ9mLLmPXsO8QvsjWm1L9tI9SQrtAKBXpcIHp+Q3OaPT42o0Lbyzw7JvSliuFOsnmhea9PDzspGpEKPkOEbXbfo6gTIv1JMGy/XmLTrR7R0Wrlp+ZiH/08liL/TolpkMoQVMqVlJn3oDe0OxfHNuo6QVk4dufVSG7QCidGdq4cNfSja8emuxvOBWHo5Q4EytycDEHq31sEr3Mpz8VWR7c5hgpLcQvBCFbfusNlXSr8NoKhs35avJ+bAIxQql1AjVbHCTW29wMoaTyYS+bje0j8ZPUeY+MLS1e711PbS2UGutwgeb9BueGNvbtP7+qZEPvsGpK/ZNpRDfJ5hlp8ZOwQ0Gj+nodhMRUEiyxX0Qo6Xw2chXWIb0WQG28Prtny0cmHKvPrY2ot7JllwqlKZef6Y59JHwumHNB5XrffPaxScV0P/jZvvSuT69FKEF3CaVy4/zKnvKblidTKAevft35YLpUCVe85+3pu269pdNT76Y6T6BpzlXCxxyn9djL4So9yoJP66nUvKPJvHDWRqPkKL3zVvqFOU852YHXX1lZitZeoKjtcvpT/Z5dIJRmNVDzgljLheftp4BJv3OjQsnPVwvfyeXnSansxEac/DyLImKuzYTSZDtcYOq/QXmu39XrL+uZ7bZco6n2fjxrszUSqzZVdc7S3pJszqdPYy36wlmNypogChcCUr2qxAMlsaZ2X/dj4irJljCvNerbhUKpaeVnumOf0S33Vj375e+8KXffxYNXRWO6NXd9qLztmje9EaEEnSmUVDF82p1Gkf78zlWVzx/c/sCkCuWrF1xWt+J1QgVq4jyBpjlXS5HTXCATLd7k5DSyExvRkaCSY60VGNtogM6tfcPeRp1Xc1BMGJnJGcvJh6NQCKXWDdTUW23nkojWb60ATCaRrfJiQZYCKf/bFn2PkolzBX7hvAsbHVUwpiDOrq37MJGkctjugRoLM7RE/dvUpFUDm9qWm+iRELL6YKYRWQmX2IILqotqn4u8i0mvA9C+8g1hXVJ6nc5h9dwLJ9Xtqda9DhJKTS8/0x37XHHla6p9dEnw/Pw3BxuK6Ww7Qgk6Vii9930rJ3z241/+sPL5J+//eNVnyklVfqrElI7V575ife8ne9OPb7grveTl50c0Xrt4oPz31771lapz7PjG9vSOkdvK5xhZ+4H0O/v/fsJ9bNv5UOVYnVf737nmjvIwr+2zZ98T5W06j3o2tF+bVqCmO1cvXGxp2CIrKDXT1MNp126GQ+1wodSSgZpGlmKC20zBm4RxKLyLCiWNSOVNKtfoqK2qFTOJq9hiEO1WlliYoTP98XS0qdae1pu31GxT+23Xbkad6+S2vB2EkmIuHwfa/+/93Keq9suL6RR7+e0STLbdjlVG0qaHRsvnlz2w7TMTspTsPNpPsaTiQcWXtdIAEUow60JJBdY+9wJGBdxS6rxddPFFFSEkcRMLaOw6qgiqaLF9NJzrJweqhyLJ5ktd+opLK/vZtfzIl7dQ3HW7UOpUo+Gb2bIk0aKRSM1DU+qPFgzRyGOe+FZKT/hupVo95to3NpKpAE1iTdezJcYlvrTKI2UJuk0o0ZYTzzXDTBxp3tGRsV+ml79mYUXwhB3WsThL4iZvu45TdlLvZa+a8Lm2+fnvdh5N9fCxYbPnPFHGYNJCSSJHFcNMhdU+++gn1kQrloZnJUY0uc+Ek3oVJKQkrLSfzqvtqnz6+/M7HqyauyRTjqt6GK6/YVllm0aHQqFk59f+qkj/duJX5ZRAL8J0L75XxPdqIJRwrgRqGGUJqH/UP4TS0bIwsoUZ1EHtBUsSLOqQF9Opw9pvV+xo29XhbWl92q74T5lIJsY03ykUShZbKrXvyoFF5cwmhBK0hFDKMwkTiSEb4ZEIstVRvIDSdqtwEj1h3qofsXryucdzRZiJJV3DVlLxQknHxiYWSnjFFpGwyo9QwrkSqGGUJaD+Uf8QSufNdzSrk9leGWPb3vb2G3LnKIVZSLE5Sj7W0zQLn2Zn25898NQEoRSm/SGUoCWE0tVL31AeWjVTQVWqm32ueUFhAVclUuE2sxxVpcLVqlS+QoRLj6sXIqxYJpQknvLu3/JlzWzoVj0XCCWcK4Ea5YSyBNQ/6h9CaWJHs6Y0xKY7+A7ryQglxZH2HTTnyOIzpdPZdutY93Ghn3+OUIKWEUqxOUrhUpAaNdKwaVLg3Uu1KpVEl+2r9LlwQp99JsHmK22YM+t7PvJMo1wIJZwrgRrlhLIE1D/qH0LpaOWdR0mB18PYXKPJCKW8+Uux8/t9i7z0FqEELSOUlLrmC69fIUU9AX5JcTO/+l2sUvlehrDnwE8YtKHgPKEkszRAzVuK3UsjFQ6hhHMlUKMsUZaA+kf963ShpHngRYSSFl3wo02NCCX/fiZ1gsfiM1v9zgslfz2EErSsUNJIj4SKTdDTv9oWm/xnozuaF6TRIr9CXqxSKSc1cQs5WKXQuTV5L8nmRunvekJJKYNJlmJno1M6n+Y66f40AoZQwrkSqFFOKEtA/aP+IZTOx3e2grAWW5CICc13klundaNCycd6Ek1+ioViNKXj2fQLL5R4jxK0pFCqZ36JRr2nKAnmKdkqJhJRfpWSvEql4+wcqqgSTH6lPVsdr55QUgW2Y5QmqHvxc6sQSjhX2gPKEmUJqH/UP4TSedP876TOwgle5Lz1prfUjelsXriyfCSCbO6Rn++kjnQt4GUiTaNV1sGNUIK2FUoSPirYfihU//dLcCduhTz/QtlalUqjRbFz6HpaicXvW0soyVQhbZQrcctLNrr+PkIJ50qgRlmiLAH1j/rXyULJryQcLtbgzTJ8/KIOeTGdn1KRuAXANBUj9t5NnSf2HiWEErRcxYoNucqUPqd0unCxBW8aOdKcIg2pfuWJL01407L1Suh84ep2Zqp86t2odQ69EVrnsGUkY/Yv//Gzynk0ilSr8iOUcK4EahhlCah/1L9uFEoW59WKqexlsbavzT2vFdMpbrQYLJwfruOUKbTpodHy+5nCeUja366FUIK2rFjdYAglnCuBGmWpi8vScGBDBY4ZiBw3t84xvZFj+ql/GG058VyXxHlAxaIC4Vxnxp779ZP+NztDE0BZIlCbennJbKzAMbsjx9UTPSORYzZT/2jLacuJ5xBKQMWiAtXjrN3D917agwOtY1//4Zf8b3aMJoBAjUANodQi0JY3YLtf2Nlo2SOewxBKMGVHh03eZovjdg/bv70FB1rHNvzNPf43e54mgEAN0Y1QahFoyxuwrbvu9+XhIM03cVgHx3kwi5yj4DfFTs/ib7jL7uOdq5fjQOvYNcNv8L/bJpqAeKD25b1foLzUsU1f/lS3i26EEm35rNmqD9/iy8N2mm/iuQ6O82AW2U7lmrKpF37dLP6GN9q9XHDBBeVebpxo3L6w+7Phb7eIJiAeqN3xV++lzNSxG951fbeL7t2BFQlW10SO66lzzLLIMSs78HnSlhe0vf/6zfTil13s69+NNN/Ecx0c5wHAFDlkFXrBFZelTx16DGcaSZO6tGe+b/j2UGwmsNKej4IQBSOUnbg9/qNHysGsK0+DFB+gLZ+VToqjJZtD0QEAgDwWJ25+SU/vK8qjJzjU83bfl9aFvY+nkvNLDkM1CjYq6XfLb72B8hOx/b95Kr36+tczPwJoy2fYNHfyz97zlrC3fwXFBgAA6rEqCYbdX3fNH6effvivyr3f3eZQH/3+9vSeLR9JrxjoD52qVidbRnGpWY4qz+sv1r+fAC0QSZo/EpSpJRQboC2fPvvqvm3ltijICpDtpLgAAEBRlpfsZEI+cZ4dJ6gtRNWEe40skYb3TDkNKlgMpFMXFQDa8nYwzakk5W5yDCTVc/7EaDJxLuCO5PwLnmOszPbZVOfcofmOSpt/qFH5A8n5eUD+BdR2TwMRH9XIAh4adXzaXUfnnec+3+6ew7Ia977GXT/PegrcT2yu5s5s+5zge+4Knok935XB/a7NuUaR+1mWxOd+hs/f/l5S4/ew+xvNrl3rWdn1+rPjD2b2aHJ+dD2JnKOH6gtTpSerWGdxpFWr1WwPGkbIRw31If8MlbqoBR60Gl43LR2udyVpqeZ3f/Ad6e9c+NthuXqaogK05bPS4XU7xWNKDDvfaKtVHqzxzGPP+2j2mUY/+yLnzrORbL+7k/GR0zFXzg85sWD3FIq1Rt6dZatmns2OOZ1MTJkeS8aX/x6pI86TOt+vv9HOyMB2B99T9nDk+W4O7vecExf+Gv0NPKOw4y98/vb34UDQ+d9j2D3f/jrPanPWzlmn0InMLPtnSVa2xrK/i34fgEL0Zr0z+1wj0C12LnOmT2e9LPMpDpMSS7sJynJtK0UEaMtntD3XAjyrk+redZiaUNocCYp9IGoL/ISvP1jihFatkfWRSPAvFmW/q+YLD7qOgcPZ/muaKJSOBN9rXjI+F3dJRCiFrHKB+0AgYCbL7sj3GnRisSe4zjl37TyhZCJzJoRSmgndekIpVubCsrA6cu3NgTBt9PsAtAXzsh4naG9WOaeCnXe6wxQLoD2HLhBKy5L4KrGPJuOjQwriTyTxNMg8oWSBcJi2J7GwPBnP/miGULLsiB1ObPRm33NuHaG0OBkfyVgRXL/ZQknPz0ZWet11zgbCI08onU2qR+ymUyidyURybxOE0irnW1dkv8n87L57EUrQyYzkNHDQfqgBX5n17hxPuisVSA7hWHI+f5x3tQDtOXS6UDqRBb2WDncsCE7nZe3iycw37EvyVx/ME0p7s+3L69yn3dNJd09jDQql5YHPOpkJPf+dYkKpx23fFBFqaXBPsqLp2Bb0H0jG594cctv8dQ6657C6hlB6OPuepzKhMZ1CaTSpTj+filCam1Sn+kt4awTz9pxnhlCCjuFQwvuKoHncnbAUMADtOUy3UHox+79+c0ut2xoJqndkf1t63v4GhNLzSbEXBds9HXGC4WCDQkko1W97Jvp8J1h/jlCa4+5xb+R8aSBizHY0KJTCTrlNSfVcarvGQDKeqrgyRyiNZL+TpaxNp1DS8znqfsOpCCUTS2szkXgmib/IHaEEHcVAUv1Ga+YIAYEaAO05tL5Q8oHoPCciFmXbXkzG30eoz2zyfbioQy2hZKl7dwfblep2e9K81Lu52bH+eN2jjYKN5gilnU6gzashlCaLT72b7/zbrsh1THCYCDpaQyjp+x4P9mtEKG3NEUqDwd9iKPv/sSkKJZ17hXvOc7MyYKN/CCXoSB4OekrW8khgCiwKAjVWLwSgPYfpF0pzXCA85ETzqaR6JOV4Eh+RyBNKK5L46mn7s+3rmiSU5ifjIzFe4I8G9xtb9e5UjaC82XOU+rLr+TlGoVDyIihPKIkbk8ZX4bPf4/ngtzcR3BsRSkkm7NIpCqWnI+3K/GR8lA2hBB2Hn5CYusYQYLJsCsrTah4JAO05TItQ2psFpfucSDqelYXt2d/rg/MtTsbnN80pIJSSZDy1TdfQCNMhd475TRJKPhA/nol+3cvZTEANRoSSzWc6kkx8789oIJRi7wYaSMaXtN5eUCgJW9RA6Y49EaEklhcQSv47FxUW81w91++gEbUXI/U9FErzncCbrFCyBUPOZZ/pN7LRMFa9g45kRRKfFL+YRwOTJFx173keCQDtOTRVKD2dTFycQKZRHku7O5xt64ucc3/22bIg+M8TDHOz7addoHzAXcvf01Bw7FhSvRR2PRGwO6le0CF8F9chF+iP1bCnC+wzlD2fkzkCMcm+d+x77c62P+yuEy4QsSvbvi54xqvcPr2ZAM37rWIsceLIL6rQF/k9PKuD32Mo576HapSF1YGfP5N9z3kIJehE9uU41h08GpgES5L4e1V6eTQAtOfQNKE0mygYnzMD11GQPVNz7Na0aT2Zkz2nObNw7Z4awg6hBB1BbzL+pu3QNDzLiwKhUXbklKd1PBoA2nOYklDSaM5YG3+PgSSe/ma2bBbrzuGk+GjOdLO9xjMabfHf2NIYzyCUoBNYn9R+J80qHhE0QGx+BPMkAGjPobkCA6HU2bSzUOoJ7reHnxPamWN1HOsBHhE0wI11ytMiHhEA7TkAAECrM1THqea9ZwEgjz11ytNWHhEA7TkAAECrs6uAYw3ftAyQh+Y/nKlTlo7zmABozwEAAFqZeQWC2vClZAC1WF2wPJGDDkB7DgAA0LKMFHSqZsM8MqjD/oJlaSePCoD2HAAAoFU51KBj3cMjgxpoVZtzBcuSlimewyMDoD0HAABoNQYadKoyvSF7Po8Ocri7wfK0gkcGQHsOAADQajw8CccqW8ujgxzo0QagPQcAAOhYzIGe5lFAAyxKJtejPY9HBzDt7TmLNwAAAEyReThWmCSbksn1aK/m0QEglAAAAFqdfhwrTFOgdo5HATBr9e8UjwIAAAChBK0ZqKU8CgDqHwAAAI4VgPIEQP0DAADAsQLULE+MUAJQ/wAAANqW+Qmr3gGBGgD1DwAAAKrox7HCNAVqCG8A6h8AAEDbsgihBNMUqJHKCUD9AwAAaFuGnVM9yOMAAjUA6h8AAAAglIBADYD6BwAAABMYcU51D48DCNQAqH8AAABQLZR28ziAQA2A+gcAAABJstk51a08DiBQA6D+AQAAQLVQ2szjAAI1AOofAAAAJMku51TX8jiAQA2A+gcAAADn5yWZUx3hcQCBGgD1DwAAAJJkv3OqK3kcQKAGQP0DAACA8+9OMqc6zOMAAjUA6h8AAAAglIBADYD6BwAAABM47pzqIh4HEKgBUP8AAAAgScacU+3ncQCBGgD1DwAAABBKQKAGQP0DAACACZxyTrWHxwEEagDUPwAAAMCpAmUKgPoHAAAAOFWgTAFQ/wAAAACnCpQpAOofAAAA4FSBMgVA/eNRAAAA4FSBMgUA1D8AAACcKlCmAID6BwAAgFMFyhQAUP8AAGBWWZdUv1sI6zw7WbLbKVNYG5cpAIQSAADMOGcJ+romsKVMYe1apoCOCozOCgCAWe2NwzrbKFNYu5YpoKMCo7MCAGD2hNJ//2//J9ZhNttCid+AMgV0fmF0VgAAIJQwglrKFGUKaNMx6iAAAE4Vw6FSpihTQJuOUQcBAHCqGA6VMkWZIkijTceogwAAOFUMh0qZwgjSaNMx6iAAAE4Vw6FSpjCCNNp0jDoIAAA4VRwqZQojSKNNx6iDAACAU8WhUqYwgjTadIw6CAAAOFUcKkIJI0ijTceogwAA0AVO9cCe76Y3vfmG9OKLLirf+8tfdkn6nptvSX994H+t2m/BZX1l0/44VMpULfvG33w5XfqnS9ILL7ywfO8qN3eNfCQde/HfqsqdlSmCNII02nSMOggAgFNtKXvib7+azrngguhbyiWcfvbcP0xwKPu++RQOlTKVa2vv/Iu8t96nV77mDytiSeWo24NQgjTa9Omwl378z+md770tfcX8S8v3rjZeHRfPfeNbVfupQ0wdFQ9t/Cx1kDoIAIBT9fbL/f9U6fF/+w1/lv7HTw+Xt+9/8pn0lT2/X97+5qE3IZQQSoXtqa/uqtzvhr/6RFkU/df//M90x5aHK4L8/ns+jVAiSKNNnyb7+fd+VGm/Y6a6aPu+acnSSl2lDlIHAQBwqs4+/dG7K2lRCmbD1CmJp8+t2zDBoXzrkcfSNXd8oDw6cPXrBsv7+mN//J196Tv/7Oby5zr3DW96c9U+EmLvW7mq7LDloF+z8PLy3zjU9i9T6rXWver3Dz/7xIc/Vu7B1ihmKJQk2vWZyoLKncqQP1ZlTumh+nxg0R+Vz+/3UVlSGdI59a/KXjv0khOk0aY309SOq+wn2eitRJONMFndVGq1jeoilKiDAAA41RyTyNG9SvQ04lCUkqeUDkvrkEn82CiVzXVSr6bv2Xz20d3lfXZu/Zvy35e9qrfymYJkHGr7lykbNfraF/627r5eKKmcqDz4eXLH//l/VkSS7Sfhrc+sHNooqMRRWKYQStBt9U+pdb7zwX+mOacSRhJFv/mnF6uEkjrNtqy/v9wJIdM+vvNM+4/8+fvLn6kODl71J+WRYdtH9VB1UPMQVV+1n4SZXYc6CAAAbedU5fAa6U30okYOUiaHqW0aLbDgVyMCcqp2nEaU/HVMKMk0YqVeTz8XCofavmUqaSA90wslEzUKrCwdVGl82rb9c1vLKaCWMnTiX//3ikg38W1CSUJNx2mhCII06Lb6p3ZY9yqhUmR/E0rqdJBpxNa+r6XIqp3X+bRN9U7/tw4Ra/clwqyDwzo71GkRZipQBwEAcKodL5QsgPUBapg6p9EA7SdHar38oVCSQ20HR4pQmn6hZKNHvlyqnIQ94krb0wioiSnbx8qhn1NHmYJuq39WDySAGhFKEkhWB20xFmUcWNrerStWlju8LGVPHVz+OiaUZNpXf1uWQRfVwYPOBks2Gmwz21SyuZHjl2efH4h8frCGjQbneL5kp0p2omT7SrbYfW73tDw4/9PZ9t6C33V1yQ6X7HR2nf3ZdzZ2uOcwWOPedwT7x2ywwP2MFHg+/ve40R27MrJv3vkeLtm8yHOrdd955+mJ/Pbe9mf3kUSea6879/PB72bPeyTymx0q2cnMDmXbGuHukh0p2ZmSjWXff1HO/Q0W+E1qPb/eoFyM4I5wqrM2n0SOLbbQg3rwLbUpLwgOhZL2txGkJEuVyhNK7bYsNEKpvpmA2Xb/lgmfKXCSeNaIUK3FHEKhpNFGG7lMsrkXdh2l+dQS7JQpQCjVF0pKmbNtee2zRJJS+ySSrD7GhFK7vT6iiXUwzQLQ3SUbyP7VtmNZoHc0Exbatity/H53H7cHn41ldiL7/Kzbtj3bZ4377FAW0KZZUDuQ7WP3NBI5v7b3F/ieq9x3tUDXruMDeG0bLtmQu1ezc9nnzwf7n4jsO1TgnjZnx5+KHL89+O6yRwOxY9t3B+ez3+5Idu40E5/hcxurcd9p9n1t+xl37h4nzNKsfIy586bu/v1z6g/apz1un+Fs22a37eGgbLzofoP1DYgk+933ZELZfrN5kfur9bvvKvD8+jJBdSDyfQCnOjMm4ZIE80HM1GufZHNHGhFKSrmznn17D5PtEwol9WIS1HZWmTKRHAvUbE6cFmVoRChZ2o96uk242z5WFhFKQP0bX6BHnVOxhR4kcvzcpdhiDqFQ0nF+FFf+wupkTCiF79/rMqF00P0dEyXzs2D1dHBsXxZEnoicx9Of83lPFoB7UZRkQbYXBs0QSnaOZW7bnmzbuohQio2cncvudTBHADTK5pzvFbtvBfrH3faTTrzsrnG+eU4shc+tXrkYc39rtHBvtn1HIJS8GBhw1xusI5RSN0oWCqVBJ2j6I4LqRMFn/Hy2/6LItpUFfsc17vn3N/D8hhFKONVZMwWdNtdDPYSa76H5QspND3PQiwolc7x2nHohLe89FEpFez0RSu1TpjRqlLgUHK1M95P/sb/8/yRI3SwqlKws2uiRyqgFbeEcJYQSUP/i6a9KW7XPbDW8IkJJK5aa+LKVJsM2HKFUWCgtyUTC88Gxm7J917qRoIEGhNLqIND3wf1AnXtqVCjZ6MSRLPiNHZMnlAacKLm9xkjJdAqlp921BrL/HygglBZn935sikIpcdc9XkMo+We9qYZQOuVGqOZEhMWOGiNHjTxvE8OWstdb43cPzzuUlftzQZlAKOFU28Ox2ipioSk1z9Kkigoly123FxxKiNnkXlvZDqHU2WVKCzPkvcRYo0KxOUq1hJKNUqmcqsxIJFmZshQ/hBJQ/6pHiZQNoPRpiSL9a51iapcbGVGyzAObs2QvqkUoFRZKoZ1OqucNWcB4NhsZWpdMTLmqJ5Q2Fwwm8+4pbUCo6B6PBscdS6rnu8SE0rxsP23f+v+3d77Bdl3led8zVrBiroNwRK1iY+RE1AooIFwzyEV15BhQsAET5KCAGC5EBXuqxk4sXAGGirFDBBGJQgx2wA6iqCCC6orUNGpx4Q4IqlA3OI3DuB11qk7dqTqjD/qgD/qgD6fnOd7P1XvWXfucfe7d997z5/ebeUe6++y9z977vGu961nvWmtXNLBTOzWgUMrZluTe7yz/3Rme9Z4KoZTa+aJ7ztSpGtdddR/R56qE0nRyXTmhNBNEzP0ZYfFk5reYTmyqxjNem7nfp0I2qUoorSmzSNq+q0Kg93p+CCWC6vKbVgfTkA0FUGV/NDRKDdB0oQU1QmXujYzvr/GKZB7eoXPJlJ3SHBMvHRvfoxTf0USjdrx8Slmk977zXa0N1/1SZ06R3nvkjFB8MaZ9Km6Xn2ibJ4Mr8+mliTWkU0uPKyulfbQiXs4P8SmY1PKn+txD41KTeBp06J3mHBVhfqBiRHwFBEKpr1A6Xv7/YGi0Smh4wYZbym1HQsPyQpkpWLlIQile08GQ5ambYVhRNpAPJo3d7RVCaUXI2hwr/84JpQPlPdjuGVAoHU2O3xvuyfe+oXy+h8v9z4XfIBVK8Tk9UR4X5xadCs+96rqXQiitLgX4+cy5/NxvydVrA/7uK0tBfDiIn/g7p9e3srg4lyk3L6/O80MoEVQxGrX4FD6FUKJOb840EkCdXcrGuoNBnWFxcR6PAFAHQ+zEcEdWXOBBw/Z0LgkrzVeSGFNHiPbT//0eJVn6HQilOcPBjiai4nCPbMj2mkJpZ5EfereuFB+b+lzTIEPvpovulfaKkKV5okIoPRgyT7nsxVIOvdN3nCiF6NlSSGwp+g+9E48U3XOx5jv0bl1Rb+jdg0X/oXcziQ+cTs61PzmHxYfs6ZrPfao8/50V1/dIxfX5mZ/IiOO6zw+hRFDFaNTiU/gUQok6HZsAobSizFJYBK0uLi7ucDDJXuQEUZVQWlOe53xxcVifvstDsvY3KJQ87C4uMb696F4RLgolf5YuNDGoUNJnVzcklPaF33zPAELpSANCaUUQXPt7CKWrw/k31BBKRSlIWkV+MYczRfeQz3WlUKv7u58pM2rxHLvL4x/KXN/uINyqlp1HKBFUMQIqPoUhlKjTsQkWSnHJak+810pjq4qLc2QOJOdaETID62sIpZjV8RLQJ0NDdXWDQskLR1wov+d4cXHZ560ZoRSXD88tAx33zy0Pfk+PzEwqlHLLgx/OCKWt4Tff1EMo5X67s6Hh32t5623huuPy4GdDdm1VIpTi8uB+pvt6CMrUFzaE4/Zmns+FUkzF5cGfC9fRiz3Bv2aCKLtQ5FflO5W5J9vxAZ4fQomgitGoxafwKYQSdTo2ZkKp6oWzj4SG7kPlthsy59tdzH3R5pqi+0WtKX7h7KlSKD2aiJ+mXji7vWzsulGbnjO+cLbXy2Rnkv1zNh2yJVVCqdcLZw8k9657nCouvqx1RXHx5aj9Xjj7aCJce70wNYrGaMfKRv+q5HdL9zsSxEL6XOP7qg7U8BtxW5mpjL/ZrpoiKYrxE+EcHraYu75ez+bwAM8PoURQxWjU4lP4FEKJOh0bM6EEzaKG/lEew8SBUCKoYjRq8Sl8CqFEnY6NgVBKVwEbRXplf/Ys0zWtKEXSOqqkRWFNn999ehl9sYVQIqhiNGrxKXwKoUSdjo12GcwtST2KTPewzVQJY8lUn9994zL6osvUFn4mgmqX6R1Eeo9FlcWlW+drPtfffOeHixKAfA9aWrZqH313eh16maH+1otLadQ251O9/Knf71TH9L4VnSe+0LJJy/lKznQf6XX4OF0jPgUIJYwyCAAwwkFVjbqixxu005d0LiRALNYLAn0PeoFh1T65FxXmXoKIUCqavv451ut3qmNH/+xrXS+qbNrqvtQyfWFmvHddIz4Fy1H+1LnVq6OiiQ4GdzIttNOj3z3066hLO8nU6aW/dX0IJQAAgmpjIuPO936g0yhMrYmX+g2rUPreN5/oNGiH7Q3v4yKUvvQHn8v61KkTfzu2QknXJjv5w5/gU7As5U+dW706KpooN+5kWminR7976NdRl9b96vTS37o+hBIAAEG1MZHRL6vit6jrDeuPf+mrnUB07TUv77w9XQ1fNYr/4as3tl5x7S92RFdsDMdedvdE6u3raa+7GpfqQdQb3mXTv/GeOUOf1HC947bbO9+ta9C15ISShj69+aZf7Xz2tjf9Wme/tPGbvi3+R9/6D7P3+J2vH+06Xn/H60g///ET323sjfHjIpT6ZVUOfHLf7PP/9Ec/2dr4ql9ubbjulzq9wv/3J/+98zvob5k+zwkl/c7yJfmD/EK/Q/wO/a3t8kud/6P/7N45Qk3neMuvvmm2JzrnKzL5uHxO55Lf6zrThqd9QL6kv3Vvvsd4vHz77757Ys5Qvnh++1QTw19ppE2eUFJ9nOuoaGII9LAKJd2f6gd1giGUAAAIqo0JJTUUFWxSc2POvecvuvznWisuuaRznP7VNjXs0m1q6KUB4oWXXda6YtWLO6a/ta8apdpHjVeJI3/H31v9ks7/9a8DuwSIt+tc/r5LL720K1iq4evr8Lm8T9Fj6J0b4C+9cs3s/fg4XfNz//mnsyLJ59d2nd/X1UTmbFyE0t2/dVfWp9LGkH8fPXcfK59Ktz38+3/Y9Tvpc/0O2kf+kPqLhEbcLl9wb7OEWJWv+Jj4W+7/xIOz23wuH9dr6J17uGMZ8T43bXpDl2hMz+99m8gA0EibPKHUL6ui+tTCXmVFHT7yNXUASejnOoNSoaQODAl5+XfaoSFTOXMniM+jTre4j+p+1RWq/32OnFBSudbf6hTRkLuvff7ROULJHRPqZEk7L3T8e9/5rs7xEpHquIjXoVjnz3V+ncudaek1UwYBACZMKFWZG3wWSnGbeue9TUErbRimAUI9+wqcMgVgbfN4eQUk/a1g6R5/B0tlqOL3qSGZCrgYLH1uNUQtbtSDX1coxUb5Dx7/y9ltDpbKYPj8bnDnzs8cpbylDTqJAosbPVMLYTfMvE2NmPR3+uTuj842cuzLanRpmxplsbEVxbgbSTlf0ffE31K/scW9fVHnUuOvrlCSqJMvaZsagu4ocGPS5/e1V52fRhp1el2hpDo411HhbEscYqoyKIHgTgJ3PsRtKjupUNJn3s/niqMTVOfb/3U99vs/f+Tg7D4u39qu79C+/k6XXY02cKeJPovnKvoMvfN16XhZ7NTySAGd3/WHzu96wuWyicwZZRAAYISFknry3PMWzUOILEgUONI5GgqydSa4x5XAFChjY9HiQ4HV8zzUCI4B2o3adCiSMwUOZv7bQ+pSwdNPKClQx/M7GLvxmzu/nhNCqfv69ZvmfCpt0CmbGSdx+9h0m4+Nv5OFqkwiI4pvN7bkR/YpN9wkbOv6Svxt47AlZ5nqCKU4gT6eP/07DsdzlgmhRJ3e5Bwl13dRKHmbOoO8zfVs3Gb/dN2putnz8dzZpXKpbarv/XcaRxRv0nO7zGhfZ/Jd5h0LYsZYnR1pJ1kvoeSOFnVCOO6p8y1ee+yEy52fMggAMKFCqd8cpZwAqrstJyBiViAG3pxZiFWtUpcOvygyc2TqrHpXtUiAz+/z1T0/c5S+NvA8BDd0em2LQyR7+WivxqKHvdX5LVNfTcV+HaEkgZY7f64s2DxXCqFEnT4foaRGf66jwp0C0Q+d7YzbcpmntO6MQ9yUkY3+H0cJuKMizv/T0D8LIGVP4z2k2WB1phTJkG5ZmvHpJZRiR5075lz3+/zOGNucfUIoAQAglBZdKHk+kkxD9WL2xoFLgTA3ATkGM2cDPGwpzSjlhksMklHqJ5R8/jgeP/aMIpSWRijJ4sIMzihZBDmjpGGUqT+519i/pYda5nwlNhTdMx6H0NURSvF+UqGUa5zGXm6EEnX6YsxRygmgutuqFnNwJkidCPb9KtOwt6pV6tK6oW4nWS+hFOtln9/nqzq/h/ghlAAAJlwoKXC41y+ae+GaEEqaQKshGWrcOjB5iFXsffRQDg2V0HkcvNwwlTByD2icJ+Vg5uFV6qXUudTTaZHVhFDyHBb1NqpBIJHkgIpQunj9WhUu51MWG00IJQlriWUNx/EchrSXOM6Nkw/Kx/zeFa1yV8dXfG799jqPesP9my9UKMUGmYboSSyp198dAGMulPYmNl3jmOnMcav6HLMxc8zGcazTl1IoeU6dh7TF7I3rdJWtXOeXyppXjoxzWuMwa5ebNMM0n4xSL6HkmKHviRkyd7YglAAAJlwoFX3eudGEUPJKZu51VBDyZFoFWTcWtT1O3PU+ClyeZFuEFfjS4RFxtTN9rv9rHwfVhQolZSPihGA3BhBK9RZzcGOniaF3+m3jCnTaZqEtwWE/iKst6rfzEu4Sbd7Hk8hzviJhlfpdk0JJ2VbfQywvEyCU0mubqXHMTOa4tTXEVXrM9DjW6f0Wc5Cpvm1CKKmcuNPKcwlVPlRXxzlKzpTqu3WMhFCavY2rWrosuNx4vl7sJLMQa0Io6bsdL9Qhp++IHSYIJQCACRVKCm65cey2OKE3fadL3W3x/UIaNqde89y7idTAvfdDuzpBTj3/6uVzUIz7aBy5z+Gey3QJVzWAdbzOoyyAgqTv1Y3kqvcopYtF+Lg47ErnUJDVdk3qlzgrMhPyJ1Eo9fInmRo98T1K/jsu79trW3zflRpg8gNNDtfvnL4jRvtqu4SsTBmo3D7KLOkc8pmcr1jMqIGn/dSTrv2qfD19j1K8n7gsc7wO+bp8W9eroam5TgeEEkJpoYs5FJkhpfMVSrFzy/uoPkxXvUuXxo/7xFUm46p7UShJeKncxf1iZ8VChZKyxB7+bdNKfqx6BwAw4UIJG9zUmFXj3CsmxTlKCuAjGlDxqWU0CTA1KuO7XTysNK4AiFBCKPUzd0D0Mon1KNjdIVB3mzuZJO71fwkT+amHtMY5pBJF8m/to3/9Kon0XUuaW6hzKLuT62BQFkydEjqPO8l8r+4k6/UepdjpEV92Ha/D2yWgtL/FXfrOJYQSAABCCaswj6svyiF3Cu6eT+IlaBFK2CDmORjqJdeQHzUa03fEIJQQStjimASS6nGNeLCgkjBzvZ6OfkAoAQAglLAKU8+jeizjfCkNFVFvZ1yFDaGE1TX5jUR2nPumYUVxZcUxbaRNJ7a1xjFbM8dN9TlmXeaYddTpmOczumNCQ+6UtfLwv3ShCYQSAABCCVtGQyhhY+JTQPkbGdOQQM19jQvAKNO70MV5KIMAAARVDKGE4VNA+cMogwAABFUMoYThU0D5wyiDAAAEVQyhhOFTQPnDKIMAAARVDKGE4VNA+cMogwAAIxpUsbE3fAobVZ8ChBKGUAIAWHIu0NibCDuLT2Ej7FNARwVGZwUAwJKzn4bt2Nv5tt2DT2Ej7FNARwVGZwUAACwyu8oAcxuPAvApoPMLo7MCAADgeY6XQeYQjwLwKQA6KgAAAKAo1hXdvXFTPBLApwBGGjoqAAAAGuD+onvowg4eCeBTACMLHRUAAAANcTJp1D7JIwF8CmBkoaMCAACgAW4o5k6G1YTnNTwawKcARhI6KgAAABrgQJFfOYhVgwCfAhg96KgAAABogBVtO13RqH2KxwP4FMDIQUcFAABAA9xS9H4fxToeEeBTACMDHRUAAAANcahPo/ZBHhHgUwAjAx0VAAAADbCybef6BNWTPCbApwBGBjoqAAAAGmBHn4Bq28yjAnwKYOihowIAAKAhnqjZqH2IRwX4FMDQQ0cFAABAA6wunl8utk5QPVM8P0EYAJ8CGF7oqAAAAGiAXTUDqu02HhngUwBDCx0VAAAADXF8wEbtIR4Z4FMAQwsdFQAAAA2wbsCAKjvftikeHeBTAEMJHRUAAAANcP88GrWyHTw6wKcA6KgAAACYNGIQBcCnAEYDOioAAABo1AI+BQCUPwAAAIIq4FMAQPkDAAAgqAI+BQCUPwAAAIIq4FMAQPkDAAAgqAI+BQCUPwAAAIIq4FP4FADlDwAAgKAKgE8BUP4AAAAIqgD4FADlDwAAgKAKgE8BUP4AAAAIqgD4FADlDwAAgKAKgE8BUP4AAACAoAr4FADlDwAAAAiqgE8BUP4AAACAoAr4FADlDwAAAAiqgE8BUP54FAAAAARVwKcAgPIHAABAUAV8CgAofwAAAARVwKcAgPIHAABAUAV8CgAofwAAAARVwKcAgPIHAABAUAV8CgAofwAAAARVwKcAgPIHAABAUAV8CuI2qvMAADAvSURBVAAofwAAAARVAHwKgPIHAABAUAXApwAofwAAAARVAHwKgPIHAABAUAXApwAofwAAAARVAHwKgPIHAABAUCWoAj4FQPkDAAAAgirgUwCUPwAAACCoAj4FQPkDAAAAgirgUwCUPwAAACCoAj4FAJQ/AAAAgirgUwBA+QMAACCoAj4FAJQ/AAAAgirgUwBA+QMAACCoAj4FAJQ/AAAYVla0bUvbDrRtpm0n23Y2CT7YaJl+v1NtO9G2R9u2o22r8CdsRH0KAKEEAABLLpB2tu00jcCJsPNlA3cN/oSNiE8BIJQAAGDJWdu2Z2joTaSdK57PBuBP2DD7FCys02JLQVaXrC4AAAzM5radiRXw6lVXtN73lm2th+/7VOvYga+2/uqxv2j99OvfxUbU9Pt953Nfa33pI59u7do23Vr/8nW5oHsP/oQNqU/B/AUSWV2yugAAME/WxZ7Fy1b+bOu+HXe1fvIvj9EYHHOTaLnqJWvSIDuNP2EN+9RWqtllYW1BVpesLgAAzJuVbXs29vp/8/cepsE3YVmB66/bEIPrhbZtwp+wBn3qLL3bSw5ZXbK6ZHUBABbI7tjz//i+LxJ8JtCe+vK30wD7bCl68Cds3g24JLN0gOp2ySCrS1aXrC4AwAKZisH0gQ99mCAzwfa9z3+jdfllUzGw7sWfsIU22JKs0hTVLqMEMLK6AACjwHZXotdceVXrbw99hwAz4SZxEwLr6QGzSvgTNseSTOU2ql1GCWBkdQEARoFDrkDvftdvEViwjiWBdTv+hC3ENG8i+NMjVLuMEsDI6gIAjAKnXIHS64jZ7n33B2NgPYQ/YQsxTTIP/jRDtcsoAYysLgDAKHDelSeTfDHb1x94KAbVE/gTthDTilzBn05R7TJKACOrCwAwCsxWngQTLI5rL7qHauBP2HL4EzBKACOrCwCAUMKGy4ruZWXxJ2w5/AkYJYCR1QUAQChhCCUMfwLqdIysLgAAQRVDKGH4E1CnY5RBAACCKoZQwvAnGmnU6RhlEABgwoJq8u6dSluMiv8dN21dkkDzule+pvN9+rffvr9/157Z6/vKx/+os03/ptsQSnnTRHQ9536mpcybnPDs8y7FRPjvP3xk9vv03f32f99btnX21b/ptltvvJlGGiCUMMogAABCCaE07kIpPqte1uRvn/vNlmqCtb57Pv7nbSp/NNIAoYRRBgEAhlgorV51RaeRWWWTLJSUQdB2mf6PUKonlG654Q2d94Dk7KHdD0y0UFJGTf7/gbe+i0YaNFoG5Vt1srpNZl+XOkOqcuf7GCQDHMtrbhtlEACAoJoVSnV7tg/t/VwnsGjIkd4e/9nf/kQnSCo4qwHplXo+/v67W+958zs6/+rvKqH0wy/+m9Z9O+6q3Dd+rxrY3q9KsOgafL4HPvThzlK7VUIpXr9e8njswFdrC6Wnvvztrm2yfveh7/vMro91fV88t845TkKpbgNEv5nMyyJ/8/ce7pzne5//xuw++kw+oO3pc0p/M31ugd/rmeo7vV+V3/m3jufrJ5R8/fp9q4SSfnedJ96jriFuq3sf6fepTOk8TYh6GmmjV6erXq2T1W2yU2GpM6TxRayDdGzouNS34zaEEgAAQXVBQslB+FW/cF3r+us2dFXil6382Y7wUHYqbr/2pdd0NUS9ff3L17VefPmLuvbVsd/6zGNd4uJtm984J8jru/74d/Z2XZv+1vb0u1/xsmvnNFTVmNT3x30vueSS1q+8dlOtoXcx+Er0pN+r+4iN4Krvk7Dy3xaakyaUvL+ErfwqPh+JT4ny6FN61jEjFYWSROjll03N/n3pz7ygtXfn784RPur5jvet/dRgkr/FfXWsPvN+OvdH3rcre4/f/uzBjr/F895x862z5aTf0Ds3/rRN503vQyI7Xlvu+7a/8W2z5aVObzuNtPEWSioPVVndJuqbURZKek6yJrPbCCUAgDEVSmqU6v85U8MwF4T1mbInGj4Ur0mNXTUwNfTK27RfruL3vjqHrkHbrrnyqtkGqzJVsdGpxrSDshqP7kVX77nFiv6VeNF3xjlYseEY7+PNr7+p0zDVv7ke135CqSiHmekcN264fnbbXb/+3tnviw1zf198PuMolHoNvcsJZ/1uV17xkk5DPwpobZfY1XOzj+hz+0gUSjLtq+cdhUZcdCGKYf0//maxIfXwfZ/qEkg6p0V3KpSU8Yq+pnPq3L7eQYSShZnPEcWSM0u9vs/lAKGEUKpTpzhL6n2dvZXFl9+qs0flWxnMfkJJnURV+6aLv3i/tJMil/31cMFeQinNPlcJJd9zrIt03XFbnfvQd/h5uXz63Ms4SgAAgKBaLNFiDmkvnLfH4BEFVxyeFnu604pfWYIYSP7J2989R5Co4ay/N7/mdV2ByQ1pixGJIh8bM03qdU8bqjreDdh43hjwBxFKaqTGIO1ze7y+gq63qTFb9X3jJpR6WbzXKLztD8pMevtr1r1ythGlHvL0HFEoSUx5X/32Fh3K6mjb1x94KCve7XtRjDgTJJGkcznDGYWVhZKyPbnyErfXFUrpOeTj3q7GmLapw8DbYkdGfG4IJYRSnTollh9lV2IHg+pf1fPywSj6lR2PQzujP6ssxX2V9XT5iQIpzbCrgyxdRVJlMR1RoPpA8ST3vFT3x+tPM8D9ht45jul+43f4PtwxF8tb+n16hoNm1SmDAABDLpQkPDxXJrU46TcG4VyDX8Gk38IN3qYAGPeNjVgFN/XmpRkbm4ccWego2+R9Yy+oLB16F78nHZYVg2pdoZSOcfcz9fepcVv1fVHgjZtQ0m9UNZE8NrK8v4Yh5s6j4Xe5Rl1OKP3Zx/Z3XY+zdhJAqRhRA9A9vzouPUcqeHOZJjeEYqMq7UW2qB9EKMXnExte9r9e/u5higglhFLV0LvYgI/lRz4vIZJmMt0hFIfGxs6v2OHjejl2KEhwxSyNy4QztbGjLcabWK60T5opjs9L9azLrK5ZnSZpBriuUFJWVudQ/RHPoXPmvk/XpM9U50WRiFACABgToTToHKV0/6oFE3oJpXTVu1R8RIFRZRZmVQIud2295tHUXR48XmvMTOSEUt3vm/Q5SrHBUnWefkIpfYZRUCgbVGeiuxfoqBLCasil1+bzqoGV3t+gc5TS3yHnf2qIugG6kFUeaaSNt1CqsugbsfzEOjkODdZQZmdUXb8pI5QTSrG8xqHTriedFZYQcaZJnQsWJBYjKoMWHTq/M8XK6kSxlF6vjonzXOOQ6rpCKQq2eM+xvFpMxvvQvvH7EEoAAAileQslBbZ0ZbuYeYliRD2LuWXLlR3SsXFRhLRHX0M64rXFhm4ctiRT9mJQoZQGw1QoKXPhfdMXrcbGMUJp4ULJ/lCVAY091FXzp+QfavDkes5TP02FUu4Z5hYTWahQisOR0rkdqb/TSJtcoaTsUC6jG19+XJWRjUM+45CznO96mwRMOl/I53Bm1uVB2SlndGWuw32OmElNF1yI/p9mbtPh1LH81BVKem79vs9zAdPh1HHEAkIJAAChNG+hpJ64KA6i2PH8p1zwU/BVr50WgHDDMQb7OLwtBi1fm453kFOwdkNT/8Zx800JJX2fe0D1mYdV6Rxx0QKE0sKFUlxAQ7+nh2g68yjfKDJz7NQDrfNom4ey+XfUsVGMxAakry32nMfGpoYZxV7xpoRS/L44Jy/ui1BCKA06RynOJaryx15CKRdH0vowXSE0Zxqm1+v9aLlrK2qOVKgjlFKxlesIqfN9CCUAgDERSr1WvZO5p75JoaTv1NAhTf6NPXaxN09DPopktTgPO5JpvojFiBd+0HmVBZCQikM04rXFlfo0NErBUt9bzGPVu35CKW3Y6pr0nRKKcenpcRNKukcvwZtafNFqk0LJS4rr+Ni48bnj6ogSyRJGyiB57oU+0+peaWNM/ikBpAxkbg6CrsXb5YcSL/LNOKejSaGUrvKoTgZlCWIjFKGEUBpUKMX95yOUND8uPb87g5x58Rw6dUD0WhWz19zDWH97m+vSdE5hXNCnrlBKxU9OKLm8x3lLVUNzKYMAACMulIqaLydsUihJIFncFGHloxisq96jpCCVzh3RkKj0vUxqqHrserw2CasouNzgjKuqNSmUdB/pKkoSZlFATdKqd9F/mhRK6RLvFmxxOKZETBQ70adidkbHpO8KS78jXltcmMMmf1yMOUpV96EJ9C5TCCWE0lILJVlcjCGKFNX3cV/V9ekiKTJfQxyyHDPFcfhevDZ3Ssj/YwZ4Pqve1RFK/j51fMUlxuPICIQSAMCIB1U1tqpWu4vm4WISI/o7fdmrlnXV9nR5Vx/vZY3jNo17V4DRUsdeiSldwSv20ilToP30b7rcrE3nU4NV++kaFTB9zem1OTj7nLpHWXrPuW1qRMf7yD3T3PdpX233sK+4zPM4CCX9LlVZpKqMUu7Fj/E80Xf0f2935idu0++thpEEgxpkykjm3mUif5Jw9ZwNHRsbeFFQ6xw6n0y+5UUh0mtzOZCQ0jl1fv2m+o21b5yfpv+nz0H37/NWPdPU7+VPuj41zrRUcZx4Hpetp5E2mUJJvlhVBl3emhZKyhTp3CoL8YXILl/x++S38mHVw84Ixbmrnm+nDgGVvTRTHK8tCiKdQ1kolb2YZW1SKMW629mxtKMGoQQAMOJBFVsaU6+jgrAaFLH30YE1t1raKAolbGnMgkxCKwpyCSVnVdNFKGikTZ5QKmq8H69JoaTsqYfWRXM2KbcyXDQdG69BQ75zc5qiMIvnzWWA47YmhVK6ol4RlmRHKAEAIJSwASwOH9TQEAX6OHykiUYtQmlyTItPFGHOmxqiavDFRmG6UhiNtMmo053B7Gf2j1yWtleGM5cN9TZ9t4SO6jMNTdPwZmU602uUqNHLmJX50QI6qg81vC5+f8yoqv7UUGUJE2WV4jXnMsAqB9pf/9e2XNY6t0335Puoeqa5UQmeIyhhx2IOAAAIJWxA0/AvBfl0TomGm+hdP7nhYQglrJepEZiba+Ve7WX0J6BOH3tT54TqbnVQaHVLb48vr84Nv6YMAgAQVLEegklj9dUrqnlKTQkkhNJkmnqv1Wvt1cK0/Hk6Z45GGnU61rzF1wQoe6Y5Utrm4XvK9DZRv1MGAQAIqhhCCRsffwLq9Ino9IpzpYpkBdV0wSPKIAAAQRVDKGH4E1CnT4xpdIBXZJVp7lVurhVlEACAoIohlDD8CajTMcogAABBFUMoYfgTUKdjlEEAAILqPE3LtWqS66033kzwQyg1+mLb9KWvGI006nSMMggAQFAdmcreKwFp8ivBD6HUxLyAosF3lWA00qjTMcogAABBFaFEUEUoYTTSqNMbe7EtZYUyCABAUF1CofRXj/1FJxDrpX56y7terqkX+flzvStGDWSZ3iUTj9WKQ/7sW595rOut7dqml3HqvHoju94QH4/VMC7toxcF6jxaxUjvxojnQSiNplDSb613V+lYmYbvxc/10kj5Unx5ZPQpfSaLPhPPqX9Tf/K7jmRaTlh+nftuGmnU6aNWBiWQyIJRBgEACKpLLJTUiFy96ors+ynce6nGrLdJ+MTj9dJNf6bGq8/pa4i2/uXrWt9/+Mic4P+ada/sfOb9JNQQSqMrlCR8cz6lF0X69/dvf8kll8xZ3vfGDdd3PrvyipfMbvv6Aw+1rrnyqq7z6e90zpQ/k+iO15AKfBpp1OkIJYwyCACAUOppbpSqUam3nUv4qEHre3aP/+bXvG628Rp78n38tS+9ZrbXX//Xthdf/qLOOy/uffcHO29O17ZbbnhDNvirwaxFKHQdo9CoRSjlTUJIL4C0T71t8xu7XhSpv9NzKpsZs0nyBW1XdtHCOz2nfVzbY8bI57z0Z17QJdBppMGkCSVlVNWR4Kyu/o7ZWWdf4/Y04xs7tvziV58zlw3W5z6vRyNoX22nDAIAEFRHTiip919iJg610zbfs4Kctn32tz8xu837xkatxJC2PXzfp2b303A7nzMe7yAag7+uwQGcOUqjK5Q0bFLDN5UljA0piyWJcG9zhkhiO87F8HepkRX9RGLbDTc1vHx8nLcRhbeuW0NAfR4aaTApQkmdDLGzwGXiPW9+x+zQaHdexfKX1utxBIE60tJzvvn1N3VlhFU/xLrC56GzAgCAoDrSizk8vu+LrY+//+5OQFbWKBVKMbDecfOtcxq1bhQrQEfx4/lL+r+3SzSlwX9UGrMIpfqLObj3WSLaPql//bmGWKb+4wylhJb3sz/qM/dWy+SHRTJEz+dTdpJhPwOxNrE1NY5ZkzluRZ9jpjLHTFGnNyeUPrPrY13DXfWaiDgUWh1h2k+iqcgMf431uuKCtik2eNsrXnZtRyBZNMWyFoWSs8DF+A+nBgBAKI2rUFJ2KM79UPDz0LkolGTKFBTlkDplftyojT2SMaBXmXomx2HcPUIpb8okqvHk3uRoUSjFuW/yidiTrYZZxXPOWrrvKK4OtsyNtPT7Z2ocM5M5bm2fY6Yzx0xTpzcrlDTEOb5HT5nYVLRoTmmRGf7qel2CyFl+1fmu6531j/WCxVcUSjpe36tyng7howwCACCUhl4oqcffWSI1bNX7r8xRDIBRKMXAqqyQG7Wx8RwzShqG5/Hx0XJD7xBK4yGU4rBNzWtTA8wvrE2FUpz7Jv9zT7bEeuzhjsN3tEhDzhBKCCWE0nezK0iqHv7AW9/VtdCJP5eYibEidlZ4OLQWUvGxyg7HrK4XTPEQvSiUYmcHZRAAAKE0ckIpNoDjfKK4kl0USjGwuodRwyviRN04R8lD7NyAVg+nAm268hlCaXyEkofDqQEV55tpKF1OKMW5a175MC74IfPiInF+k4+V38ZhmwglhNKkCyWVO2WN4siAaFEoKZNbhOGv7qyQWPJw2FhGq8zZqyiUnGWiDAIAIJSGUiipZ17/z5kCpBqYRRjL7oUd4tjyNNjFuUZFWMUsBmkHaJ1HAVvf5Z5HzSdR1gqhNJpCSZkfv/gyNfVGe3imGloa1qnfOgrvVCjFuW850Z425vROLvntH//O3tme79xiDgglhNKkCqVYBtVxobISh95FoRQzSCpnHnanTG8cShvr+1xG1x0oUSilnWyUQQAAhNJQCaVe5mAZh2TEYGixlE7CjYG1CCvgpe9myl2DRJKGcYzLu0EmUSj1Mg3F0fDN3NykKJ4tlG2eVF6E+W+p+P6V127KfqfOG+c/IJTmzUxiB2occyBzXL9FILZmjtlKnd6cUPIiC17hzvV2TijJlMF1RtdlV/Oc4kucizD0LnZyqBNEdbqXGEcoAQAglIbe1IPo1eaqzKsZeUEHBU+JIgsZZZK0n4bSped3ozeuNpaagqh6It3jqMCbNpDVqPb1IJSG1zzHqJ95XpF+V62KpaGf+le+FM+hJcSrhFhs3KViyZPUdV71fMtf03fA+Dvi5HQaaTBOQqmq/FnEuJNLw6RVJmUqh1VCKQ6XrurM8PL+yv5qf4knZ65iFhihBABAUJ1oi8MwlI3iLe7jL5QW2zQR3M8kZhzxJ6BOr7+aqOek5kYJSPw4yy/RlHZCeGh0VTZWHR1xnyIzPwmhBABAUJ1Y03A8zWPy0AwN7xjFdx8hlIbD1COtBp0aWUVYzAF/opFGnT7XlCHtl9F1RknCRyvOKeuqOluZH9XVyshqv1zWNmaHcsOpvcS4VjWVIFNZ1XDYOETP2WRfj8QVZRAAgKA6MULJz0HDLzS8b9IzIQil+Vt8h5J9SsvP40800qjTl9YkrPwuvV7DqSmDAABAUO3xPg4NpVBjNh2/PommJdGDb5zDnwY3rbKl+RKaED4qL6QcQn8C6vR5meYaqfx56f4is3gPQgkAAPpxwRVnuhoXNrmmBQlCQH0Gf8IWYnGp/radpNpFKC1FR0V8FhqmF9+Jh1ACAIA6nHLFma7WhU2uaehhCKjH8Cdsob37xWDvMAKE0oLFuVclVX026R03ZHUBAObHUVee6oGjUYfJ4nK7bXsQf8KaWvmvqPcOI2CUAEZWFwBg2Zl9k/ykr8qFXQyofqljaevwJ2y+pjl/ydLLW6h2GSWAkdUFABgFVrftvCtQVnvDtOzuAgIq/oR12X077or+dLptK6h2GSWAkdUFABgVHnQFqp5fvQOGwEIwLZ4fwrMRf8Lma1pFUi8DDT61k+qWUQIYWV0AgFFiVdnT26lEr33pNTRuJzCQ3nHzremqSA/iT9h87fF9X0wbZ8+QTWKUAEZWFwBgFLkhBlc1cAiwk2F6f5R6nhORdAR/wub7gs8HPvThNJN0pm1rqWYZJYCR1QUAGFW2xcZtUQ7d0Dh3TQpmBaXxyR7p99Tvev11G1KBJDvUUI8j/jRBPqUMkl7oec2VV6X+dLZtm6leGSWAkdUFABiHzNLpTOMZG387twi9jfjTZNuzxWCrJkKzZY+sLlldsroAAIvQG/lgmg3Axtb0O2sVpDX4E9aQKYu0u20rqU6XFbK6ZHXJ6gIALBKaFKwVlLTcrN7NcYEG4NgII/2eh8vfdzX+hDXkUxq2ub1tU1SfQwNZXbK6ZHUBAGDBqHH3DI8B8CkYM8jqktUFAABYEH7/yBYeBeBTMIaQ1SWrCwAAMC+OFxdXfQPApwBGGzK6AAAADbC+6O6ZW8UjAXwKYKQhowsAANAA+4ruYQx38kgAnwIYacjoAgAALBC9gC9dFeopHgvgUwAjCxldAACABrityE+O3cCjAXwKYCQhowsAANAARysatQd4NIBPAYwcZHQBAAAaYE1RvUTumYJ3UAA+BTBqkNEFAABogN1F73dTbOMRAT4FMFKQ0QUAAGiAZ/s0ao/xiACfAhgZyOgCAAA0wKai/9vOFXCv5lEBPgUwEpDRBQAAaIBHazRqZffzqACfAhgJyOgCAAAskKm2navZqD3F4wJ8CmDoIaMLAADQANM1G7S2LTwywKcAhhoyugAAAA1wfMBG7SEeGeBTAEMLGV0AAIAGWD9gg1Z2vm2reHSATwEMJWR0AQAAGmDfPBq1sjt5dIBPAQwlZHQBAAAWkRhEAfApgNGAjC4AAACNWsCnACCBjC4AAACNWsCnAIDyBwAAQFAFfAoAKH8AAAAEVcCnAIDyBwAAQFAFfAoAKH8AAAAEVcCnAIDyBwAAQFAFfAqfAqD8AQAAEFQB8CkAyh8AAABBFQCfAqD8AQAAEFQB8CkAyh8AAABBFQCfAqD8AQAAEFQB8CkAyh8AAAAQVAGfAqD8AQAAAEEV8CkAyh8AAAAQVAGfAqD8AQAAAEEV8CkAyh+PAgAAgKAK+BQAUP4AAAAIqoBPAQDlDwAAgKAK+BQAUP4AAAAIqoBPAQDlDwAAgKAK+BQAUP4AAAAIqoBPAQDlDwAAgKAK+BQAUP4AAAAIqoBPAQDlDwAAgKAKgE8BUP4AAAAIqgD4FADlDwAAgKAKgE8BUP4AAAAIqgD4FADlDwAAgKAKgE8BUP4AAAAIqgRVwKcAKH8AAABAUAV8CoDyBwAAAARVwKcAKH8AAABAUAV8CoDyBwAAAARVwKcAgPIHAABAUAV8CgAofwAAAARVwKcAgPIHAABAUB0f7mnbmeRZY+Nlp9u2HVen/GGUQQAAgipGQK3PeZ75xPgVDB+UP8ogAAAQVLEhDag878kxGD7wS8ogAAAQVLEhDaiz3/n0/z6HjZnRSBudOh1/pQwCAABBFRuugIpP4VNAnY5RBgEACKoYARWfwqeAOh2jDAIAEFQxAio+hU/RSKNOxyiDAAAEVYyAik9hNNKo0zHKIAAAQRUjoOJTGI006nSMMggAQFAlqBJQ8SmMRhp1OkYZBAAAgioBFZ/CaKRRp2OUQQAAIKgSUPEpjEYadTpGGQQAAIIqhlDCaKRRp2OUQQAAIKhiCCWMRhp1OuWPMggAAARVDKGE0UgDyh9lEAAACKoYQgmjkQaUP8ogZRAAgKCKIZQwfAoofxhlEACAoIohlDB8Cih/2ISVwZVtW7EE3zNFFQEABFUMoYTRSKNOxyiDvTgVbFPb9ifbbI+2bXXm+O3l589mBMipHrY/7LezbSfD/TzdtlvC576mbcn5j5fbr655r3vadib5nq3h88PhOWzqce2Hk/1ztinz/cfD59uopgAIqhhCid8AnwLqdGx4y2CrbLjvbdvath0stx0v/3+sFEHadiRz/Ey4jjszn8lOlJ+fDdv2lPvcX34mAXOobU+Uf18IYsPXNJ0RYq3yuvuxs9xXgmxfec4Lpa1L7mVL2zaGa7WdLz9/Itn/RGbfjZlruKdtRyvuBQAIqhhCCcOngPKHDZlQmgl/50SJhsSdK4VOZH25r4XUUxXfsTbzPUWZCbpQiqQ1Yfvucvu+BoWSz7E5bNtXnmNnRiilTAdBtzbZf+0Az3saoQRAUO3Yj579f61/959+2rFRC0K69i8//p3Wo3/+l9nr//HJMyNzb+MklA489o3WXb/7sdYDf/SnQ/Fsf/B3/6fWft849qPW++68p/X6zTd3TP8/OvPXc/YZpntDKFGn50w+KhuGum/mb/5Xrf1UX9+/73OtN976jtYNN/7j1ta339H61Ocea/2X/3l2aO9tCITS7lIAyLYlmZQoNLR9R5mB0v9vGEAoOcvzSLJd85Sm+lyT7LkBhIozV8+V2bMtxdz5UFVCaVPIPm3J7L85ua6rEUoABNW+pgbfsPRq/tsf/NfWvR//VC2B9Ovvnm694NKVXYHola++vtOQ9X4SUKPSYztOQuntv7Gjc041dpbbn/7Rljd1GlX99v3g3f88/Q06dsmKFa29f/CFOeVlue8NoUSdXue3Vx24XP4ngfORB/+wdfnPvajvvuqQeOnV12TL4C9f/7quzo5huLchEkqpnSkzSFHMnC4zTVM9RE8vobS33L63ZjaoyuoIJV3jseQ4Zcj29BFKa8r71PZdFcIqtVMIJQCC6sgIpS8efqIjfPo1QBV8X/u6G7vEkf62aLrshS+c7WlEKE22UPJ19BNKsQxIgH/h0NHWZ//0X3V8y2JJoguhBAil+qZ6uM79KZNkkaR/VcZ03eq8UNnTdpVlhFJWKB0tBYzs0TKbIsGwqtzv9rCfhMrGcp9zxdxFHZoSSvGa9pZCZ9ChbxvLY2fK640CKBVKGnLo+VUHM+eaCZmu6WDbEEoABNVGhJKGTtQZvrSQoRB1G6DKOGk/CaMYKI88+ePWqit+vvPZHe/dWSmUdI3pUI70Hv7jX/+PvmKt3z4IpYu/pTKA+n1lenaf/sJXOsPa9FvG5ygxon0kVORz+lz7aShfOvxN+2nYpbfpPP4O/a3PLKhv3vrWnkPlrvz7V3X5Tcxc/vxLruw03nTNqZ/Ga9TnqV/pOu/+6AOd80qsSXzFffS3zvfvf/zfOudRwzAd6odQok5vQiipPMjXVMZk8kv57Z985V93HWeflO/rsx07d80pp7E86//ernKrbfJ7bXddLfP23LUq6+Q6XWUhfrbrvn/R2a7McO7e9J26D92POzOiANP3fuCf3jt7H/H8rm/0bL727e93zhOzxyM4R0kcLi4OsxNP9Mjw3FlTKO0o8otEbCxF0e19rmmQOUp7M9kui71jFULJ33uiyC9bzhwlAIJqs0JJDUw1DpXl0bAHf66x4w6MCjjaR/squFqk6N8YbPSZ94vfq22yr37re13HKyh6e+561/7iP+jsp8CXfqbGqsa4O2BGoaTP1OjV/zUURPtF4aPzKRvl/XU9cR8FUV3XfZ/8zGzv52++/06EUg2hFHuWb7rl17ruVb+JG2H2xZet/YXZ38p22zt/s2uOQtrLnIpiX0O/e1TjzZ/nGnJpB0Gva1Q2Kvqie8OjxWu2H6277pWzn6eiEKFEnd6EUHJ5UB2eDlmWyEh9UvOD4j6qM10+YnmOnWMq787gxn1sVZld1wm6tly2SZa7tziyoChHE7ju1zHXverVc65B+6hTLZZlxTjX/fr/CAslZZGeLrcrW3J1yDDFDM8jRX5RhyqhtKrMCulct4QsjofI3d+gUHqq3Hdn2HZPue1wRij5M93jmopz1hFKOtcmhBIAQbWWUHKwVPBQT56Cpht96vWOwVJBV6ZgFwOXevri+XXOqkBeVyhJpA3SoIyNZ51fjW0LP92Pg7waCt5HDQrds49z4HVDQ9en56KGg7MMCKX6Qkk+ogaXh1pqm4V19EVlgdSgUW+zfc+/eR2hVDejJN/zcWmDrF95UVnQvfl6dJ0+h79bvdjaR6I7/Z5YzpyZ6pXtRChRpy9UKHnOneo1lYu0brZPqn5T2VCmxXWmhqLWFUqDZJTU6VDUGCKb3ps6zXR9Oq+zwspAuVyrflEnhOoRZWotnLxPLMuq81XX6LgRn6Mke6YUMl4YYV/mfBYvN9QQSs7qeAicFlo4V1xcSW+qQaG0NfkeLwQRlyGPQmmm6D0vquizz97wfE8hlAAIqgMJpdiL70Cr4JoGS2VZvJ97It1IriOU6g69i99ZZ3x6bDyrYe7skRoB2uagqP0UPKM4c+D10BTfv45VBiTX04lQ6i+U4jO2mHADyT4g4RCfrXqaozCqI5TqzlEadI6e95d/5PwyNhydjfKwwHSfxchMIpSo03sJJXVoxaFyVXV/zDJJjHg/+XIdoTTIHCV/56BCKXZ++P5ineAyqLpE9Y4Fn/eJZbLuyphDJJR2lsIktT3FxflJe8tt6zPn21F+FufprA7nyLGheH4elK7jyVKIrcpc0+bkuP3l9tU171XHHyouvuvoYPndZk+4rz0Vz8FW9NnHwwY1rPAkQgmAoDqQUIqByI3TXOM3jl9Xr78FRdNCScP90oxVXaEUG965+3NjVo3WOBRKy9PmhCKLOcxPKPVqWFX5Sup7TQql2FjMzbFL57Tl/DR3f+qx1/dbcMdheKlQanpeBEIJoVQllPqVGftkWr/G8zUtlLxoSjpH0KMI0nlLuc6y3P0pQyRxlA6B9fdU1TdDLJROlFmZlRSdxtlZijOhIYu7EUoABNXGhFLsjXPwVXBqWijJPJY89nja1ODUXCNnLapWvUvvT2LIw8A0PEOB1EMBezU0EEqLI5Q09yeeT0PSYm94k0JJvuuGVCpYJJDkK/IF+0odoSRRboGkDKuyklHkp0Jpsd7JhFBCKM1XKMWyoDJShHl8VeU5zRDXFUqqsx0j0qGnHrKq8ubP6gglLxChukTn0HWn9cEICiXblhH2+VM9bP8yXZMWgHiouDjH6VR41gglAITSwoWSh7XFVemUlbEIKZKhSsrezFcoacJ8UU6mjysueYWyIgwZrCuUfFy8Xzd0EUpLL5Rij7Z+Vy/g4f1SX/TqWFVCqd/QNu8nP7DIVqNMx/mcGn5UVyjpHP7bjbu4DaEEwy6U9MJl+67FiofExrmiXnlSYsqdWDmh1GuYsuYQubNCZc77qsy5wypm8usIJV1/EebUxm0jKpTistZrRtjnt/Sw9UNyjdvCs15HNQWAUFqwUFJDVkFLPecWHVqSNW0gavEDTar1nJOcUNLxOk/VUuPa7oCs71VmSXOk3JhWwPWqRnWFks/nuVbxmXgBAYTS0golZfjkJ/5dPTcsXYBBDSiZG1TxXix0dKyG4KRDeOLy9/4ei3rPYyuSIUF1hFLMUuke1Fiz8C7CAiEIJer0YRVK7uxS2bIvu06Pq49qf51T/3o/l2eVqyK87y6KltRU9xZhZbpYXlQW49LfdYSSy77iib7XC1cUmTlKIyKUAABglIVSvD4FRWd71CsZ5/z4cwsqB7s4Ybhf41G9mDGQxgAbVy2qK5Ri5sCNdF+zh/ghlOqbl1LXsLO4lLwsihV97iXX04aL359i/4qLQMinJF7cMPPn/o44Cd1+pnNZQFcNwZMfxCXi5WP6/dN3H8V7q7o/3YvFls6p+/H92ke9DH/TK23RSEMoxdcvxDKpf71Prsy4bpQo8ipxKjsqG7EcqA72vvpc59UQulie4wiAIlk+v2oFyrikt8q3VqNL34+U3lvu/iTS3BFTlMt+KzOmfRzDql5dQRkEACCoznnpXhQmfulgDFB+yaeHREWhpP08P0j/psMs1AjVmPH4ub8jLgShIXzaR43Kfi/e1Dk0rE/7K4jr/OlLYNOXkPa6P2W7dC6/uFCNau3jYYV+YWN8ySlC6dyiiHY33PQb93qxr8R4nRf/9nvJ8HxeOjzIuZpc8ptGGnX6YlraieQVPnstsNNv9U/V/xIuda9B+3uu30LvR987yHdTBgEACKqNNQCLHquFYQilhQoljEYadfryCiWMMggAQFBFKCGUEEr4FCCUEEqUQQAAgurCTcMjNG9Jtlgv6yOgTpZPaSidRHfVogsYjTTq9MU1lT2VwbiiKEYZBAAgqGIIJQyfAup0jDIIAEBQxQio+BQ+BdTpGGUQAICgihFQ8Sl8ahg4mNieGsfsyRy3us8xmzPHbKZOxyiDAAAIJYyAik/hU8PYSEuvbabGMTOZ49b2OWY6c8w0dTpGGQQAQChhBFR8Cp9CKCGUMMogAABCCSOg4lMYQgmhhFEGAQAQShgBFZ/CEEoIJYwyCACAUCKoElDxKWzkG2lrE1tT45g1meNW9DlmKnPMFHU6RhkEAEAoYQRUfAqfopFGnY5RBgEACKoYARWfwmikUadjlEEAAIIqRkDFpzAaadTpGGUQAICgSlAloOJTGI006nSMMggAAARVAio+hdFIo07HKIMAAEBQJaDiUxiNNOp0jDIIAADzCKrY2Bs+hY2qTwHlD6MMAgAsORcINBNhZ/EpbIR9Cih/GGUQAGDJ2U9gHXs737Z78ClshH0KKH8YZRAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGCUWd22O3kMMIZ+vbdtT7TtybY91LYN4fPNbTtYYXuSc20qj58pz6fPVw1wLbeX17Kxx+dV13J7uc/O8u9bkmP3l9sjK9u2q21Hyms+1LYdbVtR495l68M1bU7OvafcvhoXA6jkzgHrCAAAGCLUkNretqNtu9C2Fo8Exsy/ny39+kzbTpf/v1CKHjFdbsvZTDjX3rBd5zpf/v90KSjqcLA8Zrri8709rmVvco5T5f2ZU0n5XdO2Z8L9PhfOdaTGvcu2hGtKr3mm3L4WNwOopFXWFYfK8gQAAEOOepNvKRtc5zKNI4BxYVvp0zHTsrtCLBzscZ4t5T5ni4uZHImUh8rtTzcslKZrnEO2r4dQOhLuyz3a64Ng2lrzuxFKAAsTStFUTu9v29U8GgCA4ULDfQ4U3T3LLYQSjDG3lT79bCkMVpYm4bNmAKFkcbI789nJ8rMbllgoqZf6QnExmxWF0tXl/58rLg6zMzeU978KoQSw5ELJprJ7rHi+M2cljwkAYHlQI+b+4uLwozoGMC5IJBwPvi1x8WTZOCkSoXSuFBvRvN9T5T4b5yF+5iOUzmSuZVNyjvuL7uGBUShZIB4J515d1ge21QMKpWfL77KdRSgBzFsoRVN5VyfmBh4XAMDi40UZjg8gjhBKMK54QYOZ4uK8ojjnx0LpdCIEZB6e5qzRuiUSSs9mrmVjco61pejT/3ckQml7+f9Dme+2HUQoAQyFUIr2VMECEAAAi9IYTBdlwLBRsL2LWC40BG1LaHS4nKiMnEuEUq+hd8fKfW4r/9b5PHTPYuX2BoVSnaF3EijrS/F3pjQLpY3l/58Jx3nFvaPzFErLNfRuL2UEm0BjAQgAgIaYLvKLMmDYpAslL7awO+lUOBtERR2htKvcR6JIw/n2lA2Z/UF0TS2DUBIPFvmMsLNg25Lj70EoYdhImTo81tHUAQBYWM+5Gm9PE1QwhNIsG0shI3u0/K4T5fc+kQilk8Xc9wjtD+LKS20/VZ4nLopyf83rscg5nvmu1UEU5D7fUyGUVgZRFIXSbeHej5SC6skiv2IeQgnDhs+Ol+VuiiYOAECzjcN9Rf/V7ZijBJPAtqQsWDikq97l7FQ4z+pSpJxPPj9X2q4BhFLO1vYRBTMVQkncUlF+txRz5yo+Vd7zihERSgCjzKDx93QZv9fz6AAAFh81lNSTfhahBBPO1WWjfqG9syvK81ho6f+a93PbEN/7VEP3DgDNC6ULoQ5ZwSMDAFh6NDzn9rIyPo9QAmgcZ5yqbDOPCAChFEwrSWr+5BoeEwDA8KAVu3YWF4fOIJQAEEoAsPhCScN0NcpjE48GAGD4yS0CAQAAAM0JJRZmAAAYcbwIBAAAACwcFmYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIBB+f9eJZbvYpg/CgAAAABJRU5ErkJggg==)\n",
        "\n",
        "Image source: http://www.tensorflow.org"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gENrH_GK5vRN"
      },
      "source": [
        "**EXERCISE 1** Complete the code of the `Model` class:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "FAy_g782RgJ2"
      },
      "outputs": [],
      "source": [
        "class Model(nn.Module):\n",
        "    def __init__(self, input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob=0.):\n",
        "        super(Model, self).__init__()\n",
        "        self.hidden_dim = hidden_dim\n",
        "        self.n_layers = n_layers\n",
        "        self.embedding = nn.Embedding(input_dim, embedding_dim)\n",
        "        self.gru = nn.GRU(embedding_dim, hidden_dim, n_layers, batch_first=True)\n",
        "        self.fc = nn.Linear(hidden_dim, output_dim)\n",
        "        \n",
        "        \n",
        "    def forward(self, x, h):\n",
        "        emb = self.embedding(x)\n",
        "        out, h = self.gru(emb, h)\n",
        "        out = self.fc(out)\n",
        "        return out, h\n",
        "    \n",
        "    def init_hidden(self, batch_size):\n",
        "        \" Initialize the hidden state of the RNN to zeros\"\n",
        "        hidden = nn.Parameter(torch.zeros(self.n_layers, batch_size, self.hidden_dim)).to(device)\n",
        "        return hidden\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "egeSLqS78IK3"
      },
      "source": [
        "Now we can instantiate the model as follows:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Jdw_334VOY4",
        "outputId": "a9c63151-4ff8-4bd2-ee5e-09c95c744215"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model(\n",
            "  (embedding): Embedding(65, 300)\n",
            "  (gru): GRU(300, 1024, num_layers=2, batch_first=True)\n",
            "  (fc): Linear(in_features=1024, out_features=65, bias=True)\n",
            ")\n"
          ]
        }
      ],
      "source": [
        "input_dim = len(dataset.uniq_chars)\n",
        "embedding_dim = 300\n",
        "hidden_dim = 1024\n",
        "output_dim = len(dataset.uniq_chars)\n",
        "n_layers = 2\n",
        "drop_prob = 0.2\n",
        "\n",
        "model = Model(input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob).to(device)\n",
        "\n",
        "print(model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zQObv2NoBirq"
      },
      "source": [
        "As expected both inputs and outputs have the same shape, and the predicted characters are just random since the model is not trained yet."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BiQfapKgB2HQ"
      },
      "source": [
        "## Train the model\n",
        "\n",
        "Training a language model is essentially a classification problem. Given the previous RNN state, and the input character at this time step, predict the class of the next character.\n",
        "\n",
        "**EXERCISE 2** which loss function do you think we shall use to train our model?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "qIpHzp4-Vmsg"
      },
      "outputs": [],
      "source": [
        "def train(dataloader, model, batch_size, sequence_length, num_epochs):\n",
        "    model.train()\n",
        "\n",
        "    criterion = nn.CrossEntropyLoss()\n",
        "    optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
        "\n",
        "    losses = []\n",
        "    for epoch in range(num_epochs):\n",
        "        state = model.init_hidden(batch_size)\n",
        "\n",
        "        for batch, (x, y) in enumerate(dataloader):\n",
        "            x = x.to(device)\n",
        "            y = y.to(device)\n",
        "            optimizer.zero_grad()\n",
        "            \n",
        "            y_pred, state = model(x, state)\n",
        "            loss = criterion(y_pred.transpose(1, 2), y)\n",
        "\n",
        "            state = state.detach()\n",
        "\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            if batch%30 == 0:\n",
        "              print({ 'epoch': epoch, 'batch': batch, 'loss': loss.item() })\n",
        "              losses.append(loss.detach().cpu().numpy())\n",
        "    return losses"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ufa42rQ6Cz8F"
      },
      "source": [
        "Now train the model for 20 epochs and visualize the training loss curve."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "MaBXfqTXZsgp",
        "outputId": "55edbe8d-e847-4566-dfc9-8482f52a3e41"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'epoch': 0, 'batch': 0, 'loss': 4.174623012542725}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.3052425384521484}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 1.9674627780914307}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.8028618097305298}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.7222049236297607}\n",
            "{'epoch': 0, 'batch': 150, 'loss': 1.6480488777160645}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.554150104522705}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.5735938549041748}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.497287631034851}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.4575371742248535}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.4505882263183594}\n",
            "{'epoch': 1, 'batch': 150, 'loss': 1.463680386543274}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.3714479207992554}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.4225115776062012}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.4018032550811768}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.4137080907821655}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.3751800060272217}\n",
            "{'epoch': 2, 'batch': 150, 'loss': 1.3775219917297363}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.3021734952926636}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.3244497776031494}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.31913423538208}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.2958651781082153}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.3075731992721558}\n",
            "{'epoch': 3, 'batch': 150, 'loss': 1.3164814710617065}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.2252668142318726}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.2563577890396118}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.285258173942566}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.2795898914337158}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.2695810794830322}\n",
            "{'epoch': 4, 'batch': 150, 'loss': 1.2714207172393799}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.1779470443725586}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.2181473970413208}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.2299141883850098}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.233696460723877}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.2541046142578125}\n",
            "{'epoch': 5, 'batch': 150, 'loss': 1.228747010231018}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.1724560260772705}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.204108715057373}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.2056714296340942}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.2120623588562012}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.1543102264404297}\n",
            "{'epoch': 6, 'batch': 150, 'loss': 1.2192343473434448}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.1174296140670776}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.1354674100875854}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.1357295513153076}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.1659789085388184}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.1959025859832764}\n",
            "{'epoch': 7, 'batch': 150, 'loss': 1.188941478729248}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 1.0505238771438599}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.0729459524154663}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 1.121421456336975}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.1321232318878174}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.1172329187393188}\n",
            "{'epoch': 8, 'batch': 150, 'loss': 1.1590057611465454}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 0.9981228113174438}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 1.041314959526062}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 1.057134985923767}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 1.067977786064148}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 1.088416576385498}\n",
            "{'epoch': 9, 'batch': 150, 'loss': 1.1305261850357056}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 0.9976438283920288}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 1.0107066631317139}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 1.0350852012634277}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 1.0311623811721802}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 1.059543251991272}\n",
            "{'epoch': 10, 'batch': 150, 'loss': 1.0693832635879517}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 0.9167323112487793}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 0.9547595381736755}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 0.9620925188064575}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 0.9982872605323792}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 1.014163851737976}\n",
            "{'epoch': 11, 'batch': 150, 'loss': 1.034753441810608}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 0.8921875953674316}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 0.9249266982078552}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 0.936279296875}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 0.9336860179901123}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 0.9693022966384888}\n",
            "{'epoch': 12, 'batch': 150, 'loss': 0.979124128818512}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 0.856057345867157}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 0.9008118510246277}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 0.8953336477279663}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 0.9155623912811279}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 0.9227483868598938}\n",
            "{'epoch': 13, 'batch': 150, 'loss': 0.9558104872703552}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 0.8035022020339966}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 0.8232898116111755}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 0.8473919630050659}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 0.8802714347839355}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 0.9010949730873108}\n",
            "{'epoch': 14, 'batch': 150, 'loss': 0.9100486040115356}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 0.7718803286552429}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 0.791857898235321}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 0.8248854279518127}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 0.8302310109138489}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 0.8589054942131042}\n",
            "{'epoch': 15, 'batch': 150, 'loss': 0.8820571899414062}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 0.7231884002685547}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 0.7718822360038757}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 0.7965180277824402}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 0.7915220856666565}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 0.8081531524658203}\n",
            "{'epoch': 16, 'batch': 150, 'loss': 0.8413552641868591}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 0.6986593008041382}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 0.7251306772232056}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 0.7447558045387268}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 0.7773243188858032}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 0.8086738586425781}\n",
            "{'epoch': 17, 'batch': 150, 'loss': 0.8225288987159729}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 0.666923463344574}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 0.7111355662345886}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 0.7442524433135986}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 0.7569013833999634}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 0.7753151059150696}\n",
            "{'epoch': 18, 'batch': 150, 'loss': 0.7976306080818176}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 0.6198318004608154}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 0.6909986138343811}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 0.7031348943710327}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 0.7353765368461609}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 0.7460477352142334}\n",
            "{'epoch': 19, 'batch': 150, 'loss': 0.7731480598449707}\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV5bn+8e+TmSQkgUyEBAjzjEwiiFUUVASUnmpbPGrVDvZY+1NbT3uqto7ntLWn1rF1aLVq9VhnBUUrAs6CBmQeg0xJgISEBEjI/P7+2DsxI9kJgWSF+3Ndudh7r7X3fhaL62blWe96lznnEBER7wvq6AJERKR9KNBFRLoIBbqISBehQBcR6SIU6CIiXYQCXUSki1Cgy0nHzJ4ys/8OcN0dZjbjWD9H5ERQoIuIdBEKdBGRLkKBLp2Sv9XxCzNbY2bFZvaEmSWb2dtmdsjM3jOzHnXWv8jM1ptZoZm9b2bD6ywbZ2Yr/e97AYho8F1zzGyV/72fmtmYNtb8IzPLNLMCM5tvZr39r5uZ3WdmuWZ20MzWmtko/7JZZrbBX1u2mf1nm/7CRFCgS+d2MXAuMAS4EHgbuAVIxPdv93oAMxsCPA/c6F+2EFhgZmFmFga8DvwD6Am85P9c/O8dBzwJ/BiIBx4D5ptZeGsKNbNzgN8B3wFSgJ3AP/2LzwPO9G9HrH+dfP+yJ4AfO+e6A6OAJa35XpG6FOjSmT3knNvnnMsGPgKWO+e+dM6VAq8B4/zrfRd4yzm3yDlXAfwR6AacDkwGQoH7nXMVzrmXgS/qfMc1wGPOueXOuSrn3NNAmf99rXEZ8KRzbqVzrgy4GZhiZulABdAdGAaYc26jc26P/30VwAgzi3HOHXDOrWzl94rUUqBLZ7avzuMjTTyP9j/uje+IGADnXDWwG0j1L8t29Weh21nncT/gJn+7pdDMCoE+/ve1RsMaDuM7Ck91zi0BHgb+DOSa2eNmFuNf9WJgFrDTzD4wsymt/F6RWgp06Qpy8AUz4OtZ4wvlbGAPkOp/rUbfOo93A//jnIur8xPpnHv+GGuIwtfCyQZwzj3onJsAjMDXevmF//UvnHNzgSR8raEXW/m9IrUU6NIVvAjMNrPpZhYK3ISvbfIp8BlQCVxvZqFm9i1gUp33/hX4DzM7zX/yMsrMZptZ91bW8DxwtZmN9ffff4uvRbTDzE71f34oUAyUAtX+Hv9lZhbrbxUdBKqP4e9BTnIKdPE859xm4HLgIWA/vhOoFzrnyp1z5cC3gKuAAnz99lfrvDcD+BG+lsgBINO/bmtreA/4DfAKvt8KBgLz/Itj8P3HcQBfWyYf+F//siuAHWZ2EPgPfL14kTYx3eBCRKRr0BG6iEgXoUAXEekiFOgiIl2EAl1EpIsI6agvTkhIcOnp6R319SIinrRixYr9zrnEppZ1WKCnp6eTkZHRUV8vIuJJZrazuWVquYiIdBEKdBGRLkKBLiLSRSjQRUS6CAW6iEgXoUAXEekiFOgiIl2E5wJ9895D3PvuZvIPl3V0KSIinYrnAn1b3mEeWpLJ/sPlHV2KiEin4rlADw32lVxRpRu7iIjU5cFA990asqxSgS4iUpfnAj1MR+giIk3yXqCHKNBFRJriuUCv6aGXq+UiIlKPZwNdR+giIvUFHOhmFmxmX5rZm00sCzezF8ws08yWm1l6exZZV03LpbzKHa+vEBHxpNYcod8AbGxm2Q+AA865QcB9wD3HWlhzwtRyERFpUkCBbmZpwGzgb82sMhd42v/4ZWC6mdmxl9dYaIjvY9VyERGpL9Aj9PuBXwLNpWgqsBvAOVcJFAHxDVcys2vMLMPMMvLy8tpQroYtiog0p8VAN7M5QK5zbsWxfplz7nHn3ETn3MTExCbvcdqi0BC1XEREmhLIEfpU4CIz2wH8EzjHzJ5tsE420AfAzEKAWCC/HeusVdtD1xG6iEg9LQa6c+5m51yacy4dmAcscc5d3mC1+cCV/seX+Nc5LsNQaoctVmqUi4hIXSFtfaOZ3QVkOOfmA08A/zCzTKAAX/AfF8FBRnCQqYcuItJAqwLdOfc+8L7/8W11Xi8Fvt2ehR1NaLCp5SIi0oDnrhQFX9tFJ0VFROrzZKCHhwSp5SIi0oAnA11H6CIijXk20HWELiJSnycDPSwkiApNziUiUo8nAz00OEi3oBMRacCTgR4WrHHoIiINeTPQNcpFRKQRTwa6ToqKiDTm2UDXsEURkfq8G+ga5SIiUo8nA11XioqINObJQA8NNrVcREQa8Gig6whdRKQhTwa6hi2KiDTmyUDXlaIiIo15MtB1hC4i0pg3Az1Yk3OJiDTkyUAPDQ6iqtpRVa1QFxGp0WKgm1mEmX1uZqvNbL2Z3dnEOleZWZ6ZrfL//PD4lOsTGmIAaruIiNQRyE2iy4BznHOHzSwU+NjM3nbOLWuw3gvOuZ+2f4mNhQX7/h8qr6omIjT4RHyliEin12KgO+cccNj/NNT/06G9jrAQX6BXaKSLiEitgHroZhZsZquAXGCRc255E6tdbGZrzOxlM+vTzOdcY2YZZpaRl5fX5qJD6xyhi4iIT0CB7pyrcs6NBdKASWY2qsEqC4B059wYYBHwdDOf87hzbqJzbmJiYmKbi64J9IpKnRQVEanRqlEuzrlCYCkws8Hr+c65Mv/TvwET2qe8ptW0XHSELiLytUBGuSSaWZz/cTfgXGBTg3VS6jy9CNjYnkU2FBbsG+WiCbpERL4WyCiXFOBpMwvG9x/Ai865N83sLiDDOTcfuN7MLgIqgQLgquNVMNRpuegIXUSkViCjXNYA45p4/bY6j28Gbm7f0ppXO8pFgS4iUsuzV4qCWi4iInV5O9B1hC4iUsuTgR5e23LRsEURkRqeDHSdFBURacyjga5hiyIiDXk00NVDFxFpyJOBHq5hiyIijXgy0DVsUUSkMW8Guo7QRUQa8WSghwVr2KKISEOeDPSaUS5larmIiNTyZKCbGaHBppaLiEgdngx08LVddAs6EZGveTbQQ0OCNA5dRKQO7wZ6cJBaLiIidXg20MOCgyjXPUVFRGp5N9BDdIQuIlKXZwM9NNh0paiISB2B3CQ6wsw+N7PVZrbezO5sYp1wM3vBzDLNbLmZpR+PYutSD11EpL5AjtDLgHOcc6cAY4GZZja5wTo/AA445wYB9wH3tG+ZjYVplIuISD0tBrrzOex/Gur/aXg2ci7wtP/xy8B0M7N2q7IJocFBarmIiNQRUA/dzILNbBWQCyxyzi1vsEoqsBvAOVcJFAHx7VloQ2FquYiI1BNQoDvnqpxzY4E0YJKZjWrLl5nZNWaWYWYZeXl5bfmIWr5RLhq2KCJSo1WjXJxzhcBSYGaDRdlAHwAzCwFigfwm3v+4c26ic25iYmJi2yr20ygXEZH6Ahnlkmhmcf7H3YBzgU0NVpsPXOl/fAmwxDl3XA+fNcpFRKS+kADWSQGeNrNgfP8BvOice9PM7gIynHPzgSeAf5hZJlAAzDtuFftplIuISH0tBrpzbg0wronXb6vzuBT4dvuWdnRhGuUiIlKPh68UVctFRKQujwe6RrmIiNTwbKCrhy4iUp93A90/bPE4D6YREfEMzwZ6aLCv9MpqBbqICHg40MNCfKXrxKiIiI9nA73mCF1DF0VEfLwb6P4jdJ0YFRHx8WyghwfXtFzUQxcRAQ8HemiIb7p1tVxERHy8G+jBOikqIlKX5wNdR+giIj6eDfQwnRQVEanHu4Fe03LREbqICODhQA/VKBcRkXo8G+i6UlREpD7PBnposG/YYplaLiIigIcDPUzDFkVE6vFuoKvlIiJST4uBbmZ9zGypmW0ws/VmdkMT60wzsyIzW+X/ua2pz2pPGocuIlJfizeJBiqBm5xzK82sO7DCzBY55zY0WO8j59yc9i+xabpSVESkvhaP0J1ze5xzK/2PDwEbgdTjXVhLvr6wSMMWRUSglT10M0sHxgHLm1g8xcxWm9nbZjaymfdfY2YZZpaRl5fX6mLrClPLRUSknoAD3cyigVeAG51zBxssXgn0c86dAjwEvN7UZzjnHnfOTXTOTUxMTGxrzcDXwxbVchER8Qko0M0sFF+YP+ece7XhcufcQefcYf/jhUComSW0a6UNBAcZZgp0EZEagYxyMeAJYKNz7k/NrNPLvx5mNsn/ufntWWgT30lYcJAm5xIR8QtklMtU4ApgrZmt8r92C9AXwDn3KHAJcK2ZVQJHgHnOueN+tjIsOEg9dBERvxYD3Tn3MWAtrPMw8HB7FRWo0JAgtVxERPw8e6Uo+I7QKyo1bFFEBDwe6KEhph66iIiftwNdJ0VFRGp5OtB9LRcFuogIeD3QQ3SELiJSw9OBHh0eQtGRio4uQ0SkU/B0oPftGcnugpKOLkNEpFPwdqDHR7L/cDmHyyo7uhQRkQ7n6UDv1zMKgF35OkoXEfF0oPftGQnAroLiDq5ERKTjeTvQ432BvlNH6CIi3g702G6hxEWGslMnRkVEvB3oAP000kVEBOgCgd43PkotFxERukCg9+sZSXbhEU2jKyInPc8Het+ekVRVO3IKj3R0KSIiHcr7ga6RLiIiQBcI9H41ga4ToyJykvN8oCd3jyAsJEgjXUTkpNdioJtZHzNbamYbzGy9md3QxDpmZg+aWaaZrTGz8cen3MaCgoy+PSPZma+rRUXk5NbiTaKBSuAm59xKM+sOrDCzRc65DXXWuQAY7P85DXjE/+cJ4Qt0HaGLyMmtxSN059we59xK/+NDwEYgtcFqc4FnnM8yIM7MUtq92mb07RnJroISnNMNo0Xk5NWqHrqZpQPjgOUNFqUCu+s8z6Jx6GNm15hZhpll5OXlta7So+gXH0lJeRX7D5e322eKiHhNwIFuZtHAK8CNzrmDbfky59zjzrmJzrmJiYmJbfmIJtWOdFEfXUROYgEFupmF4gvz55xzrzaxSjbQp87zNP9rJ8TI3rEArNh54ER9pYhIpxPIKBcDngA2Ouf+1Mxq84Hv+Ue7TAaKnHN72rHOo0qOiWBYr+58sKX92jgiIl4TyCiXqcAVwFozW+V/7RagL4Bz7lFgITALyARKgKvbv9SjO3NIIk99soPiskqiwgPZLBGRrqXF5HPOfQxYC+s44Lr2KqotzhycyOMffsWyr/KZPjy5I0sREekQnr9StMbE9B50Cw3mQ7VdROQk1WUCPSI0mMkDeqqPLiInrS4T6ABnDUlkR36Jhi+KyEmpSwX6mUN8Y9vVdhGRk1GXCvT+CVH06dmN9zcr0EXk5NOlAt3MmDUqhaWbc9m4p00Xs4qIeFaXCnSAa6cNJKZbKP/91gZN1iUiJ5UuF+hxkWH8bMYQPsnM572NuR1djojICdPlAh3g30/ry6CkaH67cCPlldUdXY6IyAnRJQM9NDiIX88ezvb9xTz5yfaOLkdE5ITokoEOMG1oEueOSOaB97aSdUB3MxKRrq/LBjrAHReNxAxuf2O9TpCKSJfXpQM9Na4bP5sxhMWbcvnX+r0dXY6IyHHVpQMd4Oqp6QxPieHOBRsorajq6HJERI6bLh/oIcFB/Gb2cPYUlfLSiqyOLkdE5Ljp8oEOMGVgPBP69eCRpZkaxigiXdZJEehmxv87ZxA5RaW8ulJH6SLSNZ0UgQ6+qXVPSYvlz+9nsj6niFtfW8t3H/uMoiMVHV2aiEi7OGkC3XeUPpjdBUeY/eDHvLQiiy92FHDfoi0dXZqISLtoMdDN7EkzyzWzdc0sn2ZmRWa2yv9zW/uX2T6mD0/iR9/oz3/NHMaym6dz+eR+PPPZDtZlF3V0aSIixyyQI/SngJktrPORc26s/+euYy/r+DAzbp09gmunDaRnVBg3nTuUHpFh3PbGOqqrdeGRiHhbi4HunPsQKDgBtZxwsZGh/OqCYazcVcizy3d2dDkiIsekvXroU8xstZm9bWYjm1vJzK4xswwzy8jL6xx3Fbp4fBrfGJzAbW+s5y/vZ2qKABHxLAskwMwsHXjTOTeqiWUxQLVz7rCZzQIecM4NbukzJ06c6DIyMlpf8XFQWlHFL19ew/zVOVwyIY1vjk2lf2IU+w+V8f7mPNZmF3HbnBH0jY/s6FJF5CRnZiuccxObWhZyrB/unDtY5/FCM/uLmSU45/Yf62efKBGhwdz/3bH07RnJw0szebnOFaVmEGRGdHgw988b14FViogc3TEHupn1AvY555yZTcLXxsk/5spOsKAg4z/PH8r3Tu/Httxivtp/mOjwEM4YlMDjH33FXz/8ip+eM5hBSdFUVzteX5XNyN6xDO3VvaNLFxEBAgh0M3semAYkmFkWcDsQCuCcexS4BLjWzCqBI8A85+FGdFL3CJK6RzBlYHzta9d8YwDPfLqTh5ds5f554/j9O5t4/MOvADhzSCJnDk4gY8cBvtx9gBnDk7n9wpGEhZw0Q/xFpJNoMdCdc5e2sPxh4OF2q6gTio8O53tT+vHXj74iLjKMpz7dwaWT+pIaF8FTn+7kwy15pMZ1Y0RKDM8t30Vm7mEevXwCPaLCOrp0ETmJBHRS9HjoTCdFA7H/cBnfuGcpRyqqmDW6Fw9dOp7gIKOssoqC4nJ6xURgZrz+ZTa/fGUNESFBRIQGU1JexZWn9+MX5w/r6E0QkS7guJ4UPVkkRIf7x6wf4J6LxxAcZACEhwSTEtutdr1vjkulb3wkzy7bSVhwEFkHjvDnpds4Z1gyE/r16KjyReQkoCP046y4rJLp935Aj6gwFvx0KiHBvt66cw4z6+DqRMRrdITegaLCQ7j9whFc+9xKnvxkO/FR4TzywTZ2F5TQO64baT268e2JfZg9OqX2qP9gaQXrsw+yPqeIkCDjqqn9O3grRMQLFOgnwMxRvZg2NJHfLtwEwPCUGK6Y3I89B0vZkHOQ65//kocWb2XqoAQydhawPucgdX9xGpYSw+QB8c18uoiIj1ouJ0jWgRLufXcLs0enMH14Um27pbra8dbaPTyweCu78ksY1zeOKQPjGdsnjkFJ0XzrL5+SnhDFC9dMrtei2V1Qwu/e3sjeolLSekTSp2c3RqfGMq5vD5K6h1NeVU1VtSMyTP9ni3QlR2u5KNA7CeccVdWutsde46lPtnPHgg383w9P4/RBCVRWVfPEx9u5770tBJsxOi2WnMJScgqPUOmfMTLIoGbyyEcvn8DMUb0CrqGy2hEarDH0Ip2VeugeYGaEBDc+STpvUl8e/eAr7ntvC4ndw7nppdWsySpixvBk7po7kt5xvhE2pRVVbNhzkC93FVJQXEZkWAgvZezmvkVbOG9EMkFBzZ+AraiqZuHaPTz58XbWZhfx3VP78rNzB5PUPeK4ba+ItD8doXvAM5/t4LY31hMSZMR0C+XuuaOYPSalxfe9/mU2N76wikcvH8/MUb71C4rL+XTbfj7bls/6nIPkF5ex/1A5Ryqq6J8Qxbi+ccxflUNYSBC3zRnBvEl921RzcVklUeE6XhBpbzpC97jvntqHFzN206dHJHfNHUVi9/CA3nfhKb15cPFWHlicyfkje/Huhn3c8M8vKa2oJjo8hDFpsaTH96BnVDhTB8Vz9tAkgoJ8t+r79etrueW1tfSKjWDa0CTA17f/cnchpRVVHCmvIutACTvzS0jrEckts4bVtovuWrCBl1bsZvFNZ+koX+QE0hF6F/fKiixuemk1c8f2Zv7qHMakxXH7hSMYkxrbqF9fV0l5JRc/8hlZB0p47SdT+SRzP797eyOlFdW164SFBJEa143t+4v58VkDuPmC4SzdlMvVT30BwE/PHsR/nj+0xRoLisv53cKNrM85SP+EKAYmRfP9qenERWrqBJGGdFL0JFZZVc05937AroISzhuRzAPzxtEtLDig92YdKOGihz/hcFkl5ZXVnDUkkV+cP5TYbqFEhAYTHxVGUJDx69fX8uyyXdx50UgeWrKVhOhwesd1Y8XOA3x28znNjrRxzrFgzR7umL+eQ6UVnNY/nqwDJezIL+HGGYO5ccaQ9vyrEOkSFOgnuYwdBazaXcjVU/vXXrwUqM+3F/CrV9fwgzP68++T+jZ5dWt5ZTWX/nUZK3YeICwkiDf/3xkcKq3g4kc+486LRnLl6emN3pN7sJRbX1/Hog37OCUtlnsuGcOwXjEAfOexzygsKefdn53VqlrLK6u5551N9O0Z2eR3inQF6qGf5Cam92Ries82vXdS/54suWnaUdcJCwnikcvG88NnMrj8tH4MSfbNET++bxxPfLydyyf3Y212ER9szqOiqpqyyipe+GI3ZZXV3HzBMH5wRv967Z/Zo1O4ff56MnMPMSip6fnmS8orWZNVRGL3cAYmRlNcVsl/PLuCj7buJz4qjCsm9zvqyJ6mVFZVk19cTnKM+v7iTQp0aRdJMRHM/+kZ9V770TcGcO1zKznzD0vJLjwC+MbImxmnpvfgt/82mgGJ0Y0+64JRvbhjwXreWrOXG2bUD/RPt+3n929vYn3OQar8g+0HJ0UTZEZm3mFmjuzFO+v3si6niDFpcQHXX/MfwvLtBSz9z2mkxnVr+U0inYwCXY6b80b2YlzfOKqqHdedPYgLT0mhe0Roi+9Liong1H49Wbh2DzfM+Pr2tF/sKOAHT2WQHBPOtWcNZHy/OHbll/D2ur3syC/mscsnMK5vHP/asJf3N+cFHOgFxeVc/ffPWZtdRLWDt9bkcM2ZA9u83SIdRYEux01wkPHaT6a26b2zRvfijgUbyMw9zKCkaNZmFfH9v39BSmwEL/x4Sr2hmw0nLxuTGsv7m3O5fvrR71V+pLyKV7/M4pH3t5F3qIzHrpjIQ0u2Mn+1Al28SYEundIFo1O4880NPP/5LiLDgvn7JzuI7RbKsz88rcVx+GcNTeLhJVspLClvNPRxd0EJy77K54sdBSzasI8DJRWMTo3l/u+OZWJ6T3bmF/Pfb23kq7zDTbaDGso7VMbijftYnVXIln2HuWXWcM17Lx1GgS6dUnJMBBP79eCJj7cDvr76LbOG1051cDRnD03kwcVb+XDrfi46pTf5h8t4eUUWC9bksC77IACx3UI5Y1ACV56ezqnpPWpH78wZ05v/WbiRBavrt3saqqp2PPPZDu59dwuHyyqJiQihtLKa/1u+S4EuHSaQm0Q/CcwBcp1zo5pYbsADwCygBLjKObeyvQuVk8/PZgxh4bo9fG9Keu3ImUCMSYujR2Qo72/OZWTvGK7423Jyiko5pU8cv549nDOHJDIoMbrJUTC9YiOYlN6T+auzuX76oNqg/2JHAX96dws78ouJjw6jpKyKr/YX843BCdwyazjDenXnxhdWsXRzLlXVLuDhodXVjsWbcnn60x2M6B3DLbOGB7ydIg0FcoT+FL6bQD/TzPILgMH+n9OAR/x/ihyT0wclcPqghFa/LzjIOHNIIos35vLB5jwA3rhuKqf0Cewk6YWn9ObXr69j2VcF5BeX8cqKLJZuziOpezhnDE6gsKSC4rBKfn7eEGaPTqkN/RnDk3ljVQ5f7jrQ4jDRnfnFLFidwysrs9m+v5iQIOOLHQXcMH2w5sCRNmvxX45z7kMzSz/KKnOBZ5zvCqVlZhZnZinOuT3tVKNIq00bmsgbq3LoHRvBP354GgMD6IfXmDU6hTvmr+fSvy4DoEdkKP81cxhXnZ5+1KtszxqaSEiQ8d7G3GYDPfdQKTe9uJqPtu4HYGK/Hvz83CH0jArjsr8tZ+nmXOaM6d2KLRX5WnscCqQCu+s8z/K/1ijQzewa4BqAvn3bNoufSCAuGJXCzvwSvj2xT6vHlPeMCuP2C0eQe6iMaUMTOSUt7qjz3tSIiQhlUv+eLN64j19dMKzR8hU7D/CT51ZQdKSCX5w/lG+OS62traraER8Vxjvr9gYc6Fv2HeKjrftZvbuQQ6UV/Pmy8bqhyUnuhO5959zjwOPgu/T/RH63nFwiQoOPaS6YK6akt+l904cnc/ebG9iVX0Kfnt1YsGYPn2/PZ1tuMRk7C+gd142nrp7E8JSYeu8LDjLOG5nM/FU5lFZUERHa/G8Czjke+/Ar/vDOJqodJESHs/9wGQvX7uWSCWmtqre8sppXV2YxY0QyCdGBzeIpnVd7BHo20KfO8zT/ayInnRnDk7j7zQ0sXLeHrfsO88rKLLpHhDAoKZrLTuvHz2YMITay6YurZo5K4fnPd/Px1v3MGJEM+MI7M/cwK3cdwDnoHhHK/NXZ/Gv9PmaPTuE3c0aQHBPOtD++zysrsloV6NvyDnPDP79kXfZBrtp7iDsuGtkufwfScdoj0OcDPzWzf+I7GVqk/rmcrPrFRzE4KZp73tmEc3DjjMFcf87ggOaVmTIgnpiIEN5et5czhyTy0JKt/N/yXeQXl9dbLyTI+M2cEXx/anrtCdlLxqdx76It7C4ooU/PyCY/v6rasXx7Ppv2HCIz7zCvrcwmIjSIwUnRvLdxH7dfOKLJydfEOwIZtvg8MA1IMLMs4HYgFMA59yiwEN+QxUx8wxavPl7FinjB3LG9eXhpJvd+e2xAd5aqERYSxIzhySzasJetuYdYk1XE+SOTmT4smYnpPYgIDeZQaSXdI0Iajcf/t/Gp3LtoC6+uzOaGGYMpLqtkweocIsNDSI2LYPXuIp76dAe7CkoAiIkI4cwhCdw1dxSLN+Zyy2tr2bLvMEN7tTw8NLvwCK+uyGJNdhEbcg4yoV8PHrx0XOv+kuS4CGSUy6UtLHfAde1WkYjH/WTaIL5/Rv82naCcOaoXr36Zza6Cknq3DmxJWo9ITh8Yzysrs7j6jHS+//cvyNh5oN46E/r14JczhzJ5QDzxUWG1R+PThyfBa/Dexn1HDfRNew/yl6XbeGvtHqqdo39CFDHdQlmwJodfzxne6rtT5R4sZW12EecMS9JvBu1Ep8RF2llQkLV5tMn04cn87lujOXtoEr1iWxeQl0xI4+cvrmbOgx+TU3iEB+aNZXhKDNkHjpDYPZxRqbFNvi85JoIxabG8t3Ef1509qNHyqmrH4x9+xb3vbiYiNJjvT03nqqn9SY3rxua9hzj//g/517q9AZ9IzjtUxqMfbOPZZTspq6zmlWtP19W17aTlsVgicsIEBxmXTurb6jAH39F9VFgwe4qO8JfLxjN3bCpDkrtz9rCkZsO8xvXRgcUAAAwLSURBVPRhyazaXcj+w2X1Xt++v5jL/7ace97ZxHkjk/n4v87m1tkjaodbDkmOZlBSNG+tbfm0WVW14++fbGfa/y7l759sZ9boFIKDjKWbclu9reCb8ri6WoPl6tIRukgXERkWwqNXTCAqPITxfVt3xDtjRBL3vbeFJZty+c7EPuQUHuGhJVt5MSOL8JAg/nDJGL49Ia1Ra8TMmDU6hYeWbCX3UGmjtktm7mE27z1ETuER5q/OYW12EWcNSeT2C0cwIDGa7ANHWLo5N6B7z9ZwzvHkJzu45+1N3HTeEH58Vutnxqyo8t0bNzSA6wu8RIEu0oV8Y3Bim943IiWG3rERvPiFb9jkwrV7CDLjisn9uO7sQUed4XL26BQeXLy1XtulvLKaexdt5vEPv6LmLpcpsRE8/O/j6k2XMG1YIn94ZzP7DpYGdKeorAMl/Pr1dby/OY+w4CBe+zI74ECvrKrmrbV7eHf9Pj7cksfApGhev65t0zt3Vgp0EcHMmD48mX8s20n3vYe48vR0rp6aTlqPpodA1lW37XLFlHTW5xTxq1fWsja7iEsn9eWKyf1I7dGN2G6Nx9+fPTSJP7yzmQ825/GdU7++nKW0oopnl+1k895D9IqNoHtECIs35rJ8ewFhIUHcNXckFVWOu9/cwPb9xfRPiDpqjdvyDnPTi6tZtbuQhOhwBiRFs2p34VGHeTal5j+FTzPzue3CEZ1u3p3OVY2IdJgbZgxmYnoPZgxPblVQ1W27fOexz/h8ewGx3UIDGqUzrFd3esVEsHRzLt85tQ/OOV7KyOJPi7aw92ApCdFhFBSXU+2gf0IUPz93CN8an0paj0hyCo9w95sbeGfdXq6d1vRRunOOpz/dwe/f2UR4SDAPzBvLhWN6s7OghLP/+D5LN+fyvQBO5haXVfLGqhwe/3AbO/J9Qz/PGprIrNGBD0utqedf6/cxODm6VfMLBUqBLiKAbwqBuWNT2/Tei05J4c9LM8k+cISbLxjGvFP7NntFbF1mxtnDElmweg/lldX8/u1NPPnJdsb2ieP+eWOZPCCeyqpqCo9U1BtqCdA7rhunpMXyzro9TQZ67qFSfvHSGj7YksfZQxO55+IxJPnbOv0TouifEMXijc0HemlFFSt2HmDh2j28sSqHw2WVjEqN4S+XjeeXL6/hk8z9rQr0zNxD3DF/Ax9n7ueKyf24+5uNZiM/Zgp0ETlmg5K689Evzyape3hAE5nVNW1oEs9/vptrn13B4k25XD01ndvmfH3VakhwULPzzMwclcI972wiu/BI7cibwpJyXvhiN499+BXFZZXc/c1RXH5a30YndM8ZlsQ/lu2kpLyy3jDTfQdLuWvBBhZv2kdpRTXhIUHMHpPCZaf1ZXxf381QXl6RxWfb8lvctlW7C/kkcz8ZOwr4aOt+IsOCuePCEVw+uV+r/o4CpUAXkXYRyN2kmjJ1UAKhwcbiTbl8Z2Iav5kd+BQE549M5p53NvGvdXu5aGxvHnhvKy+t2E1pRTVTBsRz9zdHMiip6Yulpg9L4omPt/NJZj7n+ufOWbRhH798eTWlFdV8d2IfzhqayGn94xu1oE4fGM+STbnsKTpCSmzj7T5cVsndCzbwQoZvItpBSdF8b0o61509kPjjOAmaAl1EOlR0eAiXT+5HeWU1d80dFdC8NzUGJEYzNLk7T3y8nfsWbeFIRRWXTEjjytPTG81o2dDE9J5Eh4ewZNM+ZgxP4o/vbubPS7cxIiWGh/593FF73FMGxgPwaWY+F9eZEK2guJyPM/dz77ub2V1Qwk+mDeRH3xhAj6iw5j6qXSnQRaTD3X5h22d6nDMmhXsXbeHsoYncOnsEg5ICO9kYFhLEmUMSWLIplzsXbOCpT3cw79Q+3Dl3JOEhzU9fDDC8Vww9IkP5dJsv0PcWlXLtcyv4clchAH16duOf10xhUv+j37mqvSnQRcTT/mPaQM4b2SugicUaOmdYMgvX7uWpT3fw/an9+c2c4QG1e4KCjCkD4/ls236cc/z69bVs3HOQm84dwtTBCYxJjW31uYT2oEAXEU8LDQ5qU5iD78Ro79gILp6Qxs/PHdKqScKmDExg4dq9/HlpJu9tzOXWWcP50ZkD2lRHe1Ggi8hJq2dUGJ/86pw2zfZ4ur+P/sd3t3BKWixXT01v5+par2tNZCAi0kptnbp3QEIUyTHhhAQZv794TIe0WBrSEbqISBuYGbfMGk5VtWtxRM2JokAXEWmjtl5Ze7x0/O8IIiLSLgIKdDObaWabzSzTzH7VxPKrzCzPzFb5f37Y/qWKiMjRBHKT6GDgz8C5QBbwhZnNd85taLDqC865nx6HGkVEJACBHKFPAjKdc18558qBfwJzj29ZIiLSWoEEeiqwu87zLP9rDV1sZmvM7GUz69PEchEROY7a66ToAiDdOTcGWAQ83dRKZnaNmWWYWUZeXl47fbWIiEBggZ4N1D3iTvO/Vss5l++cq7ld+N+ACU19kHPucefcROfcxMTEtt37UEREmhZIoH8BDDaz/mYWBswD5tddwczq3rbjImBj+5UoIiKBaHGUi3Ou0sx+CvwLCAaedM6tN7O7gAzn3HzgejO7CKgECoCrWvrcFStW7DeznW2sOwHY38b3dkZdaXu0LZ2TtqVzasu2NHu7I3POHVs5HcDMMpxzEzu6jvbSlbZH29I5aVs6p/beFl0pKiLSRSjQRUS6CK8G+uMdXUA760rbo23pnLQtnVO7bosne+giItKYV4/QRUSkAQW6iEgX4blAb2kq387MzPqY2VIz22Bm683sBv/rPc1skZlt9f/Zo6NrDZSZBZvZl2b2pv95fzNb7t8/L/gvRuv0zCzOPw/RJjPbaGZTvLpfzOxn/n9f68zseTOL8NJ+MbMnzSzXzNbVea3JfWE+D/q3a42Zje+4yhtrZlv+1//vbI2ZvWZmcXWW3ezfls1mdn5rv89TgV5nKt8LgBHApWY2omOrapVK4Cbn3AhgMnCdv/5fAYudc4OBxf7nXnED9a8Mvge4zzk3CDgA/KBDqmq9B4B3nHPDgFPwbZPn9ouZpQLXAxOdc6PwXQw4D2/tl6eAmQ1ea25fXAAM9v9cAzxygmoM1FM03pZFwCj/3FdbgJsB/FkwDxjpf89f/JkXME8FOh6fytc5t8c5t9L/+BC+0EjFtw01E5o9DXyzYypsHTNLA2bjm78H891t9xzgZf8qntgWM4sFzgSeAHDOlTvnCvHofsF3BXg3MwsBIoE9eGi/OOc+xHfFeV3N7Yu5wDPOZxkQ12Aqkg7V1LY45951zlX6ny7DNz8W+Lbln865MufcdiATX+YFzGuBHuhUvp2emaUD44DlQLJzbo9/0V4guYPKaq37gV8C1f7n8UBhnX+sXtk//YE84O/+9tHfzCwKD+4X51w28EdgF74gLwJW4M39Uldz+8LrmfB94G3/42PeFq8FepdgZtHAK8CNzrmDdZc53zjSTj+W1MzmALnOuRUdXUs7CAHGA48458YBxTRor3hov/TAd6TXH+gNRNH4V35P88q+aImZ3YqvDftce32m1wK9xal8OzszC8UX5s855171v7yv5tdE/5+5HVVfK0wFLjKzHfhaX+fg60PH+X/VB+/snywgyzm33P/8ZXwB78X9MgPY7pzLc85VAK/i21de3C91NbcvPJkJZnYVMAe4zH19MdAxb4vXAr3FqXw7M3+P+Qlgo3PuT3UWzQeu9D++EnjjRNfWWs65m51zac65dHz7YYlz7jJgKXCJfzWvbMteYLeZDfW/NB3YgAf3C75Wy2Qzi/T/e6vZFs/tlwaa2xfzge/5R7tMBorqtGY6JTObia9VeZFzrqTOovnAPDMLN7P++E70ft6qD3fOeeoHmIXvzPA24NaOrqeVtZ+B71fFNcAq/88sfL3nxcBW4D2gZ0fX2srtmga86X88wP+PMBN4CQjv6PoC3IaxQIZ/37wO9PDqfgHuBDYB64B/AOFe2i/A8/j6/xX4fnv6QXP7AjB8I9+2AWvxje7p8G1oYVsy8fXKazLg0Trr3+rfls3ABa39Pl36LyLSRXit5SIiIs1QoIuIdBEKdBGRLkKBLiLSRSjQRUS6CAW6iEgXoUAXEeki/j8hLe+W2jp7wwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "num_epochs = 20\n",
        "\n",
        "losses = train(train_loader, model, batch_size, sequence_length, num_epochs)\n",
        "\n",
        "# Visualize the loss evolution during training\n",
        "plt.plot(losses)\n",
        "plt.title('model loss')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KFsHyTW_9TWK"
      },
      "source": [
        "## Try the trained model\n",
        "\n",
        "Let's try again to predict some characters given the input sentence 'ROMEO:'. Now the predicted characters should be much better."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y9PHhGw5-K7m",
        "outputId": "05ebf045-b686-4eb6-c61e-c299a255008d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input shape: torch.Size([1, 6])\n",
            "Output shape: torch.Size([1, 6])\n",
            "Input string: ROMEO:\n",
            "Output string: IKEO:\n",
            "\n"
          ]
        }
      ],
      "source": [
        "with torch.no_grad():\n",
        "  input_str = 'ROMEO:'\n",
        "\n",
        "  state = model.init_hidden(batch_size=1)\n",
        "  input = torch.tensor([[dataset.char_to_index[char] for char in input_str]]).to(device)\n",
        "  output, state = model(input, state)\n",
        "\n",
        "  logits = output[0].cpu().numpy()\n",
        "  predicted_indexes = np.argmax(logits, axis=1)\n",
        "  \n",
        "  predicted_str = ''.join([dataset.index_to_char[idx] for idx in predicted_indexes])\n",
        "\n",
        "  print(f'Input shape: {input.size()}')\n",
        "  print(f'Output shape: {input.size()}')\n",
        "  print(f'Input string: {input_str}')\n",
        "  print(f'Output string: {predicted_str}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WyYzqxX1DFrM"
      },
      "source": [
        "## Generate new text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nH-z8OVnV8Kw",
        "outputId": "8c5902ea-3734-457f-b994-6523b8c54ec0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ROMEO:\n",
            "To wrink yourself to see the churchyard:\n",
            "Go, sirrah, lead the queen's plexic,\n",
            "Give him down and flatter them;\n",
            "And Romeo do impage into this kindness,\n",
            "Who bear a discontented ground of Nepulorate;\n",
            "Since I am chief to-day, my lord;\n",
            "And for your voices neglignarless caught of him\n",
            "And spurn upon their horses. And for ever must die tomorrow.\n",
            "\n",
            "ISABELLA:\n",
            "In aught\n",
            "Be here proclaim'd,\n",
            "By honour' government than for ours:\n",
            "No, not our prison.\n",
            "Now is gone, away! thy word: he is grace that he shall ha'?\n",
            "\n",
            "QUEEN MARGARET:\n",
            "Hermione, I come;\n",
            "And that my remedy swords again;\n",
            "The hope of his greyhounds,\n",
            "Make us that promise to be commanded:\n",
            "The exchequer e remains; but first lead this\n",
            "triumph, very welcome, general, if he did come\n",
            "That you shall know how charrivals i' the care,\n",
            "There shall no form of merry boy.\n",
            "3 KING HENRY VI:\n",
            "I was adverse past the oweath been packed on me. Nathaniel,\n",
            "God pardon him! he is mut redomes among the elf?\n",
            "O, what happy works me from the king your father,\n",
            "That shares your ho\n"
          ]
        }
      ],
      "source": [
        "def predict(dataset, model, text, next_chars=1000):\n",
        "    model.eval()\n",
        "\n",
        "    chars = [char for char in text]\n",
        "    state = model.init_hidden(1)\n",
        "\n",
        "    # we start by feeding all the input characters to our model\n",
        "    x = torch.tensor([[dataset.char_to_index[char] for char in chars[0:]]])\n",
        "    x = x.to(device)\n",
        "\n",
        "    for i in range(0, next_chars):\n",
        "        \n",
        "        y_pred, state = model(x, state)\n",
        "\n",
        "        last_char_logits = y_pred[0][-1]\n",
        "        p = torch.nn.functional.softmax(last_char_logits, dim=0).detach().cpu().numpy()\n",
        "        char_index = np.random.choice(len(last_char_logits), p=p)\n",
        "        chars.append(dataset.index_to_char[char_index])\n",
        "\n",
        "        # the output of the model (a single character index) becomes the input at next iteration\n",
        "        x = torch.tensor([[char_index]])\n",
        "        x = x.to(device)\n",
        "\n",
        "\n",
        "    return chars\n",
        "\n",
        "predicted = predict(dataset,model,'ROMEO:')\n",
        "print(''.join(predicted))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v9QGZjqwYUXO"
      },
      "source": [
        "While some of the sentences are grammatical, most do not make sense. The model has not learned the meaning of words (we would need a much larger corpus for that, and probably a word level language model), but consider the following:\n",
        "\n",
        "* The model is character-based. When training started, the model did not know how to spell an English word, or that words were even a unit of text. Now it does a pretty decent job on spelling.\n",
        "\n",
        "* The structure of the output resembles a play, blocks of text generally begin with a speaker name, in all capital letters similar to the dataset.\n",
        "\n",
        "* The model is trained on small batches of text (100 characters each), and is still able to generate a longer sequence of text with coherent structure."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wX8aV2_cDLxa"
      },
      "source": [
        "# HOMEWORK\n",
        "\n",
        "A) Build two new models: one using Simple RNN cells (torch.nn.RNN) instead of GRUs(torch.nn.RNN), and the other using LSTMs (torch.nn.LSTM). Train all three models (GRU, LSTM, SimpleRNN) for 30 epochs and compare the loss curves. You can optionally try different configurations: embedding dimension, hidden layers dimensions, number of layers, etc. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "TspQOPOhJsMQ"
      },
      "outputs": [],
      "source": [
        "class Model(nn.Module):\n",
        "    def __init__(self, input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob=0., rnn_cell='RNN'):\n",
        "        super(Model, self).__init__()\n",
        "        self.hidden_dim = hidden_dim\n",
        "        self.n_layers = n_layers\n",
        "        self.embedding = nn.Embedding(input_dim, embedding_dim)\n",
        "        self.rnn_cell = rnn_cell\n",
        "        if self.rnn_cell == 'GRU':\n",
        "            self.rnn = nn.GRU(embedding_dim, hidden_dim, n_layers, batch_first=True)\n",
        "        elif self.rnn_cell == 'LSTM':\n",
        "            self.rnn = nn.LSTM(embedding_dim, hidden_dim, n_layers, batch_first=True)\n",
        "        else:\n",
        "            print('Using default Simple RNN cells')\n",
        "            self.rnn = nn.RNN(embedding_dim, hidden_dim, n_layers, batch_first=True)\n",
        "        self.fc = nn.Linear(hidden_dim, output_dim)\n",
        "\n",
        "    def forward(self, x, h, c=None):\n",
        "        emb = self.embedding(x)\n",
        "        if self.rnn_cell == 'LSTM': # in LSTM we have a cell state and a hidden state \n",
        "            out, (h, c) = self.rnn(emb, (h, c))\n",
        "        else:                       # in GRU and RNN we only have one hidden state\n",
        "            out, h = self.rnn(emb, h)\n",
        "        out = self.fc(out)\n",
        "        return out, h, c\n",
        "    \n",
        "    def init_hidden(self, batch_size):\n",
        "        \" Initialize the hidden state of the RNN to zeros\"\n",
        "        weight = next(self.parameters()).data\n",
        "        if self.rnn_cell == 'LSTM': # in LSTM we have a cell state and a hidden state\n",
        "          return weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device), weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device)\n",
        "        else:                       # in GRU and RNN we only have a hidden state\n",
        "          return weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device), None\n",
        "        \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "LjMcod9s3Npe"
      },
      "outputs": [],
      "source": [
        "def train(dataloader, model, batch_size, sequence_length, num_epochs):\n",
        "    model.train()\n",
        "\n",
        "    criterion = nn.CrossEntropyLoss()\n",
        "    optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
        "\n",
        "    losses = []\n",
        "    for epoch in range(num_epochs):\n",
        "        h_state, c_state = model.init_hidden(batch_size)\n",
        "\n",
        "        for batch, (x, y) in enumerate(dataloader):\n",
        "            x = x.to(device)\n",
        "            y = y.to(device)\n",
        "            optimizer.zero_grad()\n",
        "            \n",
        "            y_pred, h_state, c_state = model(x, h_state, c_state) # in LSTM we have a cell state and a hidden state\n",
        "       \n",
        "            loss = criterion(y_pred.transpose(1, 2), y)\n",
        "\n",
        "            h_state = h_state.detach()\n",
        "            if c_state is not None:\n",
        "                c_state = c_state.detach()\n",
        "           \n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            if batch%30 == 0:\n",
        "              print({ 'epoch': epoch, 'batch': batch, 'loss': loss.item() })\n",
        "              losses.append(loss.detach().cpu().numpy())\n",
        "    return losses"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "lQ0-KxfC0-6a",
        "outputId": "b7174eb0-74eb-4d3d-b426-0a7be1436d9b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "GRU\n",
            "{'epoch': 0, 'batch': 0, 'loss': 4.167050361633301}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.2713570594787598}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 1.9921437501907349}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.8394345045089722}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.7387173175811768}\n",
            "{'epoch': 0, 'batch': 150, 'loss': 1.630863070487976}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.5582793951034546}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.544957160949707}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.5077643394470215}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.5014588832855225}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.4500964879989624}\n",
            "{'epoch': 1, 'batch': 150, 'loss': 1.4747288227081299}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.3899999856948853}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.356325387954712}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.3591517210006714}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.3543100357055664}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.3469254970550537}\n",
            "{'epoch': 2, 'batch': 150, 'loss': 1.3720893859863281}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.3101726770401}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.349153995513916}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.3310356140136719}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.3337314128875732}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.322176218032837}\n",
            "{'epoch': 3, 'batch': 150, 'loss': 1.330328106880188}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.250676155090332}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.2726595401763916}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.2806168794631958}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.2466559410095215}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.29193913936615}\n",
            "{'epoch': 4, 'batch': 150, 'loss': 1.3186205625534058}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.1642565727233887}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.2078529596328735}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.2479186058044434}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.278781533241272}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.241901159286499}\n",
            "{'epoch': 5, 'batch': 150, 'loss': 1.2745193243026733}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.1590789556503296}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.1908187866210938}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.2197586297988892}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.2037854194641113}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.1856337785720825}\n",
            "{'epoch': 6, 'batch': 150, 'loss': 1.199881672859192}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.1275074481964111}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.1458549499511719}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.150879144668579}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.1457782983779907}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.169141411781311}\n",
            "{'epoch': 7, 'batch': 150, 'loss': 1.15751314163208}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 1.0550588369369507}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.1072139739990234}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 1.1069303750991821}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.165892243385315}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.1530643701553345}\n",
            "{'epoch': 8, 'batch': 150, 'loss': 1.1490522623062134}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 1.0273900032043457}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 1.0576705932617188}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 1.0712507963180542}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 1.0889109373092651}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 1.0663520097732544}\n",
            "{'epoch': 9, 'batch': 150, 'loss': 1.0836430788040161}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 0.990277886390686}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 1.024768352508545}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 1.0261130332946777}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 1.0693919658660889}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 1.090410590171814}\n",
            "{'epoch': 10, 'batch': 150, 'loss': 1.0679413080215454}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 0.9283795356750488}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 0.9381945729255676}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 0.9987283945083618}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 1.0174540281295776}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 1.0295387506484985}\n",
            "{'epoch': 11, 'batch': 150, 'loss': 1.02616548538208}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 0.8881756663322449}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 0.9045882225036621}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 0.9610442519187927}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 0.9585075974464417}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 0.9743355512619019}\n",
            "{'epoch': 12, 'batch': 150, 'loss': 0.996215283870697}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 0.8716973066329956}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 0.8944802284240723}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 0.8884627819061279}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 0.897365391254425}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 0.9793960452079773}\n",
            "{'epoch': 13, 'batch': 150, 'loss': 0.949839174747467}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 0.8004927635192871}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 0.8334848284721375}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 0.8744498491287231}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 0.8859649896621704}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 0.9078472256660461}\n",
            "{'epoch': 14, 'batch': 150, 'loss': 0.9130677580833435}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 0.7639613151550293}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 0.8037351369857788}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 0.8220301866531372}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 0.8355042338371277}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 0.8643753528594971}\n",
            "{'epoch': 15, 'batch': 150, 'loss': 0.8717214465141296}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 0.7257886528968811}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 0.7966061234474182}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 0.7910654544830322}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 0.8186885118484497}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 0.8210775256156921}\n",
            "{'epoch': 16, 'batch': 150, 'loss': 0.8668349981307983}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 0.6964035630226135}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 0.7468717694282532}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 0.7691527605056763}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 0.7996819019317627}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 0.8090390563011169}\n",
            "{'epoch': 17, 'batch': 150, 'loss': 0.8170597553253174}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 0.6736180186271667}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 0.7224011421203613}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 0.7288708686828613}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 0.764789879322052}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 0.7949107885360718}\n",
            "{'epoch': 18, 'batch': 150, 'loss': 0.8177458047866821}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 0.6109585165977478}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 0.6715065240859985}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 0.703477680683136}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 0.7306092977523804}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 0.7576403617858887}\n",
            "{'epoch': 19, 'batch': 150, 'loss': 0.7733680009841919}\n",
            "{'epoch': 20, 'batch': 0, 'loss': 0.5922969579696655}\n",
            "{'epoch': 20, 'batch': 30, 'loss': 0.656453549861908}\n",
            "{'epoch': 20, 'batch': 60, 'loss': 0.6984224915504456}\n",
            "{'epoch': 20, 'batch': 90, 'loss': 0.7244151830673218}\n",
            "{'epoch': 20, 'batch': 120, 'loss': 0.7357956171035767}\n",
            "{'epoch': 20, 'batch': 150, 'loss': 0.7623684406280518}\n",
            "{'epoch': 21, 'batch': 0, 'loss': 0.5807475447654724}\n",
            "{'epoch': 21, 'batch': 30, 'loss': 0.6463744640350342}\n",
            "{'epoch': 21, 'batch': 60, 'loss': 0.6725956201553345}\n",
            "{'epoch': 21, 'batch': 90, 'loss': 0.6805664896965027}\n",
            "{'epoch': 21, 'batch': 120, 'loss': 0.6890583038330078}\n",
            "{'epoch': 21, 'batch': 150, 'loss': 0.7237626910209656}\n",
            "{'epoch': 22, 'batch': 0, 'loss': 0.5641435384750366}\n",
            "{'epoch': 22, 'batch': 30, 'loss': 0.6315405368804932}\n",
            "{'epoch': 22, 'batch': 60, 'loss': 0.6373888254165649}\n",
            "{'epoch': 22, 'batch': 90, 'loss': 0.6754648089408875}\n",
            "{'epoch': 22, 'batch': 120, 'loss': 0.710915207862854}\n",
            "{'epoch': 22, 'batch': 150, 'loss': 0.7106446623802185}\n",
            "{'epoch': 23, 'batch': 0, 'loss': 0.537674069404602}\n",
            "{'epoch': 23, 'batch': 30, 'loss': 0.6122360229492188}\n",
            "{'epoch': 23, 'batch': 60, 'loss': 0.6556695699691772}\n",
            "{'epoch': 23, 'batch': 90, 'loss': 0.6511493921279907}\n",
            "{'epoch': 23, 'batch': 120, 'loss': 0.6557720899581909}\n",
            "{'epoch': 23, 'batch': 150, 'loss': 0.6902406811714172}\n",
            "{'epoch': 24, 'batch': 0, 'loss': 0.5474849939346313}\n",
            "{'epoch': 24, 'batch': 30, 'loss': 0.6040984988212585}\n",
            "{'epoch': 24, 'batch': 60, 'loss': 0.6185278296470642}\n",
            "{'epoch': 24, 'batch': 90, 'loss': 0.6388144493103027}\n",
            "{'epoch': 24, 'batch': 120, 'loss': 0.6831761002540588}\n",
            "{'epoch': 24, 'batch': 150, 'loss': 0.6911196112632751}\n",
            "{'epoch': 25, 'batch': 0, 'loss': 0.554847776889801}\n",
            "{'epoch': 25, 'batch': 30, 'loss': 0.5731790065765381}\n",
            "{'epoch': 25, 'batch': 60, 'loss': 0.5943899750709534}\n",
            "{'epoch': 25, 'batch': 90, 'loss': 0.6077479720115662}\n",
            "{'epoch': 25, 'batch': 120, 'loss': 0.6609426140785217}\n",
            "{'epoch': 25, 'batch': 150, 'loss': 0.6705785393714905}\n",
            "{'epoch': 26, 'batch': 0, 'loss': 0.5185422301292419}\n",
            "{'epoch': 26, 'batch': 30, 'loss': 0.5699060559272766}\n",
            "{'epoch': 26, 'batch': 60, 'loss': 0.600277304649353}\n",
            "{'epoch': 26, 'batch': 90, 'loss': 0.6347965598106384}\n",
            "{'epoch': 26, 'batch': 120, 'loss': 0.6306111812591553}\n",
            "{'epoch': 26, 'batch': 150, 'loss': 0.6652852892875671}\n",
            "{'epoch': 27, 'batch': 0, 'loss': 0.5133016705513}\n",
            "{'epoch': 27, 'batch': 30, 'loss': 0.5856882929801941}\n",
            "{'epoch': 27, 'batch': 60, 'loss': 0.5948677062988281}\n",
            "{'epoch': 27, 'batch': 90, 'loss': 0.6096813678741455}\n",
            "{'epoch': 27, 'batch': 120, 'loss': 0.6353678107261658}\n",
            "{'epoch': 27, 'batch': 150, 'loss': 0.6442930698394775}\n",
            "{'epoch': 28, 'batch': 0, 'loss': 0.49839118123054504}\n",
            "{'epoch': 28, 'batch': 30, 'loss': 0.5832352042198181}\n",
            "{'epoch': 28, 'batch': 60, 'loss': 0.5731289386749268}\n",
            "{'epoch': 28, 'batch': 90, 'loss': 0.6094217896461487}\n",
            "{'epoch': 28, 'batch': 120, 'loss': 0.633284866809845}\n",
            "{'epoch': 28, 'batch': 150, 'loss': 0.6252644062042236}\n",
            "{'epoch': 29, 'batch': 0, 'loss': 0.481663316488266}\n",
            "{'epoch': 29, 'batch': 30, 'loss': 0.5701886415481567}\n",
            "{'epoch': 29, 'batch': 60, 'loss': 0.6032031774520874}\n",
            "{'epoch': 29, 'batch': 90, 'loss': 0.6229734420776367}\n",
            "{'epoch': 29, 'batch': 120, 'loss': 0.6147874593734741}\n",
            "{'epoch': 29, 'batch': 150, 'loss': 0.6424421072006226}\n",
            "LSTM\n",
            "{'epoch': 0, 'batch': 0, 'loss': 4.169376373291016}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.3480660915374756}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 1.9676072597503662}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.7971948385238647}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.670941948890686}\n",
            "{'epoch': 0, 'batch': 150, 'loss': 1.6218247413635254}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.5061646699905396}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.5369410514831543}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.4985229969024658}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.4332284927368164}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.4102267026901245}\n",
            "{'epoch': 1, 'batch': 150, 'loss': 1.4016419649124146}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.3205947875976562}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.4323172569274902}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.379148006439209}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.3332709074020386}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.3238428831100464}\n",
            "{'epoch': 2, 'batch': 150, 'loss': 1.392701268196106}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.298825979232788}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.3120026588439941}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.3025403022766113}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.3181252479553223}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.2853331565856934}\n",
            "{'epoch': 3, 'batch': 150, 'loss': 1.319556474685669}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.1824952363967896}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.2576351165771484}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.2365128993988037}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.2536170482635498}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.2141413688659668}\n",
            "{'epoch': 4, 'batch': 150, 'loss': 1.227230191230774}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.1617985963821411}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.2369550466537476}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.2261757850646973}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.2114243507385254}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.2229629755020142}\n",
            "{'epoch': 5, 'batch': 150, 'loss': 1.2157515287399292}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.1149178743362427}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.1594130992889404}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.151824951171875}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.148339033126831}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.174168586730957}\n",
            "{'epoch': 6, 'batch': 150, 'loss': 1.2125052213668823}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.058665156364441}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.1308146715164185}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.095563530921936}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.1257431507110596}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.1197434663772583}\n",
            "{'epoch': 7, 'batch': 150, 'loss': 1.173472285270691}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 1.04282808303833}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.0674972534179688}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 1.0536284446716309}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.0934168100357056}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.116072654724121}\n",
            "{'epoch': 8, 'batch': 150, 'loss': 1.0979721546173096}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 0.9760805368423462}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 0.9916712045669556}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 1.0439419746398926}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 1.0144119262695312}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 1.0417392253875732}\n",
            "{'epoch': 9, 'batch': 150, 'loss': 1.0858625173568726}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 0.9080249667167664}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 0.9371868371963501}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 0.9767210483551025}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 0.9827964901924133}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 0.9884134531021118}\n",
            "{'epoch': 10, 'batch': 150, 'loss': 0.9989727735519409}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 0.8734987378120422}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 0.8977590203285217}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 0.9132729172706604}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 0.9323700070381165}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 0.95942223072052}\n",
            "{'epoch': 11, 'batch': 150, 'loss': 0.9533751606941223}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 0.8132612109184265}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 0.8259541392326355}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 0.8760473728179932}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 0.8717076182365417}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 0.8808798789978027}\n",
            "{'epoch': 12, 'batch': 150, 'loss': 0.9011396169662476}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 0.741771399974823}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 0.7846753597259521}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 0.7926867008209229}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 0.850797712802887}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 0.8193399906158447}\n",
            "{'epoch': 13, 'batch': 150, 'loss': 0.8589548468589783}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 0.6824501752853394}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 0.7301725149154663}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 0.7506439089775085}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 0.752583384513855}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 0.7822996377944946}\n",
            "{'epoch': 14, 'batch': 150, 'loss': 0.7636393904685974}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 0.5983060002326965}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 0.6604644060134888}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 0.7004697918891907}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 0.7115514278411865}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 0.7106010913848877}\n",
            "{'epoch': 15, 'batch': 150, 'loss': 0.7386829257011414}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 0.5649768710136414}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 0.6264505386352539}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 0.634280800819397}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 0.6568852066993713}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 0.666958749294281}\n",
            "{'epoch': 16, 'batch': 150, 'loss': 0.6954241394996643}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 0.5157065987586975}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 0.5603552460670471}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 0.6059373617172241}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 0.609451949596405}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 0.6405109167098999}\n",
            "{'epoch': 17, 'batch': 150, 'loss': 0.6145621538162231}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 0.46823301911354065}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 0.5381501913070679}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 0.5429209470748901}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 0.5460667610168457}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 0.5803481340408325}\n",
            "{'epoch': 18, 'batch': 150, 'loss': 0.5778624415397644}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 0.42238327860832214}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 0.4785059988498688}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 0.5108713507652283}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 0.5291213393211365}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 0.5288814902305603}\n",
            "{'epoch': 19, 'batch': 150, 'loss': 0.5398117303848267}\n",
            "{'epoch': 20, 'batch': 0, 'loss': 0.3941326141357422}\n",
            "{'epoch': 20, 'batch': 30, 'loss': 0.45000773668289185}\n",
            "{'epoch': 20, 'batch': 60, 'loss': 0.47242459654808044}\n",
            "{'epoch': 20, 'batch': 90, 'loss': 0.4742947816848755}\n",
            "{'epoch': 20, 'batch': 120, 'loss': 0.4959624111652374}\n",
            "{'epoch': 20, 'batch': 150, 'loss': 0.5216468572616577}\n",
            "{'epoch': 21, 'batch': 0, 'loss': 0.36982059478759766}\n",
            "{'epoch': 21, 'batch': 30, 'loss': 0.4182158410549164}\n",
            "{'epoch': 21, 'batch': 60, 'loss': 0.43673279881477356}\n",
            "{'epoch': 21, 'batch': 90, 'loss': 0.4573594033718109}\n",
            "{'epoch': 21, 'batch': 120, 'loss': 0.47855186462402344}\n",
            "{'epoch': 21, 'batch': 150, 'loss': 0.49508798122406006}\n",
            "{'epoch': 22, 'batch': 0, 'loss': 0.3392237424850464}\n",
            "{'epoch': 22, 'batch': 30, 'loss': 0.41722190380096436}\n",
            "{'epoch': 22, 'batch': 60, 'loss': 0.41212740540504456}\n",
            "{'epoch': 22, 'batch': 90, 'loss': 0.42822757363319397}\n",
            "{'epoch': 22, 'batch': 120, 'loss': 0.4416590631008148}\n",
            "{'epoch': 22, 'batch': 150, 'loss': 0.4520238935947418}\n",
            "{'epoch': 23, 'batch': 0, 'loss': 0.31873923540115356}\n",
            "{'epoch': 23, 'batch': 30, 'loss': 0.39050623774528503}\n",
            "{'epoch': 23, 'batch': 60, 'loss': 0.3968644440174103}\n",
            "{'epoch': 23, 'batch': 90, 'loss': 0.4057907462120056}\n",
            "{'epoch': 23, 'batch': 120, 'loss': 0.41194579005241394}\n",
            "{'epoch': 23, 'batch': 150, 'loss': 0.43709880113601685}\n",
            "{'epoch': 24, 'batch': 0, 'loss': 0.2985824644565582}\n",
            "{'epoch': 24, 'batch': 30, 'loss': 0.36323559284210205}\n",
            "{'epoch': 24, 'batch': 60, 'loss': 0.37539592385292053}\n",
            "{'epoch': 24, 'batch': 90, 'loss': 0.3859059512615204}\n",
            "{'epoch': 24, 'batch': 120, 'loss': 0.40185120701789856}\n",
            "{'epoch': 24, 'batch': 150, 'loss': 0.41471606492996216}\n",
            "{'epoch': 25, 'batch': 0, 'loss': 0.2779896557331085}\n",
            "{'epoch': 25, 'batch': 30, 'loss': 0.35627540946006775}\n",
            "{'epoch': 25, 'batch': 60, 'loss': 0.3554399907588959}\n",
            "{'epoch': 25, 'batch': 90, 'loss': 0.37194859981536865}\n",
            "{'epoch': 25, 'batch': 120, 'loss': 0.3661558926105499}\n",
            "{'epoch': 25, 'batch': 150, 'loss': 0.4090491533279419}\n",
            "{'epoch': 26, 'batch': 0, 'loss': 0.26635250449180603}\n",
            "{'epoch': 26, 'batch': 30, 'loss': 0.320465624332428}\n",
            "{'epoch': 26, 'batch': 60, 'loss': 0.33425217866897583}\n",
            "{'epoch': 26, 'batch': 90, 'loss': 0.36987408995628357}\n",
            "{'epoch': 26, 'batch': 120, 'loss': 0.37416794896125793}\n",
            "{'epoch': 26, 'batch': 150, 'loss': 0.3835069239139557}\n",
            "{'epoch': 27, 'batch': 0, 'loss': 0.23732933402061462}\n",
            "{'epoch': 27, 'batch': 30, 'loss': 0.32484397292137146}\n",
            "{'epoch': 27, 'batch': 60, 'loss': 0.3483760952949524}\n",
            "{'epoch': 27, 'batch': 90, 'loss': 0.3443152904510498}\n",
            "{'epoch': 27, 'batch': 120, 'loss': 0.3516073226928711}\n",
            "{'epoch': 27, 'batch': 150, 'loss': 0.37649551033973694}\n",
            "{'epoch': 28, 'batch': 0, 'loss': 0.24775443971157074}\n",
            "{'epoch': 28, 'batch': 30, 'loss': 0.32179176807403564}\n",
            "{'epoch': 28, 'batch': 60, 'loss': 0.3340861201286316}\n",
            "{'epoch': 28, 'batch': 90, 'loss': 0.33278658986091614}\n",
            "{'epoch': 28, 'batch': 120, 'loss': 0.3338683247566223}\n",
            "{'epoch': 28, 'batch': 150, 'loss': 0.3657086491584778}\n",
            "{'epoch': 29, 'batch': 0, 'loss': 0.25779855251312256}\n",
            "{'epoch': 29, 'batch': 30, 'loss': 0.3203544318675995}\n",
            "{'epoch': 29, 'batch': 60, 'loss': 0.31967854499816895}\n",
            "{'epoch': 29, 'batch': 90, 'loss': 0.343328595161438}\n",
            "{'epoch': 29, 'batch': 120, 'loss': 0.33408766984939575}\n",
            "{'epoch': 29, 'batch': 150, 'loss': 0.3436097204685211}\n",
            "Using default Simple RNN cells\n",
            "RNN\n",
            "{'epoch': 0, 'batch': 0, 'loss': 4.18678617477417}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.2403953075408936}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 2.0280137062072754}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.8903584480285645}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.76908540725708}\n",
            "{'epoch': 0, 'batch': 150, 'loss': 1.7488000392913818}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.6270822286605835}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.6566016674041748}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.6149219274520874}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.6054601669311523}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.604115605354309}\n",
            "{'epoch': 1, 'batch': 150, 'loss': 1.5448925495147705}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.5036016702651978}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.4800077676773071}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.5039523839950562}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.4908806085586548}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.4611130952835083}\n",
            "{'epoch': 2, 'batch': 150, 'loss': 1.4183533191680908}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.4296704530715942}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.4183037281036377}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.392906665802002}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.4398876428604126}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.4335764646530151}\n",
            "{'epoch': 3, 'batch': 150, 'loss': 1.4405653476715088}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.3464634418487549}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.3593472242355347}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.3330671787261963}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.3911867141723633}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.3862066268920898}\n",
            "{'epoch': 4, 'batch': 150, 'loss': 1.3925280570983887}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.3128852844238281}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.315048336982727}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.3426893949508667}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.3051085472106934}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.3824337720870972}\n",
            "{'epoch': 5, 'batch': 150, 'loss': 1.3209854364395142}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.3156601190567017}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.3602569103240967}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.358678936958313}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.3177028894424438}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.3218928575515747}\n",
            "{'epoch': 6, 'batch': 150, 'loss': 1.3377970457077026}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.2897121906280518}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.2766560316085815}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.3313548564910889}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.308375358581543}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.3054989576339722}\n",
            "{'epoch': 7, 'batch': 150, 'loss': 1.336159110069275}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 1.2353440523147583}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.2491350173950195}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 1.2913575172424316}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.3181949853897095}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.319349765777588}\n",
            "{'epoch': 8, 'batch': 150, 'loss': 1.2784218788146973}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 1.2650874853134155}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 1.2417786121368408}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 1.2597726583480835}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 1.2393507957458496}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 1.2726860046386719}\n",
            "{'epoch': 9, 'batch': 150, 'loss': 1.328070878982544}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 1.2407383918762207}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 1.2523045539855957}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 1.2808564901351929}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 1.2655020952224731}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 1.2258930206298828}\n",
            "{'epoch': 10, 'batch': 150, 'loss': 1.2567131519317627}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 1.1960067749023438}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 1.2419273853302002}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 1.2499855756759644}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 1.258946180343628}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 1.2487272024154663}\n",
            "{'epoch': 11, 'batch': 150, 'loss': 1.235841155052185}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 1.179039478302002}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 1.201910138130188}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 1.2353359460830688}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 1.2052485942840576}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 1.24488365650177}\n",
            "{'epoch': 12, 'batch': 150, 'loss': 1.2325936555862427}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 1.1852006912231445}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 1.229587435722351}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 1.225367784500122}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 1.2034777402877808}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 1.2124030590057373}\n",
            "{'epoch': 13, 'batch': 150, 'loss': 1.1962610483169556}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 1.1537433862686157}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 1.1787126064300537}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 1.1979900598526}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 1.2119907140731812}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 1.2014994621276855}\n",
            "{'epoch': 14, 'batch': 150, 'loss': 1.2112289667129517}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 1.137086272239685}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 1.1570255756378174}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 1.1956489086151123}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 1.163644552230835}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 1.2586749792099}\n",
            "{'epoch': 15, 'batch': 150, 'loss': 1.2230181694030762}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 1.1360368728637695}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 1.1466073989868164}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 1.1775110960006714}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 1.1608481407165527}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 1.1958990097045898}\n",
            "{'epoch': 16, 'batch': 150, 'loss': 1.1855512857437134}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 1.1297931671142578}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 1.169324278831482}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 1.1465831995010376}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 1.1693423986434937}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 1.1460245847702026}\n",
            "{'epoch': 17, 'batch': 150, 'loss': 1.206349492073059}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 1.1240730285644531}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 1.1351239681243896}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 1.1348882913589478}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 1.160815715789795}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 1.1671853065490723}\n",
            "{'epoch': 18, 'batch': 150, 'loss': 1.1464505195617676}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 1.1075342893600464}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 1.1383581161499023}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 1.1299632787704468}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 1.1113083362579346}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 1.1420563459396362}\n",
            "{'epoch': 19, 'batch': 150, 'loss': 1.1389188766479492}\n",
            "{'epoch': 20, 'batch': 0, 'loss': 1.095133900642395}\n",
            "{'epoch': 20, 'batch': 30, 'loss': 1.0928303003311157}\n",
            "{'epoch': 20, 'batch': 60, 'loss': 1.0823839902877808}\n",
            "{'epoch': 20, 'batch': 90, 'loss': 1.1001322269439697}\n",
            "{'epoch': 20, 'batch': 120, 'loss': 1.139404535293579}\n",
            "{'epoch': 20, 'batch': 150, 'loss': 1.1602956056594849}\n",
            "{'epoch': 21, 'batch': 0, 'loss': 1.043352484703064}\n",
            "{'epoch': 21, 'batch': 30, 'loss': 1.0906180143356323}\n",
            "{'epoch': 21, 'batch': 60, 'loss': 1.0864906311035156}\n",
            "{'epoch': 21, 'batch': 90, 'loss': 1.121894121170044}\n",
            "{'epoch': 21, 'batch': 120, 'loss': 1.1126935482025146}\n",
            "{'epoch': 21, 'batch': 150, 'loss': 1.0849744081497192}\n",
            "{'epoch': 22, 'batch': 0, 'loss': 1.0645396709442139}\n",
            "{'epoch': 22, 'batch': 30, 'loss': 1.0718666315078735}\n",
            "{'epoch': 22, 'batch': 60, 'loss': 1.0773295164108276}\n",
            "{'epoch': 22, 'batch': 90, 'loss': 1.1058540344238281}\n",
            "{'epoch': 22, 'batch': 120, 'loss': 1.1103416681289673}\n",
            "{'epoch': 22, 'batch': 150, 'loss': 1.135990023612976}\n",
            "{'epoch': 23, 'batch': 0, 'loss': 1.0258842706680298}\n",
            "{'epoch': 23, 'batch': 30, 'loss': 1.0639679431915283}\n",
            "{'epoch': 23, 'batch': 60, 'loss': 1.1040476560592651}\n",
            "{'epoch': 23, 'batch': 90, 'loss': 1.0929937362670898}\n",
            "{'epoch': 23, 'batch': 120, 'loss': 1.0706866979599}\n",
            "{'epoch': 23, 'batch': 150, 'loss': 1.0965653657913208}\n",
            "{'epoch': 24, 'batch': 0, 'loss': 1.006087303161621}\n",
            "{'epoch': 24, 'batch': 30, 'loss': 1.0783991813659668}\n",
            "{'epoch': 24, 'batch': 60, 'loss': 1.0944030284881592}\n",
            "{'epoch': 24, 'batch': 90, 'loss': 1.0676164627075195}\n",
            "{'epoch': 24, 'batch': 120, 'loss': 1.0752958059310913}\n",
            "{'epoch': 24, 'batch': 150, 'loss': 1.113402009010315}\n",
            "{'epoch': 25, 'batch': 0, 'loss': 1.0308837890625}\n",
            "{'epoch': 25, 'batch': 30, 'loss': 1.0192391872406006}\n",
            "{'epoch': 25, 'batch': 60, 'loss': 1.0578416585922241}\n",
            "{'epoch': 25, 'batch': 90, 'loss': 1.0650458335876465}\n",
            "{'epoch': 25, 'batch': 120, 'loss': 1.0620192289352417}\n",
            "{'epoch': 25, 'batch': 150, 'loss': 1.1149276494979858}\n",
            "{'epoch': 26, 'batch': 0, 'loss': 0.9937272071838379}\n",
            "{'epoch': 26, 'batch': 30, 'loss': 1.040947675704956}\n",
            "{'epoch': 26, 'batch': 60, 'loss': 1.0575908422470093}\n",
            "{'epoch': 26, 'batch': 90, 'loss': 1.0777125358581543}\n",
            "{'epoch': 26, 'batch': 120, 'loss': 1.090766191482544}\n",
            "{'epoch': 26, 'batch': 150, 'loss': 1.0953022241592407}\n",
            "{'epoch': 27, 'batch': 0, 'loss': 0.9966517686843872}\n",
            "{'epoch': 27, 'batch': 30, 'loss': 1.0223560333251953}\n",
            "{'epoch': 27, 'batch': 60, 'loss': 1.0340051651000977}\n",
            "{'epoch': 27, 'batch': 90, 'loss': 1.0795228481292725}\n",
            "{'epoch': 27, 'batch': 120, 'loss': 1.082619309425354}\n",
            "{'epoch': 27, 'batch': 150, 'loss': 1.0636677742004395}\n",
            "{'epoch': 28, 'batch': 0, 'loss': 1.0024142265319824}\n",
            "{'epoch': 28, 'batch': 30, 'loss': 0.9764649868011475}\n",
            "{'epoch': 28, 'batch': 60, 'loss': 1.036929726600647}\n",
            "{'epoch': 28, 'batch': 90, 'loss': 1.0678203105926514}\n",
            "{'epoch': 28, 'batch': 120, 'loss': 1.0385535955429077}\n",
            "{'epoch': 28, 'batch': 150, 'loss': 1.0675363540649414}\n",
            "{'epoch': 29, 'batch': 0, 'loss': 0.9710636734962463}\n",
            "{'epoch': 29, 'batch': 30, 'loss': 1.0503263473510742}\n",
            "{'epoch': 29, 'batch': 60, 'loss': 1.0442029237747192}\n",
            "{'epoch': 29, 'batch': 90, 'loss': 1.0610065460205078}\n",
            "{'epoch': 29, 'batch': 120, 'loss': 1.032930850982666}\n",
            "{'epoch': 29, 'batch': 150, 'loss': 1.063539981842041}\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hU1dbA4d+ekl6AkJAQSELvECD0ItIEKeJFUcSGIorYyydgr1cFwYYgV7koeEEQRJpIka6UAKGXUENCIJ30ZDKzvz8mCYQkEDUhDKz3efLcmTl7zlkZvGt21tlFaa0RQgjh+AyVHYAQQojyIQldCCFuEJLQhRDiBiEJXQghbhCS0IUQ4gZhqqwLV69eXYeEhFTW5YUQwiHt3LkzQWvtW9KxSkvoISEhhIeHV9blhRDCISmlTpd2TEouQghxg5CELoQQNwhJ6EIIcYOotBq6EEKUlcViITo6muzs7MoO5ZpxcXGhVq1amM3mMr9HEroQ4roXHR2Np6cnISEhKKUqO5wKp7UmMTGR6Oho6tSpU+b3SclFCHHdy87OxsfH56ZI5gBKKXx8fP7yXySS0IUQDuFmSeYF/s7v63AJPTI5ki92f0FiVmJlhyKEENcVh0vo2/cuZ8beGZw8s7+yQxFC3ETOnz/PfffdR926dWnbti2dOnXi559/Zv369Xh7exMaGkrjxo156aWXCt/z1ltvMWnSpCLnCQkJISEhoUJidLiEnpkcD0DKhbhKjkQIcbPQWjNkyBC6d+/OiRMn2LlzJ/PmzSM6OhqAbt26ERERwe7du1m2bBlbtmyplDgdLqEbDU4AWPJyKzkSIcTN4vfff8fJyYknnnii8LXg4GCefvrpIu1cXV0JDQ0lJibmWocIOOCwRbPRPibTkpdTyZEIISrD20sPcPBsarmes2lNL94c1KzU4wcOHKBNmzZXPU9ycjKRkZF07969PMMrM4froZuMzgDkWSWhCyEqx9ixY2nVqhXt2rUDYNOmTbRq1YrAwEBuu+02/P39gdJHqlTUiB2H66GbjPaSS26epZIjEUJUhiv1pCtKs2bNWLhwYeHzqVOnkpCQQFhYGGCvoS9btoyTJ0/SsWNHhg0bRmhoKD4+PsTGxhY5V1paGlWqVKmQOMvcQ1dKGZVSu5VSy0o45qyU+lEpdUwptU0pFVKeQV7KZLIndKtVauhCiGujZ8+eZGdnM23atMLXMjMzi7WrU6cO48aN46OPPgKge/fuLFmyhLS0NAAWLVpEq1atMBqNFRLnX+mhPwscArxKOPYokKy1rq+Uuhf4CLinHOIrpiChW6TkIoS4RpRSLF68mOeff56PP/4YX19f3N3dCxP3pZ544gkmTZrEqVOnaNmyJU899RRdu3ZFKYWfnx/ffPNNhcVZpoSulKoFDADeB14oockdwFv5j38CvlRKKa21Lo8gL+VUUEOXkosQ4hoKCAhg3rx5JR7r0aNH4WNXV9cio1wef/xxHn/88YoODyh7yeVT4P8AWynHA4EzAFrrPOAC4POPoyuByewCgNUmJRchhLjUVRO6UmogEKe13vlPL6aUGq2UCldKhcfHx/+tczjll1zybNJDF0KIS5Wlh94FGKyUOgXMA3oqpeZc1iYGqA2glDIB3kCxxVa01jO01mFa6zBf3xL3OL0qJ1NBD10SuhBCXOqqCV1rPV5rXUtrHQLcC/yutb7/smZLgIfyH9+V36bc6+cA5vyEnmeVhC6EEJf62+PQlVLvAOFa6yXAt8BspdQxIAl74q8QTk7SQxdCiJL8pYSutV4PrM9//MYlr2cDd5dnYKUpLLnovGtxOSGEcBgON/XfxckVAKtNEroQ4trx8PAo9tqRI0fo0aMHoaGhNGnShNGjR/Pbb78RGhpKaGgoHh4eNGrUiNDQUB588EHWr1+PUqrIWPSIiAiUUsWW2f07HG7qv7mw5CIJXQhRuZ555hmef/557rjjDgD27dtHixYtuO222wD7+PRJkyYVLhGwfv16mjdvzvz58xk1ahQAc+fOpVWrVuUSj+P20KXkIoSoZLGxsdSqVavweYsWLa76nuDgYLKzszl//jxaa1auXEn//v3LJR6H66E7O7kB0kMX4qb16zg4t698z+nfAvp/+Jff9vzzz9OzZ086d+5M3759GTlyZJkW3rrrrrtYsGABrVu3pk2bNjg7O/+dqItxuB66c8HiXFgrORIhxM1u5MiRHDp0iLvvvpv169fTsWNHcnKuvs7UsGHDWLBgAXPnzmX48OHlFo/D9dCNRgNGrbFJD12Im9Pf6ElXpJo1a/LII4/wyCOP0Lx5c/bv30/btm2v+B5/f3/MZjOrV6/ms88+448//iiXWBwuoQOYNFi19NCFEJVr5cqV9OrVC7PZzLlz50hMTCQwMLBM733nnXeIi4sr16V0HTKhG9HYpOQihLiGMjMzi9wAfeGFF4iOjubZZ5/FxcU++m7ixImFuxVdTefOncs9RsdM6BqsurSFH4UQovzZbCXnnMmTJ5f6nvXr1xd53qNHjyJL7RZ46623/kFkFzncTVGwfwtJD10IIYpyyIRukBq6EEIU45AJ3agVtlL32hBCiJuTYyZ0wCY1dCGEKMJBE7r00IUQ4nIOmdANGqyS0IUQogiHTOjSQxdCXGtGo5HQ0FCaN2/OoEGDSElJAeDUqVMopfjiiy8K2z711FPMmjULgIcffpjAwMDCJQESEhIICQmpkBjLskm0i1Jqu1Jqj1LqgFLq7RLaPKyUildKReT/jKqQaPMZtMJKhexwJ4QQJXJ1dSUiIoL9+/dTrVo1pk6dWnjMz8+Pzz77jNzc3BLfazQamTlzZoXHWJYeeg7QU2vdCggF+imlOpbQ7ketdWj+zzclHC83BhQ2SehCiErSqVMnYmJiCp/7+vrSq1cvvvvuuxLbP/fcc0yZMoW8vIpdg+qqM0XzN3tOz39qzv+p1GxqRGFVUnIR4mb00faPOJx0uFzP2bhaY15p/0qZ2lqtVtauXcujjz5a5PVXXnmF/v3788gjjxR7T1BQEF27dmX27NkMGjSoXGIuSZlq6Eopo1IqAogDVmutt5XQbKhSaq9S6ielVO1yjfIyBgzSQxdCXFNZWVmEhobi7+/P+fPn6dOnT5HjdevWpUOHDvzvf/8r8f3jx49n4sSJpS4hUB7KtJaL1toKhCqlqgA/K6Waa633X9JkKTBXa52jlHoc+A7oefl5lFKjgdFg/8b6u4woLJLQhbgplbUnXd4KauiZmZncdtttTJ06lWeeeaZImwkTJnDXXXdxyy23FHt/gwYNCA0NZf78+RUW418a5aK1TgHWAf0uez1Ra12wqvs3QImLAWutZ2itw7TWYb6+vn8nXiC/h67+9tuFEOJvc3Nz4/PPP+eTTz4pVhNv3LgxTZs2ZenSpSW+99VXXy2XzaBLU5ZRLr75PXOUUq5AH+DwZW0CLnk6GDhUnkFeTm6KCiEqU+vWrWnZsiVz584tduzVV18lOjq6xPc1a9aMNm3aVFhcZSm5BADfKaWM2L8A5mutlyml3gHCtdZLgGeUUoOBPCAJeLiiAgZ7D12W5hJCXEvp6elFnl/aC9+//2IFulWrVkXq5AXj0QssWrSoYgKkbKNc9gKtS3j9jUsejwfGl29opTNixKqkhy6EEJdyyJmi0kMXQojiHDKhG5UBq9wUFeKmYp8Sc/P4O7+vQyZ0A0bpoQtxE3FxcSExMfGmSepaaxITEwv3Ki0rh9xT1CA9dCFuKrVq1SI6Opr4+PjKDuWacXFxKbIpdVk4aEI3SkIX4iZiNpupU6dOZYdx3XPQkovJXnK5Sf78EkKIsnDIhG5UBixKgWxDJ4QQhRw0odt76Npa8trDQghxM3LMhG4woZUiNzerskMRQojrhkMmdIMyApCTm13JkQghxPXDIRO6UdkH5+RaJKELIUQBx0zoBntCz5aSixBCFHLQhG4GpIcuhBCXcsyErvITutTQhRCikGMmdGN+Qs+TkosQQhRwyIRuKrwpmnOVlkIIcfMoyxZ0Lkqp7UqpPUqpA0qpt0to46yU+lEpdUwptU0pFVIRwRYwGZ0AsORJQhdCiAJl6aHnAD211q2AUKCfUqrjZW0eBZK11vWBKcBH5RtmURdLLpLQhRCiwFUTurYr2EzPnP9z+apYdwDf5T/+CeillKqw9RCNBnsPPU8SuhBCFCpTDV0pZVRKRQBxwGqt9bbLmgQCZwC01nnABcCnPAO9lNlk76FLyUUIIS4qU0LXWlu11qFALaC9Uqr537mYUmq0UipcKRX+TxaqNxX00GVxLiGEKPSXRrlorVOAdUC/yw7FALUBlFImwBtILOH9M7TWYVrrMF9f378XMWA2OQNgsUoPXQghCpRllIuvUqpK/mNXoA9w+LJmS4CH8h/fBfyuK3DzP5MxP6HnWSrqEkII4XDKsgVdAPCdUsqI/QtgvtZ6mVLqHSBca70E+BaYrZQ6BiQB91ZYxIDJJCUXIYS43FUTutZ6L9C6hNffuORxNnB3+YZWOnNBQrdJD10IIQo45ExRs8kFAKv00IUQopBDJnQns72GLj10IYS4yDETurGghy4JXQghCjhmQneSHroQQlzOIRO62eQKgNWWV8mRCCHE9cMhE3phD11LD10IIQo4ZkI3uwFgkx66EEIUcsiE7uKUf1NUS0IXQogCjpnQ8ycWWW3WSo5ECCGuHw6Z0J3yl8+VHroQQlzkkAndxViQ0KWHLoQQBRwyoZtN9iVobNJDF0KIQg6Z0A3KgEFr6aELIcQlHDKhAxg12CShCyFEIcdN6IANW2WHIYQQ1w3HTehabooKIcSlHDehIz10IYS4VFn2FK2tlFqnlDqolDqglHq2hDY9lFIXlFIR+T9vlHSu8mTUCqskdCGEKFSWPUXzgBe11ruUUp7ATqXUaq31wcvabdJaDyz/EEtmQG6KCiHEpa7aQ9dax2qtd+U/TgMOAYEVHdjVGLWSkosQQlziL9XQlVIh2DeM3lbC4U5KqT1KqV+VUs1Kef9opVS4Uio8Pj7+Lwd7KSNgRf+jcwghxI2kzAldKeUBLASe01qnXnZ4FxCstW4FfAEsLukcWusZWuswrXWYr6/v340ZAAPSQxdCiEuVKaErpczYk/kPWutFlx/XWqdqrdPzH68AzEqp6uUa6WXsN0Wlhy6EEAXKMspFAd8Ch7TWk0tp45/fDqVU+/zzJpZnoJez99AloQshRIGyjHLpAjwA7FNKReS/NgEIAtBaTwfuAsYopfKALOBerXWFZlsDBmxKFucSQogCV03oWuvNgLpKmy+BL8srqLIwIiUXIYS4lMPOFDWgkFHoQghxkcMmdIURLaNchBCikMMmdJQTKCtUbKleCCEchsMmdI2TveSSlVzZoQghxHXBYRO6zeCCVQEZ/2zGqRBC3CgcNqErowt5KEiPq+xQhBDiuuCwCd3JyZ0cgyLnwvnKDkUIIa4LDpvQa3gGcd5oJD4+qrJDEUKI64LDJvT6vs2wKcXxpOOVHYoQQlwXHDahh/o3AeBUenQlRyKEENcHh03orQMaoDREWyp0DTAhhHAYDpvQ3Z3c8LEaOEt6ZYcihBDXBYdN6AA1rC7EGC2VHYYQQlwXHDqh++FNtElhtcoyukII4dAJPcBcgxyD4nTC0coORQghKp1DJ/Qgz2AAIqLCKzkSIYSofGXZgq62UmqdUuqgUuqAUurZEtoopdTnSqljSqm9Sqk2FRNuUfWrNwbgSNzBa3E5IYS4rpWlh54HvKi1bgp0BMYqpZpe1qY/0CD/ZzQwrVyjLEWQfwN88/KITJXJRUIIcdWErrWO1Vrvyn+cBhwCAi9rdgfwvbbbClRRSgWUe7SX8alRizbZORzNPU0Fb2EqhBDXvb9UQ1dKhQCtgW2XHQoEzlzyPJriSR+l1GilVLhSKjw+/p8ve+vk5UdYdg4XyCI6TWaMCiFubmVO6EopD2Ah8JzWOvXvXExrPUNrHaa1DvP19f07pyjKaKaRxQWAHed2/PPzCSGEAytTQldKmbEn8x+01otKaBID1L7kea381yqcr0dDPK2aDWe2XovLCSHEdasso1wU8C1wSGs9uZRmS4AH80e7dAQuaK1jyzHOUjnVakv7rCx2nZehi0KIm1tZeuhdgAeAnkqpiPyf25VSTyilnshvswI4ARwD/gM8WTHhFufToD3tcrJJscQTk35N/igQQojrkulqDbTWmwF1lTYaGFteQf0VxsDWtM/KAWBb7Db+1eBflRGGEEJUOoeeKQqAdy1qanc884xsPLOpsqMRQohK4/gJXSmyqrcgLNPKn2e3kmeThbqEEDcnx0/ogFfddvTPSiTTms7+hP2VHY4QQlSKGyKhO9VuQ+fsTNCKTTGbKzscIYSoFDdEQqd2B7xtGt9sd349vhaLVTa9EELcfG6MhO7hh9WvGX0yLJzJOMbQpUM5kHigsqMSQohr6sZI6ICx3q28lH4K17iHSMtN4+PtH1d2SEIIcU3dMAmderdi1haaJOfSwbcfe+L3kJabVtlRCSHENXPjJPSgzmijE92N+8lKqY9VW9kWe/mikEIIceO6cRK6kxsqqCN9XQ6w86gn7mZ3tpzdUtlRCSHENXPjJHSApndQ23KKdhd+p1nVtmyJ2cJPR3/i0d8eJSErobKjE0KICnVjJfS2I7EEduBt8yw8MwKJzYjl7T/fZvu57Ty/7nlyrbmVHaEQQlSYGyuhG4yYh36N2QCdj2zE3ezBPY3u4aNuHxERH8Ho1aOZf2Q+m2M2czxF9iEVQtxYrrraosOpVofkuoPod2wp1N/EsPYNAEjOSea7A9/x7tZ3C5tO6z2NroFdKytSIYQoVzdWDz1fQNhgvFQW29evxGazbx49oskIfhv6G8vvXM7s/rMJ8gzi4x0fY7HJrFIhxI3hhkzoqu6tWA1mGqb+yZgfdrL1RKL9daUI8goi1C+Ul8Je4uSFk8w/Mr+SoxVCiPJRli3oZiql4pRSJS5jqJTqoZS6cMluRm+Uf5h/kbMHKqQrQz0OEHhiPmrW7SzfXHSLuh61e9ApoBOf7/qcyOTIyolTCCHKUVl66LOAfldps0lrHZr/884/D+ufMzTsh0/2Kd7QX9PBcJh6qx5m7sZ9HI9PB+y99Xe7vIub2Y2nf3+a5OxkAHKsOcSmX5PtUIUQolxdNaFrrTcCSdcglvLVqD+YXCF0BFn3LKC+4SzVVz9Lr082MHn1UQBquNfgs1s/Iz4znvuW38fiY4sZumQofRf25d5l97L+zPrK/R2EEOIvKK8aeiel1B6l1K9KqWalNVJKjVZKhSulwuPj48vp0qWoGgwvH4MhX+HapC+q56v0Me7kmfrxTF13jN1RyXy1/hhrdjvzn77/QaN5fcvr5FhzGBs6lsy8TJ75/Rm+P/B9xcYphBDlRNn3d75KI6VCgGVa6+YlHPMCbFrrdKXU7cBnWusGVztnWFiYDg8Pv1qz8pObCZ+HklelDl3Ov0RihoW8/BEw61/qQXUvzW+nfqN3cG+8nb3JzstmwuYJrD69mpfCXuKhZg+VemqtNUrZ99G22qwYDcZr8isJIW4+SqmdWuuwko794x661jpVa52e/3gFYFZKVf+n5y13Tm7Q7SVM0Vv5vu1xAqq48PrAphgNirk7ovDQmqEnd+Ftsc8mdTG5MOmWSfQJ7sPknZNZe3otvxz7hZ8jfy6stwO8+cebdPhfB4YvG07/hf1pO6cte+L3VNZvKYS4if3jiUVKKX/gvNZaK6XaY/+SSPzHkVWEtg/B3h9ptPUVNjW7E/yGE9HIk5/Co3nZdwemP7+EC9Ew7DsADMrAu13e5UTKCZ5b/1zhaYzKyP1N7qdelXosilxE18CuWKwWAjwCSMpOYuHRhbTybVVZv6UQ4iZ11ZKLUmou0AOoDpwH3gTMAFrr6Uqpp4AxQB6QBbygtf7jahe+5iWXAlYLbPoENk0Gaw7p3o1oef51toTMJODc7/Y2w+fZb6rmizm9mWU7v6Brt1cxGMz8eORHFkYuBKCNXxtm3jazsMzy2ubXWBu1lnXD1uFicik8x5m0M8w/Mh9/d3+sNisnU09yb6N7aVStUclh2qxk5WXh4eRRQR+EEMIRXankUqYaekWotIRewJIF4f+F38bzrtsrvJQxhfRGQ/FN2QtZyTBmC7hVA+DCfwbjHbMB64ifMTboCcCqU6tYFLmINzu9SYBHQOFpt8Zu5bFVjzHplkncFnIbYK+xj1o1iu3nthe2Uyg6B3Zmeu/pJYb32a7PmH9kPkuGLMHH1afw9TxbHkZlLKzZCyFuLpLQS2O1wJTmWK25GLOSGGkdT6axCnP0BHY6tWV+vY9pwjEeOzwKgFOBgwgeNZsle87SsIYnTQK8ip/SZqXvwr64mdzQaIK9grml1i28u/VdJnSYQJ/gPgAsOLqAryK+YumQpYR4hwCQnpuOq8mVbGs2vRf0Jt2Szj2N7uG1jq8B8MOhH5i4YyIA3Wt155Men2A2mK/BByWEuF5cKaHfeItz/RVGM4SNxLj+39icPKjdqA/a6Mzq+Ke5PeZTEo68hZ81lnSTF7vMrQmLWc0XK/cweUMMSkGvxjVwNhlwdzbSr7k/uXma4/HpDKxzB/898A1h/mFsi93GxuiNNKzakLsb3o3JYP/I7254NzP2zmDekXmMaz+O9WfWM27TOFpUb0GXml1It6TTtkZbfjr6E/c1uY+U7BQm7phIWI0wQrxD+PHIj/x7278Z3348exP2MuvALGzaxqe3flqY5COTI9kUs4nhjYfjanIFYOWplaw6tYoPun5QpCT0T2yO2UxTn6ZUc6lWLucTQvw9N3cPHSDtHExpBo0HFt4MRWtY/iJ69xyUNQd6vcFhc1Mar7yXz/L+RTNfJ5SXP7/FuHCXXgXWHEZnP0M7wxFeN81mS4cvGdCnM15OXhxOOsynuz5lbKuxtPBtAcCuqGRe/Xk/fnUXcij1D5r5NGPn+Z3U9qxNVFoUCkUL3xZ8fuvnDPh5ADnWHEzKhJ+bH/MGzsPTyZMpO6cwc/9MFAqNxtPJk7TcNB5r8RjPtHmGXGsudy29i5MXThLsFcwDTR4gITuB6XvsJZ4pPabQO7h34ceQlZfFv7f9m1WnV5FrzWV44+G8FPbSVUs722O38+iqR7mz/p280+W6mCQsxA1NSi5Xc2IDVKsLVWoXfT03E84fgMA2gCL5302oajmHNphQtjx7G3dfdHYqWa7+uGSexWCzMMN0H4+Mn4rJaB8Veuk49UW7onll4V4sVk3PFjYyvOfgbHSmmU8znm3zLP87/D+m7JzCpz0+pVdwL/Yn7GfN6TUkZCUwsvlI6lWpB4BN25hzcA5pljSCPIPoFdSLD7d/yOJji/mg2wdEpUYxbc80xoaOZfGxxcSkxwDQO6g3u+J20c6/HZNumQRAfGY8T659kiNJRxhcbzAZlgzWRK1hXPtxjGgyotSPzWK1MHTpUE5eOImryZV1w9bhbnYHpNYvREWRksvV1L2l5Ned3KB2u8KnVe//LyREopoNgawUSIiEkK6oqD9xmzsc/FuQnpZCk5S9LN8Xy6mETH47cI7j8ek83bM+XRv48srCvbQLqYbRoDgancHmET8WuWRtwwCqJFSDzOZYbZrVu52o7zeM57rWLNLOoAw82OzBIq+Naz+OfQn7GL9pPAB9g/vyRKsnGNViFAlZCWTmZRLiFcL7W99n6YmlZFoyMRlMPLf+OU6nnmZqr6l0q9UNm7bx3Lrn+HjHxxxMPMi9je4t/OviUjP2zeDkhZM80eoJpu+ZzsqTKxnacCj7E/bz7O/P4uHkwcPNHmZwvcEy2UqIa0B66OUlPQ5cvLGtep3c7f+lRfY3vGuaSZCH5qdqozCc3kyY6ST/cRnJomf7sGDnGd5bfojtr/bCz9Ney54ffoZxC/diNCgMStG+TjU2RSYQ7OPG+pd6lKm3a7FZ2By92T7apuVjVHctPseroEzyesfX2R23m2UnljG5x+TCG7YAmZZMPgn/hGUnlpGZl8mIJiPwMHvwc+TPdAnsQk2PmkyNmMqguoN4v+v73PnLnTgZnegb0pfpe6ZT3bU6nk6eHE46zIgmI3i2zbO8v/V96njX4dEWjwJwNPko22O3s+PcDvbE76GWZy161O7BgDoDiowcKsmx5GPU9KiJm9ntr/wr/WMrT61k1v5ZTO4xmZoeNa/+BiHKmZRcrqVDS+HH+/nK9XGezPq62OELQX3wvv87Irf/ytBlNiY+0J3bMpaSsn0uX59rQGztAbx4d29GzdxCSsI5goLrsuN0CmtfvIV6vuUzJt1qs9JrQS8Ss+3zv8a0GsOToU+W2DbDksEXu7/gh0M/ANDOvx27z+8mT+fRN7gvH3b/ELPBzPcHvmdiuH0ETqeATnzY/UOqOlfl4x0fM+fQHAI9AolJj8GojCwavIg1UWv4YvcXAAR6BBLqF8rJCyc5mHgQhWJQvUG81+W9wi+xX0/+yrqodYzvMJ7NMZuZsHkCriZX+tfpz6sdXsVsMPPb6d9IyU4hxDuE9v7tMajyX+7/nmX3cDDxILU9a/Ndv+/wdfMt92sIcSWS0K+ljESYWBdMLmAww8gVsG8+unZHcpPO4Lx6HBidwZrDL7YuHGv/Hs/tG0JGTh5eZKCVARXcBVvsHgw5qVhdfRif+i8a9nuSUd3qciHTwpgfdnI+NZunetZncKtAjAYFkash5TS0G1WmMJceX8qBxAPcWf/OUic3XWpP/B6cjc40rtaYEyknCD8fzr8a/Ktw1E6ONYeVJ1fSpkYbantevBdhtVl5Yf0LbI7ZzLgO45gcbu/ZHks5Rp/gPrzQ9oUiPd3otGhmHZjFj0d+5MNuHzKg7gB+jvyZN/94E40m0COQ85nnaVm9JXW867AwciEPNH2A+lXq8+Yfbxaep51/O+5qcBf7E/cT4hXCHfXvwNnoXOR3stgs/B71O/vi9xHmH4aryZXw8+H0D+lP3Sp1i30GJ1JOcMcvdzC43mDWnF5D6xqtS51H8FflWnM5lHRIZhiLq5KEfq191QniDkKnp+C294se2zjJXnu3WbDtX8Rvzv3on/MrDxve598P9SHg6A9weAXUbg/+LWH3HBLjYoNSDHoAACAASURBVHgmYA4Th7XhoZnbOZ2YSbCPG5Fx6dT1dWdsSCyD9z2FDQPmV6MxmJ1LjquSWG1WUnNTqepSlW/3fcunuz4lxCuEHwf+WGLJxGqzMmLFCOIy4+gd3Ju5h+fSpWYXHmr2EC9vfJmqzlWZc/scvJ29eX/r+8w7Mg+zwUxYjTDe6fIOG6M38unOT0mzpGEymMiz5VHdtTpf9vqSZj7NsGkbvxz7hS8jviQuMw6DMmDTtsLrdwroxIy+MwD7jd/3t71Pg6oNSMxK5Nv937L27rUsOLKAr/Z8xcqhKwn0CCz2O+yJ38Nnuz6jZfWWDKo3qPBmdmk+3vExsw/OZt6AeTSrXuqCpUJIQr/mfh0HO/4Dz0QUHzlTIDOJ7EnNcbFlsNPWEMvDK+lY16d4u/0L4adHGG55g5PuoWTk5PH1g23pWMeHVQfPsWrlL7yV9jYmZcONbP7ouYDO3fsWO022xcryvbE0MUXT5OzPqL7vg7HoPfHkjFyik7NoHuhVYaNTcqw5TI2YypB6Q0rsBRfYF7+PEStGoNEMbzycF8NexNnozIWcC5gN5sIvguy8bO5bcR+pOanMHzS/cCx8YlYi0enRNKnWhIi4CF7f8jpWbWVyj8l8tOMj9sbvpWX1ljze6nHa+7dnV9wucq257E/Yz9d7v2bBoAU0qtqI17a8xpLjSwBwMjgR5h/G132+JjY9ltsW3sbjrR5nbOhYLFYL7217j5j0GPqF9OOT8E8wGoxk5GbgZHTilyG/UM2lGstOLOOWWrfg4+rDxuiNuJpc8XPzY8gvQ8iz5ZU4/FNrTXRaNJEpkTSo2qDIX0Di5iMJ/VrLSobk01Az9IrNIn98lQaHvmR96Cf0GFJKqSQnHevH9fghpxvRzvV4xTQXY0BLqNEMUOjtM7B61YY7p2Oa1Y8Z7o/zyAsfsjEynqPn00nLtlCziivf/3Ga4+eTWer0Gk0MUWQ/uBKXup0KL6O1ZtjXf7LjVDJ1q7vToa4PIT5uPNQ5BBdz5YxQWXVqFT6uPrSt0faK7TItmeTpPLycis/cLXAo8RAP/vog2dZsPM2ejO8wnoF1Bxb74krNTaXPgj50COiAh9mDpSeW8mSrJ4nPimfB0QWFZSCAJ9Y8wbHkY8zqN4t3t77LH2f/oJpLNZKykwj0CGRWv1lYrBbuXHIn3Wt1x9nozLITy6jqXJXm1ZuzKWYTAP7u/lzIuUCngE5sObuFtXevxdvZG4CErATGbRrHtthtAJgNZh5t8SijW44udZawTdu4kHMBb2fvq95H+PPsn3y842Na+7XmjU6Vv3ukuDpJ6NcpmyWHs7t+pVb7O+AKPWI9/yEsR9ditmWjfBuDwQiJxyE3DZoMhju+BBdvMj5syJqMenzq9izd01ewxNqJVIM3NWzxNHLPYELjWBoc+ByAw02fofGwdwuvsXTPWZ6eu5vh7WtzKiGTI+fTSMrIZeJdLbk77MboEa6NWsvyE8t5MezFEsskBSbumMj3B7/HpEw80uIRngp9CrCPymlYtWHhl8Dq06t5Yf0LgH0Fzjc7vcmAugNYG7WW1n6t8Xf3B2DG3hmFN4Dva3wfe+P3cjj5MGNajeFCzgVmH5zNM22eoWtgV+5eejf9QvqRmZdJWm4ap1NPk2HJYGzoWFpUb8H8o/P59eSvDK43mFfav8IbW94gKi2KsBphDGs4DC9nL8auHcvhpMOYDCaeb/N8seGtBabvmc7UiKkYlAEXowub792M2Vj0SyLPlodC/aVhpytOrGBN1Bre6/LeNR+FdDOQhO7oDiyGBQ+Bd214fGPhomFYssF8cfq+dd4DnD+yjcVOA3gyZybWGi2g01OoZc9iyMsGQDceyNHD+8G1Co1e2cDO00mcSsjkk1VHqOLmxNL7a2Fc/wHavTp9w8NoXK8uXwxvXRm/daVJzk7mvwf+e9WykMVmYdKOSQS4B9AruFeppZBcay6P/vYojao14tUOr2LTNtIt6UV64T4uPiileOjXh9gVt4sgzyACPAJwNbnyVOhTRW5cT9szja8ivqKqc1XSLGm08WvD3vi95Npy8XbyJtuazagWo9h5fid/nv2T6X2m07lm5yIx/XLsF17b8hqD6g6ie+3uvLzhZb7p+w0dAjoUtjmQcIAn1jyBTdvoUrMLL4S9UPgldbkzaWcIPxfOtnPbWH5iOUCRobBaa6LSoqjlUesvz0mw2CxM3zOdnkE9aeZz5fsL3+77Fi9nL+5uePdfuoYjkYTu6CzZsOw56PDElcs4f3wJq15Fu/mgXKtCShRYcyEgFLo+b38eeh+bZ02gXdwitt69i4fm7APAyWhgee8EGvzxiv1ceVlkKxfu0R+w6PWH7SNp8p1OzODA2VR6N6mBk8lA0sk9ePnXxeTqWWJYEWdS2Budwj3tauNskglGV5KUnURKdgp1vOuUeh9Da83bf77NylMrmXzLZDoHdiYlO4Wv937N1titvNflPZpVb0amJZMRK0ZwPuM8bWq0IdAjkM41O7M1divzDs+jrX9bpvWehsVqoeu8rgxvPJyRzUfyc+TPOBudmbFvBh5mD8JqhLH69GpcTC5M6TGFNjXaFIln7em1jN88nqy8LEwGEw82fZCFkQvpHtidD7p9AMC0iGl8tecrqrlUY2DdgYxpNYYjyUf4es/XjO8wnjreddBas+3cNhYdXcTAegPpXqs7Wtu3hvzl+C+08WvDd/2/K/WzK5hf4efqx5q71xT7/LTWRMRHsOb0GqJSowjwCGBChwklnivTkklqbip+bn4VMvz1n5CEfrOI2gYz82+I3r8QNBC5Cnq9Ac4Xx7Af2vgTTX5/lIetr+Lp5sZLd91CVWsCXvPvti9zMPRbyElDT+vCp5Y7ufWJyYTWrsKh2FS+++MUC3ZGY7VpGvt7crv7EZ6KfpltwaPp9MjHRcLJybPy7rKD/LAtCq2hYQ0Pvhjehkb+JSd+UXZaa3JtucWGYl4uKjWKj3d8TFxmHKdST5GVl4VRGRlcbzAvt3sZTyf7v8XoVaOJzYilmks1dsXtAqCGWw3+2++/1PaszYmUEzz9+9OcSTvDsEbDaFG9BYeTDnM46TDh58NpUb0F73V5jyCvIEwGExM2TWBjzEbWD1vP8hPLeW3La/Ss3ROjwcia02vwcfUhOTsZq7YyrOEwXu/0OuM3jWfZiWUYlRGbtnFXw7uIzYhlc8xmGlVtxJHkIywcvJCGVRsW/n6puakcTzmOs9GZlze8TEx6DFZtZdHgRTSoenEnzIJ7BcdSjuFsdMbb2Zu4zLgiq52CvcT03Lrn2BC9AYDmPs15ud3Lxb7EAM5lnONM2hna+bfDarOy7dw22vu3LxzKe6k8W17h6/sT9lPXu+7fLkdJQr9ZWLLh37XAr4m9NFNKD8+alYr+KIQk7YmfSrG/aHKBKkHw6CpwrQpA3tc92RtzgRXtZ3M8Pp11R+KpaUqlb1gzWof4MGPZZmZbXqSaSmOLoS2dXluL4ZKe/ISf9/G/bVE80qUO7UKq8uri/TSs4cG80Z1KCktUsBxrDjvP76SWRy2CvIKKHJtzcA4f7fgIgPe7vk/HgI54O3sX+cJIy01jasRU5h6ei03bcDW50qBqA8JqhDGm1Zgiq3euOb2G59c/z+MtH+fbfd/a/xroNQ2z0cye+D28v/V9QrxDyLXmsj12O9P7TGfEihEMbzycp1o/xUfbP2LJ8SUEegQyqN4g7mt8H70X9ObOBnfyWsfXyMrL4vl1z7Pl7JYiv8cHXT9gwuYJvNj2RR5u/jD7E/Yzfc90NkRvoLZnbR5r8Ri3hdxGWm4afX7qw5jQMYxpNabw/V/s/oIZe2dwf5P78XPzY86hOSRkJTCn/5wiy18cSDzAk2ueJCk7icdaPEZkSiTrz6xnQocJDG88nK2xWzmSdIQA9wAWHVvEznM7eaPTG7iZ3Xhl4yvcUe8OXu/0+t/6d/xHCV0pNRMYCMSVskm0Aj4DbgcygYe11ruuFpQk9Aqy7yfwbQT+xddeuVTc572onhyB4dZx9tUlT/8BA6dAtToXG637AOuGibTOnk6mwZOJ7TMYsmc0qmF/uHU8tp9GoVKjSfJoiCXxJKce2kmLQG8izqQQcSaFib8dYUyPerzSrzEA7y47yOytp9n3Vt9ipRetNftjUtl6IhEvVxOhtatKT/4aikqNYsDPAxhQdwAfdvvwim1j0mPIseYQ7Blcaj0805JJt3ndyLXlUr9Kfb7v/33hXwOXKtgQxs/NjwxLBqvuWlU4WikrL6tw2WeAVze/yprTa/iy15d8f/B7NkZv5NHmjxLqF0qONQd3kzudAztz5y934uPqw20ht/HOn+/g5eTFyOYjeaDpA0W+oEauHElidiK/3PELUWlRLD2+lBl7Z3BH/Tt4t4t9wEBabhoDfx5IiFcIs/rNQinFseRjjFgxAm9nb0L9Qvn15K8YlAFvJ2+CvIL4pu839PmpDyk59s5SVeeqBHgEFM6Cbl69OV/2+vJvLzf9TxfnmgV8CXxfyvH+QIP8nw7AtPz/FZWhxV1laub30PeQlw0+V5jwUr83xg0f0dv5EP+651G6rrnD3ns/+iscWY7BtRoMn4tndAROa19n0oZdvJ7sRGRcOgDdGlTnpb4Xb+Z1qFONbzefZM+ZC7SvY/+PWWvNmkNxTPrtCEfOpxW29XF3YservYv0+EXFCfIKYu6AuUXKGaW50gihAm5mN24NupVd53fxVa+vSkzmAO392+Pv7s+5jHOMbDayyNDTS5M5wINNH2Rt1Foe+e0RgMLe8OW6BnZlzqE5RMRF0LlmZyb3mFy4Cuil+tfpz7tb3+XNP95k8bHFaDRdA7syvv34wjaeTp6MDR3Lu1vfZd2Zddxa+1Y+3vExJoOJ7/t/Tw23GnQK6ESgRyD7Evbx6a5PmbF3Bik5KXza41OquVajUdVGmI1mpuycQlJ2Em90fKPCRv+UqeSilAoBlpXSQ/8aWK+1npv//AjQQ2sde6VzSg/dAVjz0BPrYqvbE6OzO+yeAw8shrwc2PM/6PMOVA2Bkxvhu0E8mPsK242t+WhoS+pW96CJvzum1Cg4fxDObMV6eCVfxTVD9XyNp3ra65vjF+1j7vYo6lZ3Z1S3uvRpWoMV+2J5c8kBfn22W4m7QgHYbJoftp3mzxOJTB4WWmlj5UXpsvNHVl1tI5Xpe6Yzc/9Mlt25DD83vyu2zbRksiF6A3m2PAbVG1Rim4JefzWXaiwcvLDEBerAPprp1vm3YtVWBtcbzNOtny5xFE+eLY+hS4aSkpPCPY3uYdqeafxfu//jgaYPFGl3Nv0sty20bzvZoGoDFg5aWCET9Cp6+dxA4Mwlz6PzXyuW0JVSo4HRAEFBQZcfFtcbowlVtwfGgz/bn3ccC/VutT9u1O9iu/zyTlf3szx+z2i6BHvAosfg2BqwZNrbGMwYXaswwryBZ0+M4ameDdgdlczc7VE82CmY1wc2xZy/fnzPxn68ueQA204kFkno2RYrziYDm48lMGX1UXZF2f+kvaddErc0LL5Ils2meXDmdgK8XXjvzuYywuYaK+uOWKNajGJYo2FlKkG4md3oX6f/Fdu08WtDr6Be3Nf4vlKTOUBVl6pM6DABd7N74WSxkpgMJqb0mMKz655l2p5pBHkGcW+je4u1q+lRkzZ+bdgVt4sRjUdUyl4A13Q9dK31DGAG2Hvo1/La4m/q/Cy4+0GbByCglIWjXKtClSBGB6ZD3Wqw8FE4tATCHoWarcGvKfg1hgOLqfbLk6RG7cVibc97yw/h6+nMK/0aYzYo+PMr2DyZ2l41ecWjBdtO+vNwF3tN/7M1kUxZcxSzUWGxavw8nfngzha8teQAm47Gl5jQ/zieyOZjCQCcTsxk1iPtcHOSLQCuNyaDqVy3L3QyOvHprZ+Wqe2wRsPK1K5ulbrMGziPGXtn0DOoZ7EJWAXub3o/OdYcbq97e5njLU/l8V93DHDpjIpa+a+JG0Gttvafq/FvCWcjYOUrcGAR9H4buj5XtE1+776dNYJhX//J7qgUPvxXC9wtSbB0nH3dmpBukJnI6Ly5dD/RH601JxMymLruGJ3r+dAi0Jt6fh7cEVoTZ5OR5fvOsikyocSQZm89RTV3J17q24gJP+9jxb5z3NW21j/9RMRNyt3szvNtn79imz7BfYrsK3CtlceI+SXAg8quI3DhavVzcQMKaAXJJ2H7DPsqk12eLd7GqyZ51RrSw7SfxPRcXuziw7D0OfB5azj4C9z6Gjy4BPq8gxErQdkHiYxL562lB3E2Gfj03lDG396EYWEXJyh1a+DLkfNpnE/NLnKp2AtZrD54nnva1ebedrXxdDGx83RyqeEfik1l6rpjjF+0lwNnL5TrRyPEtXLVHrpSai7QA6iulIoG3gTMAFrr6cAK7EMWj2EftjiyooIV17G6PWDTJ/aeeccnSm1matCTLqnfsaHjTtTGSZCXZd+gu/fbUL2+vVHt9mhloIPhEPd8/SfJmRbeHNS0cGcnwL742W8TGHV6K1/xAT/tjCYqMROlwM/LhfVH4tDAfe2DMBgUbYKqsquEhJ6Ukcv0Dcf5ZtMJbNo+dN9mg4/ualmuH48Q18JVE7rWuvi4oKLHNTC23CISjql2e5hw1r5w2JXU64naNh1+f9eeyHu+bq+vX8rFG/xb0CPuGEtcnXh1QFOGuu2GmWPAqyZciIHoHYDGpG3c4naaib954GI24Go2kpxpoVENT94a1Iza1ezDw8KCqzJ5zVEuZFnwdjWTmm3hxfl7+P1wHFabZnj72vzfbY15ccEedkaV3pPPybPyx7FEIs6k0KFONTrXL/2mG0Bieg7x6Tk09i99JUghyovcIRLlpyyLLoV0gyaDoEFfaP1AqbNZVXAXWsXPZO1znVCZiTB1LDi5Q1qs/SZs1+egxTCY1okRteJJV768NbgZQdXcyLbYcE06BHE7QAdDdDijIh5jKU+wOyqZbg18eWbubjZHJjCqWx2GhAYWjqZpG1yV3w/HkZKZSxU3pyIxxaVl89j3O9lzxj66pnmgF8ue7lbqr5ptsTLim23EpeWw87XelTLqQdxcJKGLa8vJDe6Zc/V2wZ1RW7+CqK3wx+dgs8DDy4pPhPJtTEfnU3Qc0b7wJVdlgXn32bfkO7URDq/ANTOBW4x72Xn6VtYfiWf9kXjev7M5IzoEFzld22D7sge7opLp2bgGABeyLPwSEcP09cdJzrQw5Z5WHDybyrebT5Kek4eHc/H/G2mteXfZQQ6fs0+WikrKJNin+OQWIcqTJHRxfQrKX+9l9hDQNrh9UsmzWgPbwuHl9uULCnrAf35pT+YN+8Gu78GlCrhWpZP1LGM3nSTLYuWRLnWKJXOAVrWqYDQowk8l06ludWZuOcn09cdJy8mjWU0vvn4gjBa1vKnmHs9/Np0kIiqFrg0ull0Oxaby/I8RRMalY7VpejTyZf2RePbFXCiW0AtG8Px5IpE/jyeSnJnLNw+2w9Wp9L90lu+NZeaWk3xydytCqssXhChKErq4PrlXh7YjITcdQkdcnNB0ucC2sHu2fYSN1vbNsjd9Yi/rDJsNe+fba/TrPqB51OHCZP76wCZFz2O1gLbh6uRMs5pebD2RSPjpZLafTKJ3kxo806s+LWtVKWzeJqgKBgXhp5NwdTLw3vJD1PR25ffDcXi5mni8e11qV3PjjtCatHp7FftiLjCw5cXNsBPTcxg9e2fhyBsvFxOp2XmEn06iW4PiY+qzLVbGLdzL4oizAGyMjJeELoqRhC6uX4PKMDmkVv4M6K3TIXymvTTj0wD6vm/vsbe6x37cvyW+kauYcW9T+rQKKVrPvhANPwyD9PPQYxxhQe2Z+Uc0AJ/eE8qQ1sXXLvF0MdPI34vwU8lsOZbAsbh04tNyaBtclcnDWuHndXFETiN/T/bHXBwKeSI+nUdm7SD2QjavDWhCz8Z++Ho60+rtVew4lVwsoadk5vLY9+HsOJXM870b8u3mE0SeT7/ixxJ5Po354WcY0jqQZjW9r9hWa43VpjEZr691v8VfJwldODbfJmB2g+1f29eVeWBx0RUjCwS0RGkbfasngqoD2RfsXwAXYuDwMsjNsO/TuuIlhrV4mZm05sU+DUtM5gXahVTlh21RWG2atwY1LZzVerkWgd6s2HcOm00zc8tJJv52BFcnI/97rANtgy/OkGwS4MWOk0lF3nshy8KIb7YReT6dqfe1YUDLgPz9YtMuvwwAWblWXv9lPz/ttH8hpedY+fe/Sl95c290CmPm7KJzPR8m3l3KTGDhMCShC8dmNEHNNnB2F9z7v5KTOVxctiB2D6Sdg2XPQ0Zc4bIFDJlmX6Lgq440ytrD4rFP0apWKT3bjARY8gzDDfX43tYRH3cn7mlX+tpEzQO9mbv9DB+tPMzXG0/Qu0kN3r+zOTW8iq510i6kGvN2RGGx2sjMtbI3OoVP10Ry9HwaMx4M49ZG9oWrGvh5sPrg+WLXOZOUyeOzd3LoXCpP3FKPrScSORibWmJMO08nsyQihrnbz5BrtbHlWMmzbS9ls2kyLdYSbwKL64P8ywjHN3AKWPJ72KXxrm2/ObrvJ4gJB9/GMGK+fa2ZSwW2RUWuIrSW98WbrFYLnNoEJ9bbb9Ae/AVSomhQtR5mYydGd697xRuZLQLtXwxfbzxBtwbV+c+DbUscwti+TjVm/XGK3w6c47XF+0nJtGA2Kj67t3VhMgeo7+fBvB1nSEzPwcfDvr735sgEnp67izybZuZD7bi1sR/vLjvInK2nybPaCssp2RYrby89wNztZ3A2Gejfwp8Ab1embzhOckYuVd2disUFkJ6Tx0Mzt5OYnsO6l3qUGH+e1cZ3f54mPi0HX09nHukSIkM1rzFJ6MLx+V59DW+UgoCW9qV+PWrYSzPuPsXb1WwNET/Y6+pVakNGIvy3HyQcBaMTGEzg4QdNh2A6uJiNz4Th71f8JiZgv0kb/i1N0+IxG1vhYjLy0dCWpSa5sBD7kMkX5u/ByWjgvyPb0bp2lWLj4RvUsK8tfiwuHR8PZxbtiubln/ZSz9edGQ+EFd4sbRrgRU6ejVOJGdT38yTbYmX4f7ayOyqFMT3qMfbW+ng4m9gcmcD0Dcc5FJtabKJUTp6VvdEXmLjySOEN3OjkrMIJWwVsNs24Rfv4aWd04QJqner60LRm6ROqsi1Wlu+NZWCrgFJXwszJs7LtRBL7Yi5we4sA6siN4CuShC5uHjVb2xP6kGklJ3Owl28Azu62J+4fR9iXGRj6LTQeAOb8TReO/gYHFxOQfQxUCWt4p52D39+D3bMxKSPjem+mToAPNau4Fm+bz8/ThRAfN04lZvL+kOZFeuWXauBn3x82Mi6dEwkZjF+0j871fJjxYFiRckhBMj1wNpV6vh6MW7iXiDMphbX4Ak0C7F8QBy9J6D/vjmbe9jNEnEkhJ8+G2agY06Me09YfZ/eZlCIJXWvNBysO8dPOaJ7r3YDBrWrS85MN7D97odSEnpNnZcycnaw7Ek9Ono37OhQvWeXm2bjvP9sKv0hOJmQw6Qp1/qjETDZGxtO0phdtgqqW2g7sXyZp2Xn4el55T1ZHIwld3Dy6PGcfmx7cufQ2NZrZe+Fnd9m35Yv6E+6aCc2HFm0XEGr/39g9Rc93bj+seBmi/rA/D+kGpzbxaMNsCKxR8jUvGUP/cOcQTidlXnFVyABvF9ydjGw4Gs/Go/H5ZZywYpt81PP1wMlo4GBsKheyLCyOOMtLfRsWSeYAPh7O1PBy5uDZVHLyrLyx+AA/hp+hgZ8H93cMpn2darQPqYaHi4mZm0+y50wKg1tdHIL51frjfLP5JA91CubZXg3QGtydjByIuQBh9oVYj8Wl8+Gvh3F3NuLmZCTizAUOxabibDKw41RSiQn9378eYufpZN4b0py1h86z7WRiqZ/Jfzae4P0VhwBoVtOL5c+UPoM3JTOX4f/ZRmJ6DlvH97rirljZFitJGblX/CL+q04mZBDg7VIhm7JIQhc3D7dqV07mAGYXe1I/vBySTtjHwl+ezAE8a4CHvz2hW/PsSwaf3Ah75oFrFej5GjQaYD/f563h3D4ILL5zPCc2wMJR9mv0eafUkTKXUkpRv4Ynqw+ex9lk4MOhLUtMDk4mAw1qeLDtRBInEzLo1qA6Y2+tX+I5mwZ4cTA2lS9/P8aP4Wd4umd9nuvdEONlya55/p6xOXlWvlh7jPDTSWw9kcQdoTV5c1AzlFIoBc1qerP/rP2GbFq2hdGzw4lPy8Hb1UxWrpXa1dz45O5WrDl0nu2XjexJz8nj87WR/HfLKR7pUof7OwaTm2dj3ZF4YlKyCLwsuf5/e+ceHVV1/fHPyYOAgRBCSAjkQcIjSCCRGEIAhVhFAXmqRaAVaavYWqrYH1XRZdVaV31g+6urKFVrq5UCKvgTLSoWsYiF8Ca8Q3jkBQRCCK+EkMf5/bFvyGQyd5IqZGbi+ayVNXfOnJm759zJ95y7zz775B4/x4uf7eNGK/zzvc2FlF+sbpD7/kJVDb9bsYdT5VXkFJ+9tII398Q5+kQ23h6voLScJ5fvYm1uCTW1mi/nZDZyM9V9bt2K4fT4MEb07tKog9BaX8oh9MXe4zy4eBt3XBvNU+PdzPl8Q4ygGwzOdBsIm/8GgcFww2P29aJSRNBXPS3pCYJCIHkyjHym3qVTWyvlx7Lr31d1AQ6sghN74cvnZLI261Vx88z4GGw2T3Ckd0R7theUcc/18Y0EzpGkbiG8u0lCGB8Z1dfWf9+vWwhr9pdQUFrOrclR/I/DXrCOXBMTyjvr8/jb14f50+pcUqI7ct/wBObckthAyCSyR0I6H1maTd7JchbeM5iMhIaurtMVVXyy8xhHyipoG+jPwvV5vL1eJlbvTIvh0dGSuK3ufVkHT3Jbav3dy9kLVTy2bAft2vjz3O3J7CgqY/HGArYXnGZIT3nPRN1ufAAAErdJREFUhaoa7n17E2tzS+ge2o7K6lqeGNuPZz7ezZa8Uw0Evbqmlo+yj/Dkh7vQwLjkbizdUkjWodJGgp57/Cw/X7iVfcVnCfBTvPrlgUburL3HzvDU8l2sP1hKZEgQx89WktQthJnDE2yv2bfBCLrB4Ey3VBH0YQ+KH92OqBTI/RxO5kLyneKbd05Q5ucHkf1lhA4i8O/eBftXyvPYoTBlIex4Dz55GIo2Q2xGkyZmJnZhz9Ez/CzT9Yi7jn5W0rFxKd3o391+gVG/qI7U1Goqqmp46Cb7SeaUmFD+svYQv/88hyEJnVk007Wt/buHUFFVw9LNhazYcYw5N/dpJObApc3Cv86VidkDJ+ROYsEPr72UVwegb9cOdGwXyHpL0MvKL/LSyhze31xIRVUN876fQpcOQQyMqc/FM6RnZ85VVvPTv29mbW4JL9yRzGTLBaS15k9f7GdL/immpMey4VApn+8+xj+zj3Lk9AUGdO/I/GmpRHdqx8rdx9icV8od10Zz9kIVQQH+FJVVMOW1LEDz1x8NYnB8GOnPrmLdwZJLgv7pzmP8YtEWgoMCmHVDL/JKywlv34aHb+nrNirq22AE3WBwJmmiTGoO/YX7elEpEsao/ODGJ+2zTXYdIBts19bISH7/Ssn/njJVOgylxOXyycPis3cl6GeOyGrWk7nQoStjpy5irBs/cR0jEiNI3lrEnJvdRwIlWZOXkwZG08uadHXFwBhJf1BZXctDI+0/s67zeOafuwlpG8CPbFxJV0eF0D4ogGdX7KGsvIrXp6cxsl/juQY/P8Xg+DDWHTzJko35vPDpPsoqqrgjNZppg2NJsezqFNyGhC7BbM0/Ren5i0x/M4s9R8/y4h3JfD+tfmM1pSRH/ua8U/wnt4Rpb2TRxt+PjJ6deXpCf77XN+KSuyktrhMbD5/ifGU1I178kovVtbQJkDDQd+/LoFeEjPBT4zqx6bBM4H6y4yizFm0lObojb949yDYc9HJjBN1gcKZtR8h8pOl63VNB+csOTR3tV5QSlQwbzsO2f8CqZyBpkoz+Hd0fweGSsiA/q76stlbSEZw6DMtnSSdz7d2QtQByPoWIqxudypn48GCWz7quyXo9woN5eepAhvd2n989ulM7IkOC6BXR/tLo2hU9u0h++rMXqrk/syfBNouR/P0UqXGdWJNzgluSIl2KeR2DEzqzcncxjyzdQWpsKO9MGtBgE/E6UmMlBfLcZdnkFJ/jjelp3NC38Z1WalwnVu09zvOf7SOiQxBfzMl0uWgqrUcYq/ft4+11eZSev8iYAV0pOlXBbycOuCTmAIOsnPsnz1XyxIe76N8thL//ZHCLLsRq1pmUUqOAPwL+wBta6+ecXp8BvEj9XqJ/0lq/cRntNBi8j5BucP9611kgHelqLb3/6AFJTzDuZdd54GMHy2Rs1QVYdCcc/lpy04D48+9aJqP3/SuhcFPj958uhL0rQNdA8S44sBpufAJSGu9Q3witG0Su2KGUYsnMIYRe5d7P7++n6BcVws6iM8wY2sNt3cw+Xdiad4onxvZzW29schQbD5UycWA3bknqajsfkBrbifc3F/LZrmIeHd3XpZgDDIyVUf32gjIeG9PXVnjTLNfPH1flkBjZgfnTUl2ee1B8GFrDvJU5lJyr5He3DWjxVbXN2YLOH5gPjAQKgY1KqeVa691OVZdorWddARsNBu+lOYuauvSVUEjlD99/C9raLLaJyRDXzKrfyKrUtB9LxE3HGNmEO8SabIseZK1atcIda6ok5n39q1BTKXXaWCPHXR+4F/Sj2RJlE5sB419u1ldubpbH2Tf1oeRcZYNEZa6YMbQHkwfFNCl+kSFtWXBX0xuWp8aJUKfEhHLPdfZRQynRkjGzfVAAU9PtUzekxIQS6K+4UFXL1PQY247kGqveog35hLcPIjPRZsHZFaQ53Uc6kKu1PgiglFoMTACcBd1gMLgiIAiG/0qEPcrNXqV1OeDXz5dIm1t/73okHz0IspfIiDy4C7w3A3I+EZ/88F9JeGZgMPzzl7DnI3Hd+DllUqytgaw/w7+ekk6g/CSM+6PtDlJUnIKs1+S4Oe4oYHif5gman5+6rCPZxMgOzB3dlzEDotxmkAwOCuAHg+NI7NqBDm3t7zjaBvrTv3tHdh85w6RU+/UBbQP9GdC9I1vyy7g9tTuBHshe2ZxW7A4UODwvBAa7qHe7Umo4kAM8pLUucK6glJoJzASIjbXvEQ2GVkfmo03X6dwTrgqH8hIY/rC9uNalDC7cADuXiZiPmQfp9zasF5shueJLciQnfHWl+N+P7YTju6F4p2wFGD0IVj8rOeXDnMLpaqolk+Xq38HFs4CCIfdDUOPYbW9BKcV9I5pwg1k8M7F/s+o9MqovJeckjt4d6fGd2ZJf1mACtiW5XN3iR8AirXWlUuo+4C3ge86VtNavAa8BpKWl6ct0boOhdaAU9B4pAtxnlH29yP4Q0BbWzBNhvumpxmIO9SP+/HUSjfP+j+HEHskueVW4pDPof7v421c/CwUbGwp63n8k8ubYDhH++BGw8nGJl48f7tq2ilOw/19wphCGPtj4zsBHcRVy6Yr7hieQkRDmNlLoStIcQS8CHLubaOonPwHQWjuuyX0DeOHbm2YwfAeZMF/cIe6E0D9QXDL56yAiCYbYhFeGJYhL5vBXsP4VqCiDae9Bn5sb1ou4WnzuhRtkQ5CyfPj81+J/D4kWv3+/CSLWKx+Hwo0NBV1ryPsaNr4Bez6un8iNHSoTvXYc/lp2l7p1XuM7Ax+lU3AbMm1y8LQEzRH0jUBvpVQ8IuRTgGmOFZRSUVrro9bT8cCey2qlwfBdwc/fPp7dkZh0EfRbX5Kc8K5QStwuO5fKc1diXnfO7qlQkCVb+C35IaAgcy4MfUA29gbxzXfuLRE2547Du9PF9VJ+UhZEtQ2FQfdA75vgndtlg25nQS8rkI4jf710ALoWdiyFEb9y/R1qa+DAF5KyeOBd7jsIkLuHgLbNCulsjTQp6FrraqXULOAzJGzxTa31LqXUb4BNWuvlwANKqfFANVAKzLiCNhsMhmGzoddIiBvivl5MhkyM9h3rWswv1RsMX82DD+6Dzr1g2hLo6GICMHqQrI7d8JqIcmR/qK2WjuWaH9Rno4zsD4e+kknaHe/DvhVQtEX89CBRP8l3SkeQtxZwEvSS/RLxk70EzlpjxYvn3At68W746xjZqOTeVe7bpeqC3El48VzAN6FZPnSt9QpghVPZrx2O5wJzL69pBoPBlqvCIL7plaJcPQ4OroZRz7mvF5Muo+WL58W37krMAaKvhe3/gHWvQOJomLrIdb0e18Pmv8pofulPoEM3uQtInwk9hknET0CQZKbc+o6EXlZXyt3EtoVyt6D8ZU5h9AuS/Ozw1w0yUwJQkgvZi8W1lLUAqsrh6Db5Hm2cwit3LoWCDXBin9zdBEfA7Gz7yWeQSec9H4krrE3j5FwNKD0k6ww8uKmHWSlqMLRmOsXBD5c2XS8mXQTuhrkSEWNH9CB5rDoPGT+zrxd/vSQcWzZT/PP3r5MslM7EDZPRftEW+PQRcZmEJ8LI30DyFMlqCVBRKj79kv1wfBesmy/ifmSLdEQgG5AMfxjWvCB+/oTM+vOsmQdfPANt2kOnePm+h9ZIKoXw3lLnbLHMN7SPkPmCPR/DjnflteQ7IdFmolprSdC29g/SGQ64o/HrF07Xf//TRZIuIiSq8Wd9S4ygGwwGSXcwJ6fp0WVEkmzKHZYgo3A74oaJaJUeEPeQKzGvqwfw2VwR87H/C9fOaGxH3bkO/Vvy4dRUiWto6AMw5OfWyN1PRv1fzYO8dSLoZfmwfoHE9g+YDJMWyJzBiRyYP0hG6uG9Ydf/wcezRcjr8G8jKRqy/iznTRwFlWdh93Lx6ys/+awzRdI5KD/pdBwF/dwJWSG8b4WM3gPaSaTRsNkw8mn3bf0NMIJuMBiE5rgK/ANg4isQGue+frtQWd16fA9k3G9fr30XCO8jvvSIJEi92/XnhiVI/vk18+DcMZj8tkTeuCKyv2wwsusDCdXUWvz7416un3AO7w3twiR3TmQSvHe3RA5NXSyun8CrJGVDYFtZTXvwS3HjvDIEThdAhyjpPGpr5DHzMTh/QtxHF8th2b2Sb0drmS8YMkvy61dVwDXTxF11BTCCbjAY/juSJjWv3s2/lUVSHewTbgEySi/JkbwzduGaSkGP62Dn+5IKIfFWN583FDa/JZOkUSki/qFOCxnrIoDy18k5A6+C6ctdp2VIGCEratf+QcR88ttw9fjGHc/BL2Hj6xLaufdj6DdROqLkyS0WdWME3WAwXBmaM2kLMnrt3Mv9YiqoF/T0mfahmiALqrIWQM1FmP5hYzGvI2awuELOHJEFVnY5duJHyONXL8l77O4M4oZJ6OamN8W9ctvrENAyaXPrMIJuMBg8S3gvCG9GXr+kSeITT/uR+3pxw8T/nXE/dHWztL9uJW11BaTeZV8vKkWE+kKZ+Ozt8A+UTil7MXzviRYXczCCbjAYfIV2oXDTk03Xa98FHtgqoZLu6HYN+AdJJFCMm/h2P3/xeR/NhsQx7j/zuofEzZJ0W9N2XgGMoBsMhtaHXRy9IwFBcMuzkhStqQnhcS9LnvmmctNE9HUf9nmFMYJuMBi+u7hKauYKD7hPvgmtIxWawWAwGIygGwwGQ2vBCLrBYDC0EoygGwwGQyvBCLrBYDC0EoygGwwGQyvBCLrBYDC0EoygGwwGQytBaa09c2KlTgB53/Dt4UDJZTTnSuIrtvqKneA7tvqKneA7tvqKnXDlbI3TWndx9YLHBP3boJTapLVO87QdzcFXbPUVO8F3bPUVO8F3bPUVO8EzthqXi8FgMLQSjKAbDAZDK8FXBf01TxvwX+ArtvqKneA7tvqKneA7tvqKneABW33Sh24wGAyGxvjqCN1gMBgMThhBNxgMhlaCzwm6UmqUUmqfUipXKfWop+2pQykVo5RarZTarZTapZR60Cp/SilVpJTaZv01sYdVy6CUOqyU2mHZtMkqC1NKfa6U2m89dvKwjYkO7bZNKXVGKTXbW9pUKfWmUuq4UmqnQ5nLNlTCy9bvNlspleoFtr6olNpr2fOBUirUKu+hlKpwaN8FHrbT9norpeZabbpPKXVLS9npxtYlDnYeVkpts8pbpk211j7zB/gDB4AEoA2wHejnabss26KAVOu4A5AD9AOeAuZ42j4X9h4Gwp3KXgAetY4fBZ73tJ1O1/4YEOctbQoMB1KBnU21ITAG+ARQQAaQ5QW23gwEWMfPO9jaw7GeF9jp8npb/1/bgSAg3tIGf0/a6vT6S8CvW7JNfW2Eng7kaq0Paq0vAouBCR62CQCt9VGt9Rbr+CywB+juWav+ayYAb1nHbwETPWiLMzcCB7TW33R18WVHa70GKHUqtmvDCcDbWlgPhCqlolrGUte2aq1Xaq2rrafrgWZsxHllsWlTOyYAi7XWlVrrQ0AuohEtgjtblVIKmAwsail7wPdcLt2BAofnhXihaCqlegADgSyraJZ1W/ump90YDmhgpVJqs1JqplUWqbU+ah0fAyI9Y5pLptDwn8Mb2xTs29Dbf7s/Ru4g6ohXSm1VSv1bKXW9p4xywNX19uY2vR4o1lrvdyi74m3qa4Lu9Sil2gNLgdla6zPAq0BP4BrgKHIb5g1cp7VOBUYDP1dKDXd8Uct9olfEtCql2gDjgfesIm9t0wZ4Uxu6Qyn1OFANLLSKjgKxWuuBwC+BfyilQjxlHz5yvZ2YSsMBSIu0qa8JehEQ4/A82irzCpRSgYiYL9RaLwPQWhdrrWu01rXA67TgLaE7tNZF1uNx4APEruI6N4D1eNxzFjZgNLBFa10M3tumFnZt6JW/XaXUDGAs8AOrA8JyYZy0jjcjvuk+nrLRzfX21jYNAG4DltSVtVSb+pqgbwR6K6XirVHbFGC5h20CLvnM/gLs0Vr/3qHc0U86Cdjp/N6WRikVrJTqUHeMTI7tRNrybqva3cCHnrGwEQ1GO97Ypg7YteFyYLoV7ZIBnHZwzXgEpdQo4GFgvNa63KG8i1LK3zpOAHoDBz1jpdvrvRyYopQKUkrFI3ZuaGn7XHATsFdrXVhX0GJt2lIzwpfrD4kWyEF6uMc9bY+DXdcht9fZwDbrbwzwd2CHVb4ciPICWxOQ6IDtwK66dgQ6A6uA/cC/gDAvsDUYOAl0dCjzijZFOpmjQBXiv/2JXRsi0S3zrd/tDiDNC2zNRXzQdb/XBVbd263fxTZgCzDOw3baXm/gcatN9wGjPd2mVvnfgJ861W2RNjVL/w0Gg6GV4GsuF4PBYDDYYATdYDAYWglG0A0Gg6GVYATdYDAYWglG0A0Gg6GVYATdYDAYWglG0A0Gg6GV8P+Zp/TL9p0N5QAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "input_dim = len(dataset.uniq_chars)\n",
        "embedding_dim = 300\n",
        "hidden_dim = 1024\n",
        "output_dim = len(dataset.uniq_chars)\n",
        "n_layers = 2\n",
        "drop_prob = 0.2\n",
        "\n",
        "num_epochs = 30\n",
        "\n",
        "all_losses = {}\n",
        "for cell_type in ['GRU','LSTM','RNN']:\n",
        "    model = Model(input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob, rnn_cell=cell_type).to(device)\n",
        "    print(cell_type)\n",
        "    all_losses[cell_type] = train(train_loader, model, batch_size, sequence_length, num_epochs)\n",
        "\n",
        "# Visualize the loss evolution during training\n",
        "plt.figure()\n",
        "for k in all_losses.keys():\n",
        "  plt.plot(all_losses[k],label=k)\n",
        "plt.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b2Cqzue_I-rD"
      },
      "source": [
        "B) Train your best model on a different dataset. Use the 'inaugural' corpus of the [NLTK](https://www.nltk.org/) library. Once trained generate some text with as we did for the model trained on the Shakespeare dataset. Compare the generated texts with both models using the smae \"seed\" (input) text. What differences do you appreciate? Why are they different?\n",
        "\n",
        "The Natural Language Toolkit, or more commonly NLTK, is a suite of datasets, libraries and models for symbolic and statistical natural language processing (NLP). The Inaugural Address Corpus contains the US presidential inaugural addresses from 1789 to 2017. The following code loads the 'inaugural' corpus:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hCtftivuXzSN",
        "outputId": "9d60a259-dd2f-4395-b517-5088ac5f97bc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[nltk_data] Downloading package inaugural to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/inaugural.zip.\n",
            "Length of text: 807276 characters\n"
          ]
        }
      ],
      "source": [
        "import nltk\n",
        "nltk.download('inaugural')\n",
        "\n",
        "inaugural = nltk.corpus.inaugural\n",
        "text = inaugural.raw()\n",
        "\n",
        "print(f'Length of text: {len(text)} characters')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "D1ts8t_q8bV1"
      },
      "outputs": [],
      "source": [
        "class InauguralDataset():\n",
        "    def __init__(self, sequence_length,):\n",
        "        self.sequence_length = sequence_length\n",
        "        # The whole sequence of characters in the text file (including chracters like the end of line character '\\n', punctuation marks, etc.)\n",
        "        self.chars = self.load_chars()\n",
        "        # The list of unique characters (our vocabulary)\n",
        "        self.uniq_chars = self.get_uniq_chars()\n",
        "\n",
        "        # A dictionary with indexes as keys and characters as values\n",
        "        self.index_to_char = {index: char for index, char in enumerate(self.uniq_chars)}\n",
        "        # A dictionary with characters as keys and indexes as values\n",
        "        self.char_to_index = {char: index for index, char in enumerate(self.uniq_chars)}\n",
        "\n",
        "        # The whole sequence of character indexes\n",
        "        # We want to vectorize our text using one-hot vectors at character level\n",
        "        # But instead of using the one-hot vector we just keep the index of the 1 value\n",
        "        self.chars_indexes = [self.char_to_index[w] for w in self.chars]\n",
        "\n",
        "        # Split the sequence into chunks of `sequence_length` characters\n",
        "        # `x` is the input to the model and `y` is the output of the model\n",
        "        # `y` is the same seuqence as `x` but one character shifted to the right.\n",
        "        self.x, self.y = self.get_data()\n",
        "\n",
        "    def load_chars(self):\n",
        "        inaugural = nltk.corpus.inaugural\n",
        "        text = inaugural.raw()\n",
        "        print(f'Length of text: {len(text)} characters')\n",
        "        return text\n",
        "\n",
        "    def get_uniq_chars(self):\n",
        "        char_counts = Counter(self.chars)\n",
        "        uniq_chars = sorted(char_counts, key=char_counts.get, reverse=True)\n",
        "        print(f'Unique characters (vocabulary size): {len(uniq_chars)}')\n",
        "        return uniq_chars\n",
        "\n",
        "    def get_data(self):\n",
        "        x,y = [], []\n",
        "        for index in range(0, len(self.chars_indexes)-self.sequence_length,self.sequence_length):\n",
        "            x.append(np.array(self.chars_indexes[index:index+self.sequence_length]))\n",
        "            y.append(np.array(self.chars_indexes[index+1:index+self.sequence_length+1]))\n",
        "        \n",
        "        return  np.array(x), np.array(y)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0y7-IN6m8bV-",
        "outputId": "ffc26208-f78d-4f77-c268-a896bbbfd10b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Length of text: 807276 characters\n",
            "Unique characters (vocabulary size): 85\n",
            "We have 8072 input sequence samples of length 100 and 8072 label sequences of length 100\n"
          ]
        }
      ],
      "source": [
        "# Create the dataset\n",
        "sequence_length = 100\n",
        "dataset_inaugural = InauguralDataset(sequence_length)\n",
        "\n",
        "print(f'We have {dataset_inaugural.x.shape[0]} input sequence samples of length {dataset_inaugural.x.shape[1]} \\\n",
        "and {dataset_inaugural.y.shape[0]} label sequences of length {dataset_inaugural.y.shape[1]}')\n",
        "\n",
        "# Create the Torch DataLoader\n",
        "batch_size = 64\n",
        "train_data_inaugural = TensorDataset(torch.from_numpy(dataset_inaugural.x), torch.from_numpy(dataset_inaugural.y))\n",
        "train_loader_inaugural = DataLoader(train_data_inaugural, shuffle=True, batch_size=batch_size, drop_last=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "69K7j5rlIRpa",
        "outputId": "507a3815-a9d4-4a41-c431-dc3e0d4a1ef5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'epoch': 0, 'batch': 0, 'loss': 4.177688121795654}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.310201644897461}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 1.9924647808074951}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.7654776573181152}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.6761081218719482}\n",
            "{'epoch': 0, 'batch': 150, 'loss': 1.6658620834350586}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.5302478075027466}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.4898676872253418}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.454801082611084}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.4531478881835938}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.4542973041534424}\n",
            "{'epoch': 1, 'batch': 150, 'loss': 1.4565742015838623}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.3400923013687134}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.4107811450958252}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.3439074754714966}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.385619044303894}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.3458285331726074}\n",
            "{'epoch': 2, 'batch': 150, 'loss': 1.3356677293777466}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.3230584859848022}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.2634892463684082}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.318861961364746}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.3321880102157593}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.3232018947601318}\n",
            "{'epoch': 3, 'batch': 150, 'loss': 1.295006513595581}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.2283244132995605}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.2220855951309204}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.2579224109649658}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.2614448070526123}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.2695839405059814}\n",
            "{'epoch': 4, 'batch': 150, 'loss': 1.2977403402328491}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.2252403497695923}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.202052116394043}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.2082034349441528}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.2116135358810425}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.2164431810379028}\n",
            "{'epoch': 5, 'batch': 150, 'loss': 1.2150657176971436}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.1342414617538452}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.1709365844726562}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.1391559839248657}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.1684659719467163}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.1654483079910278}\n",
            "{'epoch': 6, 'batch': 150, 'loss': 1.1640501022338867}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.0940343141555786}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.119788408279419}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.1511486768722534}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.1544042825698853}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.1490755081176758}\n",
            "{'epoch': 7, 'batch': 150, 'loss': 1.1416935920715332}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 0.9988027811050415}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.05087149143219}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 1.09348464012146}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.0672461986541748}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.0659668445587158}\n",
            "{'epoch': 8, 'batch': 150, 'loss': 1.1028884649276733}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 0.9857053160667419}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 1.0313841104507446}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 1.0238596200942993}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 1.0335294008255005}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 1.032943606376648}\n",
            "{'epoch': 9, 'batch': 150, 'loss': 1.070932388305664}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 0.9273853302001953}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 0.9487407803535461}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 0.9653026461601257}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 0.9876506328582764}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 1.0124610662460327}\n",
            "{'epoch': 10, 'batch': 150, 'loss': 1.0046578645706177}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 0.8895604610443115}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 0.8767142295837402}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 0.9118685126304626}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 0.9411937594413757}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 0.951585590839386}\n",
            "{'epoch': 11, 'batch': 150, 'loss': 0.9255717396736145}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 0.8024553656578064}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 0.8758827447891235}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 0.8598729968070984}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 0.874156653881073}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 0.9037149548530579}\n",
            "{'epoch': 12, 'batch': 150, 'loss': 0.9090826511383057}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 0.7541078925132751}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 0.7950983643531799}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 0.8048347234725952}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 0.8105976581573486}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 0.8265865445137024}\n",
            "{'epoch': 13, 'batch': 150, 'loss': 0.8439436554908752}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 0.6907413005828857}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 0.7184227108955383}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 0.7527786493301392}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 0.7536195516586304}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 0.7500025033950806}\n",
            "{'epoch': 14, 'batch': 150, 'loss': 0.80047208070755}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 0.6224033236503601}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 0.664060115814209}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 0.6843932867050171}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 0.7287766933441162}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 0.7267574071884155}\n",
            "{'epoch': 15, 'batch': 150, 'loss': 0.7384681105613708}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 0.5672542452812195}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 0.6172489523887634}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 0.6161666512489319}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 0.6578693985939026}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 0.6801603436470032}\n",
            "{'epoch': 16, 'batch': 150, 'loss': 0.7002460360527039}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 0.5213172435760498}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 0.5811964273452759}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 0.6040872931480408}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 0.6062737107276917}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 0.6272343397140503}\n",
            "{'epoch': 17, 'batch': 150, 'loss': 0.6377124786376953}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 0.47665390372276306}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 0.5197893381118774}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 0.5575060844421387}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 0.5509620904922485}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 0.5695120096206665}\n",
            "{'epoch': 18, 'batch': 150, 'loss': 0.5769457221031189}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 0.4367210865020752}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 0.5010015368461609}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 0.5089395046234131}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 0.5181630849838257}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 0.532672107219696}\n",
            "{'epoch': 19, 'batch': 150, 'loss': 0.5503288507461548}\n",
            "{'epoch': 20, 'batch': 0, 'loss': 0.38970938324928284}\n",
            "{'epoch': 20, 'batch': 30, 'loss': 0.44840770959854126}\n",
            "{'epoch': 20, 'batch': 60, 'loss': 0.4781873822212219}\n",
            "{'epoch': 20, 'batch': 90, 'loss': 0.46978238224983215}\n",
            "{'epoch': 20, 'batch': 120, 'loss': 0.5031276941299438}\n",
            "{'epoch': 20, 'batch': 150, 'loss': 0.5221077799797058}\n",
            "{'epoch': 21, 'batch': 0, 'loss': 0.37101468443870544}\n",
            "{'epoch': 21, 'batch': 30, 'loss': 0.42226773500442505}\n",
            "{'epoch': 21, 'batch': 60, 'loss': 0.4588486850261688}\n",
            "{'epoch': 21, 'batch': 90, 'loss': 0.4533916413784027}\n",
            "{'epoch': 21, 'batch': 120, 'loss': 0.4832419455051422}\n",
            "{'epoch': 21, 'batch': 150, 'loss': 0.5006250143051147}\n",
            "{'epoch': 22, 'batch': 0, 'loss': 0.3290623128414154}\n",
            "{'epoch': 22, 'batch': 30, 'loss': 0.38580456376075745}\n",
            "{'epoch': 22, 'batch': 60, 'loss': 0.4118098318576813}\n",
            "{'epoch': 22, 'batch': 90, 'loss': 0.41928327083587646}\n",
            "{'epoch': 22, 'batch': 120, 'loss': 0.46170148253440857}\n",
            "{'epoch': 22, 'batch': 150, 'loss': 0.4552209973335266}\n",
            "{'epoch': 23, 'batch': 0, 'loss': 0.31855034828186035}\n",
            "{'epoch': 23, 'batch': 30, 'loss': 0.3886123597621918}\n",
            "{'epoch': 23, 'batch': 60, 'loss': 0.40902960300445557}\n",
            "{'epoch': 23, 'batch': 90, 'loss': 0.40291571617126465}\n",
            "{'epoch': 23, 'batch': 120, 'loss': 0.42127174139022827}\n",
            "{'epoch': 23, 'batch': 150, 'loss': 0.4269025921821594}\n",
            "{'epoch': 24, 'batch': 0, 'loss': 0.29098859429359436}\n",
            "{'epoch': 24, 'batch': 30, 'loss': 0.35748887062072754}\n",
            "{'epoch': 24, 'batch': 60, 'loss': 0.3883642554283142}\n",
            "{'epoch': 24, 'batch': 90, 'loss': 0.3860117197036743}\n",
            "{'epoch': 24, 'batch': 120, 'loss': 0.39667049050331116}\n",
            "{'epoch': 24, 'batch': 150, 'loss': 0.4224803149700165}\n",
            "{'epoch': 25, 'batch': 0, 'loss': 0.27866238355636597}\n",
            "{'epoch': 25, 'batch': 30, 'loss': 0.353168785572052}\n",
            "{'epoch': 25, 'batch': 60, 'loss': 0.35056260228157043}\n",
            "{'epoch': 25, 'batch': 90, 'loss': 0.3578132688999176}\n",
            "{'epoch': 25, 'batch': 120, 'loss': 0.40184760093688965}\n",
            "{'epoch': 25, 'batch': 150, 'loss': 0.389377623796463}\n",
            "{'epoch': 26, 'batch': 0, 'loss': 0.2740265727043152}\n",
            "{'epoch': 26, 'batch': 30, 'loss': 0.3355805277824402}\n",
            "{'epoch': 26, 'batch': 60, 'loss': 0.3385190963745117}\n",
            "{'epoch': 26, 'batch': 90, 'loss': 0.3518921136856079}\n",
            "{'epoch': 26, 'batch': 120, 'loss': 0.36252760887145996}\n",
            "{'epoch': 26, 'batch': 150, 'loss': 0.3769083023071289}\n",
            "{'epoch': 27, 'batch': 0, 'loss': 0.2708403468132019}\n",
            "{'epoch': 27, 'batch': 30, 'loss': 0.32143065333366394}\n",
            "{'epoch': 27, 'batch': 60, 'loss': 0.3430485427379608}\n",
            "{'epoch': 27, 'batch': 90, 'loss': 0.35909023880958557}\n",
            "{'epoch': 27, 'batch': 120, 'loss': 0.3549739718437195}\n",
            "{'epoch': 27, 'batch': 150, 'loss': 0.3739680051803589}\n",
            "{'epoch': 28, 'batch': 0, 'loss': 0.2443719059228897}\n",
            "{'epoch': 28, 'batch': 30, 'loss': 0.30693739652633667}\n",
            "{'epoch': 28, 'batch': 60, 'loss': 0.3409916162490845}\n",
            "{'epoch': 28, 'batch': 90, 'loss': 0.3410775363445282}\n",
            "{'epoch': 28, 'batch': 120, 'loss': 0.37035664916038513}\n",
            "{'epoch': 28, 'batch': 150, 'loss': 0.35943716764450073}\n",
            "{'epoch': 29, 'batch': 0, 'loss': 0.24041405320167542}\n",
            "{'epoch': 29, 'batch': 30, 'loss': 0.3189568817615509}\n",
            "{'epoch': 29, 'batch': 60, 'loss': 0.3241327404975891}\n",
            "{'epoch': 29, 'batch': 90, 'loss': 0.3110237121582031}\n",
            "{'epoch': 29, 'batch': 120, 'loss': 0.35606762766838074}\n",
            "{'epoch': 29, 'batch': 150, 'loss': 0.3424234390258789}\n",
            "{'epoch': 0, 'batch': 0, 'loss': 4.439417362213135}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.3110156059265137}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 1.9768236875534058}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.6888766288757324}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.5974705219268799}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.5737489461898804}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.4957047700881958}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.4041591882705688}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.3388621807098389}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.288314700126648}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.2657310962677002}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.2709342241287231}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.2455347776412964}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.252485752105713}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.2404735088348389}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.196058750152588}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.2329949140548706}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.2792425155639648}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.1890469789505005}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.1825428009033203}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.1381617784500122}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.1873090267181396}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.1467258930206299}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.1399798393249512}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.150282859802246}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.0814557075500488}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.0917668342590332}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.0973072052001953}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.0971359014511108}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.1357892751693726}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.1034893989562988}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.1131987571716309}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.0942678451538086}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.1508610248565674}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.0977476835250854}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.0808955430984497}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.0606824159622192}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.0819902420043945}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.0553007125854492}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.0468415021896362}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 0.9971728324890137}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.053145408630371}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 1.0445719957351685}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.0946044921875}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.0632529258728027}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 0.9943196177482605}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 1.0155693292617798}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 0.9972057342529297}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 1.0306192636489868}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 1.0084606409072876}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 0.9725531935691833}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 0.9799688458442688}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 1.009478211402893}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 0.9600655436515808}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 0.9826303720474243}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 0.9356082677841187}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 0.9484930634498596}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 0.9764347076416016}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 0.9829170107841492}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 0.9861103296279907}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 0.9024312496185303}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 0.9349177479743958}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 0.9690760970115662}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 0.9558552503585815}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 0.9580821990966797}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 0.8661354184150696}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 0.9198608994483948}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 0.9262510538101196}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 0.9324275851249695}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 0.9269142150878906}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 0.8513760566711426}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 0.8977979421615601}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 0.8936630487442017}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 0.934389054775238}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 0.9014080166816711}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 0.8281607627868652}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 0.8291680812835693}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 0.8924095034599304}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 0.8282455205917358}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 0.8730523586273193}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 0.8050616383552551}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 0.8273236751556396}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 0.8395265340805054}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 0.866788923740387}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 0.844879150390625}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 0.7674445509910583}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 0.8125975131988525}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 0.8359628319740295}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 0.809914231300354}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 0.8127236366271973}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 0.7350136041641235}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 0.7639482617378235}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 0.789013683795929}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 0.7832973599433899}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 0.7956662178039551}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 0.6709619760513306}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 0.7533866167068481}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 0.7324537038803101}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 0.7478659152984619}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 0.7412253022193909}\n",
            "{'epoch': 20, 'batch': 0, 'loss': 0.6531429290771484}\n",
            "{'epoch': 20, 'batch': 30, 'loss': 0.7026596069335938}\n",
            "{'epoch': 20, 'batch': 60, 'loss': 0.705381453037262}\n",
            "{'epoch': 20, 'batch': 90, 'loss': 0.7234333753585815}\n",
            "{'epoch': 20, 'batch': 120, 'loss': 0.750062882900238}\n",
            "{'epoch': 21, 'batch': 0, 'loss': 0.6071523427963257}\n",
            "{'epoch': 21, 'batch': 30, 'loss': 0.6708714365959167}\n",
            "{'epoch': 21, 'batch': 60, 'loss': 0.6945395469665527}\n",
            "{'epoch': 21, 'batch': 90, 'loss': 0.7098187804222107}\n",
            "{'epoch': 21, 'batch': 120, 'loss': 0.713795006275177}\n",
            "{'epoch': 22, 'batch': 0, 'loss': 0.5782675743103027}\n",
            "{'epoch': 22, 'batch': 30, 'loss': 0.6330506205558777}\n",
            "{'epoch': 22, 'batch': 60, 'loss': 0.6493471264839172}\n",
            "{'epoch': 22, 'batch': 90, 'loss': 0.6711493134498596}\n",
            "{'epoch': 22, 'batch': 120, 'loss': 0.6752078533172607}\n",
            "{'epoch': 23, 'batch': 0, 'loss': 0.5736527442932129}\n",
            "{'epoch': 23, 'batch': 30, 'loss': 0.5865537524223328}\n",
            "{'epoch': 23, 'batch': 60, 'loss': 0.6077756285667419}\n",
            "{'epoch': 23, 'batch': 90, 'loss': 0.6546825170516968}\n",
            "{'epoch': 23, 'batch': 120, 'loss': 0.6288166642189026}\n",
            "{'epoch': 24, 'batch': 0, 'loss': 0.5088222026824951}\n",
            "{'epoch': 24, 'batch': 30, 'loss': 0.5503177642822266}\n",
            "{'epoch': 24, 'batch': 60, 'loss': 0.5831526517868042}\n",
            "{'epoch': 24, 'batch': 90, 'loss': 0.6011425256729126}\n",
            "{'epoch': 24, 'batch': 120, 'loss': 0.6084146499633789}\n",
            "{'epoch': 25, 'batch': 0, 'loss': 0.4830125868320465}\n",
            "{'epoch': 25, 'batch': 30, 'loss': 0.5345537066459656}\n",
            "{'epoch': 25, 'batch': 60, 'loss': 0.5613396167755127}\n",
            "{'epoch': 25, 'batch': 90, 'loss': 0.572844386100769}\n",
            "{'epoch': 25, 'batch': 120, 'loss': 0.557729184627533}\n",
            "{'epoch': 26, 'batch': 0, 'loss': 0.45635029673576355}\n",
            "{'epoch': 26, 'batch': 30, 'loss': 0.4944843351840973}\n",
            "{'epoch': 26, 'batch': 60, 'loss': 0.5385901927947998}\n",
            "{'epoch': 26, 'batch': 90, 'loss': 0.5359697341918945}\n",
            "{'epoch': 26, 'batch': 120, 'loss': 0.5444927215576172}\n",
            "{'epoch': 27, 'batch': 0, 'loss': 0.4080926477909088}\n",
            "{'epoch': 27, 'batch': 30, 'loss': 0.49378764629364014}\n",
            "{'epoch': 27, 'batch': 60, 'loss': 0.49949610233306885}\n",
            "{'epoch': 27, 'batch': 90, 'loss': 0.5141317248344421}\n",
            "{'epoch': 27, 'batch': 120, 'loss': 0.5010579824447632}\n",
            "{'epoch': 28, 'batch': 0, 'loss': 0.39465153217315674}\n",
            "{'epoch': 28, 'batch': 30, 'loss': 0.45985865592956543}\n",
            "{'epoch': 28, 'batch': 60, 'loss': 0.473318487405777}\n",
            "{'epoch': 28, 'batch': 90, 'loss': 0.490490198135376}\n",
            "{'epoch': 28, 'batch': 120, 'loss': 0.49008727073669434}\n",
            "{'epoch': 29, 'batch': 0, 'loss': 0.3675345480442047}\n",
            "{'epoch': 29, 'batch': 30, 'loss': 0.4275418519973755}\n",
            "{'epoch': 29, 'batch': 60, 'loss': 0.4551703631877899}\n",
            "{'epoch': 29, 'batch': 90, 'loss': 0.46426019072532654}\n",
            "{'epoch': 29, 'batch': 120, 'loss': 0.46788036823272705}\n"
          ]
        }
      ],
      "source": [
        "\n",
        "# Train models (one on the Shakespeare dataset and the other on the Inaugural dataset)\n",
        "\n",
        "input_dim = len(dataset.uniq_chars)\n",
        "embedding_dim = 300\n",
        "hidden_dim = 1024\n",
        "output_dim = len(dataset.uniq_chars)\n",
        "n_layers = 2\n",
        "drop_prob = 0.2\n",
        "\n",
        "num_epochs = 30\n",
        "\n",
        "model_shakespeare  = Model(input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob, rnn_cell='LSTM').to(device)\n",
        "losses_shakespeare = train(train_loader, model_shakespeare, batch_size, sequence_length, num_epochs)\n",
        "\n",
        "input_dim = len(dataset_inaugural.uniq_chars)\n",
        "embedding_dim = 300\n",
        "hidden_dim = 1024\n",
        "output_dim = len(dataset_inaugural.uniq_chars)\n",
        "n_layers = 2\n",
        "drop_prob = 0.2\n",
        "\n",
        "num_epochs = 30\n",
        "\n",
        "model_inaugural  = Model(input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob, rnn_cell='LSTM').to(device)\n",
        "losses_inaugural = train(train_loader_inaugural, model_inaugural, batch_size, sequence_length, num_epochs)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VBXMCP1cIstW",
        "outputId": "6d935bc6-0d40-4683-c811-93eebf2475d7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'epoch': 0, 'batch': 0, 'loss': 4.447967529296875}\n",
            "{'epoch': 0, 'batch': 30, 'loss': 2.268130302429199}\n",
            "{'epoch': 0, 'batch': 60, 'loss': 1.9006407260894775}\n",
            "{'epoch': 0, 'batch': 90, 'loss': 1.7229788303375244}\n",
            "{'epoch': 0, 'batch': 120, 'loss': 1.5253018140792847}\n",
            "{'epoch': 1, 'batch': 0, 'loss': 1.5060218572616577}\n",
            "{'epoch': 1, 'batch': 30, 'loss': 1.3984498977661133}\n",
            "{'epoch': 1, 'batch': 60, 'loss': 1.4047789573669434}\n",
            "{'epoch': 1, 'batch': 90, 'loss': 1.3213326930999756}\n",
            "{'epoch': 1, 'batch': 120, 'loss': 1.319733738899231}\n",
            "{'epoch': 2, 'batch': 0, 'loss': 1.3036012649536133}\n",
            "{'epoch': 2, 'batch': 30, 'loss': 1.2740365266799927}\n",
            "{'epoch': 2, 'batch': 60, 'loss': 1.229321002960205}\n",
            "{'epoch': 2, 'batch': 90, 'loss': 1.242007851600647}\n",
            "{'epoch': 2, 'batch': 120, 'loss': 1.2450318336486816}\n",
            "{'epoch': 3, 'batch': 0, 'loss': 1.1739486455917358}\n",
            "{'epoch': 3, 'batch': 30, 'loss': 1.2102473974227905}\n",
            "{'epoch': 3, 'batch': 60, 'loss': 1.1828495264053345}\n",
            "{'epoch': 3, 'batch': 90, 'loss': 1.178622841835022}\n",
            "{'epoch': 3, 'batch': 120, 'loss': 1.140101671218872}\n",
            "{'epoch': 4, 'batch': 0, 'loss': 1.1277151107788086}\n",
            "{'epoch': 4, 'batch': 30, 'loss': 1.1312943696975708}\n",
            "{'epoch': 4, 'batch': 60, 'loss': 1.1317483186721802}\n",
            "{'epoch': 4, 'batch': 90, 'loss': 1.0830048322677612}\n",
            "{'epoch': 4, 'batch': 120, 'loss': 1.1048489809036255}\n",
            "{'epoch': 5, 'batch': 0, 'loss': 1.1321629285812378}\n",
            "{'epoch': 5, 'batch': 30, 'loss': 1.081482172012329}\n",
            "{'epoch': 5, 'batch': 60, 'loss': 1.1008034944534302}\n",
            "{'epoch': 5, 'batch': 90, 'loss': 1.1079285144805908}\n",
            "{'epoch': 5, 'batch': 120, 'loss': 1.121607780456543}\n",
            "{'epoch': 6, 'batch': 0, 'loss': 1.0235981941223145}\n",
            "{'epoch': 6, 'batch': 30, 'loss': 1.0872267484664917}\n",
            "{'epoch': 6, 'batch': 60, 'loss': 1.0801142454147339}\n",
            "{'epoch': 6, 'batch': 90, 'loss': 1.0917021036148071}\n",
            "{'epoch': 6, 'batch': 120, 'loss': 1.0551881790161133}\n",
            "{'epoch': 7, 'batch': 0, 'loss': 1.0029200315475464}\n",
            "{'epoch': 7, 'batch': 30, 'loss': 1.059584617614746}\n",
            "{'epoch': 7, 'batch': 60, 'loss': 1.0802761316299438}\n",
            "{'epoch': 7, 'batch': 90, 'loss': 1.0343999862670898}\n",
            "{'epoch': 7, 'batch': 120, 'loss': 1.0991148948669434}\n",
            "{'epoch': 8, 'batch': 0, 'loss': 1.0089842081069946}\n",
            "{'epoch': 8, 'batch': 30, 'loss': 1.0250115394592285}\n",
            "{'epoch': 8, 'batch': 60, 'loss': 0.9974778890609741}\n",
            "{'epoch': 8, 'batch': 90, 'loss': 1.0245535373687744}\n",
            "{'epoch': 8, 'batch': 120, 'loss': 1.0117850303649902}\n",
            "{'epoch': 9, 'batch': 0, 'loss': 0.9533972144126892}\n",
            "{'epoch': 9, 'batch': 30, 'loss': 0.9915319085121155}\n",
            "{'epoch': 9, 'batch': 60, 'loss': 1.0000946521759033}\n",
            "{'epoch': 9, 'batch': 90, 'loss': 0.9958820343017578}\n",
            "{'epoch': 9, 'batch': 120, 'loss': 0.9870785474777222}\n",
            "{'epoch': 10, 'batch': 0, 'loss': 0.946840763092041}\n",
            "{'epoch': 10, 'batch': 30, 'loss': 0.9713196754455566}\n",
            "{'epoch': 10, 'batch': 60, 'loss': 0.9734042882919312}\n",
            "{'epoch': 10, 'batch': 90, 'loss': 0.9616274833679199}\n",
            "{'epoch': 10, 'batch': 120, 'loss': 1.0087389945983887}\n",
            "{'epoch': 11, 'batch': 0, 'loss': 0.8858215808868408}\n",
            "{'epoch': 11, 'batch': 30, 'loss': 0.9508054256439209}\n",
            "{'epoch': 11, 'batch': 60, 'loss': 0.9432718753814697}\n",
            "{'epoch': 11, 'batch': 90, 'loss': 0.9423542618751526}\n",
            "{'epoch': 11, 'batch': 120, 'loss': 0.9583218097686768}\n",
            "{'epoch': 12, 'batch': 0, 'loss': 0.8474208116531372}\n",
            "{'epoch': 12, 'batch': 30, 'loss': 0.9007896184921265}\n",
            "{'epoch': 12, 'batch': 60, 'loss': 0.9366130232810974}\n",
            "{'epoch': 12, 'batch': 90, 'loss': 0.9292249083518982}\n",
            "{'epoch': 12, 'batch': 120, 'loss': 0.9413449764251709}\n",
            "{'epoch': 13, 'batch': 0, 'loss': 0.8408361673355103}\n",
            "{'epoch': 13, 'batch': 30, 'loss': 0.8839274048805237}\n",
            "{'epoch': 13, 'batch': 60, 'loss': 0.8607394695281982}\n",
            "{'epoch': 13, 'batch': 90, 'loss': 0.9204393029212952}\n",
            "{'epoch': 13, 'batch': 120, 'loss': 0.8985666632652283}\n",
            "{'epoch': 14, 'batch': 0, 'loss': 0.7938174605369568}\n",
            "{'epoch': 14, 'batch': 30, 'loss': 0.8564662933349609}\n",
            "{'epoch': 14, 'batch': 60, 'loss': 0.8500336408615112}\n",
            "{'epoch': 14, 'batch': 90, 'loss': 0.8431919813156128}\n",
            "{'epoch': 14, 'batch': 120, 'loss': 0.8766882419586182}\n",
            "{'epoch': 15, 'batch': 0, 'loss': 0.7780991196632385}\n",
            "{'epoch': 15, 'batch': 30, 'loss': 0.7831891775131226}\n",
            "{'epoch': 15, 'batch': 60, 'loss': 0.826407790184021}\n",
            "{'epoch': 15, 'batch': 90, 'loss': 0.8278552293777466}\n",
            "{'epoch': 15, 'batch': 120, 'loss': 0.8673373460769653}\n",
            "{'epoch': 16, 'batch': 0, 'loss': 0.7408559918403625}\n",
            "{'epoch': 16, 'batch': 30, 'loss': 0.7868033647537231}\n",
            "{'epoch': 16, 'batch': 60, 'loss': 0.7664678692817688}\n",
            "{'epoch': 16, 'batch': 90, 'loss': 0.8008298277854919}\n",
            "{'epoch': 16, 'batch': 120, 'loss': 0.8143141865730286}\n",
            "{'epoch': 17, 'batch': 0, 'loss': 0.684284508228302}\n",
            "{'epoch': 17, 'batch': 30, 'loss': 0.715121328830719}\n",
            "{'epoch': 17, 'batch': 60, 'loss': 0.75391685962677}\n",
            "{'epoch': 17, 'batch': 90, 'loss': 0.779975414276123}\n",
            "{'epoch': 17, 'batch': 120, 'loss': 0.7780376672744751}\n",
            "{'epoch': 18, 'batch': 0, 'loss': 0.6503979563713074}\n",
            "{'epoch': 18, 'batch': 30, 'loss': 0.7073159217834473}\n",
            "{'epoch': 18, 'batch': 60, 'loss': 0.7196109294891357}\n",
            "{'epoch': 18, 'batch': 90, 'loss': 0.7453697919845581}\n",
            "{'epoch': 18, 'batch': 120, 'loss': 0.7480409741401672}\n",
            "{'epoch': 19, 'batch': 0, 'loss': 0.617267370223999}\n",
            "{'epoch': 19, 'batch': 30, 'loss': 0.6668949723243713}\n",
            "{'epoch': 19, 'batch': 60, 'loss': 0.7066113352775574}\n",
            "{'epoch': 19, 'batch': 90, 'loss': 0.6864144206047058}\n",
            "{'epoch': 19, 'batch': 120, 'loss': 0.708625316619873}\n",
            "{'epoch': 20, 'batch': 0, 'loss': 0.5726377367973328}\n",
            "{'epoch': 20, 'batch': 30, 'loss': 0.6395912766456604}\n",
            "{'epoch': 20, 'batch': 60, 'loss': 0.6670694947242737}\n",
            "{'epoch': 20, 'batch': 90, 'loss': 0.652586042881012}\n",
            "{'epoch': 20, 'batch': 120, 'loss': 0.6793392300605774}\n",
            "{'epoch': 21, 'batch': 0, 'loss': 0.5465243458747864}\n",
            "{'epoch': 21, 'batch': 30, 'loss': 0.5999834537506104}\n",
            "{'epoch': 21, 'batch': 60, 'loss': 0.597058892250061}\n",
            "{'epoch': 21, 'batch': 90, 'loss': 0.6107662320137024}\n",
            "{'epoch': 21, 'batch': 120, 'loss': 0.6502680778503418}\n",
            "{'epoch': 22, 'batch': 0, 'loss': 0.5047996640205383}\n",
            "{'epoch': 22, 'batch': 30, 'loss': 0.5487099289894104}\n",
            "{'epoch': 22, 'batch': 60, 'loss': 0.5647733211517334}\n",
            "{'epoch': 22, 'batch': 90, 'loss': 0.587833821773529}\n",
            "{'epoch': 22, 'batch': 120, 'loss': 0.6061327457427979}\n",
            "{'epoch': 23, 'batch': 0, 'loss': 0.46648597717285156}\n",
            "{'epoch': 23, 'batch': 30, 'loss': 0.531888484954834}\n",
            "{'epoch': 23, 'batch': 60, 'loss': 0.5479409694671631}\n",
            "{'epoch': 23, 'batch': 90, 'loss': 0.5619175434112549}\n",
            "{'epoch': 23, 'batch': 120, 'loss': 0.5625889301300049}\n",
            "{'epoch': 24, 'batch': 0, 'loss': 0.4261847734451294}\n",
            "{'epoch': 24, 'batch': 30, 'loss': 0.4945237636566162}\n",
            "{'epoch': 24, 'batch': 60, 'loss': 0.5039356350898743}\n",
            "{'epoch': 24, 'batch': 90, 'loss': 0.5136317610740662}\n",
            "{'epoch': 24, 'batch': 120, 'loss': 0.5374988317489624}\n",
            "{'epoch': 25, 'batch': 0, 'loss': 0.40003734827041626}\n",
            "{'epoch': 25, 'batch': 30, 'loss': 0.44887787103652954}\n",
            "{'epoch': 25, 'batch': 60, 'loss': 0.46674346923828125}\n",
            "{'epoch': 25, 'batch': 90, 'loss': 0.48810186982154846}\n",
            "{'epoch': 25, 'batch': 120, 'loss': 0.4944984018802643}\n",
            "{'epoch': 26, 'batch': 0, 'loss': 0.3593500852584839}\n",
            "{'epoch': 26, 'batch': 30, 'loss': 0.43797045946121216}\n",
            "{'epoch': 26, 'batch': 60, 'loss': 0.4501588046550751}\n",
            "{'epoch': 26, 'batch': 90, 'loss': 0.465201199054718}\n",
            "{'epoch': 26, 'batch': 120, 'loss': 0.48440665006637573}\n",
            "{'epoch': 27, 'batch': 0, 'loss': 0.3530940115451813}\n",
            "{'epoch': 27, 'batch': 30, 'loss': 0.4057917892932892}\n",
            "{'epoch': 27, 'batch': 60, 'loss': 0.4167594611644745}\n",
            "{'epoch': 27, 'batch': 90, 'loss': 0.43533188104629517}\n",
            "{'epoch': 27, 'batch': 120, 'loss': 0.4436938762664795}\n",
            "{'epoch': 28, 'batch': 0, 'loss': 0.32222139835357666}\n",
            "{'epoch': 28, 'batch': 30, 'loss': 0.3998240530490875}\n",
            "{'epoch': 28, 'batch': 60, 'loss': 0.39253395795822144}\n",
            "{'epoch': 28, 'batch': 90, 'loss': 0.40799450874328613}\n",
            "{'epoch': 28, 'batch': 120, 'loss': 0.42419445514678955}\n",
            "{'epoch': 29, 'batch': 0, 'loss': 0.3005695044994354}\n",
            "{'epoch': 29, 'batch': 30, 'loss': 0.344443678855896}\n",
            "{'epoch': 29, 'batch': 60, 'loss': 0.37762728333473206}\n",
            "{'epoch': 29, 'batch': 90, 'loss': 0.39891815185546875}\n",
            "{'epoch': 29, 'batch': 120, 'loss': 0.40222764015197754}\n"
          ]
        }
      ],
      "source": [
        "model_inaugural  = Model(input_dim, embedding_dim, hidden_dim, output_dim, n_layers, drop_prob, rnn_cell='LSTM').to(device)\n",
        "losses_inaugural = train(train_loader_inaugural, model_inaugural, batch_size, sequence_length, num_epochs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dHObSIffVqQT",
        "outputId": "2da150d6-93fc-4cc1-89e4-172dd05d2005"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " ---------------------------------------- \n",
            " ------------ Shakespeare --------------- \n",
            " ---------------------------------------- \n",
            "What cannot be a\n",
            "pardon him on thy bears against it not?\n",
            "\n",
            "Servant and made should have a thousand certain great\n",
            "As show noble Margaret, then that keep haven,\n",
            "I would intent?\n",
            "\n",
            "ALONSO:\n",
            "What saye command\n",
            "That is displeasure, great errand.\n",
            "\n",
            "CORIOLANUS:\n",
            "Nay, come what an assemnow, no day,\n",
            "And will in merit not\n",
            "The bawded down.\n",
            "And that do not.\n",
            "\n",
            "LUCIO:\n",
            "U must ower to a true of Angelo?.\n",
            "My like and sain,\n",
            "Because I bear\n",
            "His nobleness of these socrace?\n",
            "\n",
            "BISHOP OF OLY:\n",
            "Mustly harms,\n",
            "Which he cease, that, when she's dead-just our king and romour.\n",
            "Who come not, I will be put not, as free\n",
            "To thinking musty guest?\n",
            "\n",
            "ISABELLA:\n",
            "I am there;\n",
            "My doubt, this is so soundly.\n",
            "\n",
            "MARIANA:\n",
            "A lin our free down:\n",
            "Like call'd\n",
            "To spe:\n",
            "Thou never blood of ten. Think'st and tie out.\n",
            "\n",
            "DUKE VINCENTIO:\n",
            "It is ruled by one afform\n",
            "I will be less\n",
            "I poys the leadness be pluck\n",
            "remember\n",
            "UMINT:\n",
            "This is the favours there; I get my castle.\n",
            "\n",
            "Mowhem with love.\n",
            "\n",
            "QUEEN:\n",
            "Richmond.\n",
            "\n",
            "SAMPSON:\n",
            "The midst with my purpose,\n",
            "Betwixt you, to-morrows that\n",
            " ---------------------------------------- \n",
            " ------------  Inaugural  --------------- \n",
            " ---------------------------------------- \n",
            "What can be repeals and continued such a modern work and crises, the civil policy under eminently invoked save to themselves or by Federalize that of us now we continuance, an experiment or expenditures found a proposition from invested was our tariffs to be unknown being in having each tindependence to increasing you, and story of faith, servant, errors have been thus fell where they emphatic governments, and e obvious precognations which we enjoy the residentioundly to the destinies of the Nationality had transcends devotion of the progress. The simple the solution, to effective or by returned these United States has comes you gon. Then we have cause?\n",
            "\n",
            "The efforts today the more auspicion of supreme its own affairs. The coordinate desire forth on th rule all those who will continued in behalf of our compassion in the two centuries, whether foundations, fast on the final days. And I will defect, \"repentating with ordest predill undertaking sphere, when American industrially carrier and more.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# predict some text with the trained models\n",
        "\n",
        "def predict(dataset, model, text, next_chars=1000):\n",
        "    model.eval()\n",
        "\n",
        "    chars = [char for char in text]\n",
        "    h, c = model.init_hidden(1)\n",
        "\n",
        "    for i in range(0, next_chars):\n",
        "        x = torch.tensor([[dataset.char_to_index[char] for char in chars[i:]]])\n",
        "        x = x.to(device)\n",
        "        y_pred, h, c = model(x, h, c)\n",
        "\n",
        "        last_char_logits = y_pred[0][-1]\n",
        "        p = torch.nn.functional.softmax(last_char_logits, dim=0).detach().cpu().numpy()\n",
        "        char_index = np.random.choice(len(last_char_logits), p=p)\n",
        "        chars.append(dataset.index_to_char[char_index])\n",
        "\n",
        "    return chars\n",
        "\n",
        "\n",
        "print (' ---------------------------------------- ')\n",
        "print (' ------------ Shakespeare --------------- ')\n",
        "print (' ---------------------------------------- ')\n",
        "\n",
        "predicted = predict(dataset, model, 'What can')\n",
        "print(''.join(predicted))\n",
        "\n",
        "print (' ---------------------------------------- ')\n",
        "print (' ------------  Inaugural  --------------- ')\n",
        "print (' ---------------------------------------- ')\n",
        "\n",
        "predicted = predict(dataset_inaugural, model_inaugural, 'What can')\n",
        "print(''.join(predicted))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QaSd2_s6AXm4"
      },
      "source": [
        "We appreciate that the two models have learned the different language properties of the training data. Not only the vocabulary (the Sakespeare model tends to generate old English words), but also the topics (drama vs. politics), and the stucture of the text (play vs. paragraph).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "6lMK7S5oBQ8n"
      },
      "outputs": [],
      "source": [
        ""
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "W13_02_RNN_Text_Generation_Solutions.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}