Reuters.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Reuters.ipynb",
      "version": "0.3.2",
      "provenance": []
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "x4uE7qFSUVM2",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# The Reuters dataset\n",
        "**a multiclass classification example**\n",
        "\n",
        "You’ll work with the Reuters dataset, a set of short newswires and their topics, published\n",
        "by Reuters in 1986. It’s a simple, widely used toy dataset for text classification. There\n",
        "are 46 different topics; some topics are more represented than others, but each topic\n",
        "has at least 10 examples in the training set.\n",
        "\n",
        "Like IMDB and MNIST, the Reuters dataset comes packaged as part of Keras. Let’s\n",
        "take a look."
      ]
    },
    {
      "metadata": {
        "id": "GR9DaDbVU8pi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.12 Loading the Reuters dataset"
      ]
    },
    {
      "metadata": {
        "id": "EvL5grtDTmKW",
        "colab_type": "code",
        "outputId": "4e9b5b9a-5a98-4cd8-896e-426545306ea1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "from keras.datasets import reuters\n",
        "\n",
        "(train_data,train_labels),(test_data,test_labels)=reuters.load_data(num_words=10000)#making the dataset of 10,000 most frequently occurring words"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://s3.amazonaws.com/text-datasets/reuters.npz\n",
            "2113536/2110848 [==============================] - 0s 0us/step\n",
            "2121728/2110848 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "16tIe0ObVo4S",
        "colab_type": "code",
        "outputId": "bae21211-87e1-4b25-ccff-5357632fe451",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "len(train_data)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8982"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "metadata": {
        "id": "ZGQCRzJNVvZj",
        "colab_type": "code",
        "outputId": "6124034d-2383-4d64-e133-14995f9a005f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "len(test_data)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2246"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "metadata": {
        "id": "q3YJSfnDVxk6",
        "colab_type": "code",
        "outputId": "e2c9f05e-67d0-4b6c-f73a-77f4ad5d03a8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "cell_type": "code",
      "source": [
        "print(train_data[0])# first review"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1, 2, 2, 8, 43, 10, 447, 5, 25, 207, 270, 5, 3095, 111, 16, 369, 186, 90, 67, 7, 89, 5, 19, 102, 6, 19, 124, 15, 90, 67, 84, 22, 482, 26, 7, 48, 4, 49, 8, 864, 39, 209, 154, 6, 151, 6, 83, 11, 15, 22, 155, 11, 15, 7, 48, 9, 4579, 1005, 504, 6, 258, 6, 272, 11, 15, 22, 134, 44, 11, 15, 16, 8, 197, 1245, 90, 67, 52, 29, 209, 30, 32, 132, 6, 109, 15, 17, 12]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "70B6t09MYP-e",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.13Decoding newswires back to text"
      ]
    },
    {
      "metadata": {
        "id": "J3oWp06qYJs1",
        "colab_type": "code",
        "outputId": "4043129b-ddea-4769-8a3f-29ff56e0e0e6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "word_index = reuters.get_word_index()\n",
        "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
        "decoded_newswire = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])# decoding the first row back to text"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://s3.amazonaws.com/text-datasets/reuters_word_index.json\n",
            "557056/550378 [==============================] - 0s 0us/step\n",
            "565248/550378 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "CqJSOmYmYbYF",
        "colab_type": "code",
        "outputId": "31b0bce0-3903-45b9-dbcc-bf7b38711cde",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "train_labels[10]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "metadata": {
        "id": "VSVIs44QZuEs",
        "colab_type": "code",
        "outputId": "0c4adb5e-055c-401e-b57a-49f093b071b7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "cell_type": "code",
      "source": [
        "decoded_newswire"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "u'? ? ? said as a result of its december acquisition of space co it expects earnings per share in 1987 of 1 15 to 1 30 dlrs per share up from 70 cts in 1986 the company said pretax net should rise to nine to 10 mln dlrs from six mln dlrs in 1986 and rental operation revenues to 19 to 22 mln dlrs from 12 5 mln dlrs it said cash flow per share this year should be 2 50 to three dlrs reuter 3'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "metadata": {
        "id": "dLwPRYnYYkdz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##3.2  Preparing the data"
      ]
    },
    {
      "metadata": {
        "id": "cxNk4t_KYoFz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.2.1 Encoding the data"
      ]
    },
    {
      "metadata": {
        "id": "SKRhnAOLYe6G",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "0-FMP-tPYvbF",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def vectorize_sequences(sequences, dimension=10000):\n",
        "  results = np.zeros((len(sequences), dimension))\n",
        "  for i, sequence in enumerate(sequences):\n",
        "    results[i, sequence] = 1.\n",
        "  return results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "rI2jtTsAYxQ1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "x_train = vectorize_sequences(train_data)#Vectorizer train data\n",
        "x_test = vectorize_sequences(test_data)#vectorizer test data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "5IdUoAyCcHR_",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from keras.utils.np_utils import to_categorical"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "EOU68eDdcL4V",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "## Vectorize train and test labels using built-in keras function\n",
        "one_hot_train_labels = to_categorical(train_labels)\n",
        "one_hot_test_labels = to_categorical(test_labels)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "fjp1LiOeaJLZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.2.3 Building your network"
      ]
    },
    {
      "metadata": {
        "id": "TMlIGdW5Y-YE",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from keras import layers,models"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "eYzqccqOaOBC",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "\n",
        "# Hidden layer 1\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
        "\n",
        "# hidden layer 2\n",
        "model.add(layers.Dense(64, activation='relu'))\n",
        "\n",
        "# ooutput layer\n",
        "model.add(layers.Dense(46, activation='softmax'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "LdLTXWR1a8rA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "loss='categorical_crossentropy',\n",
        "metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "XY8-T1VtbQ3_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 3.3 Validating your approach\n",
        "\n",
        "Let’s set apart 1,000 samples in the training data to use as a validation set"
      ]
    },
    {
      "metadata": {
        "id": "keO4x9k3bSbG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###3.3.1  Setting aside a validation set"
      ]
    },
    {
      "metadata": {
        "id": "kjbJS4scbLHZ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "x_val = x_train[:1000]\n",
        "partial_x_train = x_train[1000:]\n",
        "\n",
        "y_val = one_hot_train_labels[:1000]\n",
        "partial_y_train = one_hot_train_labels[1000:]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ivLVyCCoci7E",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.3.2 Training the model"
      ]
    },
    {
      "metadata": {
        "id": "aQjhp052bicA",
        "colab_type": "code",
        "outputId": "bd05eeb6-9006-456f-d4e0-3d3fcd1b1fdc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 734
        }
      },
      "cell_type": "code",
      "source": [
        "history = model.fit(partial_x_train,partial_y_train,epochs=20,batch_size=512,validation_data=(x_val, y_val))#traing for 20 epochs"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/20\n",
            "7982/7982 [==============================] - 2s 203us/step - loss: 2.4969 - acc: 0.4904 - val_loss: 1.6813 - val_acc: 0.6510\n",
            "Epoch 2/20\n",
            "7982/7982 [==============================] - 1s 150us/step - loss: 1.3915 - acc: 0.7038 - val_loss: 1.2790 - val_acc: 0.7180\n",
            "Epoch 3/20\n",
            "7982/7982 [==============================] - 1s 147us/step - loss: 1.0476 - acc: 0.7707 - val_loss: 1.1178 - val_acc: 0.7610\n",
            "Epoch 4/20\n",
            "7982/7982 [==============================] - 1s 151us/step - loss: 0.8235 - acc: 0.8282 - val_loss: 1.0226 - val_acc: 0.7780\n",
            "Epoch 5/20\n",
            "7982/7982 [==============================] - 1s 148us/step - loss: 0.6591 - acc: 0.8637 - val_loss: 0.9719 - val_acc: 0.7950\n",
            "Epoch 6/20\n",
            "7982/7982 [==============================] - 1s 150us/step - loss: 0.5251 - acc: 0.8929 - val_loss: 0.9201 - val_acc: 0.8120\n",
            "Epoch 7/20\n",
            "7982/7982 [==============================] - 1s 146us/step - loss: 0.4295 - acc: 0.9108 - val_loss: 0.9136 - val_acc: 0.8010\n",
            "Epoch 8/20\n",
            "7982/7982 [==============================] - 1s 148us/step - loss: 0.3508 - acc: 0.9271 - val_loss: 0.8950 - val_acc: 0.8160\n",
            "Epoch 9/20\n",
            "5632/7982 [====================>.........] - ETA: 0s - loss: 0.2900 - acc: 0.9377"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 1s 146us/step - loss: 0.2908 - acc: 0.9384 - val_loss: 0.9138 - val_acc: 0.8080\n",
            "Epoch 10/20\n",
            "7982/7982 [==============================] - 1s 148us/step - loss: 0.2463 - acc: 0.9448 - val_loss: 0.9143 - val_acc: 0.8130\n",
            "Epoch 11/20\n",
            "7982/7982 [==============================] - 1s 149us/step - loss: 0.2121 - acc: 0.9485 - val_loss: 0.9514 - val_acc: 0.8100\n",
            "Epoch 12/20\n",
            "7982/7982 [==============================] - 1s 148us/step - loss: 0.1892 - acc: 0.9526 - val_loss: 0.9604 - val_acc: 0.8060\n",
            "Epoch 13/20\n",
            "7982/7982 [==============================] - 1s 151us/step - loss: 0.1668 - acc: 0.9525 - val_loss: 0.9977 - val_acc: 0.7950\n",
            "Epoch 14/20\n",
            "7982/7982 [==============================] - 1s 149us/step - loss: 0.1546 - acc: 0.9541 - val_loss: 0.9801 - val_acc: 0.8050\n",
            "Epoch 15/20\n",
            "7982/7982 [==============================] - 1s 150us/step - loss: 0.1469 - acc: 0.9546 - val_loss: 1.0238 - val_acc: 0.7980\n",
            "Epoch 16/20\n",
            "7982/7982 [==============================] - 1s 146us/step - loss: 0.1344 - acc: 0.9554 - val_loss: 1.0456 - val_acc: 0.7970\n",
            "Epoch 17/20\n",
            "7982/7982 [==============================] - 1s 149us/step - loss: 0.1272 - acc: 0.9553 - val_loss: 1.0493 - val_acc: 0.7960\n",
            "Epoch 18/20\n",
            " 512/7982 [>.............................] - ETA: 1s - loss: 0.1368 - acc: 0.9648"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 1s 151us/step - loss: 0.1181 - acc: 0.9563 - val_loss: 1.0456 - val_acc: 0.8140\n",
            "Epoch 19/20\n",
            "7982/7982 [==============================] - 1s 149us/step - loss: 0.1176 - acc: 0.9569 - val_loss: 1.0332 - val_acc: 0.8070\n",
            "Epoch 20/20\n",
            "7982/7982 [==============================] - 1s 149us/step - loss: 0.1153 - acc: 0.9575 - val_loss: 1.0541 - val_acc: 0.8030\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "h2mvPL3lcupT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.3.3 Plotting the training and validation loss"
      ]
    },
    {
      "metadata": {
        "id": "b20TnfqZcrKm",
        "colab_type": "code",
        "outputId": "b7227ae3-5763-4f63-90dc-2b83dae0d770",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 376
        }
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "epochs = range(1, len(loss) + 1)\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe8AAAFnCAYAAACPasF4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl4U1XCx/FvlqZQWpZCC4W6IFKk\nICiCgksLlaWgiB0XUHHFFwZBQDsq+IrggAgICCojDIujoMiIoOMIdESWUUD2QVYRfVV2WtpCN2iz\nvH9kGiikpdCm6U1/n+fp0+Te3Jtzkja/nHvPPcfkcrlciIiIiGGY/V0AERERuTQKbxEREYNReIuI\niBiMwltERMRgFN4iIiIGo/AWERExGIW3VGmjRo0iMTGRxMREWrRoQadOnTz3s7OzL2lfiYmJpKWl\nlfiYyZMns2DBgrIUudw98cQTLF68uFz21axZM44ePcrXX3/NiBEjyvR8f//73z23S/Paltbw4cP5\ny1/+Ui77EvEXq78LIOJPr732mud2QkICEydOpG3btpe1r+XLl1/0McnJyZe1b6Pp0qULXbp0uezt\nU1NTmT17Ng8++CBQutdWpCpRy1ukBI8++ihvvfUW3bt3Z+vWraSlpdGvXz8SExNJSEjg/fff9zy2\nsNW5YcMGevfuzeTJk+nevTsJCQls3LgRKNrqS0hI4JNPPuH+++/n9ttvZ/z48Z59zZgxgw4dOnDf\nfffx0UcfkZCQ4LV8n376Kd27d6dr16488sgjHDp0CIDFixczZMgQXn75Zbp160aPHj346aefADhw\n4AAPPPAAnTt3Jjk5GYfDccF+16xZQ8+ePYss69WrF//+979LfA0KLV68mCeeeOKiz/fNN9/Qs2dP\nunXrxh/+8Af27NkDQJ8+fTh8+DCJiYnk5+d7XluADz/8kB49epCYmMjAgQNJT0/3vLZvv/02Tz75\nJJ06deLJJ58kLy+vuLcWgL1799KnTx8SExPp1asX3377LQA5OTkMGjSI7t27c+edd/LKK69QUFBQ\n7HKRiqbwFrmInTt38tVXX9GmTRvee+89oqOjWb58OR988AGTJ0/myJEjF2yze/duWrduzbJly3j4\n4Yd57733vO5706ZNLFy4kM8++4z58+dz9OhRfvrpJ2bPns0XX3zBxx9/XGyr88SJE/z5z3/m/fff\n51//+hdXXnllkcPB//73v3n44YdJSUnhlltu4YMPPgBg0qRJdOjQgRUrVvD444+zdevWC/bdoUMH\njh49yoEDBwB3AB89epRbb7211K9BoeKez263M3z4cMaMGUNKSgoJCQlMmDABgHHjxhEVFcXy5cux\n2Wyeff3nP/9hzpw5zJs3j+XLl9OwYUMmT57sWb98+XLeeustvv76a9LT0/n666+LLZfT6eT555+n\nb9++LF++nLFjx5KcnEx2djaff/45NWvWZNmyZaSkpGCxWNi/f3+xy0UqmsJb5CLi4+Mxm93/Kq+8\n8gojR44E4IorriAiIoKDBw9esE2NGjXo3LkzAC1atODw4cNe992zZ08sFgv169enbt26HDlyhE2b\nNnHzzTcTGRlJcHAw9913n9dt69aty5YtW2jQoAEAbdu29YQtQJMmTWjZsiUAsbGxnoDdvHkzPXr0\nAKBVq1Zcc801F+zbZrPRqVMnVq5cCcCKFSvo3LkzVqu11K9BoeKez2q1sm7dOm644Qav5fdm9erV\ndOvWjbp16wLwwAMPsHbtWs/6+Ph4ateujdVqJSYmpsQvFQcPHiQtLY277roLgOuvv56GDRuyY8cO\nwsPD2bZtG9999x1Op5PXXnuN5s2bF7tcpKLpnLfIRdSqVctze8eOHZ6WptlsJjU1FafTecE2YWFh\nnttms9nrYwBCQ0M9ty0WCw6Hg1OnThV5zvr163vd1uFw8Pbbb7Ny5UocDgc5OTk0btzYaxkK9w1w\n8uTJIs9bs2ZNr/vv1q0bH374IY8//jgrVqzgmWeeuaTXoFBJzzdv3jyWLFlCfn4++fn5mEymYvcD\nkJ6eTmRkZJF9nThx4qJ1Lm5fYWFhRZ6zZs2apKenc9ddd3Hy5EmmTZvGL7/8wj333MOIESPo3r27\n1+XnHh0QqQhqeYtcghdeeIFu3bqRkpLC8uXLqVOnTrk/R2hoKLm5uZ77x48f9/q4pUuXsnLlSubP\nn09KSgpDhgwp1f5r1qxZpCd94Tnj891xxx3s3buXX3/9lV9//ZX27dsDl/4aFPd8W7duZdasWbz3\n3nukpKQwduzYi5a9Xr16ZGZmeu5nZmZSr169i27nTd26dTl58iTnzs2UmZnpadX36dOHTz/9lKVL\nl7Jr1y4+//zzEpeLVCSFt8glOHHiBC1btsRkMrFkyRLy8vKKBG15aNWqFRs2bCA9PZ38/Pxiw+HE\niRM0atSI8PBwMjIyWLZsGTk5ORfd/w033OA5F7x161Z+//13r4+z2WzcfvvtvPnmm9x5551YLBbP\n817Ka1Dc86Wnp1O3bl0aNmxIXl4eS5YsITc3F5fLhdVqJTc3F7vdXmRfHTt25OuvvyYjIwOATz75\nhPj4+IvW2Zvo6GgaNGjA0qVLPWVLS0ujVatWTJ8+nUWLFgHuIx/R0dGYTKZil4tUNIW3yCUYOnQo\ngwYNomfPnuTm5tK7d29GjhxZbABejlatWpGUlERSUhKPPfYYnTp18vq4u+++m8zMTLp06UJycjLD\nhg3j6NGjRXqte/PCCy+watUqOnfuzEcffcStt95a7GO7devGihUr6N69u2fZpb4GxT3fHXfcQWRk\nJJ07d+app57i8ccfJywsjCFDhtCsWTNq1arFbbfdVqS/QKtWrejfvz+PPPIIiYmJZGVl8dxzz5VY\n3+KYTCamTJnC/Pnz6d69O2PHjmXatGmEhITQq1cvvvjiC7p160ZiYiJBQUH06tWr2OUiFc2k+bxF\nKh+Xy+Vp0a1evZqpU6fq8KyIeKjlLVLJpKen0759ew4dOoTL5WLZsmWeHtkiIqCWt0iltGDBAubO\nnYvJZOKaa67h9ddf93SkEhFReIuIiBiMDpuLiIgYjMJbRETEYAwzwlpqapa/i1Du6tQJISOjfK8R\n9jfVyTgCsV6BWCcIzHqpTqUTERHmdbla3n5ktVr8XYRypzoZRyDWKxDrBIFZL9WpbBTeIiIiBqPw\nFhERMRiFt4iIiMEovEVERAxG4S0iImIwCm8RERGDUXiLiIgYjE8HaZk4cSJbtmzBbrczYMAAunbt\n6lmXkJBAgwYNsFjc18VNmjSJ+vXr+7I4IiJSSYwfP55t27aTnn6C06dP07BhI2rWrMW4cW9edNul\nS7+kRo1Q4uO9z3U/bdpkHnigDw0bNrqssg0e3J/nn3+Ra6659rK2rwg+C+/vv/+en376iYULF5KR\nkUFSUlKR8AaYNWsWNWrU8FURvFqyxMrUqTb27TMTE+Nk2LB8kpLsFVoGERGjKe/PzuHDh5OamsXS\npV/yyy8/M3jwsFJv26NHzxLXDx2afNnlMgqfhXe7du1o1aoVADVr1iQvLw+Hw+FpafvDkiVWBgyo\n7rm/Z4/lv/fzFOAiIsWoyM/OrVs388kn88nNzWXw4OfYtm0Lq1d/g9PppEOH23jqqf7MmTOT2rVr\n07hxExYv/jsmk5nffvs/Ona8k6ee6u9pOa9a9Q05Odn8/vtvHDp0kCFDkunQ4Tbmz/8bK1b8i4YN\nG2G32+nT5xHatGl7QVmys7N5/fXRZGdnYbfbGTbsBZo1u46pU99k7949OBwOkpLup0ePnkyd+iY/\n/7yP06fzPct8yWfhbbFYCAkJAWDRokXExcVdENyjRo3i0KFD3HTTTSQnJ2MymXxVHACmTrV5XT5t\nmk3hLSJSjIr+7Pz55/0sWLAYm83Gtm1b+MtfZmM2m3nwwV707v1wkcfu3r2Ljz/+DKfTyQMP9OSp\np/oXWX/8+DEmTXqb779fxxdffEaLFi1ZvPhTFiz4jJycHPr0+QN9+jzitRyffrqAFi1a0rfvE+zd\nu5t33pnCuHFvsm7dd/z9719gt9tZuvRLTp06ybp137Fq1UqOHMlg6dIvy/01OZ/PJyZZsWIFixYt\nYu7cuUWWDxkyhDvuuINatWoxaNAgUlJSSExMLHY/deqElHnc2H37iltuKXbwd1/z1/P6kupkHIFY\nr0CsE/i3Xr767IyICCMsrBohITbPfmrXDiE2tjmNGtUFoF69Wjz33ECsVisnT2ZitTqoUSOY0NBq\n1K4dwvXXt+SKKyIAMJlMRESEYbNZqVOnBjVqBNOhwy1ERITRrFljzpzJIycnneuua0Z0dAQQQevW\nrahdO6RIPQq3/+WXfQwcOJCIiDAiIm7h5ZcP0qRJNE2aXMOrr75IYmIijz7aB5vNRpMm1zBw4MAi\ny3zJp+H97bffMmPGDGbPnk1YWNE3+N577/XcjouLY9++fSWGd3nM1BITE8KePRd+AYiJcZCaWvGz\n20REhAXcbGmqk3EEYr0CsU7g/3r54rOzsE5ZWafJzc331C8zMxeXy0RqahZHjx5hzpy5zJ37ESEh\nITz66IOkp+eQk3OGoKDTZGbm4nC4PNu6XO7b+fl2MjIKH1ed1NQsMjJyyM+3k56eg93u9GxTUOAg\nMzO3yOtbuH1BgYP09BzPOrvdQWpqFm+88RY//riXr79ezqeffsZbb03njTfe4vjx3/n73xd7lpWH\nCp9VLCsri4kTJzJzpvvcxPnr+vXrR35+PgCbNm2iadOmviqKx7Bh+V6XDx3qfbmIiPjvszMzM5M6\ndeoQEhLCjz/u5ejRoxQUFJRpn1FRUfzyy8/Y7XYyMjLYu3dPsY+97rpYtm3bDMDOnTto3LgJR44c\n5tNPP6FZs+sYPHgYJ0+e9Cxr0aKFZ5mv+azlvXTpUjIyMhg27GwPwltuuYVmzZrRpUsX4uLi6N27\nN8HBwcTGxpbY6i4v7nMzeUybdrbH5NCh6m0uIlISf312Nm0aQ/XqIQwc+BTXX38DvXr9gcmTJ9Cq\nVevL3md4eF26dEnkf/7nMa66qjGxsS2K7Uj94IMPMW7cawwZ8kecTifPP/8S9epFsHPndr755l8E\nBQVx1133eJb16dMHMHPXXfdcdvlKy+RyuVw+f5ZyoENhxqA6GUcg1isQ6wSBWS9/1mnp0i/p0iUR\ni8XCY4/1YcqUd4iMLPs4I76oU3GHzX3eYU1ERKQyOXHiBP37P05QkI2uXRPLJbgrmsJbRESqlEcf\nfYJHH33C38UoE41tLiIiYjAKbxEREYNReIuIiBiMwltERMRgFN4iIlLhevfufcEAKTNmvMuCBfO9\nPn7r1s288sqLAAwf/vwF6z/7bCFz5sws9vn27/+J33//DYBRo0Zw5szpyy0699/fk9zcih+V81wK\nbxERqXB33303K1d+XWTZ6tUr6dy5azFbnDV+/JRLfr41a1Zy4MDvALz22hsEB1e75H1UJrpUTERE\nKlyPHj148MHePPPMEAD27t1DREQEERGRbNq0gdmzZxAUFERYWBh//vP4ItveddedfPXVN2zevJG3\n355MeHhd6tat55ni8/XXR5Oaepy8vDyeeqo/DRpE8cUXi1mzZiV16tTh1VdH8OGHC8nOzuKNN/5M\nQUEBZrOZ4cNHYjKZeP310TRs2Ij9+38iJqYZw4eP9FqH48ePFdl+4sTxWK2h/PnPIzlxIo38/Hz6\n9RtA27Y3X7Csfftby/T6KbxFRKq40aOD+fLL8o2Dnj3tjB59ptj1devWpWHDRuzevZPY2JasXPk1\nXbq4h8nOyspi1KixNGzYiDFjXmXDhvWeKabPNXPmu4wcOYamTWP405+G0LBhI7KyTnHzze3p3v1u\nDh06yMiRw5k7dz633NKBjh3vJDa2pWf72bNncPfdvbjzzq6sWrWCuXP/Sr9+A/jxxz289to46tQJ\nJympB1lZWRdMruVt+3fffZeePe/n5MlMpk+fRVZWFuvXr+Xnn/dfsKysdNhcRET8okuXRL75xn3o\nfO3af9Ox450A1K5dmwkTxjJ4cH+2bdvCqVPeJ/o4cuQITZvGAHDDDW0ACAuryZ49uxg48Clef310\nsdsC/PjjHm688SYA2rRpy08//QhAo0ZXULduPcxmM/XqRZCTk12q7Xfv3s1VV11Nbm4OY8aMZOvW\nTXTu3NXrsrJSy1tEpIobPfpMia1kX4mP78SHH86lS5duXHHFldSsWROAN94Yw5tvTuXqqxszZcqE\nYrc3m8+2Pwun6fj66+WcOnWK6dNnc+rUKZ5++tESSmDybFdQYMdkcu/v/IlKip8CpOj2ZrOZatWq\nMXPm39ix4weWLfuStWu/5eWXR3ldVhZqeYuIiF+EhNSgSZOmfPjh+55D5gA5OdnUr9+ArKwstm7d\nUuw0oPXqRfD777/icrnYtm0L4J5GNCqqIWazmTVrVnq2NZlMOByOIts3bx7L1q3uKT//858tXHdd\n80sq//nbt2zZ0jPPd+vWN/CnP43g11//z+uyslLLW0RE/KZLl0TGjh3FqFFjPMv+8IcHGDiwH1dc\ncSWPPPIYc+f+lf79n7lg2/79n+GVV16iQYMoz+QiHTsmMHz48+zevZO77rqHyMhI3n9/Fq1b38jU\nqW8WOXf+9NN/5I03xvDll59jtQYxYsRI7PbST3N6/vaTJk0gO9vOzJnT+eKLxZjNZh5++FGiohpe\nsKysNCWoH2maP2MIxDpBYNYrEOsEgVkv1an0+/RGh81FREQMRuEtIiJiMApvERERg1F4i4iIGIzC\nW0RExGAU3iIiIgaj8BYRETEYhbeIiIjBKLxFREQMRuEtIiJiMApvERERg1F4i4iIGIzCW0RExGAU\n3iIiIgaj8BYRETEYhbeIiIjBKLxFREQMRuEtIiJiMApvERERg1F4i4iIGIzCW0RExGAU3iIiIgaj\n8BYRETEYhbeIiIjBKLxFREQMRuEtIiJiMApvERERg1F4i4iIGIzCW0RExGAU3iIiIgaj8BYRETEY\nhbeIiIjBKLxFREQMxurLnU+cOJEtW7Zgt9sZMGAAXbt29axbt24dU6ZMwWKxEBcXx6BBg3xZFBER\nkYDhs/D+/vvv+emnn1i4cCEZGRkkJSUVCe+xY8cyZ84c6tevT9++fenWrRvXXnutr4ojIiISMHwW\n3u3ataNVq1YA1KxZk7y8PBwOBxaLhQMHDlCrVi2ioqIAiI+PZ/369QpvERGRUvDZOW+LxUJISAgA\nixYtIi4uDovFAkBqairh4eGex4aHh5OamuqrooiIiAQUn57zBlixYgWLFi1i7ty5ZdpPnTohWK2W\ncipV5REREebvIpQ71ck4ArFegVgnCMx6qU6Xz6fh/e233zJjxgxmz55NWNjZCkVGRpKWlua5f+zY\nMSIjI0vcV0ZGrs/K6S8REWGkpmb5uxjlSnUyjkCsVyDWCQKzXqpT6ffpjc8Om2dlZTFx4kRmzpxJ\n7dq1i6yLjo4mOzubgwcPYrfbWbVqFbfddpuviiIiIhJQfNbyXrp0KRkZGQwbNsyz7JZbbqFZs2Z0\n6dKF0aNHk5ycDECPHj1o3Lixr4oiIiISUHwW3r1796Z3797Frm/Xrh0LFy701dOLiIgELI2wJiIi\nYjAKbxEREYNReIuIiBiMwltERMRgFN4iIiIGo/AWERExGIW3iIiIwSi8RUREDEbhLSIiYjAKbxER\nEYNReIuIiBiMwltERMRgFN4iIiIGo/AWERExGIW3iIiIwSi8RUREDEbhLSIiYjAKbxEREYNReIuI\niBiMwltERMRgFN4iIiIGo/AWERExGIW3iIiIwSi8RUREDEbhLSIiYjAKbxEREYNReIuIiBiMwltE\nRMRgFN4iIiIGo/AWERExmCoZ3nY7fP21BYfD3yURERG5dFUyvFeutPDIIyG8916Qv4siIiJyyapk\neN9yi4PQUBczZ9o4c8bfpREREbk0VTK8a9WCxx4r4NgxM59+qta3iIgYS5UMb4ABA/IJCnIxfbpN\n575FRMRQqmx4R0W5eOCBAn7+2cyyZVZ/F0dERKTUqmx4AwwaVIDJ5OLdd224XP4ujYiISOlU6fBu\n2tRJ9+52tm61sHatxd/FERERKZUqHd4Azz6bD8A779j8XBIREZHSqfLhfdNNTm691c6qVVZ27Kjy\nL4eIiBiA0oqzre/p09X6FhGRyk/hDSQkOIiNdfD551Z++83k7+KIiIiUSOENmEwweHA+TqeJ995T\n61tERCo3hfd/3XuvnSuucLJgQRBpaWp9i4hI5aXw/i+rFQYOzCcvz8Ts2RoyVUREKi+F9zkeeqiA\n8HAnc+fayM72d2lERES8U3ifo0YN6NevgMxMEx99pNa3iIhUTgrv8/Trl09IiIsZM2wUFPi7NCIi\nIhdSeJ8nPBweeaSAQ4fMLF6sCUtERKTy8Wl479u3j86dOzN//vwL1iUkJPDwww/z6KOP8uijj3Ls\n2DFfFuWS/PGP+Vgs7ulCnU5/l0ZERKQonzUtc3NzGTNmDB06dCj2MbNmzaJGjRq+KsJlu+IKF0lJ\ndhYtCmLFCgtdu2rCbxERqTx81vK22WzMmjWLyMhIXz2FTw0erAlLRESkcvJZy9tqtWK1lrz7UaNG\ncejQIW666SaSk5MxmYofHKVOnRCs1oqbtjM+Hu66C776ysq+fWHcdptvniciIsw3O/Yj1ck4ArFe\ngVgnCMx6qU6Xz289soYMGcIdd9xBrVq1GDRoECkpKSQmJhb7+IyM3AosnduAARa++iqEMWPszJuX\nV+77j4gIIzU1q9z360+qk3EEYr0CsU4QmPVSnUq/T2/81tv83nvvpW7dulitVuLi4ti3b5+/ilKs\nW25x0Latg5QUK3v3qmO+iIhUDn5JpKysLPr160d+vvu88qZNm2jatKk/ilIik0nThYqISOXjs8Pm\nO3fuZMKECRw6dAir1UpKSgoJCQlER0fTpUsX4uLi6N27N8HBwcTGxpZ4yNyfunWzExPj4LPPrAwf\nbqJRI5e/iyQiIlWcyeVyGSKN/HluZMECK0OHVmfAgHzGjDlTbvvVOR9jCMQ6QWDWKxDrBIFZL9Wp\n9Pv0RidyS+G+++xERTmZNy+IjAx/l0ZERKo6hXcp2GwwYEA+ubkm3n9f575FRMS/FN6l9NhjBdSq\n5WL27CDyyv+qMRERkVJTeJdSaCg8+WQ+aWlmFizQdKEiIuI/Cu9L8PTTBQQHu/jLX2zY7UXXLVli\nJT4+hKioUOLjQ1iyRDOSiYiIbyi8L0FkpIs+fQr4/XczX355NpyXLLEyYEB19uyx4HCY2LPHwoAB\n1RXgIiLiEwrvS/TMM/mYzS7eecdG4UV2U6d678Q2bZo6t4mISPkrVXjv3LmTVatWAfDWW2/x+OOP\ns3nzZp8WrLJq3NhFz552du60sHq1e6KUffu8v4zFLRcRESmLUqXL2LFjady4MZs3b2bHjh2MHDmS\nt99+29dlq7QKh0wtnC40Jsbp9XHFLRcRESmLUoV3cHAwV199Nd988w0PPvgg1157LWZz1W1Vtmrl\nJC7OznffWdm2zcywYfleHzd0qPflIiIiZVGqBM7Ly2PZsmWsWLGC22+/nczMTE6dOuXrslVq57a+\nk5LszJyZR2ysA6vVRWysg5kz80hKsl9kLyIiIpeuVN2hn3/+eT788EOee+45QkNDeeedd3jiiSd8\nXLTKLS7OQevWDr76ysrPP5tISrIrrEVEpEKUKrzbt29Py5YtCQ0NJS0tjQ4dOtCmTRtfl61SK5wu\n9Omnq/OXv9iYPLn8JiwREREpSakOm48ZM4Zly5aRmZlJnz59mD9/PqNHj/Zx0Sq/u+6y07ixk4UL\ngzh2zOTv4oiISBVRqvDevXs3DzzwAMuWLSMpKYmpU6fy22+/+bpslZ7F4r7uOz/fxF//qiFTRUSk\nYpQqvAun/F69ejUJCQkA5OerJzVA794FREQ4+dvfbFTxPnwiIlJBShXejRs3pkePHuTk5NC8eXM+\n//xzatWq5euyGUK1atC/fwFZWSY++EAjqomIiO+ZXIXN6hI4HA727dtHkyZNsNls7Ny5kyuvvJKa\nNWtWRBkBSE3NqrDnulQnT8KNN4ZSo4aLzZtzCA4u3XYREWGVul6XQ3UyjkCsVyDWCQKzXqpT6ffp\nTala3qdPn2blypUMGTKEgQMHsnbtWmw2tTIL1arlnu/72DEzCxfq3LeIiPhWqcJ75MiRZGdn06dP\nHx588EHS0tJ45ZVXfF02QxkwIJ/gYBf/+7/B/OMfmk1MRER8p1ThnZaWxksvvUTHjh3p1KkT//u/\n/8uxY8d8XTZDiYpy8cEHeQQFwdNPV+ftt8/OOiYiIlKeSj08al5enud+bm4uZ85oUJLzJSQ4+PLL\nXBo2dDJ2bDDJycEUFPi7VCIiEmhKdXy3d+/edO/enZYtWwKwa9cuhg4d6tOCGVWLFk6WL8/lkUeq\nM3++jQMHzMyZk0cF9u0TEZEAV6qW9/3338+CBQu49957SUpK4pNPPmH//v2+LpthNWjg4osvcunW\nzc6aNVbuvjuEAwc0ApuIiJSPUvesioqKIioqynP/hx9+8EmBAkVoKPztb3mMHBnM7Nk2uncPYf78\nPG64QXN8i4hI2Vz2pNyluDy8yrNYYNy4M7z++mnS0kz06hXC0qXqiS4iImVz2eFtMukwcGn9z/8U\n8MEHeZhM8OST1ZgxI0g90UVE5LKV2AyMj4/3GtIul4uMjAyfFSoQdevm4Isv3B3ZXn21Gr/+ambm\nTH+XSkREjKjE8P74448rqhxVQuvWZ3uiz51r4+hRePdd9/lxERGR0ioxvBs1alRR5agyoqNd/POf\nuTz9dHWWLrVyzz0hfPRRHlFROo4uIiKlc9nnvOXyhYXB/Pl59O8PO3daSEwMYccOvRUiIlI6Sgw/\nCQqCGTPg1VdPc+SImZ49Q1ixwuLvYomIiAEovP3IZILBgwuYMycPpxP69q3O3LmalUxEREqm8K4E\neva0s3hxLuHhLoYPr8arrwbjcPi7VCIiUlkpvCuJtm2dLFuWS9OmDmbMsPHUU9XIyfF3qUREpDJS\neFciV13l4quvcrn9djvLlgWRlBTCsWMaDEdERIpSeFcytWvDJ5/k0adPAf/5j4Xu3UPYs0dvk4iI\nnKVUqIRsNpg27TTDh5/h4EEztLXiAAAehElEQVQz3buHMGaMjRMn1AoXERGFd6VlMsHzz+cze3Ye\nNWu6eOedYNq2rcH48TYyM/1dOhER8SeFdyV3zz12Nm7MYezY04SEuJgyJZibbgrlzTdtnDrl79KJ\niIg/KLwNoFo16N+/gE2bchg16jQ2m4s33wymbdtQpk61kZ3t7xKKiEhFUngbSEgIDBrkDvFXXjmD\nyQTjxgXTrl0N3n03SJeWiYhUEQpvAwoNhSFD8tm8OZuXXjpDQYGJP/+5Gu3a1WDGjCDy8vxdQhER\n8SWFdyW2ZImV+PgQoqJCiY8PYcmSopPAhYVBcnI+W7Zkk5x8htOnTbz6ajVuvrkGc+YEceaMnwou\nIiI+ZXK5XIaYizI1NcvfRSh3ERFhxdZryRIrAwZUv2D5zJl5JCXZvW6Tng7vvWdj1iwbubkmGjZ0\n8txz+Tz0UAE2W7kWvVgl1cmoArFOEJj1CsQ6gX/q5XJBQYH7x26HggKT1/vu22C3m3A6wWKBoCAX\nQUFgtfLf3+77Z5e5iIoKIzMzC6vVfXWNUWRlwaFDZg4fNnH4sJlDh9y/Dx82ERZmZfr0LKpf+NF9\n2SIiwrwuV3j7UUn/kPHxIezZc+EsY7GxDlavzi1xv6mpJqZPt/H++0Hk5Zm44gonzz+fz4MPFhDk\n43lPAvHDMxDrBIFZr0CsE1xevZxO92fBwYMmDh1yh8yhQ2YOHnSHTXa2O4DtdsjPd4fvuWHscFRc\nohaGe2Gwnw19sNlc1KwJtWq5qF3b5eU3Re7Xru2iRo3L+0KQnc0FgXz4sPt1O3LE/Ts7u/gdN20K\nKSlZhIaW4cU4j8K7EirpHzIqKtTrP4/V6uLw4dJ1Lz92zMQ779j44IMgzpwxcfXVTpKTz3DffXas\n1otvfzkC8cMzEOsEgVmvQKwTeK9XYQvw0CETBw+eDefC+0eOmCgo8B40wcEuatYs2jo+PzQLW8/F\ntZy9PdZkAofj7JeCsy31ol8MCgpMmExWcnPtnvtnW/AXbn/qVPF18cZqdYf5+cF+7u/Tpy8M5lOn\nin+OWrVcNGzopGFD9+9Gjc7eb9TISYMGLq6+uvz//ooLbx99hEtZxcQ4vba8Y2Kcpd5H/fouxo49\nw6BB+UybZmPevCCefbY606Y5SE7O5+677QQHl2epRaS8uFzw668mtm2zkJ4OP/4YzOHDZk9Lurig\nMZlcREa6aNXKSaNG7pAp/B0d7f5dt67L74eq3V9ISte71uWC3Fw4edJEZqbpnN+QmXn+sqLrDh40\nk59fcmXDwtyv0U03uX8XBrQ7mF1ERTnLtTVdHnza8t63bx/PPPMMTzzxBH379i2ybt26dUyZMgWL\nxUJcXByDBg0qcV9V5dt0ocs5530xBw+aeOstGwsWBGG3m6he3UX79g46drQTH++geXNnmf+hA7Hl\nE4h1gsCsl5HrZLfDzp1mNmywsHGjhQ0bLBw/fmGf4tDQsyHcqJGT6Gh30ERHu+9HRbkqrI9LWVTU\ne+VyQV7e+cHvHoa6sPUc5r1xe8l8UacKb3nn5uYyZswYOnTo4HX92LFjmTNnDvXr16dv375069aN\na6+91lfFMRx3QOcxbZqNffvMxMQ4GTo0/7KDGyA62sXkyWcYMiSfuXNtrF5tYdUqK6tWuf8MIiOd\nxMc7iI93h3n9+oY4oyJiSNnZsHmzxRPWW7ZYyM09++25fn0nvXoV0LatgxtvrEZoaA7R0U5q1vRj\noQ3IZHKPkRES4iIqKnA+03wW3jabjVmzZjFr1qwL1h04cIBatWoRFRUFQHx8POvXr1d4nycpyV6m\nsC7OVVe5eO0193VkR4+aWLPGwpo1VtassfDpp0F8+qm7V1vz5g7i490t8/btHYSElHtRRCqUywW/\n/WZi1y4LZrP7C2v9+i4iIlw+P4V09KiJDRvOhvXOnWaczrNhfd11Dm6+2f1zyy0Orrzy7KHtiIhq\npKaW/pSZBD6fhbfVasVaTK+o1NRUwsPDPffDw8M5cOBAifurUycEq/XCc8BGV9whkYp7frj+ehg8\n2P3BtmMHfP21+2fNGgt79liYMcOGzQa33w5durh/brwRzMWMEuDvOvlCINYJArNehXVyueD//g82\nb4YtW9w/W7dCRob37erUgQYNLv5Tr17xf/uFnE7Yswe++879s3atuyyFbDa49Vb3/9Rtt7lvh4db\ngOI/4wL5vQokFVUnw3RYy8go+fIoI6qM5+eiouCxx9w/p0/Dxo0W1qyxsHq1lZUrLaxcCSNGQHi4\nk7g4h+cwe3S0+3BUZaxTWQVinSCw6lXYuevXX0P59tszbN9uYccOC5mZRTtxNG7sPjV0/fVOLBYX\nx4+bOX7cxLFjJlJT3b/37Ck5mS0WF/XquTuFnf1xEhnpIi/PxMaN7pb1uc9dp46Lbt3Otqxbt3ZQ\nrdrZfTockJpa/HMG0ntVSHUq/T698Ut4R0ZGkpaW5rl/7NgxIiMj/VEUKUG1ahAX5yAuzsHIkfmk\npZn49lt3kK9ZY+Hzz4P4/HP3IfYmTZx07GinVy9o2ZJK1zNTAoe7RW3ihx8sbN9u4YcfzPzwg4WT\nJwvD0n38+5prnHTqZKdVKwetWzu5/noHtWpdfP/5+ZCWZuL48cIf8zm3zy77+WczO3Z47+F51VVO\nuna1ew6BN23qvGhrXeRS+CW8o6Ojyc7O5uDBgzRo0IBVq1YxadIkfxRFLkG9ei7PeXiXC/bvN7N6\ntft8+dq1FubMsTFnDgQFhdKunYNOnRx06mSnZUt9cMmlKyiAnBx3kO7YUTSoz79M6pprnCQk2Ln1\n1iCaNMmlVSvHZXfsstn476VChZ2bHMU+NjubIgFvMkG7dg4aNAicjlFSOfnsUrGdO3cyYcIEDh06\nhNVqpX79+iQkJBAdHU2XLl3YtGmTJ7C7du1Kv379StxfoB1egcA6bJSfD1u2WNi4MYSvvnKwfbsZ\nl8v9AVuvntPT8a1jR+P1Yg+k9+lc5V2vggJ30GZnm8jJgZwcE9nZ7t/uH/677tzbRde7t3HfLu7a\n3CZNnLRu7SjSoi4Mar1XxqE6lX6f3miENT8K5D/eEydM/PvfhZeiWTh27GzTu0ULd4u8Uyf3+b/K\nPlBMIL5PULZ65efD3r1mtm+3sH27mR07LOzebebMmcsbKMBmcw9pGRrqokYN9233b/coWc2bnw3q\nkq7J1XtlHKpT6ffpjWE6rImx1K1b9BD7nj1mVq1yh/mGDRZ27Qrm3Xfd117eemthmNtp0sT/Iz9J\nUWfOuN+/wsPW27db2LPHXGS4yqAgF82bO2nSxPnfAHaHb9HbZwP53Ns1amCIQUVEKhOFt/icyQSx\nsU5iY50MGlRAbi58//3ZVvmKFVZWrHD/KV5xhbvjW6dODu64w16qDkZSfvLyYPdu93nlwqDeu9eM\n3X42qG02Fy1aOD2HrVu3dnDddU4FsEgFUnhLhQsJgYQEBwkJ7o5ABw+aWL3aHeT//reVefNszJvn\nviSnTRsn7dvbCQuDatVcVKvm/l29+rn3oXp19yAbhcuqV3f/LpwsQS6UmwubNhUGtfvw948/motM\niFOtmstzuLp1a3dgX3ed0+ez04lIyRTe4nfR0S769i2gb98C7Hb4z3/MnmFbt2wxs2nT5Z8UN5uL\nhvm54V+vnnt0rcKfBg3O3q9Xz4WlgsYEKihw91g+etTEsWNmjh41ee4fPWomPd09TzK4L5Mq7KVy\n/u1Cxa0/e9s9Y9OBA+B01vBsV726ixtvPNsZrFUrJzExCmqRykjhLZWK1Qpt2zpp2zafF17IJzMT\n9u61kJsLp0+bOH3aPXhMXl7h7bO/8/IuvH/mTNFlp07B8eNm8vLAbi8+nc1m95CZDRq4uOIKqFMn\n+Jygd0//VzisZnHTqxYUuOdTPjeUjx1z/xw9evb+iRMmT898b6pVc2E2nz2CYDIVve1tmcnkumD5\nuY83m6FDB4iNzfcEddOmTp9NFSsi5Uv/qlKp1a4N7dsXf53t5XK5IDOT80LVfE64uu/v22dm+3YA\n7yd0TSZ3K70w2E0m/ttivngo16jh/nIQE+PwfBko/GLgvu8+EuCrAW/cPWPP+GbnIuJTCm+pkkwm\n91jWdeo4ue664h/ncoHNFsauXTnnhPr5QW/ml1/M7NzpDuqQkKKhHBnpPiRfGNCFh+c1Cp2IXC6F\nt0gJTCZ36z8mxklMTMmPzc52h315zQ0sIlIchbdIOVFLWkQqikacFhERMRiFt4iIiMEovKugJUus\nxMeHEBUVSnx8CEuW6OyJiIiR6FO7ilmyxMqAAdU99/fssfz3fh5JSXb/FUxEREpNLe8qZupU79cr\nT5umgalFRIxC4V3F7Nvn/S0vbrmIiFQ++sSuYmJinJe0XEREKh+FdxUzbFi+1+VDh3pfLiIilY/C\nu4pJSrIzc2YesbEOrFYXsbEOZs5UZzURESNRb/MqKCnJrrAWETEwtbxFREQMRuEtIiJiMApvERER\ng1F4i4iIGIzCW0RExGAU3iIiIgaj8BYRETEYhbeIiIjBKLxFREQMRuEtIiJiMApvERERg1F4S7lY\nssRKfHwIVivEx4ewZImGzRcR8RV9wkqZLVliZcCA6p77e/ZY/ntfs5WJiPiCWt5SZlOn2rwunzbN\n+3IRESkbhbeU2b593v+MilsuIiJlo09XKbOYGOclLRcRkbJReEuZDRuW73X50KHel4uISNkovKXM\nkpLszJyZR2ysA6sVYmMdzJypzmoiIr6i3uZSLpKS7CQl2YmICCM1NdffxRERCWhqeYuIiBiMwltE\nRMRgFN4iIiIGo/AWERExGIW3iIiIwSi8RUREDEbhLSIiYjAKb6m0CqcZjYoK1TSjIiLn0KehVEqa\nZlREpHhqeUulpGlGRUSK59OW97hx49i+fTsmk4mXX36ZVq1aedYlJCTQoEEDLBYLAJMmTaJ+/fq+\nLI4YiKYZFREpns/Ce+PGjfz2228sXLiQn3/+mZdffpmFCxcWecysWbOoUaOGr4ogBhYT42TPHovX\n5SIiVZ3PmjHr16+nc+fOADRp0oSTJ0+SnZ3tq6eTAKNpRkVEiuezlndaWhotWrTw3A8PDyc1NZXQ\n0FDPslGjRnHo0CFuuukmkpOTMZlMxe6vTp0QrNYLW2JGFxER5u8ilLvyqFP//lCzJrzxBuzeDbGx\nMGIE9OlT/eIb+0Agvk8QmPUKxDpBYNZLdbp8Fdbb3OVyFbk/ZMgQ7rjjDmrVqsWgQYNISUkhMTGx\n2O0zMgJvmkn39JlZ/i5GuSrPOt15p/vnXKmp5bLrSxKI7xMEZr0CsU4QmPVSnUq/T298dtg8MjKS\ntLQ0z/3jx48TERHhuX/vvfdSt25drFYrcXFx7Nu3z1dFERERCSg+C+/bbruNlJQUAHbt2kVkZKTn\nkHlWVhb9+vUjP999/nLTpk00bdrUV0UREREJKD47bN6mTRtatGhBnz59MJlMjBo1isWLFxMWFkaX\nLl2Ii4ujd+/eBAcHExsbW+IhcxERETnL5Dr/ZHQlFWjnRkDnfPxlyRIrU6fa2LfPTEyMk2HD8ksc\ntc0IdbocgVivQKwTBGa9VKfS79MbDY8qVYqGXRWRQKDhqqRK0bCrIhIIFN5SpWjYVREJBPrEkiql\nuOFVNeyqiBiJwluqFA27KiKBQOEtVUpSkp2ZM/OIjXVgtbqIjXUwc6Y6q4mIsai3uVQ5SUl2hbWI\nGJpa3iIiIgaj8BYpB0uWWImPDyEqKpT4+BCWLNFBLRHxHX3CiJSRBn4RkYqmlrdIGWngFxGpaApv\nkTLSwC8iUtH06SJSRhr4RUQqmsJbpIw08IuIVDSFt0gZaeAXEalo6m0uUg58MfDLpc47LiJVh8Jb\npBLS5WciUhIdNhephHT5mYiUROEtUgnp8jMRKYk+CUQqIV1+JiIlUXiLVEK+vPyscBx2qxWNwy5i\nUPqvFamE3J3S8pg27Wxv86FDy97bXB3hRAKDwlukkvLF5WcldYRTeIsYhw6bi1Qh6ggnEhj0HytS\nhfiqI5zmMxepWApvkSrEFx3hCs+j79ljweEwec6jK8BFfEfhLVKFFB2HnXIZh10DyohUPIW3SBWT\nlGRn9epcCgpg9ercMndU89V5dB2KFymewltEysQX59F1KF6kZApvESkTX5xH9+WheLXoJRDor1ZE\nysQXA8r48lC8BqmRQKCWt4iUWeF59MOHs8vlPLqvLmnzVYterXmpaApvEal0fDW2uy9a9L48P69x\n6KU4Cm8RqXSKXtLmKpdL2sA3LXpftubPfimg3L4U6ChBYFB4i0ilVN6H4sE3LXpfnZ/3xZcCXx0l\n0BeCiqfwFpEqwxctel+dn/fFlwIjfSEo3Hd5fynw5T4r8vSGvh6JSJVS3rO1DRuWX6QHe6Gynp+P\niXGyZ4/F6/LLVdFfCMryOvviygCj7LM01PIWESkDX52f98Uhfl8cJTDSaQOj7LM01PIWESkjX8y9\nXvT6eQsxMY4yXz/vi6MEvjhCAL75UmCUfZaGWt4iIpVUeY9D74ujBL66rM8XRwmMss/SUHiLiFQh\n5d2L30inDYyyz9LQYXMRESkT3582KJ9hd32/z/I5vVEaJpfL5fLpM5ST1NQsfxeh3EVEhAVcvVQn\n4wjEegVinSAw66U6lX6f3uiwuYiIiMEovEVERAxG4S0iImIwCm8RERGDUXiLiIgYjE/De9y4cfTu\n3Zs+ffrwww8/FFm3bt067r//fnr37s306dN9WQwREZGA4rPw3rhxI7/99hsLFy7k9ddf5/XXXy+y\nfuzYsbzzzjssWLCAtWvXsn//fl8VRUREJKD4LLzXr19P586dAWjSpAknT54kOzsbgAMHDlCrVi2i\noqIwm83Ex8ezfv16XxVFREQkoPhshLW0tDRatGjhuR8eHk5qaiqhoaGkpqYSHh5eZN2BAwdK3F+d\nOiFYrRcOfm90xV2Ab2Sqk3EEYr0CsU4QmPVSnS5fhXVYK+tAboEY3CIiIpfDZ+EdGRlJWlqa5/7x\n48eJiIjwuu7YsWNERkb6qigiIiIBxWfhfdttt5GSkgLArl27iIyMJDQ0FIDo6Giys7M5ePAgdrud\nVatWcdttt/mqKCIiIgHFpxOTTJo0ic2bN2MymRg1ahS7d+8mLCyMLl26sGnTJiZNmgRA165d6dev\nn6+KISIiElAMM6uYiIiIuGmENREREYNReIuIiBiMz67zlrMmTpzIli1bsNvtDBgwgK5du3rWJSQk\n0KBBAywW96VwkyZNon79+v4qaqls2LCBoUOH0rRpUwBiYmIYOXKkZ/26deuYMmUKFouFuLg4Bg0a\n5K+iXpJPP/2Uf/zjH577O3fuZNu2bZ77LVq0oE2bNp77f/vb3zzvW2W0b98+nnnmGZ544gn69u3L\nkSNHePHFF3E4HERERPDmm29is9mKbDNu3Di2b9+OyWTi5ZdfplWrVn4qvXfe6jRixAjsdjtWq5U3\n33zTc1ULXPxvtTI4v07Dhw9n165d1K5dG4B+/frRsWPHIttU9vcJLqzXkCFDyMjIACAzM5MbbriB\nMWPGeB6/ePFipk2bxpVXXgnArbfeysCBA/1S9uKc/1l+/fXX++9/yiU+tX79etfTTz/tcrlcrvT0\ndFd8fHyR9Z06dXJlZ2f7oWSX7/vvv3c9++yzxa7v3r276/Dhwy6Hw+F66KGHXD/99FMFlq58bNiw\nwTV69Ogiy26++WY/lebS5eTkuPr27et65ZVXXPPmzXO5XC7X8OHDXUuXLnW5XC7X5MmTXR999FGR\nbTZs2ODq37+/y+Vyufbv3+968MEHK7bQF+GtTi+++KLrq6++crlcLtf8+fNdEyZMKLLNxf5W/c1b\nnV566SXXypUri92msr9PLpf3ep1r+PDhru3btxdZ9tlnn7nGjx9fUUW8ZN4+y/35P6XD5j7Wrl07\npk2bBkDNmjXJy8vD4XD4uVS+EyhD306fPp1nnnnG38W4bDabjVmzZhUZP2HDhg3ceeedAHTq1OmC\n96WkIY0rA291GjVqFN26dQOgTp06ZGZm+qt4l8VbnS6msr9PUHK9fvnlF7Kysirl0YKSePss9+f/\nlMLbxywWCyEhIQAsWrSIuLi4Cw61jho1ioceeohJkyaVeSS6irJ//37++Mc/8tBDD7F27VrPcm9D\n36ampvqjiJfthx9+ICoqqsjhV4D8/HySk5Pp06cP77//vp9KVzpWq5Vq1aoVWZaXl+c5pFe3bt0L\n3pe0tDTq1KnjuV/Z3jtvdQoJCcFiseBwOPj444/p2bPnBdsV97daGXirE8D8+fN57LHHeO6550hP\nTy+yrrK/T1B8vQA+/PBD+vbt63Xdxo0b6devH48//ji7d+/2ZREvmbfPcn/+T+mcdwVZsWIFixYt\nYu7cuUWWDxkyhDvuuINatWoxaNAgUlJSSExM9FMpS+fqq69m8ODBdO/enQMHDvDYY4/xr3/964Jz\nPUa1aNEikpKSLlj+4osvcs8992Aymejbty9t27bl+uuv90MJy640XxKN8kXS4XDw4osv0r59ezp0\n6FBknRH/Vnv16kXt2rVp3rw5f/3rX3n33Xd59dVXi328Ud4ncH8B3rJlC6NHj75gXevWrQkPD6dj\nx45s27aNl156iS+//LLiC3kR536Wn9t/qaL/p9TyrgDffvstM2bMYNasWYSFFR20/t5776Vu3bpY\nrVbi4uLYt2+fn0pZevXr16dHjx6YTCauvPJK6tWrx7Fjx4DAGPp2w4YN3HjjjRcsf+ihh6hRowYh\nISG0b9/eEO/VuUJCQjh9+jTg/X0paUjjymzEiBFcddVVDB48+IJ1Jf2tVlYdOnSgefPmgLtD6/l/\nZ0Z9nwA2bdpU7OHyJk2aeDrm3XjjjaSnp1e6U4znf5b7839K4e1jWVlZTJw4kZkzZ3p6j567rl+/\nfuTn5wPuP+zCXrGV2T/+8Q/mzJkDuA+TnzhxwtND3uhD3x47dowaNWpc0DL75ZdfSE5OxuVyYbfb\n2bp1qyHeq3PdeuutniGL//Wvf3HHHXcUWV/SkMaV1T/+8Q+CgoIYMmRIseuL+1utrJ599lnPLIsb\nNmy44O/MiO9ToR07dnDdddd5XTdr1iz++c9/Au6e6uHh4ZXqag5vn+X+/J/SYXMfW7p0KRkZGQwb\nNsyz7JZbbqFZs2Z06dKFuLg4evfuTXBwMLGxsZX+kDm4WwN/+tOf+OabbygoKGD06NH885//9Ax9\nO3r0aJKTkwHo0aMHjRs39nOJS+/8c/Z//etfadeuHTfeeCMNGjTg/vvvx2w2k5CQUKk73OzcuZMJ\nEyZw6NAhrFYrKSkpTJo0ieHDh7Nw4UIaNmzIvffeC8Bzzz3HG2+8QZs2bWjRogV9+vTxDGlcmXir\n04kTJwgODubRRx8F3K230aNHe+rk7W+1Mh0y91anvn37MmzYMKpXr05ISAhvvPEGYJz3CbzX6513\n3iE1NdVzKVihgQMH8t5779GzZ09eeOEFPvnkE+x2O6+//rqfSu+dt8/y8ePH88orr/jlf0rDo4qI\niBiMDpuLiIgYjMJbRETEYBTeIiIiBqPwFhERMRiFt4iIiMHoUjGRAHbw4EESExMvGHQmPj6ep59+\nusz737BhA1OnTmXBggVl3peIlJ7CWyTAhYeHM2/ePH8XQ0TKkcJbpIqKjY3lmWeeYcOGDeTk5DB+\n/HhiYmLYvn0748ePx2q1YjKZePXVV7n22mv59ddfGTlyJE6nk+DgYM/gIU6nk1GjRrFnzx5sNhsz\nZ84EIDk5mVOnTmG32+nUqVOlm5tZxMh0zlukinI4HDRt2pR58+bx0EMP8fbbbwPuCVhGjBjBvHnz\nePLJJ3nttdcA9+x3/fr146OPPuK+++5j2bJlAPz88888++yz/P3vf8dqtfLdd9+xbt067HY7H3/8\nMZ988gkhISE4nU6/1VUk0KjlLRLg0tPTPcOHFnrhhRcAuP322wFo06YNc+bM4dSpU5w4ccIz9OvN\nN9/M888/D7inSr355psBuOuuuwD3Oe9rrrmGevXqAdCgQQNOnTpFQkICb7/9NkOHDiU+Pp4HHngA\ns1ltBZHyovAWCXAlnfM+d3Rkk8mEyWQqdj3gtfXsbfKIunXr8sUXX7Bt2za++eYb7rvvPpYsWVLs\nHM8icmn0VVikCvv+++8B2LJlC82aNSMsLIyIiAi2b98OwPr167nhhhsAd+v822+/BdyTNEyZMqXY\n/X733XesXr2am266iRdffJGQkBBOnDjh49qIVB1qeYsEOG+HzaOjowHYvXs3CxYs4OTJk0yYMAGA\nCRMmMH78eCwWC2azmdGjRwMwcuRIRo4cyccff4zVamXcuHH8/vvvXp+zcePGDB8+nNmzZ2OxWLj9\n9ttp1KiR7yopUsVoVjGRKqpZs2bs2rULq1Xf4UWMRofNRUREDEYtbxEREYNRy1tERMRgFN4iIiIG\no/AWERExGIW3iIiIwSi8RUREDEbhLSIiYjD/D2OcBseu6jEOAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f834e38ca90>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "QGvyYuBzdE1E",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.3.4 Plotting the training and validation accuracy"
      ]
    },
    {
      "metadata": {
        "id": "Bo7I7CAic3TO",
        "colab_type": "code",
        "outputId": "cf69fbb3-1716-44a9-86da-21a3b958b9fd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 376
        }
      },
      "cell_type": "code",
      "source": [
        "\n",
        "acc = history.history['acc']\n",
        "val_acc = history.history['val_acc']\n",
        "\n",
        "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe8AAAFnCAYAAACPasF4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VFXixvHvlCQQEiCBhI70EhAx\nyyoIGFooImpshBVBwQUpAoqywIJBUUQFBTuKDRBFWaKidBDbsnSRpij+QJCWQCghIcnM3N8fIwMx\nEwiQyeRO3s/z8DD3ztw752TKO+fcc8+1GIZhICIiIqZh9XcBRERE5NIovEVERExG4S0iImIyCm8R\nERGTUXiLiIiYjMJbRETEZBTeEjCSkpLo2rUrXbt2pUmTJrRv396znJ6efkn76tq1K6mpqRd8zNSp\nU/nwww+vpMiF7r777mPBggWFsq+GDRty6NAhli9fzpgxY67o+T7++GPP7YL8bUXkwuz+LoBIYXni\niSc8tzt06MBzzz1HixYtLmtfS5YsuehjRo4ceVn7Npv4+Hji4+Mve/uUlBRmzpzJ3XffDRTsbysi\nF6aWt5QY9957Ly+++CLdunVj06ZNpKam0r9/f7p27UqHDh149913PY892+pcu3YtPXv2ZOrUqXTr\n1o0OHTqwbt06AEaPHs1rr70GuH8sfPTRR9x55520adOGyZMne/b1xhtv0KpVK+644w4++OADOnTo\n4LV8n3zyCd26daNz587cc889/PHHHwAsWLCAYcOGMXbsWLp06cJNN93EL7/8AsC+ffu466676NSp\nEyNHjsTpdObZ79dff02PHj1yrbv11lv55ptvLvg3OGvBggXcd999F32+lStX0qNHD7p06cLtt9/O\nzp07AUhMTOTAgQN07dqV7Oxsz98WYNasWdx000107dqVQYMGcezYMc/f9qWXXuL++++nffv23H//\n/WRmZuYpW2ZmJiNGjKBLly506NCBZ5991nPfvn37uOeee4iPj+eOO+5g+/btF1zfoUMHNmzY4Nn+\n7PL+/ftp06YNkyZNonfv3hesK8Cbb75Jx44d6dKlC8888wxOp5PWrVuzdetWz2PmzJnD4MGD89RH\npKAU3lKibNu2jS+//JLY2Fhef/11qlevzpIlS3j//feZOnUqBw8ezLPNjh07uOaaa1i8eDH/+Mc/\neP31173ue/369cybN4///Oc/zJkzh0OHDvHLL78wc+ZMPvvsM+bOnZtvq/Po0aM8+eSTvPvuuyxb\ntoyaNWt6fhgAfPPNN/zjH/9g6dKlXH/99bz//vsATJkyhVatWrFixQr69u3Lpk2b8uy7VatWHDp0\niH379gHu8Dp06BA33HBDgf8GZ+X3fA6Hg9GjRzNx4kSWLl2aK0gnTZpElSpVWLJkCcHBwZ59/fDD\nD7z99tvMnj2bJUuWULVqVaZOneq5f8mSJbz44ossX76cY8eOsXz58jzl+fDDDzl9+jRLliwhOTmZ\nBQsWeAJ4/PjxdO/eneXLlzNo0CBGjRp1wfUXcvz4cRo3bsycOXMuWNcNGzYwf/58PvvsMxYuXMjG\njRtZtmwZ3bp144svvvDsb/ny5XTv3v2izyuSH4W3lChxcXFYre63/bhx4xg/fjwANWrUICoqiv37\n9+fZpkyZMnTq1AmAJk2acODAAa/77tGjBzabjUqVKlGhQgUOHjzI+vXrue6664iOjiYkJIQ77rjD\n67YVKlRg48aNVK5cGYAWLVp4whagbt26NG3aFICYmBhPwG7YsIGbbroJgGbNmlGnTp08+w4ODqZ9\n+/asWrUKgBUrVtCpUyfsdnuB/wZn5fd8drud//73vzRv3txr+b1ZvXo1Xbp0oUKFCgDcddddfP/9\n95774+LiKF++PHa7nQYNGnj9UdGvXz9ee+01LBYL5cqVo379+uzfv5+srCzWrl3LzTffDEDHjh35\n+OOP811/MTk5OZ5DBxeq6zfffENcXBxhYWEEBwcze/ZsOnfuTPfu3Vm0aBEul4vjx4+zbds22rdv\nf9HnFcmPjnlLiVKuXDnP7a1bt3pamlarlZSUFFwuV55twsPDPbetVqvXxwCEhYV5bttsNpxOJydP\nnsz1nJUqVfK6rdPp5KWXXmLVqlU4nU5Onz5N7dq1vZbh7L4BTpw4ket5y5Yt63X/Xbp0YdasWfTt\n25cVK1Z4umwL+jc460LPN3v2bJKTk8nOziY7OxuLxZLvfgCOHTtGdHR0rn0dPXr0onU+3549e5g8\neTK//fYbVquVQ4cOcfvtt3P8+HFcLpdnHxaLhTJlynD48GGv6y/GZrPlqnd+dU1LS8tVp9KlSwNw\n7bXXEhQUxLp16zh06BBt2rQhNDT0os8rkh+1vKXEeuyxx+jSpQtLly5lyZIlREREFPpzhIWFkZGR\n4Vk+cuSI18ctWrSIVatWMWfOHJYuXcqwYcMKtP+yZcvmGkl/9pjxX7Vt25affvqJPXv2sGfPHlq2\nbAlc+t8gv+fbtGkTb731Fq+//jpLly7lqaeeumjZK1asyPHjxz3Lx48fp2LFihfd7nxPPvkk9evX\nZ/HixSxZsoRGjRoBEBERgcViIS0tDQDDMNi7d2++6w3DyPPD7MSJE16f80J1jYiI8Owb3GF+drl7\n9+4sWbKEJUuWeHovRC6XwltKrKNHj9K0aVMsFgvJyclkZmbmCtrC0KxZM9auXcuxY8fIzs7m008/\nzbcs1apVIzIykrS0NBYvXszp06cvuv/mzZt7jgVv2rSJ33//3evjgoODadOmDc8//zwdO3bEZrN5\nnvdS/gb5Pd+xY8eoUKECVatWJTMzk+TkZDIyMjAMA7vdTkZGBg6HI9e+2rVrx/Llyz3h9tFHHxEX\nF3fROp/v6NGjNG7cGJvNxvfff8/evXvJyMggODiY1q1bk5ycDMC3337LgAED8l1vsViIiorip59+\nAtw/prKysrw+54Xq2qFDB1atWsWJEydwOBwMGTKE7777DoCbb76ZFStWsHnz5kuup8hfKbylxBo+\nfDhDhgyhR48eZGRk0LNnT8aPH59vAF6OZs2akZCQQEJCAn369Mn3OOfNN9/M8ePHiY+PZ+TIkYwY\nMYJDhw7lGrXuzWOPPcZXX31Fp06d+OCDD7jhhhvyfWyXLl1YsWIF3bp186y71L9Bfs/Xtm1boqOj\n6dSpE/369aNv376Eh4czbNgwGjZsSLly5WjdunWu8QLNmjVjwIAB3HPPPXTt2pVTp07x8MMPX7C+\nfzVo0CCeffZZbr75ZtatW8fQoUN5+eWX2bhxI08//TRfffUVHTt2ZNq0aUyZMgUg3/WDBw/mvffe\n4+abb2b37t3Uq1fP63NeqK7Nmzenf//+3HbbbXTv3p2YmBjP8fWGDRtSvnx52rRpQ6lSpS6pniJ/\nZdH1vEV8yzAMzzHR1atXM23atHxb4BLY/vnPf9K7d2+1vOWKqeUt4kPHjh2jZcuW/PHHHxiGweLF\niz2jlKVk2bhxI3/88Qdt27b1d1EkAGi0uYgPRUZGMmLECO677z4sFgt16tQp0HnFEljGjBnDpk2b\neP755z2nKopcCXWbi4iImIx+AoqIiJiMwltERMRkTHPMOyXllL+LUOgiIkJJSyvc84r9TXUyj0Cs\nVyDWCQKzXqpTwURFhXtdr5a3H9ntNn8XodCpTuYRiPUKxDpBYNZLdboyCm8RERGTUXiLiIiYjMJb\nRETEZBTeIiIiJqPwFhERMRmFt4iIiMkovEVEREzGNJO0FEcvv/wiP/+8k2PHjnLmzBmqVq1G2bLl\nmDTp+Ytuu2jRQqpWjaJ585Ze758+fSp33ZVI1arVCrvYIiJicqa5MElhzLCWnGxn2rRgdu2y0qCB\nixEjsklIcFzxfhctWshvv+1m6NARl7RdVFR4wM0cpzqZRyDWKxDrBIFZr0Cq07lssdGggbPQsgXy\nn2GtxLS8k5PtDBxY2rO8c6ftz+XMQvsjn7Vp0wY++mgOGRkZDB36MJs3b2T16pW4XC5atWpNv34D\nePvtGVSvXpmoqGosWPAxFouVvXv/j3btOtKv3wCGDh3AI4+M4quvVnL6dDq//76XP/7Yz7BhI2nV\nqjVz5rzHihXLqFq1Gg6Hg8TEe4iNbeEpw/r1a5k58w2CgoIIDw/nyScnExQUxLRpU9ixYxs2m43H\nHhtDnTr1vK4TEfE3XzS4CnufRZkt5ysxx7ynTQv2un76dO/rr9Tu3b/ywguv0KhRYwBee20mb775\nHosXf8Hp0+m5Hrtjx3b+/e8JvPHGu/znP/Py7OvIkcNMmfISw4c/yuefL+DkyRMsWPAJM2a8w6OP\njuaHHzbl2ebUqVMkJT3FK6+8SWhoGdauXcP69Ws5cuQwb775HgMHDmHlyuVe14nIpUlOthMXF0qV\nKmHExYWSnFw47aKz+7XbKbT9+qKsvtrnwIGl2bnThtNp8YTilezbF/ss6mw5q8S0vHft8v47Jb/1\nV6pevfoEB7tfvFKlSjF06ABsNhvHjx/n5MmTuR7bsGEjSpUqle++mjVrDkB0dDTp6ens37+POnXq\nEhJSipCQUjRu3CTPNuXLl+fZZ5/C6XRy4MAf/O1vfyct7RhXX30NAM2bx9K8eSwffPB+nnUixYGv\nDnOZpeXli/2aZZ9w4VC83P36Yp9FnS1nlZiWd4MGrktaf6WCgoIAOHToIPPmfcDUqS/zyitvUrly\n5TyPtdkuPJn9+fcbhoFhgNV67qWzWPJu88wzE3n44VG88sqbtGlzIwBWqw3DyF1fb+tE/M0XLSRf\n7ddXLS9f7Ncs+wTfhKIv9lnU2XJWiQnvESOyva4fPtz7+sJy/PhxIiIiCA0N5eeff+LQoUPk5ORc\n0T6rVKnCb7/txuFwkJaWxk8/7czzmNOn06lUqTKnTp1i06aN5OTk0LhxDJs2bQBg166fmDr1Wa/r\nRPzNTIHoq5aXWcLLV/X3RSj6Yp/+ypYSE94JCQ5mzMgkJsaJ3W4QE+NkxgzfDigAqF+/AaVLhzJo\nUD9WrlzGrbfefsUBGRlZgfj4rvzzn32YPn0KMTFN8rTeb7/9LgYN6s9zzz3NPff0Yc6c96hevSZX\nXVWbwYMfYNq0Kdx22x00bx6bZ53IpSrsY55mCkRftbzMEl6+qr8vQtEX+8ydLRRZtpSoU8WKmys5\nVWLRooXEx3fFZrPRp08iL7zwMtHRlQq5hJcukE7/OCsQ6wSFV6+/HvM860q+wOLiQtm5M+/hpJgY\nJ6tXZ+S73cXqdLn7vRBf1N9X+y1O+yzI+y852c706efGJwwfXjijzQt7n2f54rsiv1PFSkzLO9Ac\nPXqUAQP68uCD/ejcuWuxCG4xh8IeweyLrmhfdUX6vuVVeL16vmjR+aKsvuzVTEhwsHp1BgcOpLN6\ndUax3ac/qOXtR4HYolOdijdftLyqVAnD6cw7atJuNzhwIN3LFgVzOS0kf7XmfC2Q3oNnqU4F36c3\nJeZUMRHxzakyDRq4vHZFX+kxz4QEh09C1Vf7FSlK6jYXKUF8MWDLX6NtRUoyhbdIMeWLWat8MTLY\nX2dyiJRk6jYXKYZ8NWvViBHZXo95X2krWV3RIkVLLe8rMHDg/XkmSHnjjVf48MM5Xh+/adMGxo0b\nBcDo0Y/kuf8//5nH22/PyPf5fv31F37/fS8ASUljyMo6c7lFl2LOV5OU+OucVBEpXArvKxAf34VV\nq3JfyGP16lV06tT5ottOnvzCJT/f11+vYt++3wF44olnCAnJfz50MTdfzpd89lSZnBxMfaqMSEmm\nbvMr0LFjZwYN6s/gwcMA+OmnnURFRREVFe31kpzn6969I+vWrWPDhnW89NJUIiMrUKFCRc8lPp9+\negIpKUfIzMykX78BVK5chc8+W8DXX68iIiKCxx8fw6xZ80hPP8UzzzxJTk4OVquV0aPHY7FYePrp\nCVStWo1ff/2FBg0aMnr0+FzPv2zZYubPn4fNZqVWrbr861//xuFw8NRTSRw+fJDg4BDGjXuCiIjI\nPOuioqKL7G9cUvlqBLeIBIaACe8JE0JYuLBwq9Ojh4MJE7LyvT8iIpKqVauxY8c2YmKasmrVcuLj\nuwLnLslZtWo1Jk58nLVr1xAaGppnHzNmvML48ROpX78Bjz46jKpVq3Hq1Emuu64l3brdzB9/7Gf8\n+NG8884crr++Fe3adSQmpqln+5kz3+Dmm2+lY8fOfPXVCt5550369x/Izz/v5IknJhEREUlCwk2c\nOnWK8PBz5wtmZmYyderLhIeHM2TIP9m9+1d27NhGhQoVmDDhaVasWMp3332D3W7Psy4h4c5C/CuL\nN746Ni0igSFgwttf4uO7snLlcmJimvL999/w+uvvAN4vyektvA8ePEj9+g0A9yU5s7KyCA8vy86d\n2/n88wVYLFZOnjyR7/P//PNOHnxwKACxsS14772ZAFSrVoMKFSoCULFiFKdPp+cK77JlyzJmzEgA\n9u79P06cOM7PP/9EixZ/B6BTpy4ATJkyOc86yauwLzPp3jbTdJOJiEjRCJjwnjAh64KtZF+Ji2vP\nrFnvEB/fhRo1alK2bFnAfUnO55+fRq1atXnhhfwvRHL+pT3PTna3fPkSTp48yauvzuTkyZM88MC9\nFyiBxbNdTo4Di8W9v79eqOT8ifRycnJ44YXneO+9uVSoUJFRo0b8uY0Vlyv3hHve1kluvhoZrhHc\nIpIfDVi7QqGhZahbtz6zZr3r6TIH75fk9KZixSh+/30PhmGwefNGwH0Z0SpVqmK1Wvn661WebS0W\nC06nM9f251/S84cfNtKoUeOLljkj4zQ2m40KFSpy+PAhfvppJw6Hg0aNYti0aT0A33//LbNmveN1\nneTmq5HhIiL5CZiWtz/Fx3flqaeSSEqa6Fl39pKcNWrU5J57+vDOO28yYMDgPNsOGDCYceP+ReXK\nVTwXF2nXrgOjRz/Cjh3b6N79FqKjo3n33be45pprmTbt+Vzd7w888CDPPDORhQs/xW4PYsyY8Tgc\nF26tlStXnr///XoeeKAP9erV5x//uJeXXnqBd96Zw4YN6xg6dAA2m51x4yZQvnxEnnWSmy9HhouI\neKMLk/iRJuY3B39cZrIolMTXyqwCsV6qU8H36Y2aBiJXSHN7i0hRU3iLXCHN7S0iRU3HvEUKgUaG\ni0hRUstbRETEZBTeIiIiJqPwlhLHF9fJFhEpSvrWkhLFV7OhiYgUJbW8pUTRbGgiEggU3lKiaDY0\nEQkE+saSEiW/62HrOtkiYiY+De9JkybRs2dPEhMT+fHHH3Pdt2LFCu644w569erFnDlzfFkMEQ/N\nhiYigcBn4b1u3Tr27t3LvHnzePrpp3n66ac997lcLiZOnMhbb73FBx98wFdffcWhQ4d8VRQRD82G\nJiKBwGejzdesWUOnTp0AqFu3LidOnCA9PZ2wsDDS0tIoW7YskZGRALRs2ZL//ve/3H777b4qjoiH\nZkMTEbPzWcs7NTWViIgIz3JkZCQpKSme26dPn2bPnj3k5OSwdu1aUlNTfVUUERGRgFJk53mff+VR\ni8XC5MmTGTt2LOHh4VSvXv2i20dEhGK3573sotnld7k3M1OdzCMQ6xWIdYLArJfqdPl8Ft7R0dG5\nWtNHjhwhKirKs3zdddcxd+5cAKZOnUq1atUuuL+0tOJ7XeTLpevZXlhysp1p04LZtctKgwYuRozI\n9kt3dyC+ThCY9QrEOkFg1kt1Kvg+vfFZt3nr1q1ZunQpANu3byc6OpqwsDDP/Q888ABHjx4lIyOD\nr776ilatWvmqKGJCZ2dC27nThtNp8cyEpqlMRUR82PKOjY2lSZMmJCYmYrFYSEpKYsGCBYSHhxMf\nH8/dd99Nv379sFgsDBgwwDN4TQQuPBOaBpuJSEnn02bMo48+mmu5UaNGntudO3emc+fOvnx6MTHN\nhCYikj99E0qxpJnQRETyp/CWYkkzoYmI5E/hLcWSZkITEcmfhu5KsaWZ0EREvFPLW0RExGQU3iIi\nIiaj8BYRETEZhbeIiIjJKLxFRERMRuEtIiJiMgpvERERk1F4i4iImIzCW0RExGQU3lIokpPtxMWF\nYrdDXFyorrstIuJD+oaVK5acbGfgwNKe5Z07bX8uay5yERFfUMtbrti0acFe10+f7n29iIhcGYW3\nXLFdu7y/jfJbLyIiV0bfrnLFGjRwXdJ6ERG5MgpvuWIjRmR7XT98uPf1IiJyZRTecsUSEhzMmJFJ\nTIwTux1iYpzMmKHBaiIivqLR5lIoEhIcJCQ4iIoKJyUlw9/FEREJaGp5i4iImIzCW0RExGQU3iIi\nIiaj8BYRETEZhbeIiIjJKLxFRERMRuEtIiJiMgpvERERk1F4i4iImIzCW0RExGQU3iIiIiaj8BYR\nETEZhbeIiIjJKLxLoORkO3FxoVSpEkZcXCjJybq4nIiImehbu4RJTrYzcGBpz/LOnbY/l3X9bRER\ns1DLu4SZNi3Y6/rp072vFxGR4kfhXcLs2uX9Jc9vvYiIFD/6xi5hGjRwXdJ6EREpfhTeJcyIEdle\n1w8f7n29iIgUPwrvEiYhwcGMGZnExDix2w1iYpzMmKHBaiIiZqLR5iVQQoJDYS0iYmJqeYsUkjNn\nID3d36UQkZJALW+RK7Rjh5X33w/ik0+COH0aGjVy0aKFk9hYF3/7m5MGDVxY9TNZRAqRwlvkMpw5\nAwsX2nn//SDWrXN/jKpUcdG0qYstW2zs3Glj9mz3Y8PDDa691snf/ub+FxvromJFo8jLbBiQkmJh\nzx4L+/dbiYqCoCAbFSoYVKjgonx59CNDio3UVAtvvBHExx8HERVl0LSpi6uvdtK0qYumTZ2Ehfm7\nhP6l8JZi6/BhCzNnBrFli40bbnDSubODxo1dWCz+K9Nvv1mYNSuYjz6yc+yYO+nat3dw3305xMc7\nsNvB4YCdO61s3Gj785+Vb76x88035z5utWq5iI110qKFO9CbNHERXAjz5OTkwP79FvbssbJ3r5U9\ne6zs2XNu+fTpv/7xQj23rFaDyEjjzzDP/a9ixbzrIiMNgoKuvMwi5zt82MJrrwXz/vtBZGRYKF/e\n4JdfLGzdauPDD91vOIvFoHZtg6ZNnVx99blQj44u+h/F/mIxDMMUtU1JOeXvIhS6qKjwgKtXYdRp\n9273h3fevCCys3OHTY0aLjp3dhAf76B1aychIVf0VAUSERHOnDmZvP9+EKtXuwO4QgUXvXrlcO+9\nOdSuffGPUFoabN5sOy/QbZw4ca5uISEGV1/t7mY/G+jVqhlef6ikp/NnKLuD+VxIW9m/34LTmXej\n0FCDWrVcf/4zqFHDRalSpdi7N4ujRy2kplo4etTC0aNWjh2zkJZWsF9I5cqdH+juL8+qVQ2qVnVR\npYpBlSru20XVSgrEzxQEZr3+WqeDBy288kows2cHceaMhSpVXAwbls0//pFDUBD8+quVrVutbN1q\nY9s2K9u22Th+PPf7tFIlF1df7fKEetOmTmrV8v45Koo6FdY+vVF4+1FJ+EBeik2brLzySjBffmnH\nMCzUquVi8OBsunRx8P33NpYts7NqlZ2TJ92fxDJlDNq1c9C5s4NOnZxERRXuW/ngQQuzZwcxd24I\nBw64111/vbuVffPNjiv64eByuVvxGzbY2LTJHeY7dlhzBW+lSu7Wef36Lg4cOBfWqane+7ajotzB\nXKuWi6uucuUK66iovF9gF3qtHA44duxsoFv+EvB51x87ZsHlyv8bMjz8XKCf+999u3Jl9//ly3PF\nX7KB+JmCwKzX2Trt22fh5ZeDmTvX/WO9Rg13aCcm5lzwM2YY7l6mrVttbN3qDvNt26z88Ufuz0d4\nuJErzJs2ddGwocsnvUYKby8C7Y0Lgf2BLCjDgFWrbLzySjDff+9u1TZv7uShh7K56SYHNlvux+fk\nwNq1NpYutbNsmZ3/+z/3B9ViMYiNdbfKO3d2EBNzed3rLhd8/bWN998PYulSO06nhfBwuOuubPr2\nzaFxY9/NRHf6NPz4o+3PQLeyYYONw4fPfRHZbAbVq5/fgs4d1pfaui3M95/LBcePw5EjVg4etHDw\noIUDB6ye/w8csHDwoDVPS+l8pUsbfwl39/81arho2dJJuPfvMJ/VqTgJxHqdOhVOUlI2H30UhMPh\n/rE+YkQWd93luKJgPXrUwrZt1lyB/uuv1lw/LoOD3Z+bsz8iq1Rx5eoxutwfkwET3pMmTWLLli1Y\nLBbGjh1Ls2bNPPd98MEHfP7551itVpo2bcq///3vC+4r0N64EJgfyILWKScHPvvMziuvBLNjhzuh\n27d38NBD2bRu7SzQh8Yw3F3sZ4N83Tqbp+Vavfq5IC9I9/rRoxY+/NDOrFnB7NnjDsyrr3Zy3305\nDBhQiszMon+dDAMOHHAfr65WzUX16gb2Qhyl4o/33+nTcOiQO8jPBrr7/3Mh761nISjIoFUrJ126\nuF/Tq67y/rXljzqdOgXffWdn9WobR49aiIk5dwy2cuXC6bINpO+K3bstTJsWwvz5QTidUK+ekxEj\nsrn9dkehvr/Pd/q0exzK+V3ue/ZYL3h4qFQp7z8mzw/5ihWNXIM8AyK8161bx9tvv82MGTPYvXs3\nY8eOZd68eQCkp6dzyy23sGzZMux2O/369WPYsGE0b9483/0Fyhv3fIH0gTzrYnU6fRrmzg3i9deD\n2b/fis1mcOutDoYMyebqq6+sVZuWBqtWuYN85cpz3euhoe7u9S5dHHTs6PQMajEMWLfOxnvvBbFw\noZ3sbAulShncdpuD++7L5tpr3a33QHydoPjWKyvrXMAfPGjh55+trFhh58cfz3XDNGzo/PPHmXuM\nwNkemqKok8MBP/xgZfVqd2Bv3GjzOs4AoGJFl2eU9Nlu2zp1jEse1V9cX6tL8fPPVl58MZhPP7Xj\nclmIiYHhwzO55Za8PWxFJSPD+4/J82+npOT/YgUFGVSufK7lfv31Qdx//6lCPWsjv/D22WjzNWvW\n0KlTJwDq1q3LiRMnSE9PJywsjKCgIIKCgsjIyCA0NJTMzEzKlSvnq6JIMZCaauHtt4N4551g0tIs\nlC5t8MAD2Tz4YDY1axbO78eICLjjDgd33OEgJ8cdzGdb5YsWBbFoUZCne71lSyerVrlP6QL3r/++\nfXO4++4cIiIKpThymUJC4Kp2knZyAAAgAElEQVSrDK66yulZN3p0NgcPWli+3P16fvONjZdfDuHl\nlyEy0kXHju5W+V13+aZMe/daPGH93Xd2z2BDq9Xg2mtdxMU5aNfOSfXqLnbssOY6Duve7txXbWio\nQZMmuQO9USNXkQy+9Ift292hvXCheyxLkyZOHnkkm/vuK83Ro/6d6TE0FOrUMahTx5nvY7Kz3SPg\nzx0GOj/k3es2bLDhcllYuBBuu81ChQq+Pxrts/BOTU2lSZMmnuXIyEhSUlIICwsjJCSEIUOG0KlT\nJ0JCQujevTu1a9f2VVHEj/bssfDGG8F8+GEQmZkWIiNdPPZYNv365fj0DR4UBK1bO2nd2smTT2bl\n6l5fu9bdWrLbDW65JYe+fXNo06ZgXfXiP1WqGPTpk0OfPjlkZMB337l/nC1fbueTT9yT5AwaBC1b\nlvZ0r9eqdXnvsZMnz3WFr15t9xxKAahZ08Wtt+YQF+ekbVsH5cvn3rZ6dSedO58LgxMnYNs2W66R\n0ps2WVm//lxz0243aNDAleu0p6ZNnZQte1nFLxZ+/NHK1KnBLF7sPoB9zTVORo7MoksX92fNLHMK\nBAdDjRoGNWrkH/AOh3sOhaioMOz2ohlG5rNu8/HjxxMXF+dpfffq1YtJkyZRu3Zt0tPT6dmzJ7Nn\nzyYsLIy+ffuSlJREo0aN8t2fw+HEbvdT34pcss2b4bnn4OOP3YOZatWCkSPh/vuhTBn/li0tDf73\nP2jeHKpU8W9Z5MoZBmzaBF98AQsXwsaN5+5r3Bh69HD/a9WKfLtnHQ5Yvx6WLYPly93vD+ef39Vl\ny0KHDhAfD507Q926Vz4q/swZ2LbN/Tk5++/HH93duOerUweaNXOXISTk8v4FB1/8MYX5w3XtWpg4\nEb780r3csiU8/jh07Vq4z1PS+azlHR0dTWpqqmf5yJEjREVFAbB7925q1KhBZGQkAC1atGDbtm0X\nDO+0tIx87zOrQDiOdT7DgK1bw5k40cHXX7vfWk2aOBk6NJtbb3UPRsnIyPsF5Q8tWrj/T0m5+GMD\n7XU6K5DqVbMmDB4MSUnhbN2anqt7/bnnLDz3HEREGHTs6B770L69g2PH3F3hX39t49tvz42RsFrd\nh1bOdoXHxjpzjX4+72vtilx1lfvfbbe5l51O+O23c+cyu7vdrXz6qe+bqEFBxp8h7/7//NvuHwC5\nb5/9UXD+7ZAQg02bbJ5DBC1bOhg5Mpsbb3S3tP/6dwuk999ZRTlgzWfh3bp1a15++WUSExPZvn07\n0dHRhP15Lku1atXYvXs3Z86coVSpUmzbto24uDhfFUV8zOGAL790jxzfsgXATtu27kFo7durO1qK\nVuXKBvfe655A52z3+rJl7jCfPz+I+fODsFqNXKcO1azp4rbbcmjXzt0V7o8hODYb1K/von59F7ff\n7j4WbBjuMyFKlw7j4MF0srIsZGfj+f/821lZkJ2d+7b7//Nv5173132cfzsjw5Lr+QqqbVt3aN9w\nQ/7dzHLlfBbesbGxNGnShMTERCwWC0lJSSxYsIDw8HDi4+Pp378/ffr0wWazce2119LibFNITCMz\nEz76KIjXXgtm714rVqvBnXfCP/95mmuv9d350CIFFRoKnTu7j0EbRhZbt1pZutTd2q5QwaBdOyft\n2jkKNEueP1gsULGiQVSUe1Ii8E85DcN9eqe3Hwvn/xgoV84gJkaf/aKgSVr8yKzdRmlp8O67wcyc\nGURqqpWQEIPExBwGDcrm+uvDTFmnCzHr63QxgVivQKwTBGa9VKeC79MbXZhECmz/fgszZrjnHs7I\ncP/KfvjhLPr3zylRFwQQEfE3hbdc1I4dVl59NZjkZDsOh4WqVV2MHp1F7945Jf6yfCIi/qDwFq8M\nA9ascc85vmKF+23SqJGTIUOySUhwFMrlK0VE5PIovCUXlwsWL3aPHN+40X1SbMuW7jnHO3Z0mmZi\nBRGRQKbwFsA9cvSTT4J49dVgdu92J3S3bjkMHZrN3/+u0aMiIsWJwruEO3kS3nsvmDffDOLIEStB\nQQb33JPN4ME51K+v0BYRKY4U3iXUwYMW3nwzmPffDyI93UJ4uMHQoVkMGJBD5coaOS4iUpwpvEuY\nXbusvPaa+yIOOTkWKlVy8fDD2fTtm23qiyCIiJQkCu8SYv16Ky+/HMySJe5JmuvWdTFkSBZ33ZUT\nsJciFBEJVArvYiw52c60acHs2mWlQQMXI0a4T9MqKJcLVqyw8fLLwaxd636p//Y394VCunVzaOS4\niIhJKbyLqeRkOwMHlvYs79xp+3M586IBnp0NCxbYee21YH76yX26V3y8g6FDs2nZUhcKERExO4V3\nMTVtmvdZUKZPD843vNPTYfbsIGbMCObAASt2u8Hdd+cweHC2LhYgIhJAFN7F1K5d3vu0va0/csTC\nzJlBvPtuMCdOWAgNNRg4MJuBA7OpXl0jx0VEAo3Cu5hq0MDFzp02r+vP+u03C6+9Fsy8eUFkZVmo\nWNHF6NHZ3H9/NhERRVlaEREpSgrvYmrEiOxcx7zPGj48m82brbzySjBffGHHMCxcdZWLwYOzSEzM\noXTeTUREJMAovIsp93HtTKZPd482r1/fRXy8g9mzg/juO/fL1qyZk4ceyubmmx3Y8jbSRUQkQCm8\ni7GEBAc9ejj47DP3hUJeesl9QnZcnPtCIW3bauS4iEhJpPAuxhwOuPfe0qxcacdqNUhIcF8o5Oqr\nNXJcRKQkU3gXU4YBY8aEsHKlnRtvdDBlyhlq1dLIcRERUXgXWzNmBPH++8HExDh5771MwsL8XSIR\nESkuNEFmMbRkiY2kpBAqVXLxwQcKbhERyU3hXcz8+KOVBx8sTenSMGdOJtWqqatcRERyU7d5MXLg\ngIXevUuTmQnvvnuGa67RwDQREclLLe9iIj0d7rmnNIcOWUlKyuKmmwp+9TARESlZFN7FgNMJDz5Y\nmu3bbdx7bzaDBuX4u0giIlKMKbyLgaSkEJYtsxMX52Dy5CxNvCIiIhek8Pazt98O4s03g2nY0Mnb\nb2cSFOTvEomISHGn8PajRYvg3/8OoWJF9ylhZcv6u0QiImIGCm8/2b7dSs+eEBwMs2ZlUrOmTgkT\nEZGC0alifnD4sPuUsPR0mDnzDC1a6JQwEREpuAK1vLdt28ZXX30FwIsvvkjfvn3ZsGGDTwsWqE6f\ndl9s5I8/rEyaBLfcolPCRETk0hQovJ966ilq167Nhg0b2Lp1K+PHj+ell17yddkCjssFQ4aU4ocf\nbCQm5jB6tL9LJCIiZlSg8A4JCaFWrVqsXLmSu+++m3r16mG16nD5pZo4MYRFi4Jo08Z9lTCdEiYi\nIpejQAmcmZnJ4sWLWbFiBW3atOH48eOcPHnS12ULKLNnB/Hqq8HUq+fknXcyCQ72d4lERMSsChTe\njzzyCAsXLuThhx8mLCyM2bNnc9999/m4aIHj669tjBoVQmSk+5Sw8uX9XSIRETGzAo02b9myJU2b\nNiUsLIzU1FRatWpFbGysr8sWEH7+2Ur//qWx2eC9985Qu7ZOCRMRkStToJb3xIkTWbx4McePHycx\nMZE5c+YwYcIEHxfN/FJSLNxzT2lOnrQwbdoZWrZ0+rtIIiISAAoU3jt27OCuu+5i8eLFJCQkMG3a\nNPbu3evrsplaZib06VOa33+38thjWdx5p04JExGRwlGg8DYMd1fv6tWr6dChAwDZ2dm+K5XJuVww\nfHgpNm60ceedOTz6qP5WIiJSeAoU3rVr1+amm27i9OnTNG7cmE8//ZRy5cr5umym9eyzwXz6aRDX\nX+/gxRd1SpiIiBSuAg1Ye+qpp9i1axd169YFoF69ejz33HM+LZhZffSRnRdfDKFWLRfvvXeGkBB/\nl0hERAJNgcL7zJkzrFq1iunTp2OxWGjevDn16tXzddlM5/vvbYwcWYry5Q3mzs2gQgWNLBcRkcJX\noG7z8ePHk56eTmJiInfffTepqamMGzfO12UzlbQ06N+/FIYB776bSb16Cm4REfGNArW8U1NTeeGF\nFzzL7du359577/VZoczoxRdDOHbMyuOPn6F1a50SJiIivlPg6VEzMzM9yxkZGWRlZfmsUGazd6+F\nd94JomZNF//8Z46/iyMiIgGuQC3vnj170q1bN5o2bQrA9u3bGT58uE8LZibPPBNCdraFsWM1QE1E\nRHyvQOF955130rp1a7Zv347FYmH8+PHMnj3b12UzhR9+sLJgQRDXXOPktts0EYuIiPhegcIboEqV\nKlSpUsWz/OOPP/qkQGZiGPDEE+6mdlJSFrpKqoiIFIXLjpuzs66VZCtW2Pj+ezvx8Q5SUizExYVS\npUoYcXGhJCcX+HeRiIjIJbnshLGU8GnDHA548skQrFaDVq0cDBxY2nPfzp22P5czSUhQV7qIiBSu\nC4Z3XFyc15A2DIO0tLSL7nzSpEls2bIFi8XC2LFjadasGQCHDx/m0Ucf9Txu3759jBw5kh49elxq\n+f1m3rwgfv7Zxj33ZPPJJ0FeHzN9erDCW0RECt0Fw3vu3LmXveN169axd+9e5s2bx+7duxk7dizz\n5s0DoFKlSp4Bbw6Hg3vvvddzwRMzOH3aPX956dIGo0ZlExtbxuvjdu3SQXARESl8FwzvatWqXfaO\n16xZQ6dOnQCoW7cuJ06cID09nbCwsFyPS05OpkuXLpQp4z0Ai6MZM4I5dMjKI49kUaWKQYMGLnbu\ntOV5XIMGLj+UTkREAp3PRlWlpqbSpEkTz3JkZCQpKSl5wvuTTz7hnXfeuej+IiJCsdvzBmRRO3IE\nXnkFoqIgKSmEsmVDePxx6NUr72PHj7cRFRV+wf1d7H4zUp3MIxDrFYh1gsCsl+p0+YpsSLS30emb\nN2+mTp06eQLdm7S0DF8U65KNGRNCenow//73GbKyckhJgY4dYcYMO9OnB7Nrl5UGDVwMH55Nx44O\nUlLy31dUVDgpKaeKrvBFQHUyj0CsVyDWCQKzXqpTwffpjc/COzo6mtTUVM/ykSNHiIqKyvWY1atX\n06pVK18VodDt3m1h1qwg6tRx0adP7mlQExIcGpwmIiJFwmcjqlq3bs3SpUsB93Sq0dHReVrYW7du\npVGjRr4qQqF76qkQHA4L48ZlEeR9gLmIiIjP+azlHRsbS5MmTUhMTMRisZCUlMSCBQsIDw8nPj4e\ngJSUFCpUqOCrIhSqdeusfPllEC1aOOneXS1sERHxH58e8z7/XG4gTyt74cKFvnz6QuOeBrUUABMm\nnKGEz08jIiJ+phORC+DLL+2sX2+je/ccrrtOp3+JiIh/KbwvIifHfazbZjMYN07XMBcREf9TeF/E\n7NlB/PablT59cqhbVxdjERER/1N4X8CpUzBlSjBlyhiMHJnt7+KIiIgACu8LevXVYFJTrTz0UDbR\n0Wp1i4hI8aDwzsehQxZefz2YSpVcDByoVreIiBQfCu98PPdcMJmZFv71r2xMdM0UEREpARTeXvz0\nk5W5c4No1MhJYmLOxTcQEREpQgpvLyZODMHlsjB+fBb2Irt0i4iISMEovP/iu+9sLF9up3VrB506\nOf1dHBERkTwU3udxueDJJ0MASErK0jSoIiJSLCm8z/Ppp3Z++MHG7bfn0Ly5pkEVEZHiSeH9p6ws\nmDQphKAggzFjNA2qiIgUXwrvP737bhC//26lX78crrpKE7KIiEjxpfAGjh+HF14IoWxZg4cfVqtb\nRESKN4U3MH16CMePWxgxIovISH+XRkRE5MJKfHjv22dh5swgqld38cADmpBFRESKvxIf3pMnh5CV\nZWH06CxKlfJ3aURERC6uRIf31q1W5s+307SpkzvvdPi7OCIiIgVSYsPbMGDChBAMw0JSUhbWEvuX\nEBERsymxkfXVVza+/dZO+/YO4uI0DaqIiJhHiQxvp9M9DarFYvD44zo1TEREzKVEhvc339jYscNG\nz54OmjTRNKgiImIuJfKCl9dc42TYsCwGDdKpYSIiYj4lMrwjI2HcuGx/F0NEROSylMhucxERETNT\neIuIiJiMwltERMRkFN4iIiImo/AWERExGYW3iIiIySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiM\nwltERMRkFN4iIiImo/AWERExGYW3iIiIySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiMwltERMRk\nFN4iIiImo/AWERExGYW3iIiIySi8RURETEbhLSIiYjJ2X+580qRJbNmyBYvFwtixY2nWrJnnvoMH\nD/LII4+Qk5NDTEwMTz75pC+LIiIiEjB81vJet24de/fuZd68eTz99NM8/fTTue6fPHky/fr1Y/78\n+dhsNg4cOOCrooiIiAQUn4X3mjVr6NSpEwB169blxIkTpKenA+Byudi4cSMdOnQAICkpiapVq/qq\nKCIiIgHFZ93mqampNGnSxLMcGRlJSkoKYWFhHDt2jDJlyvDMM8+wfft2WrRowciRIy+4v4iIUOx2\nm6+K6zdRUeH+LkKhU53MIxDrFYh1gsCsl+p0+Xx6zPt8hmHkun348GH69OlDtWrVGDBgAKtXr6Zd\nu3b5bp+WllEEpSxaUVHhpKSc8ncxCpXqZB6BWK9ArBMEZr1Up4Lv0xufdZtHR0eTmprqWT5y5AhR\nUVEAREREULVqVWrWrInNZqNVq1b88ssvviqKiIhIQPFZeLdu3ZqlS5cCsH37dqKjowkLCwPAbrdT\no0YN9uzZ47m/du3aviqKiIhIQPFZt3lsbCxNmjQhMTERi8VCUlISCxYsIDw8nPj4eMaOHcvo0aMx\nDIMGDRp4Bq+JiIjIhfn0mPejjz6aa7lRo0ae21dddRUffvihL59eREQkIGmGNREREZNReIuIiJiM\nwltERMRkFN4iIiImo/AWERExGYW3iIiIySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiMwltERMRk\nFN4iIiImo/AWERExGYW3iIiIySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiMwltERMRkFN4iIiIm\no/AWERExGYW3iIiIySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiMwltERMRkFN4iIiImo/AWEREx\nGYW3iIiIySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiMwltERMRkFN4iIiImo/AWERExGYW3iIiI\nySi8RURETEbhLSIiYjIKbxEREZNReIuIiJiMwltERMRkFN4iIiImo/AWERExGYW3iIiIySi8RURE\nTEbhLSIiYjIKbxEREZOx+3LnkyZNYsuWLVgsFsaOHUuzZs0893Xo0IHKlStjs9kAmDJlCpUqVfJl\ncURERAKCz8J73bp17N27l3nz5rF7927Gjh3LvHnzcj3mrbfeokyZMr4qgoiISEDyWbf5mjVr6NSp\nEwB169blxIkTpKen++rpRERESgyfhXdqaioRERGe5cjISFJSUnI9JikpiV69ejFlyhQMw/BVUURE\nRAKKT495n++v4Txs2DDatm1LuXLlGDJkCEuXLqVr1675bh8REYrdbvN1MYtcVFS4v4tQ6FQn8wjE\negVinSAw66U6XT6fhXd0dDSpqame5SNHjhAVFeVZvu222zy3b7zxRnbt2nXB8E5Ly/BNQf0oKiqc\nlJRT/i5GoVKdzCMQ6xWIdYLArJfqVPB9euOzbvPWrVuzdOlSALZv3050dDRhYWEAnDp1iv79+5Od\nnQ3A+vXrqV+/vq+Kkktysp24uFCqVAkjLi6U5OQi63wQEREpFD5LrtjYWJo0aUJiYiIWi4WkpCQW\nLFhAeHg48fHx3HjjjfTs2ZOQkBBiYmIu2OouLMnJdgYOLO1Z3rnT9udyJgkJDp8/v4iISGGwGCYZ\nKVYYXRFxcaHs3Jn3uHlMjJPVq4u+W17dRuYQiHWCwKxXINYJArNeqlPB9+lNiZphbdcu79XNb72I\niEhxVKJSq0ED1yWtFxERKY5KVHiPGJHtdf3w4d7Xi4iIFEclKrwTEhzMmJFJTIwTu90gJsbJjBka\nrCYiIuZS4s6TSkhwKKxFRMTUSlTLW0REJBAovEVERExG4S0iImIyCm8RERGTUXiLiIiYjMJbRETE\nZBTeIiIiJqPwFhERMRmFt4iIiMmY5pKgIiIi4qaWt4iIiMkovEVERExG4S0iImIyCm8RERGTUXiL\niIiYjMJbRETEZOz+LkBJ8Nxzz7Fx40YcDgcDBw6kc+fOnvs6dOhA5cqVsdlsAEyZMoVKlSr5q6gF\nsnbtWoYPH079+vUBaNCgAePHj/fc/9///pcXXngBm83GjTfeyJAhQ/xV1EvyySef8Pnnn3uWt23b\nxubNmz3LTZo0ITY21rP83nvveV634mjXrl0MHjyY++67j969e3Pw4EFGjRqF0+kkKiqK559/nuDg\n4FzbTJo0iS1btmCxWBg7dizNmjXzU+m981anMWPG4HA4sNvtPP/880RFRXkef7H3anHw1zqNHj2a\n7du3U758eQD69+9Pu3btcm1T3F8nyFuvYcOGkZaWBsDx48dp3rw5EydO9Dx+wYIFTJ8+nZo1awJw\nww03MGjQIL+UPT9//S6/+uqr/feZMsSn1qxZYzzwwAOGYRjGsWPHjLi4uFz3t2/f3khPT/dDyS7f\n//73P+Ohhx7K9/5u3boZBw4cMJxOp9GrVy/jl19+KcLSFY61a9caEyZMyLXuuuuu81NpLt3p06eN\n3r17G+PGjTNmz55tGIZhjB492li0aJFhGIYxdepU44MPPsi1zdq1a40BAwYYhmEYv/76q3H33XcX\nbaEvwludRo0aZXz55ZeGYRjGnDlzjGeffTbXNhd7r/qbtzr961//MlatWpXvNsX9dTIM7/U63+jR\no40tW7bkWvef//zHmDx5clEV8ZJ5+y7352dK3eY+9ve//53p06cDULZsWTIzM3E6nX4ule/s27eP\ncuXKUaVKFaxWK3FxcaxZs8bfxbpkr776KoMHD/Z3MS5bcHAwb731FtHR0Z51a9eupWPHjgC0b98+\nz+uyZs0aOnXqBEDdunU5ceIE6enpRVfoi/BWp6SkJLp06QJAREQEx48f91fxLou3Ol1McX+d4ML1\n+u233zh16lSx7C24EG/f5f78TCm8fcxmsxEaGgrA/PnzufHGG/N0tSYlJdGrVy+mTJmCYZIJ7379\n9VcefPBBevXqxffff+9Zn5KSQmRkpGc5MjKSlJQUfxTxsv34449UqVIlV/crQHZ2NiNHjiQxMZF3\n333XT6UrGLvdTqlSpXKty8zM9HTpVahQIc/rkpqaSkREhGe5uL123uoUGhqKzWbD6XQyd+5cevTo\nkWe7/N6rxYG3OgHMmTOHPn368PDDD3Ps2LFc9xX31wnyrxfArFmz6N27t9f71q1bR//+/enbty87\nduzwZREvmbfvcn9+pnTMu4isWLGC+fPn88477+RaP2zYMNq2bUu5cuUYMmQIS5cupWvXrn4qZcHU\nqlWLoUOH0q1bN/bt20efPn1YtmxZnmM9ZjV//nwSEhLyrB81ahS33HILFouF3r1706JFC66++mo/\nlPDKFeRHoll+SDqdTkaNGkXLli1p1apVrvvM+F699dZbKV++PI0bN+bNN9/klVde4fHHH8/38WZ5\nncD9A3jjxo1MmDAhz33XXHMNkZGRtGvXjs2bN/Ovf/2LhQsXFn0hL+L87/Lzxy8V9WdKLe8i8O23\n3/LGG2/w1ltvER4enuu+2267jQoVKmC327nxxhvZtWuXn0pZcJUqVeKmm27CYrFQs2ZNKlasyOHD\nhwGIjo4mNTXV89jDhw9fUpdgcbB27VquvfbaPOt79epFmTJlCA0NpWXLlqZ4rc4XGhrKmTNnAO+v\ny19fuyNHjuTpfSiOxowZw1VXXcXQoUPz3Heh92px1apVKxo3bgy4B7T+9X1m1tcJYP369fl2l9et\nW9czMO/aa6/l2LFjxe4Q41+/y/35mVJ4+9ipU6d47rnnmDFjhmf06Pn39e/fn+zsbMD9xj47KrY4\n+/zzz3n77bcBdzf50aNHPSPkq1evTnp6Ovv378fhcPDVV1/RunVrfxb3khw+fJgyZcrkaZn99ttv\njBw5EsMwcDgcbNq0yRSv1fluuOEGli5dCsCyZcto27Ztrvtbt27tuX/79u1ER0cTFhZW5OW8FJ9/\n/jlBQUEMGzYs3/vze68WVw899BD79u0D3D8k//o+M+PrdNbWrVtp1KiR1/veeustvvjiC8A9Uj0y\nMrJYnc3h7bvcn58pdZv72KJFi0hLS2PEiBGedddffz0NGzYkPj6eG2+8kZ49exISEkJMTEyx7zIH\nd2vg0UcfZeXKleTk5DBhwgS++OILwsPDiY+PZ8KECYwcORKAm266idq1a/u5xAX312P2b775Jn//\n+9+59tprqVy5MnfeeSdWq5UOHToU6wE327Zt49lnn+WPP/7AbrezdOlSpkyZwujRo5k3bx5Vq1bl\ntttuA+Dhhx/mmWeeITY2liZNmpCYmIjFYiEpKcnPtcjNW52OHj1KSEgI9957L+BuvU2YMMFTJ2/v\n1eLUZe6tTr1792bEiBGULl2a0NBQnnnmGcA8rxN4r9fLL79MSkqK51SwswYNGsTrr79Ojx49eOyx\nx/joo49wOBw8/fTTfiq9d96+yydPnsy4ceP88pnSJUFFRERMRt3mIiIiJqPwFhERMRmFt4iIiMko\nvEVERExG4S0iImIyOlVMJIDt37+frl275pl0Ji4ujgceeOCK97927VqmTZvGhx9+eMX7EpGCU3iL\nBLjIyEhmz57t72KISCFSeIuUUDExMQwePJi1a9dy+vRpJk+eTIMGDdiyZQuTJ0/GbrdjsVh4/PHH\nqVevHnv27GH8+PG4XC5CQkI8k4e4XC6SkpLYuXMnwcHBzJgxA4CRI0dy8uRJHA4H7du3L3bXZhYx\nMx3zFimhnE4n9evXZ/bs2fTq1YuXXnoJcF+AZcyYMcyePZv777+fJ554AnBf/a5///588MEH3HHH\nHSxevBiA3bt389BDD/hoRwMAAAHFSURBVPHxxx9jt9v57rvv+O9//4vD4WDu3Ll89NFHhIaG4nK5\n/FZXkUCjlrdIgDt27Jhn+tCzHnvsMQDatGkDQGxsLG+//TYnT57k6NGjnqlfr7vuOh555BHAfanU\n6667DoDu3bsD7mPederUoWLFigBUrlyZkydP0qFDB1566SWGDx9OXFwcd911F1ar2goihUXhLRLg\nLnTM+/zZkS0WCxaLJd/7Aa+tZ28Xj6hQoQKfffYZmzdvZuXKldxxxx0kJyfne41nEbk0+iksUoL9\n73//A2Djxo00bNiQ8PBwoqKi2LJlCwBr1qyhefPmgLt1/u233wLuizS88MIL+e73u+++Y/Xq1fzt\nb39j1KhRhIaGcvToUR/XRqTkUMtbJMB56zavXr06ADt27ODDDz/kxIkTPPvsswA8++yzTJ48GZvN\nhtVqZcKECQCMHz+e8ePHM3fuXOx2O5MmTeL333/3+py1a9dm9OjRzJw5E5vNRps2bahWrZrvKilS\nwuiqYiIlVMOGDdm+fTt2u37Di5iNus1FRERMRi1vERERk1HLW0RExGQU3iIiIiaj8BYRETEZhbeI\niIjJKLxFRERMRuEtIiJiMv8Pqhvr6fEcxosAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f834e38c690>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "qX1Jp-D0eIQq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Observations**\n",
        "* The network begins to overfit after nine epochs. \n",
        "* Let’s train a new network from\n",
        "scratch for nine epochs and then evaluate it on the test set."
      ]
    },
    {
      "metadata": {
        "id": "4T39sVcJeTNB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.3.5 Retraining a model from scratch"
      ]
    },
    {
      "metadata": {
        "id": "8InS09a_dQCd",
        "colab_type": "code",
        "outputId": "b1664413-3a50-4888-889b-144c170342af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 377
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
        "model.add(layers.Dense(64, activation='relu'))\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])\n",
        "\n",
        "model.fit(partial_x_train,partial_y_train,epochs=9,batch_size=512,validation_data=(x_val, y_val))#Model with 9 epochs\n",
        "\n",
        "results = model.evaluate(x_test, one_hot_test_labels)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/9\n",
            "7982/7982 [==============================] - 1s 161us/step - loss: 2.4662 - acc: 0.5314 - val_loss: 1.6363 - val_acc: 0.6510\n",
            "Epoch 2/9\n",
            "7982/7982 [==============================] - 1s 121us/step - loss: 1.3477 - acc: 0.7211 - val_loss: 1.2601 - val_acc: 0.7210\n",
            "Epoch 3/9\n",
            "7982/7982 [==============================] - 1s 151us/step - loss: 1.0078 - acc: 0.7913 - val_loss: 1.1424 - val_acc: 0.7390\n",
            "Epoch 4/9\n",
            "7982/7982 [==============================] - 1s 152us/step - loss: 0.8025 - acc: 0.8290 - val_loss: 1.0140 - val_acc: 0.7890\n",
            "Epoch 5/9\n",
            "7982/7982 [==============================] - 1s 155us/step - loss: 0.6359 - acc: 0.8658 - val_loss: 0.9880 - val_acc: 0.8050\n",
            "Epoch 6/9\n",
            "7982/7982 [==============================] - 1s 155us/step - loss: 0.5115 - acc: 0.8919 - val_loss: 0.9317 - val_acc: 0.8050\n",
            "Epoch 7/9\n",
            "7982/7982 [==============================] - 1s 152us/step - loss: 0.4116 - acc: 0.9113 - val_loss: 0.9232 - val_acc: 0.8020\n",
            "Epoch 8/9\n",
            "7982/7982 [==============================] - 1s 154us/step - loss: 0.3370 - acc: 0.9258 - val_loss: 0.8980 - val_acc: 0.8110\n",
            "Epoch 9/9\n",
            "6656/7982 [========================>.....] - ETA: 0s - loss: 0.2834 - acc: 0.9404"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 1s 156us/step - loss: 0.2794 - acc: 0.9394 - val_loss: 0.9180 - val_acc: 0.8180\n",
            "2246/2246 [==============================] - 0s 114us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5afNa_0Qejbj",
        "colab_type": "code",
        "outputId": "ce5e4146-68a7-41d2-939a-e9d14487dd17",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "results"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.9799815752839575, 0.7836153161440804]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "metadata": {
        "id": "F3lUzYJKfZB3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.3.6 Comparison with random baseline"
      ]
    },
    {
      "metadata": {
        "id": "hwqO8UhmeoB6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import copy"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "YHWAy8RvfgMI",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "test_labels_copy = copy.copy(test_labels)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "V-nzPKXGfi53",
        "colab_type": "code",
        "outputId": "2b1cc6c2-d467-47be-ca64-c75199702c27",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "test_labels_copy"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 3, 10,  1, ...,  3,  3, 24])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "metadata": {
        "id": "Twy7tPhQfkpv",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "np.random.shuffle(test_labels_copy)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "-gxP099wfoe4",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "hits_array = np.array(test_labels) == np.array(test_labels_copy)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "R9g-zBLXfsKa",
        "colab_type": "code",
        "outputId": "0b63f7c0-1e04-4148-918d-9e3ec1db7d63",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "hits_array"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([False, False, False, ..., False, False, False])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "metadata": {
        "id": "P20qHHWZftt4",
        "colab_type": "code",
        "outputId": "c34b09ba-c8e0-422f-8c7b-2ec0e7cc4feb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "float(np.sum(hits_array)) / len(test_labels)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.19056099732858414"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "metadata": {
        "id": "rhE3gwnXgJQ2",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Thus a random classifier gets a result closer to 19%.\n",
        "\n",
        "This means our naive model does a pretty good job compared to a random baseline"
      ]
    },
    {
      "metadata": {
        "id": "LR_txyPwgQLf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 3.4 Generating predictions on new data"
      ]
    },
    {
      "metadata": {
        "id": "BhAYlIvdgVlF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### 3.4.1 Generating predictions for new data"
      ]
    },
    {
      "metadata": {
        "id": "vlGtTwQIfxYY",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "predictions = model.predict(x_test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IpJ2CUKGgZI3",
        "colab_type": "code",
        "outputId": "92542d9f-1455-401f-ecde-7ca81d69250a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "cell_type": "code",
      "source": [
        "predictions"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[3.0065852e-04, 4.1537313e-04, 1.1777668e-05, ..., 3.7633497e-04,\n",
              "        4.3693185e-06, 8.3263276e-06],\n",
              "       [9.7704055e-03, 2.8166387e-02, 2.3261255e-03, ..., 2.9999301e-05,\n",
              "        2.0469583e-04, 2.3240090e-04],\n",
              "       [8.9360019e-03, 7.2709453e-01, 3.5117064e-03, ..., 1.2005242e-04,\n",
              "        2.1959001e-03, 1.2024097e-03],\n",
              "       ...,\n",
              "       [2.2505754e-05, 2.0794429e-04, 2.6390850e-05, ..., 8.5873282e-05,\n",
              "        2.4255105e-06, 3.8740518e-06],\n",
              "       [5.2037216e-03, 2.5934726e-02, 2.2171566e-03, ..., 3.8686451e-03,\n",
              "        8.1460312e-04, 7.7010161e-04],\n",
              "       [1.0389665e-04, 8.6087537e-01, 4.4489494e-03, ..., 2.0292635e-05,\n",
              "        9.4074472e-05, 1.2326859e-04]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "metadata": {
        "id": "lpu9JfpigaWH",
        "colab_type": "code",
        "outputId": "40d80b02-fa4b-4387-f9c2-ca1ad3b718c4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "predictions[0].shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(46,)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "metadata": {
        "id": "u8M-JQPBgfdn",
        "colab_type": "code",
        "outputId": "1cb2d06e-c69d-4719-b932-7295d3ccc9c7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "np.sum(predictions[0])#vector sum"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    },
    {
      "metadata": {
        "id": "Lm8EV95BgmdX",
        "colab_type": "code",
        "outputId": "9fc3225b-2f3c-47a3-a52d-74e888655b28",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "np.argmax(predictions[0])#class with the highest probability"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "metadata": {
        "id": "6g5Q9ok3hley",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 4 Building the model with -\n",
        "1) Try to use only 4 units(instead of 64) in the second hidden layer of the above architecture.\n",
        "\n",
        "2) Try to use 2 hidden layers each with 32 units.\n",
        "\n",
        "3) Try to use 2 hidden layers each with 128 units.\n",
        "\n",
        "4) Try to use only 1 hidden layer (with 64 units).\n",
        "\n",
        "5) Try to use 3 hidden layers (each with 64 units.\n"
      ]
    },
    {
      "metadata": {
        "id": "hHC0kDnFh72T",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## **4.1 1) Try to use only 4 units(instead of 64) in the second hidden layer of the above architecture.**"
      ]
    },
    {
      "metadata": {
        "id": "NmDzqlf_gu2e",
        "colab_type": "code",
        "outputId": "22b2c27e-b3b2-4bef-a691-9a34b9d53a93",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 751
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
        "model.add(layers.Dense(4, activation='relu'))\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])\n",
        "\n",
        "model.fit(partial_x_train,partial_y_train,epochs=20,batch_size=128,validation_data=(x_val, y_val))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/20\n",
            "7982/7982 [==============================] - 2s 267us/step - loss: 2.6508 - acc: 0.4728 - val_loss: 1.8770 - val_acc: 0.5990\n",
            "Epoch 2/20\n",
            "7982/7982 [==============================] - 2s 219us/step - loss: 1.6147 - acc: 0.6378 - val_loss: 1.5800 - val_acc: 0.6390\n",
            "Epoch 3/20\n",
            "7982/7982 [==============================] - 2s 217us/step - loss: 1.3261 - acc: 0.6805 - val_loss: 1.4220 - val_acc: 0.6640\n",
            "Epoch 4/20\n",
            "7982/7982 [==============================] - 2s 228us/step - loss: 1.1391 - acc: 0.7309 - val_loss: 1.3527 - val_acc: 0.6990\n",
            "Epoch 5/20\n",
            "7982/7982 [==============================] - 2s 218us/step - loss: 1.0091 - acc: 0.7516 - val_loss: 1.3343 - val_acc: 0.6960\n",
            "Epoch 6/20\n",
            "7982/7982 [==============================] - 2s 224us/step - loss: 0.9180 - acc: 0.7650 - val_loss: 1.3140 - val_acc: 0.7160\n",
            "Epoch 7/20\n",
            "1408/7982 [====>.........................] - ETA: 1s - loss: 0.8716 - acc: 0.7706"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 222us/step - loss: 0.8486 - acc: 0.7829 - val_loss: 1.3383 - val_acc: 0.7150\n",
            "Epoch 8/20\n",
            "7982/7982 [==============================] - 2s 224us/step - loss: 0.7882 - acc: 0.7967 - val_loss: 1.3730 - val_acc: 0.7110\n",
            "Epoch 9/20\n",
            "7982/7982 [==============================] - 2s 225us/step - loss: 0.7393 - acc: 0.8053 - val_loss: 1.3703 - val_acc: 0.7130\n",
            "Epoch 10/20\n",
            "7982/7982 [==============================] - 2s 221us/step - loss: 0.6931 - acc: 0.8166 - val_loss: 1.4448 - val_acc: 0.7020\n",
            "Epoch 11/20\n",
            "7982/7982 [==============================] - 2s 221us/step - loss: 0.6548 - acc: 0.8249 - val_loss: 1.4841 - val_acc: 0.7010\n",
            "Epoch 12/20\n",
            "7982/7982 [==============================] - 2s 220us/step - loss: 0.6186 - acc: 0.8324 - val_loss: 1.4647 - val_acc: 0.7140\n",
            "Epoch 13/20\n",
            "4608/7982 [================>.............] - ETA: 0s - loss: 0.5772 - acc: 0.8385"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 222us/step - loss: 0.5864 - acc: 0.8379 - val_loss: 1.4997 - val_acc: 0.7150\n",
            "Epoch 14/20\n",
            "7982/7982 [==============================] - 2s 223us/step - loss: 0.5589 - acc: 0.8450 - val_loss: 1.5725 - val_acc: 0.7060\n",
            "Epoch 15/20\n",
            "7982/7982 [==============================] - 2s 221us/step - loss: 0.5348 - acc: 0.8530 - val_loss: 1.5510 - val_acc: 0.7170\n",
            "Epoch 16/20\n",
            "7982/7982 [==============================] - 2s 223us/step - loss: 0.5112 - acc: 0.8583 - val_loss: 1.5843 - val_acc: 0.7160\n",
            "Epoch 17/20\n",
            "7982/7982 [==============================] - 2s 215us/step - loss: 0.4915 - acc: 0.8633 - val_loss: 1.6853 - val_acc: 0.7170\n",
            "Epoch 18/20\n",
            "7982/7982 [==============================] - 2s 213us/step - loss: 0.4725 - acc: 0.8654 - val_loss: 1.6911 - val_acc: 0.7120\n",
            "Epoch 19/20\n",
            "6144/7982 [======================>.......] - ETA: 0s - loss: 0.4530 - acc: 0.8703"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 211us/step - loss: 0.4544 - acc: 0.8687 - val_loss: 1.7753 - val_acc: 0.7090\n",
            "Epoch 20/20\n",
            "7982/7982 [==============================] - 2s 212us/step - loss: 0.4422 - acc: 0.8736 - val_loss: 1.7870 - val_acc: 0.7030\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f834aae8cd0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 52
        }
      ]
    },
    {
      "metadata": {
        "id": "YSUGyqowjmtn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Observation :**\n",
        "* The network now peaks at ~70% validation accuracy, an 8% absolute drop.\n",
        "* This drop\n",
        "is mostly due to the fact that you’re trying to compress a lot of information (enough\n",
        "information to recover the separation hyperplanes of 46 classes) into an intermediate\n",
        "space that is too low-dimensional. T\n",
        "* he network is able to cram most of the necessary\n",
        "information into these eight-dimensional representations, but not all of it."
      ]
    },
    {
      "metadata": {
        "id": "HqfbCpNOkIjv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 4.2. 2) Try to use 2 hidden layers each with 32 units."
      ]
    },
    {
      "metadata": {
        "id": "sfpZQQShiXUd",
        "colab_type": "code",
        "outputId": "f12c83d1-6b3a-4892-e0e4-7a95f03fea8d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 751
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Dense(32, activation='relu', input_shape=(10000,)))# 1hidden layer\n",
        "model.add(layers.Dense(32, activation='relu'))# 2 hidden layer\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])\n",
        "\n",
        "model.fit(partial_x_train,partial_y_train,epochs=20,batch_size=128,validation_data=(x_val, y_val))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/20\n",
            "7982/7982 [==============================] - 2s 226us/step - loss: 2.1649 - acc: 0.5791 - val_loss: 1.4553 - val_acc: 0.6830\n",
            "Epoch 2/20\n",
            "7982/7982 [==============================] - 1s 175us/step - loss: 1.2261 - acc: 0.7258 - val_loss: 1.1673 - val_acc: 0.7400\n",
            "Epoch 3/20\n",
            "7982/7982 [==============================] - 1s 172us/step - loss: 0.9199 - acc: 0.7988 - val_loss: 1.0217 - val_acc: 0.7840\n",
            "Epoch 4/20\n",
            "7982/7982 [==============================] - 1s 169us/step - loss: 0.7062 - acc: 0.8504 - val_loss: 0.9482 - val_acc: 0.8060\n",
            "Epoch 5/20\n",
            "7982/7982 [==============================] - 1s 172us/step - loss: 0.5496 - acc: 0.8811 - val_loss: 0.9185 - val_acc: 0.8130\n",
            "Epoch 6/20\n",
            "7982/7982 [==============================] - 1s 168us/step - loss: 0.4332 - acc: 0.9068 - val_loss: 0.8983 - val_acc: 0.8110\n",
            "Epoch 7/20\n",
            "7982/7982 [==============================] - 1s 168us/step - loss: 0.3490 - acc: 0.9225 - val_loss: 0.9132 - val_acc: 0.8080\n",
            "Epoch 8/20\n",
            "3968/7982 [=============>................] - ETA: 0s - loss: 0.2733 - acc: 0.9398"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 1s 173us/step - loss: 0.2868 - acc: 0.9367 - val_loss: 0.9309 - val_acc: 0.8130\n",
            "Epoch 9/20\n",
            "7982/7982 [==============================] - 1s 172us/step - loss: 0.2409 - acc: 0.9439 - val_loss: 0.9775 - val_acc: 0.7950\n",
            "Epoch 10/20\n",
            "7982/7982 [==============================] - 1s 167us/step - loss: 0.2119 - acc: 0.9496 - val_loss: 0.9805 - val_acc: 0.8020\n",
            "Epoch 11/20\n",
            "7982/7982 [==============================] - 1s 167us/step - loss: 0.1854 - acc: 0.9538 - val_loss: 0.9777 - val_acc: 0.8080\n",
            "Epoch 12/20\n",
            "7982/7982 [==============================] - 1s 166us/step - loss: 0.1736 - acc: 0.9533 - val_loss: 1.0466 - val_acc: 0.7990\n",
            "Epoch 13/20\n",
            "7982/7982 [==============================] - 1s 164us/step - loss: 0.1579 - acc: 0.9540 - val_loss: 1.0686 - val_acc: 0.8000\n",
            "Epoch 14/20\n",
            "7982/7982 [==============================] - 1s 165us/step - loss: 0.1471 - acc: 0.9557 - val_loss: 1.1017 - val_acc: 0.8000\n",
            "Epoch 15/20\n",
            "7982/7982 [==============================] - 1s 166us/step - loss: 0.1416 - acc: 0.9572 - val_loss: 1.1526 - val_acc: 0.7940\n",
            "Epoch 16/20\n",
            " 128/7982 [..............................] - ETA: 1s - loss: 0.0477 - acc: 0.9922"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 1s 168us/step - loss: 0.1345 - acc: 0.9564 - val_loss: 1.0858 - val_acc: 0.8010\n",
            "Epoch 17/20\n",
            "7982/7982 [==============================] - 1s 166us/step - loss: 0.1275 - acc: 0.9589 - val_loss: 1.1325 - val_acc: 0.7960\n",
            "Epoch 18/20\n",
            "7982/7982 [==============================] - 1s 168us/step - loss: 0.1269 - acc: 0.9573 - val_loss: 1.1295 - val_acc: 0.8050\n",
            "Epoch 19/20\n",
            "7982/7982 [==============================] - 1s 169us/step - loss: 0.1213 - acc: 0.9572 - val_loss: 1.1867 - val_acc: 0.7980\n",
            "Epoch 20/20\n",
            "7982/7982 [==============================] - 1s 167us/step - loss: 0.1142 - acc: 0.9579 - val_loss: 1.2523 - val_acc: 0.7910\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f834a772dd0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "metadata": {
        "id": "35Ix32AclKpY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Observation :**\n",
        "* The network now peaks at ~79.1% validation accuracy, an 0.9% absolute drop.\n",
        "Better then the previous model."
      ]
    },
    {
      "metadata": {
        "id": "1Y_wYzZlloqZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 4.3. 3) Try to use 2 hidden layers each with 128 units."
      ]
    },
    {
      "metadata": {
        "id": "xaR_Ek2WkrMp",
        "colab_type": "code",
        "outputId": "40d9bcc6-7810-4cb0-85af-2db98918a9fb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 751
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Dense(128, activation='relu', input_shape=(10000,)))# 1hidden layer\n",
        "model.add(layers.Dense(128, activation='relu'))# 2 hidden layer\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])\n",
        "\n",
        "model.fit(partial_x_train,partial_y_train,epochs=20,batch_size=128,validation_data=(x_val, y_val))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/20\n",
            "7982/7982 [==============================] - 3s 373us/step - loss: 1.5474 - acc: 0.6761 - val_loss: 1.0562 - val_acc: 0.7670\n",
            "Epoch 2/20\n",
            "7982/7982 [==============================] - 2s 304us/step - loss: 0.7418 - acc: 0.8388 - val_loss: 0.8884 - val_acc: 0.8110\n",
            "Epoch 3/20\n",
            "7982/7982 [==============================] - 2s 302us/step - loss: 0.4427 - acc: 0.9047 - val_loss: 0.8261 - val_acc: 0.8310\n",
            "Epoch 4/20\n",
            "7982/7982 [==============================] - 2s 303us/step - loss: 0.2962 - acc: 0.9356 - val_loss: 0.8385 - val_acc: 0.8310\n",
            "Epoch 5/20\n",
            "7982/7982 [==============================] - 2s 299us/step - loss: 0.2233 - acc: 0.9466 - val_loss: 0.8652 - val_acc: 0.8270\n",
            "Epoch 6/20\n",
            "7982/7982 [==============================] - 2s 300us/step - loss: 0.1933 - acc: 0.9508 - val_loss: 0.9026 - val_acc: 0.8230\n",
            "Epoch 7/20\n",
            " 896/7982 [==>...........................] - ETA: 2s - loss: 0.1615 - acc: 0.9520"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 301us/step - loss: 0.1679 - acc: 0.9524 - val_loss: 1.1579 - val_acc: 0.7830\n",
            "Epoch 8/20\n",
            "7982/7982 [==============================] - 2s 302us/step - loss: 0.1523 - acc: 0.9545 - val_loss: 1.0411 - val_acc: 0.8080\n",
            "Epoch 9/20\n",
            "7982/7982 [==============================] - 2s 301us/step - loss: 0.1406 - acc: 0.9541 - val_loss: 1.0595 - val_acc: 0.8020\n",
            "Epoch 10/20\n",
            "7982/7982 [==============================] - 2s 306us/step - loss: 0.1379 - acc: 0.9548 - val_loss: 1.0770 - val_acc: 0.7950\n",
            "Epoch 11/20\n",
            "7982/7982 [==============================] - 2s 303us/step - loss: 0.1237 - acc: 0.9555 - val_loss: 1.0809 - val_acc: 0.7980\n",
            "Epoch 12/20\n",
            "7982/7982 [==============================] - 2s 303us/step - loss: 0.1217 - acc: 0.9541 - val_loss: 1.1440 - val_acc: 0.8010\n",
            "Epoch 13/20\n",
            "4224/7982 [==============>...............] - ETA: 1s - loss: 0.0891 - acc: 0.9645"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 303us/step - loss: 0.1138 - acc: 0.9557 - val_loss: 1.1400 - val_acc: 0.8010\n",
            "Epoch 14/20\n",
            "7982/7982 [==============================] - 2s 300us/step - loss: 0.1088 - acc: 0.9565 - val_loss: 1.1893 - val_acc: 0.8040\n",
            "Epoch 15/20\n",
            "7982/7982 [==============================] - 2s 299us/step - loss: 0.1043 - acc: 0.9553 - val_loss: 1.1448 - val_acc: 0.8060\n",
            "Epoch 16/20\n",
            "7982/7982 [==============================] - 2s 302us/step - loss: 0.1005 - acc: 0.9563 - val_loss: 1.2644 - val_acc: 0.7930\n",
            "Epoch 17/20\n",
            "7982/7982 [==============================] - 2s 300us/step - loss: 0.0960 - acc: 0.9553 - val_loss: 1.3550 - val_acc: 0.7810\n",
            "Epoch 18/20\n",
            "7982/7982 [==============================] - 2s 302us/step - loss: 0.0930 - acc: 0.9585 - val_loss: 1.3131 - val_acc: 0.7980\n",
            "Epoch 19/20\n",
            "4480/7982 [===============>..............] - ETA: 1s - loss: 0.0779 - acc: 0.9621"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 302us/step - loss: 0.0902 - acc: 0.9572 - val_loss: 1.3248 - val_acc: 0.8030\n",
            "Epoch 20/20\n",
            "7982/7982 [==============================] - 2s 304us/step - loss: 0.0854 - acc: 0.9592 - val_loss: 1.4230 - val_acc: 0.8010\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f834a5a5d10>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 57
        }
      ]
    },
    {
      "metadata": {
        "id": "mwL34_r5mEdm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Observation : **\n",
        "*  The network now peaks at ~80.1% validation accuracy. Better then the previous model."
      ]
    },
    {
      "metadata": {
        "id": "alWN6GKfmvc1",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 4.4 4)  Try to use only 1 hidden layer (with 64 units)."
      ]
    },
    {
      "metadata": {
        "id": "BGyijou2mDiF",
        "colab_type": "code",
        "outputId": "cb085398-914d-4791-af71-4287ad76b2cc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 751
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))#1 hidden layer\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])\n",
        "\n",
        "model.fit(partial_x_train,partial_y_train,epochs=20,batch_size=128,validation_data=(x_val, y_val))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/20\n",
            "7982/7982 [==============================] - 2s 303us/step - loss: 1.8764 - acc: 0.6457 - val_loss: 1.2151 - val_acc: 0.7520\n",
            "Epoch 2/20\n",
            "7982/7982 [==============================] - 2s 218us/step - loss: 0.9422 - acc: 0.8026 - val_loss: 0.9717 - val_acc: 0.8050\n",
            "Epoch 3/20\n",
            "7982/7982 [==============================] - 2s 215us/step - loss: 0.6399 - acc: 0.8730 - val_loss: 0.8698 - val_acc: 0.8270\n",
            "Epoch 4/20\n",
            "7982/7982 [==============================] - 2s 213us/step - loss: 0.4582 - acc: 0.9088 - val_loss: 0.8233 - val_acc: 0.8310\n",
            "Epoch 5/20\n",
            "7982/7982 [==============================] - 2s 213us/step - loss: 0.3395 - acc: 0.9283 - val_loss: 0.8120 - val_acc: 0.8280\n",
            "Epoch 6/20\n",
            "7982/7982 [==============================] - 2s 212us/step - loss: 0.2648 - acc: 0.9419 - val_loss: 0.8219 - val_acc: 0.8340\n",
            "Epoch 7/20\n",
            "2944/7982 [==========>...................] - ETA: 1s - loss: 0.2029 - acc: 0.9524"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 212us/step - loss: 0.2190 - acc: 0.9476 - val_loss: 0.8372 - val_acc: 0.8270\n",
            "Epoch 8/20\n",
            "7982/7982 [==============================] - 2s 211us/step - loss: 0.1855 - acc: 0.9508 - val_loss: 0.8864 - val_acc: 0.8210\n",
            "Epoch 9/20\n",
            "7982/7982 [==============================] - 2s 213us/step - loss: 0.1633 - acc: 0.9535 - val_loss: 0.9012 - val_acc: 0.8130\n",
            "Epoch 10/20\n",
            "7982/7982 [==============================] - 2s 209us/step - loss: 0.1479 - acc: 0.9553 - val_loss: 0.9252 - val_acc: 0.8130\n",
            "Epoch 11/20\n",
            "7982/7982 [==============================] - 2s 207us/step - loss: 0.1359 - acc: 0.9557 - val_loss: 0.9776 - val_acc: 0.8190\n",
            "Epoch 12/20\n",
            "7982/7982 [==============================] - 2s 210us/step - loss: 0.1295 - acc: 0.9574 - val_loss: 1.0138 - val_acc: 0.8140\n",
            "Epoch 13/20\n",
            "6912/7982 [========================>.....] - ETA: 0s - loss: 0.1186 - acc: 0.9580"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 216us/step - loss: 0.1239 - acc: 0.9572 - val_loss: 1.0143 - val_acc: 0.8110\n",
            "Epoch 14/20\n",
            "7982/7982 [==============================] - 2s 216us/step - loss: 0.1212 - acc: 0.9564 - val_loss: 1.0204 - val_acc: 0.8110\n",
            "Epoch 15/20\n",
            "7982/7982 [==============================] - 2s 214us/step - loss: 0.1166 - acc: 0.9580 - val_loss: 1.0806 - val_acc: 0.8070\n",
            "Epoch 16/20\n",
            "7982/7982 [==============================] - 2s 213us/step - loss: 0.1116 - acc: 0.9553 - val_loss: 1.0906 - val_acc: 0.8030\n",
            "Epoch 17/20\n",
            "7982/7982 [==============================] - 2s 211us/step - loss: 0.1099 - acc: 0.9572 - val_loss: 1.1015 - val_acc: 0.8110\n",
            "Epoch 18/20\n",
            "7982/7982 [==============================] - 2s 214us/step - loss: 0.1054 - acc: 0.9582 - val_loss: 1.1375 - val_acc: 0.8020\n",
            "Epoch 19/20\n",
            "6528/7982 [=======================>......] - ETA: 0s - loss: 0.1058 - acc: 0.9573"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 217us/step - loss: 0.1083 - acc: 0.9563 - val_loss: 1.1596 - val_acc: 0.8050\n",
            "Epoch 20/20\n",
            "7982/7982 [==============================] - 2s 216us/step - loss: 0.1047 - acc: 0.9583 - val_loss: 1.1148 - val_acc: 0.8050\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f834a591a90>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "metadata": {
        "id": "ohpKvSaYnE6Q",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Observation :**\n",
        "* The network now peaks at ~80.5% validation accuracy. Better then the previous model."
      ]
    },
    {
      "metadata": {
        "id": "XZ97lM4oncMf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 4.5 5) Try to use 3 hidden layers (each with 64 units."
      ]
    },
    {
      "metadata": {
        "id": "dj_xA2gXnEFa",
        "colab_type": "code",
        "outputId": "6f97c774-03d3-4597-bf2e-3d318c634ad9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 751
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))# 1hidden layer\n",
        "model.add(layers.Dense(64, activation='relu'))# 2 hidden layer\n",
        "model.add(layers.Dense(64, activation='relu'))# 3 hidden layer\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])\n",
        "\n",
        "model.fit(partial_x_train,partial_y_train,epochs=20,batch_size=128,validation_data=(x_val, y_val))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 7982 samples, validate on 1000 samples\n",
            "Epoch 1/20\n",
            "7982/7982 [==============================] - 2s 284us/step - loss: 1.8646 - acc: 0.6116 - val_loss: 1.2962 - val_acc: 0.7080\n",
            "Epoch 2/20\n",
            "7982/7982 [==============================] - 2s 217us/step - loss: 1.0352 - acc: 0.7632 - val_loss: 1.1645 - val_acc: 0.7280\n",
            "Epoch 3/20\n",
            "7982/7982 [==============================] - 2s 227us/step - loss: 0.7015 - acc: 0.8468 - val_loss: 0.9927 - val_acc: 0.8060\n",
            "Epoch 4/20\n",
            "7982/7982 [==============================] - 2s 228us/step - loss: 0.4871 - acc: 0.8986 - val_loss: 0.9939 - val_acc: 0.8050\n",
            "Epoch 5/20\n",
            "7982/7982 [==============================] - 2s 228us/step - loss: 0.3548 - acc: 0.9252 - val_loss: 0.9214 - val_acc: 0.8260\n",
            "Epoch 6/20\n",
            "7982/7982 [==============================] - 2s 228us/step - loss: 0.2741 - acc: 0.9417 - val_loss: 1.0876 - val_acc: 0.7820\n",
            "Epoch 7/20\n",
            "1152/7982 [===>..........................] - ETA: 1s - loss: 0.1974 - acc: 0.9592"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 222us/step - loss: 0.2311 - acc: 0.9481 - val_loss: 1.0922 - val_acc: 0.7910\n",
            "Epoch 8/20\n",
            "7982/7982 [==============================] - 2s 220us/step - loss: 0.1995 - acc: 0.9503 - val_loss: 1.1518 - val_acc: 0.7870\n",
            "Epoch 9/20\n",
            "7982/7982 [==============================] - 2s 220us/step - loss: 0.1793 - acc: 0.9546 - val_loss: 1.0897 - val_acc: 0.8020\n",
            "Epoch 10/20\n",
            "7982/7982 [==============================] - 2s 222us/step - loss: 0.1756 - acc: 0.9565 - val_loss: 1.1732 - val_acc: 0.7960\n",
            "Epoch 11/20\n",
            "7982/7982 [==============================] - 2s 219us/step - loss: 0.1671 - acc: 0.9540 - val_loss: 1.1325 - val_acc: 0.7960\n",
            "Epoch 12/20\n",
            "7982/7982 [==============================] - 2s 217us/step - loss: 0.1601 - acc: 0.9575 - val_loss: 1.3589 - val_acc: 0.7780\n",
            "Epoch 13/20\n",
            "5632/7982 [====================>.........] - ETA: 0s - loss: 0.1259 - acc: 0.9599"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 221us/step - loss: 0.1488 - acc: 0.9559 - val_loss: 1.1942 - val_acc: 0.7910\n",
            "Epoch 14/20\n",
            "7982/7982 [==============================] - 2s 223us/step - loss: 0.1455 - acc: 0.9587 - val_loss: 1.2145 - val_acc: 0.8010\n",
            "Epoch 15/20\n",
            "7982/7982 [==============================] - 2s 223us/step - loss: 0.1432 - acc: 0.9575 - val_loss: 1.2281 - val_acc: 0.7980\n",
            "Epoch 16/20\n",
            "7982/7982 [==============================] - 2s 220us/step - loss: 0.1394 - acc: 0.9563 - val_loss: 1.3685 - val_acc: 0.7940\n",
            "Epoch 17/20\n",
            "7982/7982 [==============================] - 2s 221us/step - loss: 0.1369 - acc: 0.9569 - val_loss: 1.3283 - val_acc: 0.7850\n",
            "Epoch 18/20\n",
            "7982/7982 [==============================] - 2s 222us/step - loss: 0.1321 - acc: 0.9570 - val_loss: 1.4211 - val_acc: 0.7650\n",
            "Epoch 19/20\n",
            "4992/7982 [=================>............] - ETA: 0s - loss: 0.1149 - acc: 0.9615"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "7982/7982 [==============================] - 2s 225us/step - loss: 0.1301 - acc: 0.9575 - val_loss: 1.2495 - val_acc: 0.7950\n",
            "Epoch 20/20\n",
            "7982/7982 [==============================] - 2s 221us/step - loss: 0.1270 - acc: 0.9569 - val_loss: 1.3501 - val_acc: 0.7890\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f834a096490>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 59
        }
      ]
    },
    {
      "metadata": {
        "id": "Dok4Itlpnu9i",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Observations :**\n",
        "* We can see that The network now peaks at ~78.9% validation accuracy. Not Better then the previous model."
      ]
    },
    {
      "metadata": {
        "id": "9QzHPPc3nuQK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}