Compare_LR_and_NN_Results.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "pxsNqb9E41tk"
      },
      "source": [
        "Tasks:\n",
        "\n",
        "1.   //\n",
        "2.   Import the three datasets (training, validation and test) into a google colab and analyze your given data set. Your target variable will be \"discr_r\".\n",
        "3.   Generate a Linear Regression model as a benchmark and evaluate it (for comparison the mean absolute error as a performance metric is suggested). \n",
        "4.   Generate a neural network with an arbitrary choice of hyperparameters in order to perform a comparable task to the linear regression model and evaluate it. \n",
        "5.   Compare the feature importance for the benchmark and the baseline neural network.\n",
        "6.   Perform hyperparameter tuning with Optuna in order to generate a better performing neural network. \n",
        "7.   Evaluate its results and compare them as well as the feature importance to the benchmark and the baseline neural network. \n",
        "8.   Describe your results for all three models and explain what your results means with respect to $(R_{Mt} - R_{ft})$ , SMB and HML. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3tzT1eTOy1WG"
      },
      "source": [
        "## Step **2**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        },
        "id": "7-6bwG2PfjpA",
        "outputId": "280bc177-1947-4e95-dbcf-1ece88b08be1"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-0a0c331e-9331-41e1-8d90-c6188a5e4fc6\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-0a0c331e-9331-41e1-8d90-c6188a5e4fc6\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Saving test_data.csv to test_data.csv\n",
            "Saving train_data.csv to train_data.csv\n",
            "Saving validation_data.csv to validation_data.csv\n"
          ]
        }
      ],
      "source": [
        "# upload files from local\n",
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J1GNC6Soy8lg"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import seaborn as sns\n",
        "from matplotlib import pyplot as plt\n",
        "import io\n",
        "\n",
        "## assigment data from csv\n",
        "train_df = pd.read_csv(io.BytesIO(uploaded['train_data.csv']))\n",
        "val_df = pd.read_csv(io.BytesIO(uploaded['validation_data.csv']))\n",
        "test_df = pd.read_csv(io.BytesIO(uploaded['test_data.csv']))\n",
        "\n",
        "## set date to index\n",
        "train_df = train_df.rename(columns={\"Unnamed: 0\": \"date\"}).set_index(['date'], drop=True)\n",
        "val_df = val_df.rename(columns={\"Unnamed: 0\": \"date\"}).set_index(['date'], drop=True)\n",
        "test_df = test_df.rename(columns={\"Unnamed: 0\": \"date\"}).set_index(['date'], drop=True)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QqKDuTrlk4Ap",
        "outputId": "ac0a437b-3fd1-49bd-9199-600d5facbfe1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Index: 4302 entries, 2019-06-30 to 2019-06-30\n",
            "Data columns (total 4 columns):\n",
            " #   Column          Non-Null Count  Dtype  \n",
            "---  ------          --------------  -----  \n",
            " 0   book_to_market  4302 non-null   float64\n",
            " 1   market_cap      4302 non-null   float64\n",
            " 2   beta            4302 non-null   float64\n",
            " 3   discr_r         4302 non-null   float64\n",
            "dtypes: float64(4)\n",
            "memory usage: 168.0+ KB\n",
            "\n",
            "\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Index: 4335 entries, 2019-07-31 to 2019-07-31\n",
            "Data columns (total 4 columns):\n",
            " #   Column          Non-Null Count  Dtype  \n",
            "---  ------          --------------  -----  \n",
            " 0   book_to_market  4335 non-null   float64\n",
            " 1   market_cap      4335 non-null   float64\n",
            " 2   beta            4335 non-null   float64\n",
            " 3   discr_r         4335 non-null   float64\n",
            "dtypes: float64(4)\n",
            "memory usage: 169.3+ KB\n",
            "\n",
            "\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Index: 4348 entries, 2019-08-31 to 2019-08-31\n",
            "Data columns (total 4 columns):\n",
            " #   Column          Non-Null Count  Dtype  \n",
            "---  ------          --------------  -----  \n",
            " 0   book_to_market  4348 non-null   float64\n",
            " 1   market_cap      4348 non-null   float64\n",
            " 2   beta            4348 non-null   float64\n",
            " 3   discr_r         4348 non-null   float64\n",
            "dtypes: float64(4)\n",
            "memory usage: 169.8+ KB\n"
          ]
        }
      ],
      "source": [
        "train_df.info()\n",
        "print(\"\\n\")\n",
        "val_df.info()\n",
        "print(\"\\n\")\n",
        "test_df.info()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S5DZ3hIDlUNa"
      },
      "source": [
        "#####We have confirmed that there is no outlier Dtype. Therefore, no need to do any conversion.\n",
        "#####There is no row with a null value.\n",
        "\n",
        "\n",
        "#####In the next step, we will examine the correlation between columns.\n",
        "*   If the correlation value for two columns is close to 1, it is expected that the data in this column will affect each other negatively-positively.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 355
        },
        "id": "dYhHYb1m3FQ2",
        "outputId": "808a504f-3494-43bb-f8fc-b4bb28394dc7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc025208bb0>"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAFBCAYAAAA126tDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwUVbbA8d8JJBCSkIUdIntAkE0WBR8IgiAqiDoquOPIADqI4zLznH0+zhsdHZdR1AFmRBRxGR1RFmURZVFBCGEJIEsCsi8OCSEJgRBy3h9VSTohJAGSVNt9vp9Pf+iqunX73KK7T+6t21WiqhhjjDFeCfE6AGOMMcHNEpExxhhPWSIyxhjjKUtExhhjPGWJyBhjjKcsERljjPGUJSJjjDEVJiLTROSwiGw8y3YRkZdFJEVENohI9/LqtERkjDHmXEwHhpax/VogwX2MBf5RXoWWiIwxxlSYqi4D0sooMgJ4Sx0rgRgRaVJWnTUrM0BTYXY5C2NMRcmFVtC5+2MV/s7ZuPaFcTg9mQJTVXXqObxcM2CPz/Jed92Bs+1gicgjnbs/5nUIfiE56Xmmr37Z6zA8N7rXRADeSXrJ40i8d0f3hwF4w94X3Oe+L6qTm3TOJfFcMEtExhgT6C64T3VO9gEX+SzHu+vOys4RGWNMoBOp+OPCzQbucWfP9QYyVPWsw3JgPSJjjAl8ldgjEpF3gQFAfRHZC/wRCAVQ1cnAp8B1QApwHLivvDotERljTKCrxESkqreXs12Bn59LnZaIjDEm4FXvSaJzZYnIGGMCnPp3HrJEZIwxAS/EvzORzZozxhjjKesRGWNMoPPvDpElImOMCXiV8/ugKmOJyBhjAp1/5yFLRMYYE+jUekTGGGM85d95yBKRMcYEPv/ORJaIjDEm0Pl3HrJEZIwxAc8SkTHGGC/ZZAVjjDHeskRkjDHGU/6dhywRGWNMoFOvAyiHJSJjjAl0NjRn/M2TfxzJlf06kJaWxc23Ped1OOdNVVk0Yzmp63YRWiuUYWMH0bhVgzPKHdh5mHlTFnMqN4823Vow+O5+iAg5WSf4+JUFZPyQSXSDKG586BrCI2qXWe97z8xhf+pB4ts14bbHh53xWgvfWsaGpd/x+Ovjqrz9FaGqzH/zK7av20VoWE1ufGAQTUo5Rvt3HOaTyV9wKjePhG4tGHpvX0SEhTO/YVvS99SoEUJco2hGjB9I7YhapG7Yw+L3VnI67zQ1atZg8B19aNUp3oMWlm3H+l18PuMr8vPz6TqgI31u6FFse96p08yd/DkHdx4mPKo2IyZcQ0yDugCsmL2G9Us2ExISwtX39KN1l+aF++Xn5zP99x8QFRvBre774NN/fsGBnYdBIa5xDNePG0hY7bDqa2xZ/DsPlX8bCBFpKSIbL/SFROR7EalfgXIxIvLghb5eZRCR0SLyyjmU/4WI1KnKmCrDJ3NW88CEf3odxgVLXb+L9IMZjH/+Lq69fwDzpy8ptdyCN5Zy7ZirGP/8XaQfzGDHht0ArJiTRMuO8Yx//i5adoxn5ZykcuvtfX03ho+/utTXObDjMCeyT1ZqGy9UyrrdpB3M4KEX72T4zwYw7/WlpZabN20Zw382gIdevJO0gxmkrHeOUZvO8Tz47CgeeHYUcU1iWP6Jc4zqRNXm9sev44FnR3HjAwOZ9driamtTReXn57PwzWXc9qth/OzZO9i8cjv/3ZdWrMyGJZupHVGL8S/cTa+h3Vjy3goA/rsvjc0rtzPmmTu47VfDWTh9Kfn5+YX7Jc7fQP2mscXqGnRnX+5/ahT3Pz2KuvUiWbMwueobWVEiFX94wB/vRxQDeJ6IROR8eou/APw+Ea1J2kFGxnGvw7hg29fspFPf9ogIzdo25mR2Llnp2cXKZKVnczInl2ZtGyMidOrbnm2JOwr379zvYgA697uYbYk7y623ZaeLSv0rNz8/ny/e/YaBo66oyiafsy1rdtKln9OW+ITGnDieS2aJY5TpHqP4BOcYdenXni3usWjTpTkhNZyvifiERmSmZQHQpFUDouIiAGgQH8ep3DzyTp2uxpaV70DqYWIbRRPTMJoaNWvQsXcC29fsLFZme1LRe+Diy9qwa9NeVJXta3bSsXcCNUNrENOwLrGNojmQehiAY0eySF33PV0GdCxWV606zvtCVck7lYf40XCYSsUfXqhoIqopIjNF5DsR+VBE6ojIIBFZKyLJIjJNRGoBnG19AREJF5HPRORnZ3mtvwJtRGSdiPxNHH8TkY1unSPPFqSIDBCRpSLyiYjsEJG/isidIrLK3beNW264iHzrxvm5iDRy1/9JRGaIyNfAjBJ1Xy8iK0SkvogMcZ8nicgHIhIpIhOBpsCXIvJlBY+ruQCZ6dnUrRdZuBwVF1Hql2zduKIydeMiC8tkHztOZKzzZRoRU4fsY8crXG9JaxYmk9C9ZWF9/iIzLZvoer7tjyAzLfuMMsWOUb0zywCsW/Idbbs2P2P9d6t20KRVA2qG1qjEyC9cZnoWUXG+/4+Rpb4/CsqE1AihVp0wcrJOFFtftK+ThBe//RVX3X5FqYlm3pTFTPr5GxzZf5QeQzpXRbPOT4D0iNoDr6lqB+AY8CgwHRipqp1xzjU9ICK1S1vvU08kMAd4V1XPNjb0BJCqqt1U9ZfAzUA3oCtwNfA3EWlSRqxdgfFAB+BuoJ2qXgb8C3jILfMV0FtVLwXeA37ls39H4GpVvb1ghYjc5MZ1nbvqd26Z7kAi8KiqvgzsB65S1atKBiUiY0UkUUQSp06dWkb4xgsigpznQHpmejZbVqXSc0iXSo7KfyyblUhISAid+7Yrtv7wnjQ+f2cFw8b09yiy6pWy9nvq1A2ncauGpW6/ftwgJrwymnpNY/luZUo1R/fjVdHhpz2q+rX7/G3g98BOVd3mrnsT+Dnw5VnW/91d/gR4VlVnnkOMfXES12ngkIgsBXoBs89SfrWqHgAQkVRgobs+GShIEPHA+25CCwN8++uzVTXHZ3kg0BMYoqrHRGQYTrL62v2LKAxYUV4jVHUqUJCBdNLkx8rbxZRizaJk1n25CYAmrRtx7EhW4bbMtGyiSvRIomIjOJZWVOZYWlZhmYi6dchKzyYyNoKs9Gzq1A0v2qecen0d+v4H0g8dZfJjbwNwKjePfzw6gwdeuPsCW3t+Vi1MJumLzQA0bd2QjCO+7c8uHFIrEBVX4hgdKV5m3dItbF+7i3t+e0OxXsCxI1m8/8Jn3PjgIOIaRVdVc85bVGxk4VAiQKbP/31RmQgy07KoWy+S/NP5nDyeS3hk7cL1xfeNZHvSTlKSdpK6fhenT+VxMucUc15bxPAHBxeWDQkJoUOfBL6dm0SX/h2qvqEVEChXVig5Df0oUO88Xu9rYKiIvKOqVTW13fdscb7Pcj5F7Z0EvKCqs0VkAPAnn31KjkmkAq2Bdji9HwEW+faYTPXpMbgzPQY7Qx4pa79nzaJkOvZJYH/qIWrVCTtjaCwyNoJa4WHsSzlI0zaN2PjV1sIhk4TuLUlevoU+N/QgefkWEnq0cte3KrdeX20vbcnEV39auPzc/VM8S0IAlw3pzGVuG7clfc/qhRvpdEVb9qU4bSnty7hWeBh7tx+kWdtGbFi+lcuucY/xut18PWcto/9wI6G1Qgv3OZF9kneencfVt/emefuyBii806R1Q9IOZnD08DGi4iLYvHI7N/gkDIC23VuRvHwLzRIas2VVKi06NkNEaNu9JbNfW0Sva7uRlZ5N2sEMmrRpSLOExgwY2QeAXZv3serTtQx/cDCqytFDGcQ2jkFVSUnaSb0Skxk85d95qMKJqLmI9FHVFcAdOF/I40Skraqm4AyBLQW2Ai1LWV/gD+7jVc4+ISETiPJZXu6+1ptAHHAl8MsKxn020cA+9/m95ZTd5b7eRyJyK7ASeLWgjSISATRze4EFsf/3AuOrUs88dRe9erQhJiaCzz/7Pa9OXsCsT1Z5HdY5a9OtBanrdzH5sbcJDavJ9WMHFW57/Tfvcf9TowC4ZnR/5k5dTF5uHq27tqBN1xYA9B7eg48nzWf90u+Iru9M3y6v3hlPfsSRA+mcOnGKVx6aznU/G1hsWq+/Sbi0BdvX7WbSL2YSWqsmI8YNLNw2+Yn3Gf9X55Tr9fddyceTvyAvN4+23ZrTtpvTpk+nL+P0qdPMeMoZgIhv24hhYwawakEyaYcyWPpRIks/SgTg7l8PJyLaf+bqhNQIYci9/Xj/2dlovtKlfwcaxNdj2Yff0qRVQxJ6tKJr/w7Mmfw5kx+dQXhkbUZMGAJAg/h6dLi8Lf/633cICQlhyOgrCQkp40yGwtwpi8nNyUWBhs3rcc3oAdXSzorw9x6RlNcxEZGWwHyc5NMD2IyTYPoAz+Eks9XAA6p6UkQGnWX99zhDXEeAacAPqvorSiEi7wBdgM9wzt88C1yL0zP7P1V9/yz7DQAeV9Vh7vISdznRd5uIjABeBNKBL4BeqjpARP4EZKnqc+7+o4GeqjpBRC4FZgLDgRbAM0DBRIzfub2rh4AJwP7SzhP50M7dbWgOIDnpeaavftnrMDw3utdEAN5JesnjSLx3R/eHAXjD3hfc57wvLjiLtBv2pwqPQG2b+6dqz1rl9ohU9Xvg4lI2LQYuLaX82da39Fm8r5zXvKPEql9SgV6Qqi4BlvgsDyhtm6p+gnO+quT+fyqxPB1n8gWquhbn3BA4w3W9Stl/Es6wnzHG+A//7hDZlRWMMSbw+Xcm8iwRiUg9nF5VSYNU9Ug5+3amxO98gJOqenllxWeMMYHCqx+qVpRnichNNt3Oc9/k893XGGOCjiUiY4wxXvL3WXOWiIwxJtD5dx6yRGSMMYHPvzORJSJjjAlw/j5ZwR9vA2GMMaYyyTk8yqtKZKiIbBWRFBF5opTtzUXkS/fuBhtE5LrS6vFlicgYYwJe5WQiEamBc4m2a3F+4H+7iHQsUex3wL/duxuMAl4rLzpLRMYYE+A0pOKPclwGpKjqDlXNxbmNzoiSLwfUdZ9H49wep0yWiIwxJtCdQ4fI995p7mOsT03NgD0+y3vddb7+BNwlInuBTym6D9xZ2WQFY4wJeBWfrVDi3mnn43Zguqo+LyJ9gBki0klV88+2gyUiY4wJcJU4a24fcJHPcjxFt9QpcD8wFEBVV7h37q4PHD5bpTY0Z4wxga7yZs2tBhJEpJWIhOFMRih5t+zdwCAAEekA1AZ+KKtS6xEZY0ygq6RL/KhqnohMABYANYBpqrpJRJ4EElV1NvAY8E8ReQRn4sLo8u7IbYnIGGMCXGX+oFVVP8WZhOC77g8+zzcD/3MuddrQnDHGGE9Zj8gYYwKdn1/ixxKRMcYEOrsNhDHGGC+VOVPAD1giMsaYQOfnswGknFl1pmrYQTfGVNQFj6u1uOfpCn/n7Hrr19U+jmc9Io9MX/2y1yH4hdG9JtK5+2Neh+G55KTnAZi55iWPI/HenT0eBuDNRPuM3NtzYuVUZOeIjDHGeMlujGeMMcaUwXpExhgT6Py8y2GJyBhjAp5/j81ZIjLGmEDn33nIEpExxgQ8S0TGGGO85O8/XLREZIwxgc56RMYYYzxlP2g1xhjjKf/OQ/4+u9wYY0ygsx6RMcYEOj/vEVkiMsaYQOfniciG5owxxnjKekTGGBPgJMS/u0TWIzLGGOMp6xEZY0yg8+8OkSUiY4wJeH6eiGxozhhjjKesR2SMMQHOz6/wY4nIGGMCniUiY4wxnrJEVLVEZADwuKoOq2D50cBCVd1flXFVFVVl0YzlpK7bRWitUIaNHUTjVg3OKHdg52HmTVnMqdw82nRrweC7+yEi5GSd4ONXFpDxQybRDaK48aFrCI+oXWa97z0zh/2pB4lv14TbHj/zMC98axkbln7H46+Pq/L2V7Yn/ziSK/t1IC0ti5tve87rcCqdqrLgra/Yvm4XoWE1GTF+EE1Keb/s33GY2VO+4FRuHgndWnDNPX0RETavTGHpf1bzw/50xvz5Fpq2bghA8lfb+Gbe2sL9D+0+wti/3EbjlvWrrW0Voaosems5qet3UTMslOHjzv55mTt5MXmn8mjTtQWD7yn6vMyaVPR5uWmi83nZlriDpR+uQgRCaoQw+O6+XNS+KYe+/4H5byzlZE4uEhLC/4zoQcc+CR60vDg/z0M/7skKInI+iXQ00LSSQ6k2qet3kX4wg/HP38W19w9g/vQlpZZb8MZSrh1zFeOfv4v0gxns2LAbgBVzkmjZMZ7xz99Fy47xrJyTVG69va/vxvDxV5f6Ogd2HOZE9slKbWN1+mTOah6Y8E+vw6gyKet2c+RgBhNeuJNhYwYwb9rSUst9Om0Zw8YMYMILd3LkYAYp6533S4OL4rj1kaG0uLj4R6Zz33aMe3ok454eyY0PXE1sg7p+l4TAeV+nue/r6+4fwPw3lpRabv60pVznfl7SDmaww23/itlJtLwkngdeuIuWl8SzYrbzeWnZKZ4xT49kzNOjGDZ2IPP++SUANWuFMvyBqxn77B2M+t/hLHr7K//4fIhU/OEBTxKRiLQUkS0iMl1EtonITBG5WkS+FpHtInKZ+1ghImtF5BsRae/uO1pEZovIF8DiEvX2csu3EZEeIrJURNaIyAIRaSIitwA9gZkisk5Ews8SXy/3NdeLyCoRiXJjXi4iSe7jCrfsABFZJiLzRGSriEwWkSo7rtvX7KRT3/aICM3aNuZkdi5Z6dnFymSlZ3MyJ5dmbRsjInTq255tiTsK9+/c72IAOve7mG2JO8utt2WniwirHXZGLPn5+Xzx7jcMHHVFVTW3yq1J2kFGxnGvw6gyW9fspGs/5/81PqExJ4/nklni/ZLpvl/iE5z3S9d+7dnqvi8aNIujftPYMl9j4zfbuaRP2yprw4XYtmYnnd32N0tozInjZXxe3PZ37teerWucz8u2pJ10cT8vXfpdzLY1znEJqx2GuF/auSfzCp/XaxJDXOMYAKJiI4ioG87xzJxqaWtZ/DwPeTo01xa4FfgpsBq4A+gL3AD8BrgH6KeqeSJyNfAU8BN33+5AF1VNc4fmcBPDJGAEcACYAYxQ1R9EZCTwF1X9qYhMwBnKSywtKBEJA94HRqrqahGpC+QAh4HBqnpCRBKAd3GSGsBlQEdgFzAfuBn4sDIOUkmZ6dnUrRdZuBwVF0FmejaRsRHFy8QVlakbF1n45ZN97Hhh2YiYOmQfO17hektaszCZhO4tyyxjvFXyvVDw/xpVxvuloExFbV6ZwsjHrq2cgCtZVtq5f16i4iLJSnM/LxklPi8+f7RsXb2DL99fwfFjOdz2yzOHrPenHuJ0Xj6xDaMrvV3nzM/H5rwcmtupqsmqmg9sAharqgLJQEsgGvhARDYCLwKX+Oy7SFXTfJY7AFOB4aq6G2gPdAIWicg64HdAfAXjag8cUNXVAKp6TFXzgFDgnyKSDHyAk3gKrFLVHap6GidB9S1ZqYiMFZFEEUmcOnVqBUOpWiKCnOc7NDM9my2rUuk5pEslR2V+TPamHCK0Vk0aXlTP61CqXMnPS/terRn/3J3c8sh1LPvg22Jls9Kzmf2Pzxk2dqBfXOdNzuFRbl0iQ93RnxQReeIsZW4Tkc0isklE3imvTi97RL4Dp/k+y/k4cf0Z+FJVbxKRlsASn/Il/1w7ANQGLgX24xzPTarapxLjfQQ4BHTFSeAnfLZpibIll1HVqTjJEkCnr365wi+8ZlEy677cBECT1o04diSrcFtmWvG/bsEZEjiWVlTmWFpWYZmIunXIcv8izErPpk7d8KJ9yqnX16HvfyD90FEmP/Y2AKdy8/jHozN44IW7K9wuUzVWL0wm6cvNADRt3bDYe6Ei75fy/u99bVqxnUv84GS8r8SFRZ+XpufxeclMyyIyzv28RJf4vESfOZrfvENTjk45xvHMHOpEhXPyeC7vPzeX/rdeTrOExlXRxHNXSblQRGoArwKDgb3AahGZraqbfcokAL8G/kdV00WkYXn1+vNkhWhgn/t8dDlljwLXA0+7Q3VbgQYi0gdAREJFpKBHlQlElVHXVqCJiPRy941yJ0VE4/SU8oG7gRo++1wmIq3cc0Mjga8q1sSK6TG4M/c/NYr7nxpFux6t2PjVVlSVfSkHqVUn7IyhscjYCGqFh7Ev5SCqysavtpLQoxUACd1bkrx8CwDJy7f4rC+/Xl9tL23JxFd/yoN/v4cH/34PoWE1LQn5iV5DOhdOJGjfsxXrlzv/r3u3H6RWeFipX8S1wsPYu915v6xfvpX27vuiLJqvbF6ZSic/Oz/Uc0hnxjw9ijFPj6Jdz1Yku+3f57b/rJ8Xt/3Jy7fSzufzssH9vGxYvoV23Z31aQeP4gzgwMGdP5CXd5rwyNqczjvNh3//lM59L6bD5f5zXCrxHNFlQIo7ApQLvIdzOsTXz4BXVTUdQFUPl1epP0/ffhZ4U0R+B8wrr7CqHhKRYcBnOOedbgFeFpFonHb+HWcIcDowWURygD6qmlOinlz3nNIkdzJDDnA18BrwHxG5B+c8kG+vbDXwCs55ry+BWefd6nK06daC1PW7mPzY24SG1eT6sYMKt73+m/e4/6lRAFwzuj9zpy4mLzeP1l1b0KZrCwB6D+/Bx5Pms37pd0TXd6Zvl1fvjCc/4siBdE6dOMUrD03nup8NpHWX5lXVxGr1zFN30atHG2JiIvj8s9/z6uQFzPpklddhVZqEbi1IWbebVx6ZSWitmtwwbmDhtim/fp9xT48E4LqfXsknk78gLzePtl2b07ab8/+7ZfUOPntzOceP5fDus/No1KI+d/16OAC7tuynbr1IYhv5wTmQs2jTrQUp63bxj0ed9/WwcUXv63/9+j3GPO18Xobe1585U5zPSxufz0uf4T2YNWk+65c4n5ebJjqfl62rd5C8fAshNUIIDavJTQ8NQUT4bmUKe7YcICfzBBuWfQfA8HGDaNTyzCnj/kpExgJjfVZNdUd0AJoBe3y27QUuL1FFO7eer3H+YP+Tqs4v8zULsro5P+f6OybXOQ3NBbLRvSbSuftjXofhueSk5wGYueYljyPx3p09HgbgzUT7jNzbcyJUwsBa2189V+Ev+pRnHz/r67kzj4eq6hh3+W7gclWd4FNmLnAKuA3n3PwyoLOqHj1bvf48NGeMMaYSVOLQ3D7gIp/leIpOoRTYC8xW1VOquhPYBpR5IjGoE5GIzHJ/T+T7uOZc6lDVJefYGzLGmOpVedPmVgMJ7jnxMGAUMLtEmY+BAQAiUh9nqG5HWZX68zmiKqeqN3kdgzHGVLXz/ZlGSe7vOicAC3DO/0xT1U0i8iSQqKqz3W1DRGQzcBr4paoeKaveoE5ExhgTFCrxp0yq+inwaYl1f/B5rsCj7qNCLBEZY0yA8/4ntWWzRGSMMQGu6q5+WTksERljTICzHpExxhhv+XkmskRkjDEBzs/zkCUiY4wJdF7dZ6iiLBEZY0ygs0RkjDHGS35wS6Qy+fmkPmOMMYHOekTGGBPg7ByRMcYYb1kiMsYY46XKuuhpVbFEZIwxAc6G5owxxnjKEpExxhhP+XkeskRkjDEBz88zkSUiY4wJcP4+NCfOzfRMNbODboypqAtOIz3/8kKFv3MSf/totact6xEZY0yA8/cekSUij7yT9JLXIfiFO7o/zMw1dizu7PEwAJ27P+ZxJN5LTnoegEfm2vvixWEPV0o9loiMMcZ4yxKRMcYYL/l5HrJEZIwxgc6G5owxxnjMvzORJSJjjAlw4ud3nrNEZIwxAc6/+0OWiIwxJuD5+zkiP++wGWOMCXTWIzLGmADn7z0iS0TGGBPgLBEZY4zxlCUiY4wxnvLzPGSJyBhjAp6fZyJLRMYYE+D8PA/Z9G1jjAl0IhV/lF+XDBWRrSKSIiJPlFHuJyKiItKzvDqtR2SMMQFOKmm2gojUAF4FBgN7gdUiMltVN5coFwU8DHxbkXqtR2SMMYFOzuFRtsuAFFXdoaq5wHvAiFLK/Rl4BjhRkfAsERljTIA7lzwkImNFJNHnMdanqmbAHp/lve66otcS6Q5cpKrzKhqfDc0ZY0yAO5eROVWdCkw9v9eREOAFYPS57Gc9ImOMCXSVNzS3D7jIZzneXVcgCugELBGR74HewOzyJixYj8gYYwJcJU7fXg0kiEgrnAQ0CrijYKOqZgD1C19XZAnwuKomllWpJSJARFoCc1W1UwXLjwYWqur+KgzrnKgq89/8iu3rdhEaVpMbHxhEk1YNzii3f8dhPpn8Bady80jo1oKh9/ZFRFg48xu2JX1PjRohxDWKZsT4gdSOqEXqhj0sfm8lp/NOU6NmDQbf0YdWneI9aGHFqSoL3io6FiPGn/1YzJ5SdCyuucc5FptXprD0P6v5YX86Y/58C01bNwQg+attfDNvbeH+h3YfYexfbqNxy/pn1P1j9OQfR3Jlvw6kpWVx823PeR1Olbq4QQtuuqQ/IsK3uzexOLX492TruKbcdEl/mkTVZ8baz1h/IAWAtvXiubHjlYXlGkbG8lbSZ2w8tKNa4z9XlXWJH1XNE5EJwAKgBjBNVTeJyJNAoqrOPp96LRGdn9HARsBvElHKut2kHczgoRfvZF/KIea9vpQx/3fLGeXmTVvG8J8NoFnbRrzzzDxS1u8moVsL2nSO5+pRvQmpEcKid1aw/JMkBt/RhzpRtbn98euIiovg8J4jvP30XB597V4PWlhxKet2c+RgBhNecI/FtKWM+fOZx+LTacsYNsY9Fs8WHYsGF8Vx6yNDmff60mLlO/dtR+e+7QAnCf37hc8CJgkBfDJnNe++/xV/efJ2r0OpUoLwk04DmPztLI7mZPFIv1FsPLSDQ1lphWXSczJ5Z90irmrTvdi+KUf28tzydwCoE1qL31w1mq0/7K7W+M9HZV5rTlU/BT4tse4PZyk7oCJ12jmiIjVFZKaIfCciH4pIHRHpISJLRWSNiCwQkSYicgvQE5gpIutEJFxE/iAiq0Vko4hMlcqatH8OtqzZSZd+7RER4hMac+J4Lpnp2cXKZKZnczInl/iExogIXfq1Z0viTgDadGlOSA3n7RCf0IjMtCwAmrRqQFRcBAAN4uM4lZtH3qnT1diyc7d1zU66+hyLkxU4Fl37tWereywaNLmHQ+sAAB1gSURBVIujftPYMl9j4zfbuaRP2yprgxfWJO0gI+O412FUueYxjfhvdgZHjh/jtOazdt82OjVqXaxMek4mBzL/i6qetZ6uTRLY8sP3nMrPq+qQL1hl/qC1KlgiKtIeeE1VOwDHgJ8Dk4BbVLUHMA34i6p+CCQCd6pqN1XNAV5R1V7u0F44MKy6g89Myya6XmThct24CDLTss8oUzfOp0y9M8sArFvyHW27Nj9j/XerdtCkVQNqhtaoxMgrX2Z68XZGxUWUmojKK1OWzStT6HRFwoUHa6pdTHgkR09kFi5nnMgiOjyyjD1Kd2nTdiTt21aZoQUtS0RF9qjq1+7zt4FrcGZ/LBKRdcDvcGaIlOYqEflWRJKBgcAlJQv4zs2fOvW8ZkZWi2WzEgkJCSkcgipweE8an7+zgmFj+nsUmf/Ym3KI0Fo1aXhRPa9DMR6pW6sOTaLqseWHXV6HUiH+3iOyc0RFSvbBM4FNqtqnrJ1EpDbwGtBTVfeIyJ+A2mdUXnxuvr6T9NIFB7xqYTJJXzhX1mjauiEZR7IKtx1Lyy4cUisQFRfBsTSfMkeKl1m3dAvb1+7int/eUOySIMeOZPH+C59x44ODiGsUfcFxV4XVC5NJ+rLoWPi2MzMtm6jYEsciNqLcMmezacV2LuljvaEfq6M5WcTUjipcjq4dSUZOVhl7nKlb03YkH0wlX/MrO7wq4e/3I7IeUZHmIlKQdO4AVgINCtaJSKiIFPR0MnHmy0NR0vmviEQCZ54VryKXDenM+L+OZPxfR3Jxz1ZsWL4VVWXv9oPUqhNW6pdvrfAw9m4/iKqyYflWLu7RCnBO8H89Zy2jHr+O0FqhhfucyD7JO8/O4+rbe9O8fZPqato56zWkM+OeHsm4p0fSvmcr1vsei/Dyj8X65Vtp7x6Lsmi+snllKp0C7PxQMNmTcYgGETHEhdelhoRwabN2bDrHWW+XNm1H0v4fz7Cc9Yh+PLYCPxeRacBmnPNDC4CXRSQa51j9HdgETAcmi0gO0Af4J84suoM48+yrXcKlLdi+bjeTfjGT0Fo1GTFuYOG2yU+8z/i/jgTg+vuu5OPJX5CXm0fbbs1p2805F/Tp9GWcPnWaGU85sy/j2zZi2JgBrFqQTNqhDJZ+lMjSj5wprnf/ejgR0XWquYUVl9CtBSnrdvPKI86xuMHnWEz59fuMe9o5Ftf99Eo+KTgWXYuOxZbVO/jszeUcP5bDu8/Oo1GL+tz16+EA7Nqyn7r1Ion1057hhXjmqbvo1aMNMTERfP7Z73l18gJmfbLK67AqXb4q/9m0hHGX30iICN/u2czBrDSGtuvNnoxDbDq0k4uiG/HTntcTHlqbSxq1Ymi73jyz9G0AYsOjiAmPIvXIXo9bUnF+3iFCypoVYqpMpQzNBYI7uj/MzDV2LO7s8TAAnbs/5nEk3ktOeh6AR+ba++LFYQ9DJeSRYf96qcJf9HPHPFztect6RMYYE+D8vUdkicgYYwKcv09WsERkjDEBzhKRMcYYT1kiMsYY4yk/z0OWiIwxJuD5eSayRGSMMQHOz/OQJSJjjAl0do7IGGOMpywRGWOM8ZQlImOMMZ7y8zxkicgYYwKd9YiMMcZ4yhKRMcYYT1kiMsYY4yk/z0OWiIwxJtBZj8gYY4yn/DwPWSIyxpiA5+eZyBKRMcYEOBuaM8YY46kQS0TGGGO85Od5yBKRMcYEOn8fmhNV9TqGYGQH3RhTURecRn76wUsV/s6ZduvD1Z62rEfkkTdWv+x1CH7hvl4TeTPRjsW9PScC8MjclzyOxHsvDnsYgM7dH/M4Eu8lJz1fKfX4e4/IEpExxgQ6S0TGGGO85O+z5kK8DsAYY0zVknN4lFuXyFAR2SoiKSLyRCnbHxWRzSKyQUQWi0iL8uq0RGSMMQFOpOKPsuuRGsCrwLVAR+B2EelYothaoKeqdgE+BJ4tLz5LRMYYE+AqsUd0GZCiqjtUNRd4DxjhW0BVv1TV4+7iSiC+vEotERljTIA7lx6RiIwVkUSfx1ifqpoBe3yW97rrzuZ+4LPy4rPJCsYYE+DOZbKCqk4Fpl7oa4rIXUBPoH95ZS0RGWNMgKvE3xHtAy7yWY5315V4Pbka+C3QX1VPllepDc0ZY0yAq8RzRKuBBBFpJSJhwChgdrHXErkUmALcoKqHKxKf9YiMMSbAVVaPSFXzRGQCsACoAUxT1U0i8iSQqKqzgb8BkcAH4rzwblW9oax6LREZY0yAq8zfs6rqp8CnJdb9wef51edapyUiY4wJcHatOWOMMZ7y90v8WCIyxpgA5+d5yBKRMcYEOhuaM8YY4ylLRMYYYzzl53nIEpExxgS6ED/vElkiMsaYAOfnecgSkTHGBDo/z0OWiIwxJtDZ74hMldmxfhefz/iK/Px8ug7oSJ8behTbnnfqNHMnf87BnYcJj6rNiAnXENOgLgArZq9h/ZLNhISEcPU9/WjdpXnhfvn5+Uz//QdExUZw6+PDAPj0n19wYOdhUIhrHMP14wYSVjus+hpbDlVl0VvLSV2/i5phoQwfN4jGrRqcUe7AzsPMnbyYvFN5tOnagsH39ENEyMk6waxJC8j4IZPoBlHcNPEawiNqsy1xB0s/XIUIhNQIYfDdfbmofVMOff8D899YysmcXCQkhP8Z0YOOfRI8aHnFXdygBTdd0h8R4dvdm1icmlhse+u4ptx0SX+aRNVnxtrPWH8gBYC29eK5seOVheUaRsbyVtJnbDy0o1rjr05P/nEkV/brQFpaFjff9pzX4VwwP89DwZOIRORPQBZQF1imqp97G9GFyc/PZ+Gbyxj1xA1ExUUy/Q8fkNCjFfWbxRWW2bBkM7UjajH+hbvZvGI7S95bwY0PXcN/96WxeeV2xjxzB1np2bz3108Y+9ydhIQ4F2NPnL+B+k1jOZmTW1jXoDv7UquOk3gWv/0VaxYmn5H4vJS6fhdpBzMY//xd7E85xPw3ljD6yVvPKDd/2lKuG3MVTds24v1n57Jj/W7adGvBitlJtLwknitu6ME3s9ewYnYSA2+/gpad4kno0QoR4fDu//LRywsY/9yd1KwVyvAHriaucQyZ6dlM+92/ad2lObUjannQ+vIJwk86DWDyt7M4mpPFI/1GsfHQDg5lpRWWSc/J5J11i7iqTfdi+6Yc2ctzy98BoE5oLX5z1Wi2/rC7WuOvbp/MWc2773/FX5683etQKoW/nyMKuttAqOofLiQJiaPc4+be273KHEg9TGyjaGIaRlOjZg069k5g+5qdxcpsT9pJ534XA3DxZW3YtWkvqsr2NTvp2DuBmqE1iGlYl9hG0RxIda7WfuxIFqnrvqfLgOK3oS9IQqpK3qk8xM/e2dvW7KRzv/aICM0SGnPieC5Z6dnFymSlZ3MyJ5dmCY0RETr3a8/WNc5f9duSdtLFPVZd+l3MNvdYhtUOK2xr7smidtdrEkNc4xgAomIjiKgbzvHMnGpp6/loHtOI/2ZncOT4MU5rPmv3baNTo9bFyqTnZHIg87+o6lnr6dokgS0/fM+p/LyqDtlTa5J2kJFxvPyCPxIhUvGHJ/F587LVQ0R+KyLbROQroL27brqI3OI+/6uIbBaRDSLynLuukYjMEpH17uMKEWkpIltF5C1gI8VvDOX7elki8ryIrAf6VGXbMtOziIqLLFyOiosks8QXb2Z6dmGZkBoh1KoTRk7WiWLri/bNApzezlW3X1Fqopk3ZTGTfv4GR/YfpceQzlXRrPOWlZZN3Xq+bYoo9XjULdHurDSnTHbGcSJjIwCIiKlDts+X0NbVO5j8+Ez+/be5XD924BmvvT/1EKfz8oltGF2pbapMMeGRHD2RWbiccSKL6PDIMvYo3aVN25G0b1tlhmaqQSXej6hKBOzQnIj0wLlpUzecdiYBa3y21wNuAi5WVRWRGHfTy8BSVb3J7dVEArFAAnCvqq4s42UjgG9V9bFS4hkLjAWYMmUKoZdeaAsrX8ra76lTN5zGrRqya/MZN13k+nGDyM/PZ9Gby/luZQpd+nfwIMqqJyKIz0eyfa/WtO/Vmt3f7WfZB99yx29GFG7LSs9m9j8+Z/i4QYi/nxG+QHVr1aFJVD22/LDL61DMOfKzAYwzBGwiAvoBs1T1OICIzC6xPQM4AbwuInOBue76gcA9AKp6GsgQkVhgVzlJCOA08J/SNpS4D7y+sfrlc2xOcVGxkWSmZRUuZ6ZlEeX+RV9UJoLMtCzq1osk/3Q+J4/nEh5Zu3B98X0j2Z60k5SknaSu38XpU3mczDnFnNcWMfzBwYVlQ0JC6NAngW/nJnmeiBIXJrPuy00ANG3diGNHfNuUXerxOFai3ZFxbi8oug5Z6dlExkaQlZ5NnejwM16veYemHJ1yjOOZOdSJCufk8Vzef24u/W+9nGYJjauiiZXmaE4WMbWjCpeja0eSkZNVxh5n6ta0HckHU8nX/MoOz1QxP89DgT00VxZVzQMuAz4EhgHzy9klu5ztACfc5FXlmrRuSNrBDI4ePsbpvNNsXrmdtt1bFivTtnsrkpdvAWDLqlRadGyGiNC2e0s2r9xO3qnTHD18jLSDGTRp05ABI/vw80mjefDv93DDz6+hRcdmDH9wMKpK+sGjgHOOKCVpJ/WaxlZHM8vUc0hnxjw9ijFPj6Jdz1YkL9+KqrJv+0FqhYcVDrUViIyNoFZ4GPu2H0RVSV6+lXY9WgGQ0L0lG9xjtWH5Ftp1d9anHTxaeM7k4M4fyMs7TXhkbU7nnebDv39K574X0+HyttXY6vOzJ+MQDSJiiAuvSw0J4dJm7dh0jrPeLm3ajqT9Niz3Y+Tv54gCuUe0DJguIk/jtHM4zn3UARCRSKCOqn4qIl8DBZ/KxcADwN99hub8TkiNEIbc24/3n52N5itd+negQXw9ln34LU1aNSShRyu69u/AnMmfM/nRGYRH1mbEhCEANIivR4fL2/Kv/32HkJAQhoy+snDGXKkU5k5ZTG5OLgo0bF6Pa0YPqJZ2VlSbbi1IWbeLfzz6NqFhNRk2blDhtn/9+j3GPD0KgKH39WfOlMXk5TrTt9t0bQFAn+E9mDVpPuuXfEd0fWf6Njjnh5KXbyGkRgihYTW56aEhiAjfrUxhz5YD5GSeYMOy7wAYPm4QjVqeOWXcH+Sr8p9NSxh3+Y2EiPDtns0czEpjaLve7Mk4xKZDO7kouhE/7Xk94aG1uaRRK4a2680zS98GIDY8ipjwKFKP7PW4JdXjmafuolePNsTERPD5Z7/n1ckLmPXJKq/DOm/+3iOSsmbI/NiJyG+Be4HDwG6c80SdcIbhvgY+AWrj/D89p6pvikgjnCG01jhDbQ8AB4C5qtqpnNfLUtWKJK4LHpoLFPf1msibiXYs7u05EYBH5r7kcSTee3HYwwB07n7Gqdagk5z0PFRCHvnrly9X+Iv+iasmVnveCuQeEar6F+AvZRS5rJR9DgEjSilbZhJy9/XL3pMxJrjZZAVjjDGe8vM8ZInofIjIt0DJn9DfrarJXsRjjDFlsR5RAFLVy72OwRhjKsoSkTHGGE/VsERkjDHGS36ehywRGWNMoLOhOWOMMZ7y8zxkicgYYwKd9YiMMcZ4ys/zkCUiY4wJdP5+hxJLRMYYE+BsaM4YY4yn/P1+P5aIjDEmwFmPyBhjjKf8PA/5fY/NGGPMBRKp+KP8umSoiGwVkRQReaKU7bVE5H13+7ci0rK8Oi0RGWNMgKusW4W7d61+FbgW6AjcLiIdSxS7H0hX1bbAi8Az5cZ3Po0yxhjz41GJPaLLgBRV3aGqucB7nHkj0RHAm+7zD4FBImXXbInIGGMCnJzLQ2SsiCT6PMb6VNUM2OOzvNddR2llVDUPyADqlRWfTVYwxpgAdy6z5lR1KjC1yoIphfWIjDEmwIWcw6Mc+4CLfJbj3XWllhGRmkA0cKS8+IwxxgSwENEKP8qxGkgQkVYiEgaMAmaXKDMbuNd9fgvwhaqWWbGUs91UDTvoxpiKuuCfAb2T9FKFv3Pu6P5wma8nItcBfwdqANNU9S8i8iSQqKqzRaQ2MAO4FEgDRqnqjjLrtEQUvERkrDseHPTsWBSxY1HEjkX1sKG54Da2/CJBw45FETsWRexYVANLRMYYYzxlicgYY4ynLBEFNxv7LmLHoogdiyJ2LKqBTVYwxhjjKesRGWOM8ZQlImOMMZ6yRGSMMcZTloiMMcYlIiEicoXXcQQbm6wQZERksaoOKm9dsBCRm4G+OJdd+kpVZ3kckmdE5HrgEqB2wTpVfdK7iLwhImtV9VKv4wgm1iMKEiJSW0TigPoiEisice6jJWfeTyQoiMhrwHggGdgIjBORV72NyhsiMhkYCTyEc22zW4EWngblncUi8pPybuZmKo/1iIKEiDwM/AJoCuz32XQM+KeqvuJJYB4SkS1Ah4IrA4tICLBJVTt4G1n1E5ENqtrF599I4DNV7ed1bNVNRDKBCCAPOIGTmFVV63oaWACzG+MFCVV9CXhJRB5S1Ulex+MnUoDmwC53+SJ3XTDKcf89LiJNce4f08TDeDyjqlFlbReRS1R1U3XFEwxsaC74TBOR34nIVAARSRCRYV4H5ZEo4DsRWSIiXwKbgboiMltESt5jJdDNFZEY4G9AEvA98K6nEfmvGV4HEGhsaC7IiMj7wBrgHlXtJCJ1gG9UtZvHoVU7Eelf1nZVXVpdsXhNRGqp6smC5zgTFk4UrDNFbDJD5bOhueDTRlVHisjtAKp6PFhPygZToqmAFUB3ADf5nBSRpIJ1phj7672SWSIKPrkiEo77YRKRNkBQ/tUrIr2BSUAHIAznjpPZwXRSWkQa48yaDBeRSym6G2hdoI5ngZmgYoko+PwRmA9cJCIzgf8BRnsakXdeAUYBHwA9gXuAdp5GVP2uwfn/jwde8Fl/DPiNFwF5yR0diFfVPWUUy62ueIKFnSMKMu5viQTo7f67EohS1Z2eBuYBEUlU1Z4FU5bddUE5/i8iP1HV/3gdhz8QkWRV7ex1HMHEZs0FnznAKVWdp6pzgQbuumB0XETCgHUi8qyIPELwfia+FpHXReQzABHpKCL3ex2UR5JEpJfXQQSTYP3QBbOngDkiEiEiPYAPgbs8jskrd+N8BiYA2Ti/I/qJpxF55w1gAc4PngG24fwAOhhdDqwQkVQR2SAiySKyweugApkNzQUhEbkR+BXO72h+oqrbPA7JEyISAeSoar67XAOoparHvY2s+onIalXt5Ts0KSLrgnRaf6mXNlLVXaWtNxfOJisECRGZRPFpp9FAKjBBRFDVid5E5qnFwNVAlrscDiwEgvHqy9kiUo+i2ZS9gQxvQ/JME5xLPWUCiEhdnJmVloiqiCWi4JFYYnmNJ1H4l9qqWpCEUNUs9we+wehRYDbQWkS+xjl3eIu3IXnmHxT//VRWKetMJbJEFCRU9U136OktVb3T63j8RLaIdFfVJAD3nFlOOfsEqs3ALOA4kAl8jHOeKBiJ+pyzUNV8EbHvyipkBzeIqOppEWkhImGqar+FcE7GfyAi+3GmsjfGuRVCMHoL57dDT7nLd+BcU+1WzyLyzg4RmYjTCwJ4ENjhYTwBzyYrBBkReQtnvHs2zkwxAFT1hbPuFMBEJBRo7y5uVdVTPtsGq+oibyKrXiKyWVU7lrcuGIhIQ+BlYCDOObPFwC9U9bCngQUw6xEFn1T3EYIzay6ouYln41k2PwMERSLC+e1Mb1VdCSAil3PmecWg4CacUV7HEUysR2TMWQTDVRZEJBnnr/6CnuFud7kFsCVIe0TPAv+Hc75wPtAFeERV3/Y0sABmPaIgIyINcH5DdAnOpf4BUNWBngXlv4Lhr7RgvRdVWYao6q9E5Cac+zLdDCwDLBFVEUtEwWcm8D7OF9B44F7gB08jMp6xH2mWquB78XrgA1XNCNI7pVQbu8RP8Kmnqq/jXG9uqar+FOekbNBxbwBX1rrvqy8a40fmisgWoAew2B1FOOFxTAHNElHwKZgVdkBErnfvQRPnZUAeWlHWOlW9uRpjMX5CVZ/AubpGT3cySzYwwtuoApsNzQWf/xORaOAxnJvC1QUe8Tak6mU3gzOlEZGBqvqFiNzss863yEfVH1VwsEQUZNxbP4BzHbGrvIzFQ3YzOFOaK4EvgOE4E1WkxL+WiKqITd8OMiLSCngIaInPHyKqeoNXMXnFbgZnfInIY5yZgHCfB+2PvquD9YiCz8fA6zg3w8v3OBavfS0irwNNVfVaEekI9HEnc5jgE+n+2x7oBXyCk4yGA6u8CioYWI8oyIjIt6p6uddx+AP3bqRvAL9V1a7uhS3X2m2ig5uILAOu97kNRBQwT1Wv9DaywGWz5oLPSyLyRxHpIyLdCx5eB+WR+qr6b9yeoarmAae9Dcn4gUaA70WBc911porY0Fzw6Yxzi+yBFA3NKcH5WyK7GZwpzVvAKhGZ5S7fCEz3LpzAZ0NzQUZEUoCOdhsIcHuCk4BOOBc+bQDcoqobPA3MeM59b/RzF5ep6lov4wl01iMKPhuBGMAuaQ/pQH+ck9MCbAW6eRqR8QvuzRKTvI4jWFgiCj4xwBYRWQ2cLFgZjNO3gQ+BG1R1E4CIXAm8ijN8aYypJpaIgs8fvQ7Aj4wHPhaR4UB34GngOm9DMib42DkiU4yIrFDVPl7HUV1EpA8wBeeilterql2J3JhqZj0iU1Lt8ov8uInIHIrfa6gOzmy510UkWIcpjfGMJSJTUjB0kZ/zOgBjTBFLRCboqOpSEakBfK6qwXrhV2P8hl1ZwZQUFLeiVNXTQL57SwxjjIesRxSERKQRzkUdAVapqu9viu72ICSvZAHJIrII5+ZnAKjqRO9CMib42Ky5ICMitwF/A5bg9H76Ab9U1Q+9jMsLInJvaetV9c3qjsWYYGaJKMiIyHpgcEEvSEQa4Jwr6eptZMaYYGVDc8EnpMRQ3BGC9FyhiCTg/Ii1Iz7T1lW1tWdBGROELBEFn/kisgB4110eCXzmYTxeegPnShMv4tw2/T6CNCkb4yUbmgtCInIz0NddXK6qs8oqH6hEZI2q9hCR5IKb4RWs8zo2Y4KJ9YiCjIg8o6r/C3xUyrpgc1JEQoDtIjIB2EfR7aKNMdXEhiGCz+BS1l1b7VH4h4dxLu8zEegB3AXc42lExgQh6xEFCRF5AHgQaC0ivjd+iwK+9iYqzykwA2gBhLrr/gl08SwiY4KQnSMKEu4VBGJxZok94bMpU1XTfMrFqmp6dcfnBRHZCvwSSKbotumo6i7PgjImCFkiMsWISJKqdvc6juogIl+pat/ySxpjqpINzZmSguJac64/isi/gMUUv1vtR2ffxRhT2SwRmZKCqYt8H3AxzvmhgqE5xWdGoTGm6lkiMsGsl6q29zoIY4KdTd82JQXT0Nw3ItLR6yCMCXaWiIKMiNxfyrq/+iwOqsZwvNYbWCciW0Vkg4gkl5jaboypBjY0F3x+IiInVHUmgIi8SvELfqaddc/AM9TrAIwxloiC0U+A2SKSj/NFfFRVz+glBQP7vZAx/sF+RxQkRCTOZzEK+Bjnigp/gKDrCRlj/IgloiAhIjtxpiaLz78F1O7BY4zxiiUiY4wxnrJzREFGREKBB4Ar3VVLgCmqesqzoIwxQc16REHGvaRNKPCmu+pu4LSqjvEuKmNMMLNEFGREZL2qdi1vnTHGVBf7QWvwOS0ibQoWRKQ1cNrDeIwxQc7OEQWfXwJfisgOnJlzLXAu/mmMMZ6wobkgJCK1gIKLfW5V1ZNllTfGmKpkPaIg486aG4fPrDkRsVlzxhjPWI8oyNisOWOMv7FEFGRs1pwxxt/YrLngY7PmjDF+xc4RBR/fWXMALbFZc8YYD1mPKPh8DUwB8oE09/kKTyMyxgQ1O0cUZETk38AxYKa76g4gRlVv9S4qY0wws0QUZERks6p2LG+dMcZUFxuaCz5JItK7YEFELgcSPYzHGBPkbLJCkBCRZJwb4oUC34jIbne5BbDFy9iMMcHNhuaChIi0KGu7qu6qrliMMcaXJSJjjDGesnNExhhjPGWJyBhjjKcsERljjPGUJSJjjDGe+n8TR7QkhKYRUwAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "## train data correlations\n",
        "sns.heatmap(train_df.corr(), cmap=\"crest\", annot=True, linewidth=.5)\n",
        "# There is a possitive weak correlation between beta and discr_r (target). \n",
        "# There is a negative very weak correlation between market_cap and discr_r (target). \n",
        "# There is a possitive very weak correlation between book_to_market and discr_r (target)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 355
        },
        "id": "KTG1kShu1cRf",
        "outputId": "e4180254-bdad-4fc0-d3d5-0978f5de81b7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc022424250>"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAFBCAYAAAA126tDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwV5fX48c/JRhZCQkASIBgCsi9q2ETFDVAoKK0bqGjtRvVbpYtdlNZCbevSalu1+lNbqbu4lYoLoKKgogIhKhgQBELYCRASshKSnN8fM0luQlZJMtd7z/v1uq9kZp6Ze+bJ5J77PPPMjKgqxhhjjFdCvA7AGGNMcLNEZIwxxlOWiIwxxnjKEpExxhhPWSIyxhjjKUtExhhjPGWJyBhjTLOJyHwRyRGRLxpYLiLygIhsEZF1IpLW1DYtERljjGmJJ4BJjSyfDPRzX7OA/9fUBi0RGWOMaTZVfR/IbaTINOApdXwCxItI98a2GdaaAZpms9tZGGOaS050A8PSbmn2Z84Xn/7txzgtmSqPqepjLXi7nsBOn+ld7ry9Da1gicgjw9Ju8ToEv7A+4z7mvv2A12F47g8TZwPwx2VWF7ePd+pi3jtWF/MmzG7393STTksSzwmzRGSMMYHuhNtULbIb6OUznezOa5CdIzLGmEAn0vzXiVsEXOeOnjsDyFfVBrvlwFpExhgT+FqxRSQizwPnAV1FZBcwFwgHUNVHgDeBbwFbgGLge01t0xKRMcYEulZMRKp6VRPLFfhJS7ZpicgYYwJe+54kailLRMYYE+DUv/OQJSJjjAl4If6diWzUnDHGGE9Zi8gYYwKdfzeILBEZY0zAa53rg9qMJSJjjAl0/p2HLBEZY0ygU2sRGWOM8ZR/5yFLRMYYE/j8OxNZIjLGmEDn33nIEpExxgQ8S0TGGGO8ZIMVjDHGeMsSkTHGGE/5dx6yRGSMMYFOvQ6gCZaIjDEm0FnXnPE3d8ydzjnjBpGbW8ilV97rdThfm6qS8fIH7M3MJjQinDHXjieh10nHlcvdkcOqp5dRcayc7kNSSLt8HCLC0aJSPpq/lKLcAmISYjnrBxcRER3Z6HZfuPlh4nokABDdOZZzbpgCwDt//y/lpWUAlBaU0KV3IuNmfaudaqJhqkr6Sx+wOzObsPBwxl43ni4nH19Hh3bk8PFTyyg/Vk7PISmMvMKpo+yMLax7YzX5+w4z+ddX0CWlGwAV5RWsem45uTtyQISRV4wjqX/Pdt67pqkqGS99wB73b3nGteNJqGf/c3fk8MnTy6goK6fHkBTSrqg5RlbOX0rRoQJiusRytnuMAOzfvJuMlz+gsqKSDh2jmPDz71BxrJx3/r6QyvIKKisqOfn0vgybOqa9d/t4/p2Hmn4MhIj0FpEvTvSNRGS7iHRtRrl4Efm/E32/1iAi14vIP1tQ/mciEt2WMbWGV19bw403/cvrME7Y3g3ZFB7IZ8rcmYy66jzSFyyvt1z6CysYdfX5TJk7k8ID+ezdsAOAjW9nkDggmalzZ5I4IJkNb2U0ud3Q8FAm3TaDSbfNqE5CABN+fmn1/K6pSSSf2qfN9rsl9mRmU5CTz7R5MxlzzXmsbqCOVj+/gjHXnM+0eTMpyMlnj1tH8d0TOGfWZLqd0qNW+S0rNwAw9XdXMWH2JWS8shKt9L8OoL2Z2RQcyGfqvJmMvrrhY2TNghWMvvp8ps6bSYHPMbLhrQySBiRz8byZJPkcI2XFR0l/YQXn3DCFKbdfzdk/vAiAkLBQLpg9jclzZjB5znT2btjBwax97bKvjRJp/ssD/vg8onjA80QkIl+ntfgzwO8T0dqMbeTnF3sdxgnbvS6L3qMHICJ0TU3iWEkZJflFtcqU5BdxrLSMrqlJiAi9Rw9g97pt1eunjhkIQOqYgexel9Xs7TbkWEkZ+zfvJnm4fySineuySB3j7MtJqUmUFZdRXGdfit06Osmto9QxA9j5uVNHcd0TiEvsfNx28/fmkjQgGYDI2GgioiM4tCOn7XeohXaty6L3mJq/ZVlzjpExA9j1ef3HyK7PnWMkO30zvU7rQ0xCLODUAYCIEB4ZAUBlRSWVlZXtsp9NUWn+ywvNTURhIvKsiGwUkZdFJFpExovIpyKyXkTmi0gHgIbmVxGRKBFZLCI/auC97gb6ishnIvJXcfxVRL5wtzm9oSBF5DwRWSEir4rINhG5W0SuEZHV7rp93XIXi8gqN853RCTRnT9PRJ4WkZXA03W2PUVEPhaRriJyoft7hoi8JCIdRWQ20AN4T0Tea2a9mhNQkldEdOeO1dNR8TGU5BUdXybet0zH6jKlBcVExcUAENkpmtKC4ia3W1FewdJ7XuTte1+u/rDytWvdNhIHJBMeFdFKe3liSvKKiPHZl5jOTddRTOeOx5Wpq3NyV3aty6KyopLCg0c4tOMAxYcLWzf4VlCSX0SMz75Fx8dQXGffiuvsf3R8x+pk1dAxciQnj7Lioyz7x0KW3P0iWau+rF6/srKSxXcuYOFv5pM0sBddU5PabP+aLUBaRAOAh1V1EHAE+AXwBDBdVYfhnGu6UUQi65vvs52OwGvA86raUN/QrcBWVT1NVX8FXAqcBpwKTAD+KiLdG4n1VOAGYBBwLdBfVUcD/wZudst8CJyhqqcDC4Bf+6w/GJigqldVzRCR77hxVXX6/84tkwakA79Q1QeAPcD5qnp+3aBEZJaIpItI+mOPPdZI+MYLIkJzOtIvvuM6LvrNlYy9fiIZr3xIwYH8Wst3rP2KlBH92ihK/9F37CCiO3dk8T0vkv7yB5zUJ8mtw8Dle4xohZK74wDn3jiV82+6mC8Wp3Nkfx4AISEhTJ4zg2l/vp5D23PI23PIw6i/GZrb/bRTVVe6vz8D3A5kqepmd96TwE+A9xqY/w93+lXgL6r6bAtiPBsncVUA+0VkBTAKWNRA+TWquhdARLYCb7nz1wNVCSIZeMFNaBFAls/6i1S1xGf6AmAkcKGqHhGRqTjJaqX7jxcBfNzUTqjqY0BVBtIHH7mlqVVMPb5asZ6tH2UCkJCSWOtbeEleEVHxMbXKR8XHUJznW6awukxkbDQl+UVExcVQkl9EZGxUzToNbLfqm3PHrnF069eTvF0HiD0pDoCjhSUc2r6fs380ubV3u0U2rVjPlpVOHXVJSaTIZ1+KDjddR0WHC48rU1dIaAgjLz+7enrJX18hNjG+NcI/YZtXrGer7/7nFVI1PMFp/dTet+g6+1+cV1jTCmrgGInuHEOHmF6EdQgnrEM43U7pQd7ug3TyqYOI6A4k9u/J3g07iO/RpQ33uGn+fmeF5raI6p6FzPua77cSmCRt+9XpqM/vlT7TldQk3geBf7qtth8DkT7r1O2T2ArEAv3daQHedltsp6nqYFX9QWvugGlYv3OHVQ8KSB6eyvbVm1BVDmbtIzwqovoDpEpUXAzhkREczNqHqrJ99SZ6Dk8FoOew3tVdKlmrvvSZX/92y4pLqThWAThJ5+C2vXRKSqh+r52fbqXH0N6Ehns7GHXAucOYMmcGU+Y4dZS1ytmXA1n7iIiKILpOHUW7dXTAraOsVZvo5dZFQ8rLjlF+9BgAezfuJCRUiO+e0Og67aX/ucPcwQIz6HlqKttXtfAYWbWJ5KaOkeGpHNi6l8qKSsrLjnFo+346JXWmtKCEsmLnI6e8rJx9X+6kUz3n2NqdtODlgeb+x5wsImNV9WPgapzuqB+LyCmqugWnC2wFsAnoXc/8Kr93Xw/R8ICEApwP/iofuO/1JJAAnAP8qplxNyQO2O3+/t0myma77/dfEbkC+AR4qGofRSQG6Om2AqtiP3iC8bWpe+6cyagRfYmPj+Gdxbfz0CNLWfjqaq/DarHuQ1LYk5nN6394hrDwMMbMHF+9bMldC5h02wwARl55LquecYYm9xicQvfBKQAMmjiClfOXsO3jjcQkxHLm9y9qdLtH9h1mzfPLkRBBK5VBE9OI8/nwzV77FYMvTGuv3W+WnkOdfXl17jOERYQx9tqaOnrjzgVMmePU0egZ5/LRU84Q9x5DUugxxKmjHZ9tI/3F9yktLOG9h1+nc3JXxt98CaUFJSx78DVEhOj4GM787gRP9q8pPYaksDczm9fnPUNoRO1jZPGdC5js7v/I6efWDPEfnEJ3d/8HXziClY8vYetHG6uH+APEJSXQffDJLL5zASJCnzMHE9+jC4d3H+STp5Y5IwhVOTntFHoO693u+12Xv7eIRLXxIZci0htYgpN8RgAbcBLMWOBenGS2BrhRVY+KyPgG5m/H6eI6BMwHDqjqr6mHiDwHDAcW45y/+QswGadl9idVfaGB9c4DfqmqU93p5e50uu8yEZkG/B04DLwLjFLV80RkHlCoqve6618PjFTVm0TkdOBZ4GIgBbgHqBqI8TtVXSQiNwM3AXvqO0/kQ4elWdccwPqM+5j79gNeh+G5P0ycDcAfl1ld3D7eqYt571hdzJswG1qhndJ/6rxmj63f/Pq8ds9aTbaIVHU7MLCeRcuA0+sp39D83j6T32viPa+uM+tXNKMVpKrLgeU+0+fVt0xVX8U5X1V3/Xl1pp/AGXyBqn6Kc24InO66UfWs/yBOt58xxvgP/24Q2Z0VjDEm8Pl3JvIsEYlIF5xWVV3jVbXR8Y4iMow61/kAR1XVD+6lYYwx/sWrC1Wby7NE5Cab077muuu/7rrGGBN0WjERicgk4H4gFPi3qt5dZ/nJOJfuxLtlblXVNxvbpj/e4scYY0wrUpFmvxojIqE4o54n45wzv0pEBtcp9jvgRfeGATOAh5uKzxKRMcYEuta7jmg0sEVVt6lqGc6daabVKaNAJ/f3OJw7zjTKEpExxgS85mci39uRua9ZPhvqCez0md7lzvM1D5gpIruAN6m5tVqDbNScMcYEuJYMVqhzO7Kv4yrgCVW9T0TGAk+LyFBVbfBW5NYiMsaYQNd6XXO7gV4+08nU3KWmyg+AFwHcu/FEAo0+i84SkTHGBLxWy0RrgH4ikioiETiDEeregHoHMB5ARAbhJKIDjW3UuuaMMSbAaSs1OVS1XERuApbiDM2er6qZInIHkK6qi4BbgH+JyM9xBi5cr03cS84SkTHGBLpWvI7IvSbozTrzfu/z+wbgrJZs0xKRMcYEPP++tYIlImOMCXB2ix9jjDHeskRkjDHGU37+YDxLRMYYE+D8vWvOriMyxhjjKWsRGWNMoPPzFpElImOMCXR2jsgYY4yXGr2tgR+wRGSMMYHOz0cDSBO3ADJtwyrdGNNcJ9yvlnLdXc3+zMl+6rZ278ezFpFH5r79gNch+IU/TJzNsLRbvA7Dc+sz7gPgj8vsuLh9/GwAbltqdXHXRbNbZ0N2jsgYY4yX7DoiY4wxphHWIjLGmEDn500OS0TGGBPw/LtvzhKRMcYEOv/OQ5aIjDEm4FkiMsYY4yV/v3DREpExxgQ6axEZY4zxlF3QaowxxlP+nYf8fXS5McaYQGctImOMCXR+3iKyRGSMMYHOzxORdc0ZY4zxlLWIjDEmwEmIfzeJrEVkjDHGU5aIjDEm0EkLXk1tSmSSiGwSkS0icmsDZa4UkQ0ikikizzW1TeuaM8aYQNdKPXMiEgo8BEwEdgFrRGSRqm7wKdMPuA04S1UPi0i3prZrLSJjjDHNNRrYoqrbVLUMWABMq1PmR8BDqnoYQFVzmtqoJSJjjAlwIi15ySwRSfd5zfLZVE9gp8/0Lneer/5AfxFZKSKfiMikpuKzrjljjAl0LeiaU9XHgMdO4N3CgH7AeUAy8L6IDFPVvIZWsBaRMcYEutYbrLAb6OUznezO87ULWKSqx1Q1C9iMk5ga9I1vEYnIecAvVXVqM8tfD7ylqnvaMq62oqpkvPwBezOzCY0IZ8y140noddJx5XJ35LDq6WVUHCun+5AU0i4fh4hwtKiUj+YvpSi3gJiEWM76wUVEREc2ut0Xbn6YuB4JAER3juWcG6YA8M7f/0t5aRkApQUldOmdyLhZ32qnmmgdd8ydzjnjBpGbW8ilV97rdTitTlVJf+kDdmdmExYeztjrxtPl5OOPl0M7cvj4qWWUHyun55AURl7hHC/ZGVtY98Zq8vcdZvKvr6BLinPeubKigk+eeY/cnQeorFD6jBnA0Ekj2nv3vrb+XVOYOvAcQkRYsyuTFVlray0PlVCuHDaRnnHdKC4r5bnPF5NXWsApXXoxqd9ZhIaEUFFZyZubP2Rb7i6P9qL5WvEqojVAPxFJxUlAM4Cr65T5H3AV8B8R6YrTVbetsY1+o1tEIvJ1Eun1QI9WDqXd7N2QTeGBfKbMncmoq84jfcHyesulv7CCUVefz5S5Myk8kM/eDTsA2Ph2BokDkpk6dyaJA5LZ8FZGk9sNDQ9l0m0zmHTbjOokBDDh55dWz++amkTyqX3abL/byquvreHGm/7ldRhtZk9mNgU5+UybN5Mx15zH6gaOl9XPr2DMNeczbd5MCnLy2eMeL/HdEzhn1mS6nVL7XyY7YysV5RVM/d1VfOu2K/jqw0wKDx1p691pFYJwyaDz+M/aV/n7h89wavf+dItJqFVmVPJgSsqPcu8HT/Fh9qdM7n8WAEVlpTz56Wvc/9FzvPTFW1w57EIvdqHlWnKSqBGqWg7cBCwFNgIvqmqmiNwhIpe4xZYCh0RkA/Ae8CtVPdTYdj1JRCLSW0S+FJEnRGSziDwrIhPck1tficho9/WxiHwqIh+JyAB33etFZJGIvAssq7PdUW75viIyQkRWiMhaEVkqIt1F5HJgJPCsiHwmIlENxDfKfc/PRWS1iMS6MX8gIhnu60y37Hki8r6IvOGOrX9ERNqsXnevy6L36AGICF1TkzhWUkZJflGtMiX5RRwrLaNrahIiQu/RA9i9blv1+qljBgKQOmYgu9dlNXu7DTlWUsb+zbtJHv7NS0RrM7aRn1/sdRhtZue6LFLHOH/Xk1KTKCsuo7jO37XYPV5Oco+X1DED2Pm5c7zEdU8gLrFzvdsuP1pOZUUlFWUVhISFEB4Z0eb70xp6xSVyqDiPwyVHqNBKPt/7FYO61T52B3XrQ8bujQB8sX8Lfbs4vVF7Cw5QcNSpv/2FuYSHhhEqoe27A19DK+UhAFT1TVXtr6p9VfXP7rzfq+oi93dV1V+o6mBVHaaqC5rappddc6cAVwDfx2nuXQ2cDVwCzAGuA8aparmITADuBC5z100Dhqtqrts1h5sYHsQZSrgXeBqYpqoHRGQ68GdV/b6I3ITTlZdeX1AiEgG8AExX1TUi0gkoAXKAiapa6o6Tfx4nqYEzpHEwkA0sAS4FXm6NSqqrJK+I6M4dq6ej4mMoySsiKi6mdpl43zIdKclz/nlKC4qry0Z2iqa0oLjJ7VaUV7D0nhcJCQ1h0MS041o+u9ZtI3FAMuFR34wPomBSkldEjM/fNaaz83eNbuR4ielcc7w0JCWtL7vWZfHKbf+hvKyckZefTYeYyNbfgTbQKbIj+aWF1dNHSgvpFZ9Yu0yHjuS5ZSpVKS0vIzo8kuJjpdVlhiaewp4jOVRoRfsEfiL8+w4/niaiLFVdDyAimcAyVVURWQ/0BuKAJ90PfQXCfdZ9W1VzfaYH4YzyuFBV94jIUGAo8LY4KT4UJzk1xwBgr6quAVDVI26MMcA/ReQ0oAKn37PKalXd5pZ7Hieh1kpE7hDIWQCPPvoopDYzmjbk1E3TR+jFd1xHdHxHCg/m8+4DrxLXowuxJ8VVL9+x9iv6jB3chpEaf3Nwew4SIlx21/WUFR9l6X0LSRqYTGzXuKZXDgDdYhKY1P8s5qf/z+tQmsXP85Cnieioz++VPtOVOHH9EXhPVb8jIr2B5T7l635d2wtEAqcDe3DqPVNVx7ZivD8H9gOn4nRplvos0zpl607XHRKpc99+oNlv/NWK9Wz9KBOAhJREig/XfJsrySsiKj6mVvmo+BiK83zLFFaXiYyNpiTfaemU5BcRGRtVs04D2636ttyxaxzd+vUkb9eB6kR0tLCEQ9v3c/aPJjd7f0zb2rRiPVtWOsdLl5REinz+rkWHmz5eig4XHlemru1rNtNjcAohoaFExkbTrW8Sudk534hEdKS0kLjImhag00Kq/ZFy5Ggh8ZEdOXK0kBARIsMiqltDnTp05NrTp/DS+rfILclv19i/Nj/PRP48WCGOmmGB1zdRNg+YAtzldtVtAk4SkbEAIhIuIkPcsgVAbCPb2gR0F5FR7rqx7qCIOJyWUiVwLU4rq8poEUl1zw1NBz5s3i42T79zh1UPCkgensr21ZtQVQ5m7SM8KqJWtxxAVFwM4ZERHMzah6qyffUmeg53mmA9h/Uma9WXAGSt+tJnfv3bLSsupeKY0/VwtLCEg9v20imp5sTuzk+30mNob0LDv/EDMAPGgHOHMWXODKbMcY6XrFXO3/VA1j4ioiJqdcsBRLvHywH3eMlatYlewxtvssckxLJvkzNarPzoMQ5m7adTA+eS/M2uI/vpGh1P56hOhEoIp3bvx8ac2oO6NuZkkdZzEOB0wW11R8ZFhkVw/YiLWbL5I7LzmtvJ4r3WPEfUFvz50+MvOF1zvwPeaKqwqu4XkanAYpzzTpcDD4hIHM5+/gPIBJ4AHhGREmCsqpbU2U6Ze07pQXcwQwkwAXgYeEVErsM5D+T7FWoN8E+c817vAQu/9l43ofuQFPZkZvP6H54hLDyMMTPHVy9bctcCJt02A4CRV57Lqmec4bg9BqfQfXAKAIMmjmDl/CVs+3gjMQmxnPn9ixrd7pF9h1nz/HIkRNBKZdDENOK61ySi7LVfMfjCtLba3TZ3z50zGTWiL/HxMbyz+HYeemQpC19d7XVYrabnUOfv+urcZwiLCGPstTXHyxt3LmDKHOd4GT3jXD56yhnu32NICj2GOMfLjs+2kf7i+5QWlvDew6/TObkr42++hP7nDOXjp9/ltT8+Bwp9xg6kc3JXT/axpSpVWbRxOd8fMQ2RENJ3Z5JTlMuEU8awOz+HjQeySN+dyZXDLuSX466j+Fgpz3++BICxJ59Kl6h4Lug7mgv6jgZg/tr/UVRW0thbmiaI6nG9SKYFWnodk6tFXXOB7A8TZzMs7Ravw/Dc+oz7APjjMjsubh8/G4Dbllpd3HXRbGiFjrVTfn1vsz/ot/zll+3eLvLnFpExxphW4FWXW3MFdSISkYUcP37tN6q6tLnbUNXl1B5IYYwx/sUSkf9S1e94HYMxxrQ18fNMFNSJyBhjgoJ/5yFLRMYYE+j8PA9ZIjLGmEDXdne/bB2WiIwxJsBZi8gYY4y3/DwTWSIyxpgA5+d5yBKRMcYEOrug1RhjjLcsERljjPFSiJ8nIj8f1GeMMSbQWYvIGGMCnJ0jMsYY4y1LRMYYY7xkNz01xhjjKeuaM8YY4ylLRMYYYzzl53nIEpExxgQ8P89EloiMMSbA+XvXnKiq1zEEI6t0Y0xznXAaGfnnvzX7Myf9t79o97Rld1YwxpgAJ9L8V9PbkkkisklEtojIrY2Uu0xEVERGNrVN65rzyB+XPeB1CH7h9vGzrS5w6gFgWNotHkfivfUZ9wHwXMb9HkfivavTftoq22mtrjkRCQUeAiYCu4A1IrJIVTfUKRcL/BRY1ZztWovIGGMCnbTg1bjRwBZV3aaqZcACYFo95f4I3AOUNic8S0TGGBPgWpKHRGSWiKT7vGb5bKonsNNnepc7r+a9RNKAXqr6RnPjs645Y4wJcC3pmlPVx4DHvt77SAjwN+D6lqxnLSJjjAl4rdY3txvo5TOd7M6rEgsMBZaLyHbgDGBRUwMWrEVkjDEBTlqvybEG6CciqTgJaAZwddVCVc0Hula/r8hy4Jeqmt7YRq1FZIwxAa612kOqWg7cBCwFNgIvqmqmiNwhIpd83fisRWSMMQGuNe+soKpvAm/Wmff7Bsqe15xtWovIGGOMp6xFZIwxAc7f7zVnicgYYwKcJSJjjDGeskRkjDHGU36ehywRGWNMwPPzTGSJyBhjApyf5yFLRMYYE+jsHJExxhhPiZ9nIktExhgT6Pw7D1kiMsaYQOfnecgSkTHGBDo/75mzRGSMMQHPEpExxhgv+XkeskQEICK9gddVdWgzy18PvKWqe9owrBZRVdJf+oDdmdmEhYcz9rrxdDn5pOPKHdqRw8dPLaP8WDk9h6Qw8opxiAjZGVtY98Zq8vcdZvKvr6BLSjcAKsorWPXccnJ35IAII68YR1L/nsdt15+0VV1UVlTwyTPvkbvzAJUVSp8xAxg6aUR7716buWPudM4ZN4jc3EIuvfJer8NpU6rKkic/5KvPsgmPCOPbN46ne+rxx8iyFz5h3fubKCk6ypwnZtValvnxFpa/sgYBElO6ctnNE9sp+pbz9645ewzE13M90MPrIHztycymICefafNmMuaa81i9YHm95VY/v4Ix15zPtHkzKcjJZ8+GHQDEd0/gnFmT6XZK7d3asnIDAFN/dxUTZl9Cxisr0Upt0305UW1VF9kZW6kor2Dq767iW7ddwVcfZlJ46Ehb7067efW1Ndx407+8DqNdbPlsB7n78rn579dw8Y/O443HV9RbbkBab374p8uPm39obx4fvprB9+d9h/+79yomXXdWW4d8QkSa//KCJaIaYSLyrIhsFJGXRSRaREaIyAoRWSsiS0Wku4hcDowEnhWRz0QkSkR+LyJrROQLEXlMPBi0v3NdFqljBiAinJSaRFlxGcX5RbXKFOcXcay0jJNSkxARUscMYOfn2wCI655AXGLn47abvzeXpAHJAETGRhMRHcGhHTltv0MnoK3qAqD8aDmVFZVUlFUQEhZCeGREm+9Pe1mbsY38/GKvw2gXX67NYvg45xhJ7pdEaXEZBYeLjiuX3C+J2M4xx83PeHcDoy4cSlTHSABi4qLbPOYTYYnom2MA8LCqDgKOAD8BHgQuV9URwHzgz6r6MpAOXKOqp6lqCfBPVR3ldu1FAVPbO/iSvCJiOnesno7pHENJXtFxZaLjfct0PK5MXZ2Tu7JrXRaVFZUUHjzCoR0HKD5c2LrBt7K2qouUtL6EdQjjldv+w39/9ySDJ5xOh5jI1g3etIuC3CLiutT8/TslxFCQ2/jf39ehffkc2pvH/Ln/5d+3v8KWz3a0Rfp7cd8AABwISURBVJhBw84R1dipqivd358B5gBDgbfdBk4osLeBdc8XkV8D0UACkAm85ltARGYBswAeffRR6Nvq8beJvmMHkb/vMIvveZGYhFhO6pPk91dpt5WD23OQEOGyu66nrPgoS+9bSNLAZGK7xnkdmmlnlRWV5O7L57u3T+NIbhFP/GEhN/5lBpExHbwOrV7+/i9riahG3RMfBUCmqo5tbCURiQQeBkaq6k4RmQcc9zVZVR8DHqua/OOyB0444E0r1rNlZSYAXVISKfJpqRQdLiIqvnaXQlR8DMV5vmUKjytTV0hoCCMvP7t6eslfXyE2Mf6EY29t7VEX29dspsfgFEJCQ4mMjaZb3yRys3MsEX1DrH5rPRnvOuc8e/TpRv6hmr//kdwiYhMa//v76pQQQ89TEgkNC6Vzt0506R7PoX159Oyb2OpxtwZ/T0TWNVfjZBGpSjpXA58AJ1XNE5FwERniLi8AYt3fq5LOQRHpCBx/ZrONDDh3GFPmzGDKnBkkD08la9UmVJUDWfuIiIogOq72P1Z0XAzhkREcyNqHqpK1ahO9hqc2+h7lZccoP3oMgL0bdxISKsR3T2izffq62qMuYhJi2bdpFwDlR49xMGs/nRo4l2T8z+gLh3HD3dO54e7pDByZyroPnGNk11f76BAdUe+5oIYMHNmH7A3OoNniIyUc2ptH527++4XE388RWYuoxibgJyIyH9iAc35oKfCAiMTh1NU/cLrdngAeEZESYCzwL+ALYB+wpv1Dh55DU9iTmc2rc58hLCKMsdeOr172xp0LmDJnBgCjZ5zLR08to+JYOT2GpNBjSAoAOz7bRvqL71NaWMJ7D79O5+SujL/5EkoLSlj24GuICNHxMZz53Qle7F6LtFVd9D9nKB8//S6v/fE5UOgzdiCdk7t6so9t4Z47ZzJqRF/i42N4Z/HtPPTIUha+utrrsNpEv9NT+OqzHTz4s2cJ7xDGtB9fUL3skVtf4Ia7pwPw9rMfsf6jrzhWVs7ffvIkaecP4rzLR9P31F5sXb+Th375PCEhwsRrziQ61n/PF/p5gwhR9e+huAGqVbrmAsHt42djdeHUA8CwtFs8jsR76zPuA+C5jPs9jsR7V6f9FFohj0z99/3N/qB//Yc/bfe8ZS0iY4wJcP7eIrJEZIwxAc7fBytYIjLGmABnicgYY4ynLBEZY4zxlJ/nIUtExhgT8Pw8E9kFrcYYE+CkBa8mtyUySUQ2icgWEbm1nuW/EJENIrJORJaJSEpT27REZIwxAa617qwgIqHAQ8BkYDBwlYgMrlPsU5xbng0HXgb+0lR8loiMMSbAteItfkYDW1R1m6qWAQuAab4FVPU9Va16nsgnQHJTG7VEZIwxAa4liUhEZolIus/L99G0PYGdPtO73HkN+QGwuKn4bLCCMcYEuJaMVajzpICv/54iM3EeInpuU2UtERljTIBrxeuIdgO9fKaT3Xl13k8mAL8FzlXVo01t1LrmjDEmwLXiOaI1QD8RSRWRCGAGsKj2e8npwKPAJaqa05z4rEVkjDEBrrVaRKpaLiI34TwiJxSYr6qZInIHkK6qi4C/Ah2Bl9ynOe9Q1Usa264lImOMCXCteT2rqr4JvFln3u99fm/xQ8ssERljTICze80ZY4zxlJ/nIUtExhgT8Pw8E1kiMsaYAGddc8YYYzwVYonIGGOMl/w8D1kiMsaYQOfvXXOiql7HEIys0o0xzXXCaeT7L93f7M+c+Vf8tN3TlrWIPDLvnQe8DsEvzJswm9uWWl3cddFsAJ7LuN/jSLx3ddpPARiWdovHkXhvfcZ9rbIdf28RWSIyxphAZ4nIGGOMl2zUnDHGGE/5eR6yRGSMMYHOzhEZY4zxlJ/nIUtExhgT6KxFZIwxxlM2WMEYY4ynrEVkjDHGU36ehywRGWNMoLMWkTHGGE/5eR6yRGSMMYHOWkTGGGM8ZaPmjDHGeMrP85AlImOMCXTWNWeMMcZTloiMMcZ4ys/zkCUiY4wJdCF+3iSyRGSMMQHOz/MQIV4HYIwxpm1JC15NbktkkohsEpEtInJrPcs7iMgL7vJVItK7qW1aIjLGmAAXIs1/NUZEQoGHgMnAYOAqERlcp9gPgMOqegrwd+CepuKzrrlvMFUl46UP2JOZTWhEOGdcO56Ek086rlzujhw+eXoZFWXl9BiSQtoV4xARjhaVsnL+UooOFRDTJZazf3AREdGRAOzfvJuMlz+gsqKSDh2jmPDz71BxrJx3/r6QyvIKKisqOfn0vgybOqa9d7tF+ndNYerAcwgRYc2uTFZkra21PFRCuXLYRHrGdaO4rJTnPl9MXmkBp3TpxaR+ZxEaEkJFZSVvbv6Qbbm7PNqL1qGqLHnyQ776LJvwiDC+feN4uqcef7wse+ET1r2/iZKio8x5YlatZZkfb2H5K2sQIDGlK5fdPLGdom9fd8ydzjnjBpGbW8ilV97rdTgnrBV75kYDW1R1G4CILACmARt8ykwD5rm/vwz8U0REVbWhjQZNIhKReUAh0Al4X1Xf8TaiE7c3M5uCA/lMnTeTQ9v3k75gORf++orjyq1ZsILRV59Pl96JrHj4dfZu2EGPISlseCuDpAHJDL5wBBveWsuGtzI47dtnUlZ8lPQXVnDeTy4mJiGW0oJiAELCQrlg9jTCIyOorKjgnfv+S/chKXRNTWrvXW8WQbhk0Hk8nr6QI6WF/GTsdDbmZJFTlFtdZlTyYErKj3LvB08xPKkfk/ufxfPrllBUVsqTn75GwdEiEjsm8L0R3+buFfM93JsTt+WzHeTuy+fmv1/D7i37eePxFfzwT5cfV25AWm9GXziMB3/+bK35h/bm8eGrGXx/3neI6hhJUX5xe4Xe7l59bQ3Pv/Ahf77jKq9DaRUtOUckIrMA328gj6nqY+7vPYGdPst2AXW/jVaXUdVyEckHugAHG3rPoOuaU9Xfn0gSEkeT9eY2YdvUrnVZ9B4zABGha2oSZSVllOQX1SpTkl/EsdIyuqYmISL0HjOAXZ9vA2D3uixSxwwEIHXMQHZ9ngVAdvpmep3Wh5iEWAAiY6Or9onwyAgAKisqqaysbOtdPCG94hI5VJzH4ZIjVGgln+/9ikHd+tQqM6hbHzJ2bwTgi/1b6NulFwB7Cw5QcNSpy/2FuYSHhhHa9n/SNvXl2iyGj3OOl+R+SZQWl1FwuOi4csn9kojtHHPc/Ix3NzDqwqFEdXRazTFx0W0es1fWZmwjP4ASbUu65lT1MVUd6fN6rOl3OMH42voNvCQivxWRzSLyITDAnfeEiFzu/n63iGwQkXUicq87L1FEForI5+7rTBHp7Z6cewr4AujVwPsVish9IvI5MLat968kv4iY+I7V09HxMRTn1f5gKc4rIrpWmY7Vyaq0oJioOOcDJ7JTdHXL50hOHmXFR1n2j4UsuftFslZ9Wb1+ZWUli+9cwMLfzCdpYC+/bQ0BdIrsSH5pYfX0kdJC4iJrf8B26tCRPLdMpSql5WVEh0fWKjM08RT2HMmhQivaPug2VJBbRFyXmmOhU0IMBbnHJ6KGHNqXz6G9ecyf+1/+ffsrbPlsR1uEadpAKw5W2E3tz79kd169ZUQkDIgDDjW20YDtmhOREcAM4DSc/cwA1vos7wJ8Bxioqioi8e6iB4AVqvodt1XTEegM9AO+q6qfNPK2McAqVb2lnniqm7uPPvoo9KlbwlsiNYehVii5Ow5wwexpVBwr5617X6FL7yQ6JcYTEhLC5DkzKCs+ygePLSZvzyHie3TxNvg21C0mgUn9z2J++v+8DsVzlRWV5O7L57u3T+NIbhFP/GEhN/5lBpExHbwOzTShFYdvrwH6iUgqTsKZAVxdp8wi4LvAx8DlwLuNnR+CAE5EwDhgoaoWA4jIojrL84FS4HEReR143Z1/AXAdgKpWAPki0hnIbiIJAVQAr9S3wG3eVjVxdd47D7RwdxybV6xn68pMALqkJFKUV0jV6Wan9VP7G7/TSqppFRTnFda0gmKjKckvIiouhpL8IiJjo5x1OsfQIaYXYR3CCesQTrdTepC3+yCdEuOrtxMR3YHE/j3Zu2GH3yYipwXk0wKI7Eh+ae0WwJGjhcRHduTI0UJCRIgMi6D4WKlTvkNHrj19Ci+tf4vckvx2jb21rH5rPRnvOueRe/TpRv4hnxZibhGxCcd3wTWkU0IMPU9JJDQslM7dOtGlezyH9uXRs29iq8dtWldr5SH3nM9NwFIgFJivqpkicgeQrqqLgMeBp0VkC5CLk6waFdBdc41R1XKcESAvA1OBJU2s0pw+jFI3ebWZ/ucOY/KcGUyeM4Oep6ayfdUmVJWDWfsIj4qoTjJVouJiCI+M4GDWPlSV7as2kTw8FYCew3pXd7tlrfqSnlXzh6dyYOteKisqKS87xqHt++mU1JnSghLKio8CUF5Wzr4vd9IpsXNb7u4J2XVkP12j4+kc1YlQCeHU7v3YmLOtVpmNOVmk9RwEOF1wW92RcZFhEVw/4mKWbP6I7Ly97R57axl94TBuuHs6N9w9nYEjU1n3gXO87PpqHx2iI+o9F9SQgSP7kL1hDwDFR0o4tDePzt3i2ip004paa/g2gKq+qar9VbWvqv7Znfd7NwmhqqWqeoWqnqKqo6tG2DUmkFtE7wNPiMhdOPt5MfBo1UIR6QhEq+qbIrISqKqsZcCNwD98uub8Uo8hKezNzOb1ec8QGhHGmJnjq5ctvnMBk+c4X0RGTj+XVU8vo+JYOd0Hp9B9SAoAgy8cwcrHl7D1o43EJMRy1g8uAiAuKYHug09m8Z0LEBH6nDmY+B5dOLz7IJ88tQytVFDl5LRT6Dmsd7vvd3NVqrJo43K+P2IaIiGk784kpyiXCaeMYXd+DhsPZJG+O5Mrh13IL8ddR/GxUp7/3Pk+MvbkU+kSFc8FfUdzQd/RAMxf+z+Kykq83KUT0u/0FL76bAcP/uxZwjuEMe3HF1Qve+TWF7jh7ukAvP3sR6z/6CuOlZXzt588Sdr5gzjv8tH0PbUXW9fv5KFfPk9IiDDxmjOJjo1s6O2+0e65cyajRvQlPj6GdxbfzkOPLGXhq6u9Dutr8/MbKyBNdN19o4nIb3H6KnOAHTjniYbidMOtBF4FInH+Tveq6pMikojThdYHp6vtRmAv8LqqDm3i/QpVtTmJ62t3zQWaeRNmc9tSq4u7LpoNwHMZ93scifeuTvspAMPSjjvVGnTWZ9wHrZBH7n7vgWZ/0N96/ux2z1uB3CLCbTb+uZEio+tZZz/OBVl1NZqE3HX9tvVkjAle/n6vuYBORMYYY/y/a84S0dcgIquAumNWr1XV9V7EY4wxjbEWUQBSVf++wZoxxviwRGSMMcZToZaIjDHGeMnP85AlImOMCXTWNWeMMcZTfp6HLBEZY0ygsxaRMcYYT/l5HrJEZIwxga45NzP1kiUiY4wJcNY1Z4wxxlP+/rwfS0TGGBPgrEVkjDHGU36ehywRGWNMoLMWkTHGGE/ZqDljjDGeshaRMcYYT/l5HrJEZIwxgc5aRMYYYzxl1xEZY4zxVIio1yE0SlT9O8AAZZVujGmuE+5Yey7j/mZ/5lyd9tN278izRBTERGSWqj7mdRz+wOqihtVFDauL9uHvXYembc3yOgA/YnVRw+qihtVFO7BEZIwxxlOWiIwxxnjKElFws77vGlYXNawualhdtAMbrGCMMcZT1iIyxhjjKUtExhhjPGWJyBhjjKcsERljjEtEQkTkTK/jCDY2WCHIiMgyVR3f1LxgISKXAmfj3HbpQ1Vd6HFInhGRKcAQILJqnqre4V1E3hCRT1X1dK/jCCbWIgoSIhIpIglAVxHpLCIJ7qs30NPb6LwhIg8DNwDrgS+AH4vIQ95G5Q0ReQSYDtyMc2+zK4AUT4PyzjIRuUzE3x+eEDisRRQkROSnwM+AHsAen0VHgH+p6j89CcxDIvIlMEjdfwIRCQEyVXWQt5G1PxFZp6rDfX52BBar6jivY2tvIlIAxADlQClOYlZV7eRpYAHMHgMRJFT1fuB+EblZVR/0Oh4/sQU4Gch2p3u584JRifuzWER6AIeA7h7G4xlVjW1suYgMUdXM9oonGFjXXPCZLyK/E5HHAESkn4hM9Tooj8QCG0VkuYi8B2wAOonIIhFZ5HFs7e11EYkH/gpkANuB5z2NyH897XUAgca65oKMiLwArAWuU9WhIhINfKSqp3kcWrsTkXMbW66qK9orFq+JSAdVPVr1O86AhdKqeaaGDWZofdY1F3z6qup0EbkKQFWLg/WkbDAlmmb4GEgDcJPPURHJqJpnarFv763MElHwKRORKNx/JhHpCwTlt14ROQN4EBgERAChQFEwnZQWkSScUZNRInI6NU8D7QREexaYCSqWiILPXGAJ0EtEngXOAq73NCLv/BOYAbwEjASuA/p7GlH7uwjn758M/M1n/hFgjhcBecntHUhW1Z2NFCtrr3iChZ0jCjLutUQCnOH+/ASIVdUsTwPzgIikq+rIqiHL7ryg7P8XkctU9RWv4/AHIrJeVYd5HUcwsVFzwec14JiqvqGqrwMnufOCUbGIRACfichfROTnBO//xEoReVxEFgOIyGAR+YHXQXkkQ0RGeR1EMAnWf7pgdifwmojEiMgI4GVgpscxeeVanP+Bm4AinOuILvM0Iu/8B1iKc8EzwGacC6CD0RjgYxHZKiLrRGS9iKzzOqhAZl1zQUhEvg38Guc6mstUdbPHIXlCRGKAElWtdKdDgQ6qWuxtZO1PRNao6ijfrkkR+SxIh/XXe2sjVc2ub745cTZYIUiIyIPUHnYaB2wFbhIRVHW2N5F5ahkwASh0p6OAt4BgvPtykYh0oWY05RlAvrcheaY7zq2eCgBEpBPOyEpLRG3EElHwSK8zvdaTKPxLpKpWJSFUtdC9wDcY/QJYBPQRkZU45w4v9zYkz/w/al8/VVjPPNOKLBEFCVV90u16ekpVr/E6Hj9RJCJpqpoB4J4zK2linUC1AVgIFAMFwP9wzhMFI1GfcxaqWiki9lnZhqxyg4iqVohIiohEqKpdC+GcjH9JRPbgDGVPwnkUQjB6CufaoTvd6atx7ql2hWcReWebiMzGaQUB/B+wzcN4Ap4NVggyIvIUTn/3IpyRYgCo6t8aXCmAiUg4MMCd3KSqx3yWTVTVt72JrH2JyAZVHdzUvGAgIt2AB4ALcM6ZLQN+pqo5ngYWwKxFFHy2uq8QnFFzQc1NPF80sPgeICgSEc61M2eo6icAIjKG488rBgU34czwOo5gYi0iYxoQDHdZEJH1ON/6q1qGO9zpFODLIG0R/QX4E875wiXAcODnqvqMp4EFMGsRBRkROQnnGqIhOLf6B0BVL/AsKP8VDN/SgvVZVI25UFV/LSLfwXku06XA+4AlojZiiSj4PAu8gPMBdAPwXeCApxEZz9hFmvWq+lycArykqvlB+qSUdmO3+Ak+XVT1cZz7za1Q1e/jnJQNOu4D4Bqbt739ojF+5HUR+RIYASxzexFKPY4poFkiCj5Vo8L2isgU9xk0CV4G5KGPG5unqpe2YyzGT6jqrTh31xjpDmYpAqZ5G1Vgs6654PMnEYkDbsF5KFwn4OfehtS+7GFwpj4icoGqvisil/rM8y3y3/aPKjhYIgoy7qMfwLmP2PlexuIhexicqc85wLvAxTgDVaTOT0tEbcSGbwcZEUkFbgZ64/NFRFUv8Somr9jD4IwvEbmF4xMQ7u9Be9F3e7AWUfD5H/A4zsPwKj2OxWsrReRxoIeqThaRwcBYdzCHCT4d3Z8DgFHAqzjJ6GJgtVdBBQNrEQUZEVmlqmO8jsMfuE8j/Q/wW1U91b2x5af2mOjgJiLvA1N8HgMRC7yhqud4G1ngslFzwed+EZkrImNFJK3q5XVQHumqqi/itgxVtRyo8DYk4wcSAd+bApe580wbsa654DMM5xHZF1DTNacE57VE9jA4U5+ngNUistCd/jbwhHfhBD7rmgsyIrIFGGyPgQC3JfggMBTnxqcnAZer6jpPAzOec4+Nce7k+6r6qZfxBDprEQWfL4B4wG5pD4eBc3FOTguwCTjN04iMX3AflpjhdRzBwhJR8IkHvhSRNcDRqpnBOHwbeBm4RFUzAUTkHOAhnO5LY0w7sUQUfOZ6HYAfuQH4n4hcDKQBdwHf8jYkY4KPnSMytYjIx6o61us42ouIjAUexbmp5RRVtTuRG9POrEVk6opsusg3m4i8Ru1nDUXjjJZ7XESCtZvSGM9YIjJ1BUMT+V6vAzDG1LBEZIKOqq4QkVDgHVUN1hu/GuM37M4Kpq6geBSlqlYAle4jMYwxHrIWURASkUScmzoCrFZV32uKrvUgJK8UAutF5G2ch58BoKqzvQvJmOBjo+aCjIhcCfwVWI7T+hkH/EpVX/YyLi+IyHfrm6+qT7Z3LMYEM0tEQUZEPgcmVrWCROQknHMlp3obmTEmWFnXXPAJqdMVd4ggPVcoIv1wLmIdjM+wdVXt41lQxgQhS0TBZ4mILAWed6enA4s9jMdL/8G508TfcR6b/j2CNCkb4yXrmgtCInIpcLY7+YGqLmysfKASkbWqOkJE1lc9DK9qntexGRNMrEUUZETkHlX9DfDfeuYFm6MiEgJ8JSI3AbupeVy0MaadWDdE8JlYz7zJ7R6Ff/gpzu19ZgMjgJnAdZ5GZEwQshZRkBCRG4H/A/qIiO+D32KBld5E5TkFngZSgHB33r+A4Z5FZEwQsnNEQcK9g0BnnFFit/osKlDVXJ9ynVX1cHvH5wUR2QT8ClhPzWPTUdVsz4IyJghZIjK1iEiGqqZ5HUd7EJEPVfXspksaY9qSdc2ZuoLiXnOuuSLyb2AZtZ9W+9+GVzHGtDZLRKauYGoifw8YiHN+qKprTvEZUWiMaXuWiEwwG6WqA7wOwphgZ8O3TV3B1DX3kYgM9joIY4KdJaIgIyI/qGfe3T6T49sxHK+dAXwmIptEZJ2IrK8ztN0Y0w6say74XCYipar6LICIPETtG37mNrhm4JnkdQDGGEtEwegyYJGIVOJ8EOep6nGtpGBg1wsZ4x/sOqIgISIJPpOxwP9w7qjwewi6lpAxxo9YIgoSIpKFMzRZfH5WUXsGjzHGK5aIjDHGeMrOEQUZEQkHbgTOcWctBx5V1WOeBWWMCWrWIgoy7i1twoEn3VnXAhWq+kPvojLGBDNLREFGRD5X1VObmmeMMe3FLmgNPhUi0rdqQkT6ABUexmOMCXJ2jij4/Ap4T0S24YycS8G5+acxxnjCuuaCkIh0AKpu9rlJVY82Vt4YY9qStYiCjDtq7sf4jJoTERs1Z4zxjLWIgoyNmjPG+BtLREHGRs0ZY/yNjZoLPjZqzhjjV+wcUfDxHTUH0BsbNWeM8ZC1iILPSuBRoBLIdX//2NOIjDFBzc4RBRkReRE4AjzrzroaiFfVK7yLyhgTzCwRBRkR2aCqg5uaZ4wx7cW65oJPhoicUTUhImOAdA/jMcYEORusECREZD3OA/HCgY9EZIc7nQJ86WVsxpjgZl1zQUJEUhpbrqrZ7RWLMcb4skRkjDHGU3aOyBhjjKcsERljjPGUJSJjjDGeskRkjDHGU/8fv4LljUcyVT8AAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "##test data correlations\n",
        "sns.heatmap(test_df.corr(), cmap=\"crest\", annot=True, linewidth=.5)\n",
        "# There is a negative weak correlation between beta and discr_r (target). \n",
        "# There is a positive very weak correlation between market_cap and discr_r (target). \n",
        "# There is a negative very weak correlation between book_to_market and discr_r (target)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 355
        },
        "id": "a93TqO04eFAD",
        "outputId": "5426973e-6201-4595-a327-083c7670501a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc0228b4430>"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAFBCAYAAAA126tDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXgV5dn48e+dHZJACHsEQ1hkUXbEBVEUcKmKdQUBAUVRW4u21ta+rctPautS26rVV3EpuFt9i1AUlB3BhSVoIpuEhFB2JAskBEKS+/fHTMJJyApJ5njO/bmuc+XMnGfm3PNkzrnP88wzM6KqGGOMMV4J8ToAY4wxwc0SkTHGGE9ZIjLGGOMpS0TGGGM8ZYnIGGOMpywRGWOM8ZQlImOMMbUmIq+LyD4R+a6K10VEnhORNBFJEZEBNa3TEpExxpi6mAFcXs3rVwDd3McU4H9rWqElImOMMbWmqsuBrGqKXAO8oY6vgDgRaV/dOsPqM0BTa3Y5C2NMbcmprqD3gPtr/Z3z3bq/3onTkik1XVWn1+HtTgP+6zO9w523u6oFLBF5pPeA+70OwS+kJj/DC18853UYnvv5+VMBeOUrq4s7znXqYrrVBVPcumhMbtKpS+I5ZZaIjDEm0J1ym6pOdgIdfaY7uPOqZMeIjDEm0InU/nHq5gAT3NFz5wK5qlpltxxYi8gYYwJfPbaIRORdYBjQSkR2AI8A4QCq+hLwCfATIA04DNxa0zotERljTKCrx0SkqjfX8LoCP6/LOi0RGWNMwGvcg0R1ZYnIGGMCnPp3HrJEZIwxAS/EvzORjZozxhjjKWsRGWNMoPPvBpElImOMCXj1c35Qg7FEZIwxgc6/85AlImOMCXRqLSJjjDGe8u88ZInIGGMCn39nIktExhgT6Pw7D1kiMsaYgGeJyBhjjJdssIIxxhhvWSIyxhjjKf/OQ5aIjDEm0KnXAdTAEpExxgQ665oz/uaxR0Zz4dCeZGXlcd1Nf/E6nJOmqix/53O2pWQSFhHOyMnDadOp9Qnl9m3bx4JXF1F0rIhOfRK5cOxQRIQjeUeY97+fcvCHQzRrFcsVP7uMqOioKte7f/t+lryxjMKCQiQkhLOvGsgZ53QDYOHri9m3bR+qENcujpGTLyEiKqKxq+QEqsritz8n41tnW664YzhtK6mjPRn7mP/qIooKi0jqm8gl45w6WvreStK/2UZIaChxbZpx+e3DiYqOZPfWvXw2Y4n7JnD+TwfTbVDnRt66ulFVlvjUxeVV1MXeCnVxsVsXm1el8eWsVRzYnc24R26kXVIbAIqLivns9SXsy9xPSbHSa0h3zrl6YGNvXvX8Ow/VfBsIEekkIt+d6huJyDYRaVWLcnEi8rNTfb/6ICKTROQfdSh/n4g0bciY6sPs/6zm7nte8TqMU5aZkknO3lwmPDGeSyYNY8mbSystt+SNZVxy68VMeGI8OXtzyUzdDsCaT5Lp2KsDE58cT8deHVj7cXK16w2LCOfS20cw/vGxXPOrq1n+7gqOHj4KwNCbL2DsY2MYN20MsfExpCxKbfDtr42MlEyy9+Qy+anxXHrrMBbMXFppuYUzl3HprRcz+anxZO/JJSPFqaNOZ3Zk0uM3M+nxMbRoF8fXc9cC0KpDPLc8ehMTp43h+l9fzWczllJSXNJYm3VSSuvitqfGM/LWYSyspi5G3noxt7l1sc2ti1Yd4hk19Qo6dE8oV/771VspLipm4uM3M/7/3UjK0vXk7j/Y0JtTNyK1f3jAH+9HFAd4nohE5GRai/cBfp+I1iank5t72OswTln6ugx6nN8dEaF9l3YcPVxIfk5+uTL5OfkUFhTSvks7RIQe53cnPTm9bPmeQ3oA0HNID7auy6h2vS3axRHXLg6AmBbRNG3WhIKDBQBENnFaP6pK8bEi/OUnaFpyBmcOcbYloauzLXkV6igvJ5/CI4UkdHXq6Mwh3Ulz66hT79MJCXW+JhK6tCMvOw+A8MjwsvlFx4r9vecHgK3JGfSqRV0c9amLXj510TIhnvj2LSpd97GjRZQUl1B0rJjQ0BAimnjfGvalUvuHF2qbiMJE5G0R2SgiH4pIUxEZLiLrRCRVRF4XkUiAquaXEpEmIjJPRO6o4r2eALqIyDci8rQ4nhaR79x1jq4qSBEZJiLLRGS2iKSLyBMiMk5EVrnLdnHLXS0iX7txLhSRtu78R0XkTRFZCbxZYd1XisiXItJKRC51nyeLyAciEiMiU4EEYImILKllvZpTkJeTT2x8TNl0TIto8rIrfLFk5xPjWyY+puzL53DuYaLjogFo2rwph93kXJv17knfS3FRCc3bNC+bt+C1Rbx63z/J2p1D3xG962krT01edj6xLY9vS2x8FXXUwrdMzAllAFI/30hS78Sy6d1b9/DP373DzN+/y8iJw8oSk7+qbV3E1qIufJ1xdhfCI8N46d5/Mv2XMxl0RX+axETVb/CnKkBaRN2BF1W1J3AQ+BUwAxitqr1xjjXdLSJRlc33WU8M8B/gXVWtqm/oQWCrqvZT1QeA64B+QF9gBPC0iLSvJta+wF1AT+AW4AxVHQy8CvzCLbMCOFdV+wPvAb/xWb4XMEJVby6dISLXunH9xJ31B7fMAGAN8CtVfQ7YBVysqhdXDEpEpojIGhFZM3369GrCN14QEaSWH8L8nHw+e2UhIyZfgvjcgnnk5OFM/tsk4tu3YMuqtIYK1RNfzVlDSIjQ8/wzyua179KOW/88lvGP3sjXc9dSVFjkYYTe2ZO+DwkR7vz7JO545hbWzP+GnH25Xof1o1Lb7qf/qupK9/lbwENAhqp+786bCfwcWFLF/L+707OBp1T17TrEeAFO4ioG9orIMuBsYE4V5Ver6m4AEdkKfObOTwVKE0QH4H03oUUAGT7Lz1HVAp/pS4BBwKWqelBErsJJVivdL64I4MuaNkJVpwOlGUiff+n+mhYxlfh2USrrl60HoG1SWw5l5ZW95vyyjy5XPqZFNHm+ZbLyiPFpBeXn5BMdF01+Tj5NmjVxlomLrnK9RwsKmfO3uZx33Tm079LuhPhCQkI445xurJ2XTK+hPetpq+tm3cJUUtw6apfUlkMHjm/Loawq6ijbt0xeuTLffb6Rrd9s46bfXlNpsm6ZEE94VDg/7MwqO4DvL9YtTCW1jnVxqJq6qMzGr74nqXcioWGhNG3WlIRu7dibsY84n9ay1/z9ygq1bRFVHIaec5LvtxK4XGr70/PkHPV5XuIzXcLxxPs88A+31XYn4NuOrtgO3wrEAqU/BQVY4LbY+qlqL1WdXJ8bYKrWd3hvxj42hrGPjaHzgCQ2fbEZVWX31j1ENoko62orFR0XTUSTCHZv3YOqsumLzXTunwRA536d2LhyEwAbV24qm5/Uv/L1FhcV8/Hzn9BjSA+6nd217D1UlZy9OWXP09dl0KJd5ccSGkP/Eb2ZOG0ME6eNoeuAJNavdLZlV5qzLTEV6igmLpqIqAh2pTl1tH7lZroOcOoiIyWTVZ+s49r7riQ8MrxsmZz9B8sGJ+T+cJCs3dk0axXbeBtZS/1H9GbCtDFMcOtiQy3qItKnLjas3EwXty6q0qxlLNs37ADg2NFj7N66t8pjSZ6ROjw8UNsW0ekicp6qfgmMxemOulNEuqpqGk4X2DJgM9CpkvmlHnYfL1D1gIRDOF/8pT5332smEA9cCDxQy7ir0hzY6T6fWEPZTPf9/i0iNwJfAS+UbqOIRAOnua3A0th/OMX4GtSTfxrP2QO7EBcXzcJ5D/HCS58ya/Yqr8Oqs059EtmWksnM375FeEQYIyYPL3vtnYffY+xjYwAYdstFLHjNGY7bqXciiX2c4xwDrxzIvBfns375Rmf49t2XVbveLavS2PX9bo7kHWHjio0AjLx9OK06tGLBq4soLChEgdYdWzJswrDGq4hqdO6bSEZKJq8+8BbhkWFcfvvxOpr50HtMnObU0YiJFzHvFXfIcp9Ektw6WvTmcoqLSvjg6dmAM2Bh5KRh7Px+N7PmriUkLAQRYcSEi2ga26TxN7AOkvomkp6SyWtuXVzmUxdvPPQeE9y6GD7xIuZXUhdb1qSz+K3lFBwqYNZf59L69Fbc8MAo+g0/i09fXcyM372DAmcN7UHr02scINyo/L1FJKrVn3MrIp2A+TjJZyCwASfBnAf8BSeZrQbuVtWjIjK8ivnbcLq4DgCvA/tV9TdUQkTeAfoA83CO3zwFXIHTMvujqr5fxXLDgF+r6lXu9FJ3eo3vayJyDfA3IBtYDJytqsNE5FEgT1X/4i4/CRikqveISH/gbeBqIBF4EigdiPEHVZ0jIr8A7gF2VXacyIf2HmBdcwCpyc/wwhfPeR2G535+/lQAXvnK6uKOc526mG51wRSnLk45i5xx1aO1vrjC93MfbfSsVWOLSFW3AT0qeWkR0L+S8lXN7+QzeWsN7zm2wqwHqEUrSFWXAkt9podV9pqqzsY5XlVx+UcrTM/AGXyBqq7DOTYETnfd2ZUs/zxOt58xxvgP/24Q2ZUVjDEm8Pl3JvIsEYlIS5xWVUXDVfVADcv2psJ5PsBRVT2nvuIzxphA4dWJqrXlWSJyk02/k1w29WSXNcaYoGOJyBhjjJf8fdScJSJjjAl0/p2H/PKip8YYY+pV/Z3RKiKXi8hmEUkTkQcref10EVniXsszRUR+Utl6fFkiMsaYAFdfV98WkVCcCxJcgXM6y80i0qtCsT8A/3Kv5TkGeLGm+CwRGWNMoKu/BtFgIE1V01W1EOei0ddUKKNAM/d5c5yLQVfLEpExxgS82mci3zsFuI8pPis6Dfivz/QOd56vR4HxIrID+ITjdz2okg1WMMaYAKd1aHJUuFPAybgZmKGqz4jIecCbInKWqlZ5C19rERljTKCrv665nUBHn+kOHL+AdKnJwL8A3AtlRwHVXgXWEpExxgS8estEq4FuIpIkIhE4gxEq3htuOzAcQER64iSi/dWt1LrmjDEmwNXXJX5UtUhE7gE+BUKB11V1vYg8BqxR1TnA/cArIvJLnIELk7SG2zxYIjLGmEBXjye0quonOIMQfOc97PN8AzCkLuu0RGSMMYHOLvFjjDHGS/5+9W0brGCMMcZT1iIyxphA5+ctIktExhgT6OwYkTHGGC9VO3baD1giMsaYQOfnowGkhvOMTMOwSjfG1NYp96slTvhzrb9zMt/4XaP341mLyCMvfPGc1yH4hZ+fP5XeA+73OgzPpSY/A8D0r2y/mHLuVACeW2F1MfWCqfWzIjtGZIwxxkt2HpExxhhTDWsRGWNMoPPzJoclImOMCXj+3TdnicgYYwKdf+chS0TGGBPwLBEZY4zxkr+fuGiJyBhjAp21iIwxxnjKTmg1xhjjKf/OQ/4+utwYY0ygsxaRMcYEOj9vEVkiMsaYQOfnici65owxxnjKWkTGGBPgJMS/m0TWIjLGGOMpaxEZY0yg8+8GkSUiY4wJeH6eiKxrzhhjjKesRWSMMQHOz6/wY4nIGGMCniUiY4wxnrJE1LBEZBjwa1W9qpblJwGfqequhoyroagqy9/5nG0pmYRFhDNy8nDadGp9Qrl92/ax4NVFFB0rolOfRC4cOxQR4UjeEeb976cc/OEQzVrFcsXPLiMqOqrK9e7fvp8lbyyjsKAQCQnh7KsGcsY53QBY+Ppi9m3bhyrEtYtj5ORLiIiKaOwqOSWPPTKaC4f2JCsrj+tu+ovX4dQ7VWXJ25+T8a3zf738juG0rWR/2Zuxj/mvLqKosIikvolcPM7ZXzavSuPLWas4sDubcY/cSLukNmXL7N/+AwtmLHX3DWHcIzcSFvHj+ErJTM1kxbsrKNESeg3txcCfDCz3evGxYha+tpB9mfuIio7isrsuo1mrZhzJO8L8F+ezd9teeg7pyYXjLvRoC+rGz/PQj3uwgoiczF4/CUio51AaTWZKJjl7c5nwxHgumTSMJW8urbTckjeWccmtFzPhifHk7M0lM3U7AGs+SaZjrw5MfHI8HXt1YO3HydWuNywinEtvH8H4x8dyza+uZvm7Kzh6+CgAQ2++gLGPjWHctDHExseQsii1wbe/vs3+z2ruvucVr8NoMBkpmWTvyeW2p8Yz8tZhLJy5tNJyC2cuY+StF3PbU+PJ3pPLthRnf2nVIZ5RU6+gQ/fyH5mS4hI+eXkhIyYNY9Kfx3LT764lJOzH8XVSUlLC8reXc9Uvr2LstLFs+XoLWbuyypXZ8PkGIptGcsufb6HfyH58+eGXAISGhzL42sEMuWmIF6GfPJHaP2pclVwuIptFJE1EHqyizE0iskFE1ovIOzWt05M9R0Q6icgmEZkhIt+LyNsiMkJEVorIFhEZ7D6+FJF1IvKFiHR3l50kInNEZDGwqMJ6z3bLdxGRgSKyTETWisinItJeRG4ABgFvi8g3ItKkivjOdt/zWxFZJSKxbsyfi0iy+zjfLTtMRJaLyMfuP+clEWmwek1fl0GP87sjIrTv0o6jhwvJz8kvVyY/J5/CgkLad2mHiNDj/O6kJ6eXLd9zSA8Aeg7pwdZ1GdWut0W7OOLaxQEQ0yKaps2aUHCwAIDIJk7rR1UpPlaE///uOtHa5HRycw97HUaD2ZqcQa8hzv81oavzf82rsL/k5eRz9EghCV2d/aXXkO6kuftLy4R44tu3OGG9277bTuuOLWlzeisAmsREERLy40hE+9L30bxNc5q3bk5oWCjdBncjw/0clMr4JoMe5zufky6DurBj4w5UlfDIcBK6JRAaFupF6CetvvKQiIQCLwBXAL2Am0WkV4Uy3YDfAUNU9Uzgvpri87Id3RW4EbgNWA2MBS4ARgH/A0wAhqpqkYiMAP4EXO8uOwDoo6pZbtccbmJ4HrgG2A28CVyjqvtFZDTwuKreJiL34HTlraksKBGJAN4HRqvqahFpBhQA+4CRqnrEreh3cZIawGCcf0omMB+4DviwPiqporycfGLjY8qmY1pEk5edT3Rc9PEy2fnE+JaJjyn78jmce7isbNPmTTnsfgnXZr170vdSXFRC8zbNy+YteG0R21IyiU+I54LRP7JfiUEgLzuf2JbH/6+x8c7/NabC/hLbwrdMDHnZ5ZNVRdl7ckHgw6fnUHCogO7ndGPwlQPqfwMaQF5OXvnPR4sY9mbsLVcm3+czFBIaQkSTCI7kHaFJbKW/Xf1f/f1GHAykqWo6gIi8h/Odu8GnzB3AC6qaDaCq+2paqZeJKENVUwFEZD2wSFVVRFKBTkBzYKb7pa9AuM+yC1TVty3dE5gOXKqqu0TkLOAsYIE4KT4UJznVRndgt6quBlDVg26M0cA/RKQfUAyc4bPMKp9/zLs4CbVcIhKRKcAUgJdfftmJzmMigtRyXGd+Tj6fvbKQkbcPL3fdqpGTh1NSUsKytz5ny6o0eg3t2VDhGj9SUlzCzu93M+7RGwmPCOODJ2fTtlNrEs/s6HVophJ1yUO+31Wu6ao63X1+GvBfn9d2AOdUWMUZ7npW4nz3Pqqq86t7Ty8T0VGf5yU+0yU4cU0DlqjqtSLSCVjqU77iz7XdQBTQH9iFU+/rVfW8eoz3l8BeoC9Ol+YRn9e0QtmK07j/yNJ/pr7wxXO1fuNvF6Wyftl6ANomteVQVl7Za3nZ+cS0iC5XPqZFNHm+ZbLyyn4BN23elPwcp6WTn5NPk2bOL7yYuOgq13u0oJA5f5vLededQ/su7U6ILyQkhDPO6cbaecmWiPzAuoWppLr7S7ukthw6cPz/eiir8v3lULZvmbwTylQUGx9Dh+4JNHVbCEl9E9mXuf9HkYhi4mLKfz6y88q1/AGi3c9QTHwMJcUlFBYUEhUT1dih1p86ZKIK31UnIwzoBgwDOgDLRaS3quZUtYA/d+o2B3a6zyfVUDYHuBL4s9tVtxloLSLnAYhIuIic6ZY9BMRWs67NQHsROdtdNtYdFNEcp6VUAtyCk+lLDRaRJPfY0GhgRe02sXb6Du/N2MfGMPaxMXQekMSmLzajquzeuofIJhEnfojiooloEsHurXtQVTZ9sZnO/ZMA6NyvExtXbgJg48pNZfOT+le+3uKiYj5+/hN6DOlBt7O7lr2HqpKzN6fsefq6DFq0O/FYgml8/Uf0ZsK0MUyYNoauA5LYsNL5v+5Kc/6vMRX2l5i4aCKjItiV5uwvG1ZupsuApGrfo1Pvjvyw4wDHjh6jpLiEHZt20TIhviE3q960SWpD7t5cDu4/SHFRMVtWbaFTv07lyiT1S2LTF87nZOuarZzW47Ra9x74o3ocq7AT8P210YHj39OldgBzVPWYqmYA3+Mkpir581jLp3C65v4AfFxTYVXdKyJXAfNwjjvdADwnIs1xtvPvwHpgBvCSiBQA56lqQYX1FLrHlJ53BzMUACOAF4H/E5EJOMeBfFtlq4F/4Bz3WgLMOumtrkGnPolsS8lk5m/fIjwijBGTh5e99s7D7zH2sTEADLvlIha85gzH7dQ7kcQ+iQAMvHIg816cz/rlG53h23dfVu16t6xKY9f3uzmSd4SNKzYCMPL24bTq0IoFry6isKAQBVp3bMmwCcMaarMbzJN/Gs/ZA7sQFxfNwnkP8cJLnzJr9iqvw6o3SX0TSU/J5LUH3iI8MozLbj++v7zx0HtMmObsL8MnXsT8V9zh230SSXL3ly1r0ln81nIKDhUw669zaX16K254YBRR0VEMvKwfbz/6AYiQ1DeRzhW+zP1VSGgIQ8cNZc7f5qAlSs8LetLytJZ8/dHXtOnUhqR+SfQc2pOFryzkzd+9SVR0FJfeeWnZ8m/85g0KCwopLi4mfV06o341ivgfSRKuB6uBbiKShJOAxuAc3/f1EXAz8E8RaYXTVZde3UpF9YReJFMHdT2PyVWnrrlA9vPzp9J7wP1eh+G51ORnAJj+le0XU86dCsBzK6wupl4wFephqEHX3/yl1l/0aU/9utr3E5Gf4PywDwVeV9XHReQxYI2qzhGn6fgMcDnO8fTHVfW96tbpzy0iY4wx9aA+exVV9RPgkwrzHvZ5rsCv3EetBHUiEpFZQMXO8N+q6qe1XYeqLqX8QApjjPEvfn54K6gTkape63UMxhjT0MTPM1FQJyJjjAkK/p2HLBEZY0yg8/M8ZInIGGMCXcNd/bJ+WCIyxpgAZy0iY4wx3vLzTGSJyBhjApyf5yFLRMYYE+j8/TJ5loiMMSbQWSIyxhjjpRA/T0R+PqjPGGNMoLMWkTHGBDg7RmSMMcZbloiMMcZ4yS56aowxxlPWNWeMMcZTloiMMcZ4ys/zkCUiY4wJeH6eiSwRGWNMgPP3rjlRVa9jCEZW6caY2jrlNDLo8b/W+jtnze9/1ehpy1pExhgT4Py9RWSJyCOvfPWc1yH4hTvOncp0qwumnDsVgN4D7vc4Eu+lJj8DwDvJz3ociffGDri3XtZjicgYY4y3LBEZY4zxkp/nIUtExhgT6KxrzhhjjMf8OxNZIjLGmAAnfn7nOUtExhgT4Py7PWSJyBhjAp6/HyPy8wabMcaYQGctImOMCXDWIjLGGOMpkdo/al6XXC4im0UkTUQerKbc9SKiIjKopnVaIjLGmABXX4lIREKBF4ArgF7AzSLSq5JyscC9wNe1ic8SkTHGBDipw6MGg4E0VU1X1ULgPeCaSspNA54EjtQmPktExhgT6OqQiURkiois8XlM8VnTacB/faZ3uPOOv5XIAKCjqn5c2/BssIIxxgS4uoxVUNXpwPSTeh+REOCvwKS6LGeJyBhjAlw9jprbCXT0me7gzisVC5wFLBXnTdsBc0RklKquqWqlloiMMSbASf1lotVANxFJwklAY4CxpS+qai7Qyud9lwK/ri4JgR0jMsaYwFdPoxVUtQi4B/gU2Aj8S1XXi8hjIjLqZMOzFpExxgS4+jyfVVU/AT6pMO/hKsoOq806LREZY0yA8/crK1giMsaYQGeJyBhjjJf8PA9ZIgIQkU7AXFU9q5blJwGfqequBgyrTlSVxW9/Tsa3mYRFhHPFHcNp26n1CeX2ZOxj/quLKCosIqlvIpeMG4qIsPS9laR/s42Q0FDi2jTj8tuHExUdye6te/lsxhL3TeD8nw6m26DOjbx1daOqLPGpi8urqIu9FeriYrcuNq9K48tZqziwO5txj9xIu6Q2Zcvs3/4DC2YspbCgEAkRxj1yI2ERgfExeuyR0Vw4tCdZWXlcd9NfvA6n3qkq82euYMs3mYRHhPHTu4fTPunE/WJX+j5mv7SYY4VFdOuXyOUTLygbdfb1/BRWL/iOEBG69U9k5Ljzydl/kBfuf5eWCXEAdOjalqtuH9aYm1Yj65oLTJOA7wC/SUQZKZlk78ll8lPj2b11LwtmLmX8IzeeUG7hzGVceuvFtO/Slv97Zi4ZKdvp3DeRTmd25MIbzyMkNIRl73/B13PXctHo82nVIZ5bHr2JkNAQ8nLymfmH9+nSvxMhof474LK0Lm5z62LhzKWMq6IuRrp18e9n5rItZTtJfRNp1SGeUVOvYMGMpeXKlxSX8MnLC7nizhG0Ob0VBXlHCAnz33qoq9n/Wc2776/g8cdu9jqUBpH2zXay9uTyi7+NY2faXj5+bRm3//GGE8p9/Ppyrr5jGKd1bcs7T35M2rfb6dYvkYz1O9m8dht3PTGasPBQ8nMPly3Tom1z7npidGNuTp34eyIKnE/RqQsTkbdFZKOIfCgiTUVkoIgsE5G1IvKpiLQXkRuAQcDbIvKNiDQRkYdFZLWIfCci06UeB+3XVlpyBmcO6Y6IkNC1HUcPF5KXk1+uTF5OPoVHCkno2g4R4cwh3UlLTgegU+/Ty5JLQpd25GXnARAeGV42v+hYsd/v0ABbkzPoVYu6OOpTF7186qJlQjzx7VucsN5t322ndceWtDndOU2iSUwUISGB8xFam5xOrs+Xa6DZtDaDPkOd/aJDt3YcOVzIoezy+8Wh7HyOFhTSoZuzX/QZ2p1NazIAWLPgOy4Y1Z+w8FAAops3bfRtOFn1efXthmAtouO6A5NVdaWIvA78HLgWuEZV94vIaOBxVb1NRO7B5yQtEfmHqj7mPn8TuAr4T2MGn5edT2zLmLLp2Pho8rLziYmLLlcmpoVvmRjyKnwQAVI/30iPwV3Lpndv3cP8Vxdz8Kf8TbMAABzbSURBVMAhfjJlpF+3hqD2dRFbi7rwlb0nFwQ+fHoOBYcK6H5ONwZfOaD+N8A0iENZ+TT32S+axUdzKCuf2BbR5co0i/cp09IpA3BgTw6Zm3az+P2vCQsPY+T48zitS1sAcvYf5OUH/0VkkwguHj2YxB4JjbRVgcES0XH/VdWV7vO3gP/BuVTFAreBEwrsrmLZi0XkN0BTIB5YT4VE5F44cArAyy+/jPSp9/jrxVdz1hASIvQ8/4yyee27tOPWP4/lwK4s5k1fRFKf0wPmuEhdlBSXsPP73Yx79EbCI8L44MnZtO3UmsQzO9a8sPnRKylWCvKOMnna9ezauo8Pn/2Mqc+OJyYumvuen0DT2Ch2pe/j/Wfm8bOnbyayaYTXIZfx956M4Ps2qZpWmD4ErFfV86pbSESigBeBQar6XxF5FIg6YeXlLySor3z13CkHvG5hKinL1gPQLqkthw7kHQ8+K58Yn196ADEtosu63JwyeeXKfPf5RrZ+s42bfntNpZcEaZkQT3hUOD/szCp3AN8frFuYSmod6+JQNXVRmdj4GDp0T6BpbBMAkvomsi9zvyUiP7bqs1SSF28AIKFzG3J99ouDWfnExpf/n8fGR3Mwy6fMgeNlmsVH03NwZ0SE07q2RUQ4fOgI0c2alHXXJXRuQ4u2zTmwO4eELv7zGfH3ROTffSyN63QRKU06Y4GvgNal80QkXETOdF8/hHNxPziedH4QkRjgxKOfDaT/iN5MnDaGidPG0HVAEutXbkZV2ZW2h8gmEeW6ogBi4qKJiIpgV9oeVJX1KzfTdUAS4BzgX/XJOq6970rCI8PLlsnZf5CS4hIAcn84SNbubJq1isXf9B/RmwnTxjDBrYsNtaiLSJ+62LByM13cuqhKp94d+WHHAY4dPUZJcQk7Nu2iZUJ8Q26WOUWDL+3NXU+M5q4nRtNjUBIpnzv7xY4te4hsGlGuWw4gtkU0kU0i2LHF2S9SPt9Mj4HOftFjUBLbNjjX9zywO4fiomKaxkaRf7CAkhLnM5K9N5esPbm0aNuscTe0BnaM6MdjM/Bz9/jQBuB5nOspPScizXHq6u843W4zgJdEpAA4D3gFZxTdHpyLAja6zn0TyUjJ5NUH3iI8MozLbx9e9trMh95j4rQxAIyYeBHzXnGHLPdJJKlPIgCL3lxOcVEJHzw9G3AGLIycNIyd3+9m1ty1hISFICKMmHBRWYvAXyX1TSQ9JZPX3Lq4zKcu3njoPSa4dTF84kXMr6QutqxJZ/Fbyyk4VMCsv86l9emtuOGBUURFRzHwsn68/egHIEJS30Q69+vkxSY2iCf/NJ6zB3YhLi6ahfMe4oWXPmXW7FVeh1VvuvVPZMs323n+vrcJjwzjmjsvKXvtpQffLxv1duWtF/LRS4spKiyia7/T6drvdAD6X9yT2S8t5sUH3iM0LISf3j0cESFz4y6WfrCq7DNy5eSLaBJzQqeIp/y8QYSoVuyRMo2gXrrmAsEd505lutUFU86dCkDvAfd7HIn3UpOfAeCd5Gc9jsR7YwfcC/WQR6569dlaf9HPvf3eRs9b1iIyxpgA5+8tIktExhgT4Px9sIIlImOMCXCWiIwxxnjKEpExxhhP+XkeskRkjDEBz88zkSUiY4wJcH6ehywRGWNMoLNjRMYYYzxlicgYY4ynLBEZY4zxlJ/nIUtExhgT6KxFZIwxxlOWiIwxxnjKEpExxhhP+XkeskRkjDGBzlpExhhjPOXnecgSkTHGBDw/z0SWiIwxJsBZ15wxxhhPhfh5IgrxOgBjjDENS+rwqHFdIpeLyGYRSRORByt5/VciskFEUkRkkYgk1rROS0TGGBPgRGr/qH49Egq8AFwB9AJuFpFeFYqtAwapah/gQ+CpGuNT1ZPZLnNqrNKNMbV1yh1rt33wbK2/c16/8d4q309EzgMeVdXL3OnfAajqn6so3x/4h6oOqe497RiRR6Z/9ZzXIfiFKedO5bkVVhdTL5gKwDvJz3ociffGDrgXgN4D7vc4Eu+lJj9TL+upy2AFEZkCTPGZNV1Vp7vPTwP+6/PaDuCcalY3GZhX03taIjLGmEBXh0TkJp3pNRas6S1FxgODgItqKmuJyBhjAlw9jprbCXT0me7gzitHREYAvwcuUtWjNcZXb+EZY4zxS/U4am410E1EkkQkAhgDzCn3Xs5xoZeBUaq6rzbxWYvIGGMCXH2d0KqqRSJyD/ApEAq8rqrrReQxYI2qzgGeBmKAD8R54+2qOqq69VoiMsaYAFef57Oq6ifAJxXmPezzfERd12mJyBhjApxd4scYY4yn/P0SP5aIjDEmwFmLyBhjjKf8PA9ZIjLGmEBnLSJjjDGe8vM8ZInIGGMCnbWIjDHGeMpGzRljjPGUn+chS0TGGBPorGvOGGOMpywRGWOM8ZSf5yFLRMYYE+hC/LxJZInIGGMCnJ/nIUtExhgT6Pw8D1kiMsaYQGfnEZlGoaoseftzMr7NJCwinMvvGE7bTq1PKLc3Yx/zX11EUWERSX0TuXjcUESEzavS+HLWKg7szmbcIzfSLqkNAMVFxXz2+hL2Ze6npFjpNaQ751w9sLE376Rlpmay4t0VlGgJvYb2YuBPysdefKyYha8tZF/mPqKio7jsrsto1qoZR/KOMP/F+ezdtpeeQ3py4bgLPdqCU6OqzJ+5gi3fZBIeEcZP7x5O+6QT94td6fuY/dJijhUW0a1fIpdPvAD37pp8PT+F1Qu+I0SEbv0TGTnufHL2H+SF+9+lZUIcAB26tuWq24c15qY1qMceGc2FQ3uSlZXHdTf9xetwTpmf56HgSUQi8iiQBzQDlqvqQm8jql8ZKZlk78nltqfGs3vrXhbOXMq4R248odzCmcsYeevFtO/Sln8/M5dtKdtJ6ptIqw7xjJp6BQtmLC1X/vvVWykuKmbi4zdz7OgxZvzPu/Q4txvNWzdrpC07eSUlJSx/ezmj7h9FTIsYPpj2AUn9kohPiC8rs+HzDUQ2jeSWP9/Clq+38OWHX3LZXZcRGh7K4GsHk7Uzi6ydWR5uxalJ+2Y7WXty+cXfxrEzbS8fv7aM2/94wwnlPn59OVffMYzTurblnSc/Ju3b7XTrl0jG+p1sXruNu54YTVh4KPm5h8uWadG2OXc9MboxN6fRzP7Pat59fwWPP3az16HUC38/RhTidQCNTVUfPpUkJI4a601EQk/2PU7G1uQMeg3pjoiQ0LUdRw8XkpeTX65MXk4+R48UktC1HSJCryHdSUtOB6BlQjzx7VtUuu5jR4soKS6h6FgxoaEhRDSJaPDtqQ/70vfRvE1zmrduTmhYKN0GdyNjXUa5MhnfZNDj/B4AdBnUhR0bd6CqhEeGk9AtgdCwRv031rtNazPoM9TZLzp0a8eRw4Ucyi6/XxzKzudoQSEdujn7RZ+h3dm0xqmnNQu+44JR/QkLd+ohunnTRt8GL6xNTifXJ+n+2IVI7R+exOfN2zYOEfm9iHwvIiuA7u68GSJyg/v8CRHZICIpIvIXd15bEZklIt+6j/NFpJOIbBaRN4DvgI5VvF+eiDwjIt8C5zXOVjrysvOJbRlTNh0bH01ehS+cvOx8Ylv4lok5oUxFZ5zdhfDIMF66959M/+VMBl3RnyYxUfUbfAPJy8kjJv749sa0iCG/QnLOz84vKxPiJtkjeUcaNc6GdCgrn+Y++0Wz+GgOZeWfUKaZTz01a3m8zIE9OWRu2s2rf/iQGf/vI3Zu3VtWLmf/QV5+8F/M+H8fkblpVwNviTkVUoeHFwK2a05EBgJjgH4425kMrPV5vSVwLdBDVVVE4tyXngOWqeq1bqsmBmgBdAMmqupX1bxtNPC1qt5fSTxTgCkAL7/8MvQ51S1sHHvS9yEhwp1/n8TRw0d57/FZnH5mB+LaNPc6NNMISoqVgryjTJ52Pbu27uPDZz9j6rPjiYmL5r7nJ9A0Nopd6ft4/5l5/Ozpm4ls+uNoLQcbf++aC9hEBAwFZqnqYQARmVPh9VzgCPCaiMwF5rrzLwEmAKhqMZArIi2AzBqSEEAx8H+VvaCq04HppZPTv3qujptzonULU0ldth6AdkltOXQgr+y1Q1n5xLSILlc+pkU0h7J9y+SdUKaijV99T1LvRELDQmnarCkJ3dqxN2PfjyIRxcTFkJd1fHvzsvOIjiu/vdEtosnLclpOJcUlFBYUEvUjafFVZdVnqSQv3gBAQuc25PrsFwez8omNL18HsfHRHPSpp4MHjpdpFh9Nz8GdERFO69oWEeHwoSNEN2tS1l2X0LkNLdo258DuHBK6tGnozTMnwc/zUGB3zVVHVYuAwcCHwFXA/BoWqb4Py3HETV6Nov+I3kyYNoYJ08bQdUASG1ZuRlXZlbaHyCYRxFT40o2JiyYyKoJdaXtQVTas3EyXAUnVvkezlrFs37ADgGNHj7F7694qjyX5mzZJbcjdm8vB/QcpLipmy6otdOrXqVyZpH5JbPpiEwBb12zltB6nlY0W+7EafGlv7npiNHc9MZoeg5JI+dzZL3Zs2UNk0whiK/z4iG0RTWSTCHZscfaLlM8302Ogs1/0GJTEtg07ATiwO4fiomKaxkaRf7CAkpISALL35pK1J5cWbf1/AEuw8vdjRIHcIloOzBCRP+Ns59XAy6UvikgM0FRVPxGRlUC6+9Ii4G7g7z5dc34vqW8i6SmZvPbAW4RHhnHZ7cPLXnvjofeYMG0MAMMnXsT8V9zh230SSeqTCMCWNeksfms5BYcKmPXXubQ+vRU3PDCKfsPP4tNXFzPjd++gwFlDe9D69FZebGKdhYSGMHTcUOb8bQ5aovS8oCctT2vJ1x99TZtObUjql0TPoT1Z+MpC3vzdm0RFR3HpnZeWLf/Gb96gsKCQ4uJi0telM+pXo8qNuPsx6NY/kS3fbOf5+94mPDKMa+68pOy1lx58v2zU25W3XshHLy2mqLCIrv1Op2u/0wHof3FPZr+0mBcfeI/QsBB+evdwRITMjbtY+sEqQsJCEBGunHzRj+bYYW08+afxnD2wC3Fx0Syc9xAvvPQps2av8jqsk+bvP61EVb2OocGIyO+BicA+YDvOcaKzcLrhVgKzgSic/9NfVHWmiLTF6ULrjNPVdjewG5irqmfV8H55qlqbxFUvXXOBYMq5U3luhdXF1AumAvBO8rMeR+K9sQPuBaD3gBMOtQad1ORnoB7yyBNLnqv1F/2DF09t9LwVyC0iVPVx4PFqigyuZJm9wDWVlK02CbnL/ihaT8aY4OLvvc0BnYiMMcb4f9ecJaKTICJfA5EVZt+iqqlexGOMMdWxFlEAUtVzvI7BGGNqyxKRMcYYT4VaIjLGGOMlP89DwXtCqzHGBAuR2j9qXpdc7l57M01EHqzk9UgRed99/WsR6VTTOi0RGWNMgKuvi566J/m/AFwB9AJuFpFeFYpNBrJVtSvwN+DJmuKzRGSMMQGuHltEg4E0VU1X1ULgPU487/IaYKb7/ENguNRw3SxLRMYYE+Dq0iISkSkissbnMcVnVacB//WZ3uHOo7Iy7jU9c4GW1cVngxWMMSbA1eViphXuFNAorEVkjDEBrh675nZS/sagHdx5lZYRkTCgOXCgupVaIjLGmAAXUodHDVYD3UQkSUQicG4+WvFeb3NwLjYNcAOwWGu4urZ1zRljTICrrysrqGqRiNwDfAqEAq+r6noReQxYo6pzgNeAN0UkDcjCSVbVskRkjDEBrj5PaFXVT4BPKsx72Of5EeDGuqzTEpExxgQ4u9acMcYYT3l1C/DaskRkjDEBzlpExhhjPOXnecgSkTHGBDprERljjPGUv58waonIGGMCXIhUez6p56SGE15Nw7BKN8bU1il3rL2T/Gytv3PGDri30TvyLBEFMRGZ4l7gMOhZXRxndXGc1UXj8PeuQ9OwptRcJGhYXRxndXGc1UUjsERkjDHGU5aIjDHGeMoSUXCzvu/jrC6Os7o4zuqiEdhgBWOMMZ6yFpExxhhPWSIyxhjjKUtExhhjPGWJyBhjXCISIiLnex1HsLHBCkFGRBap6vCa5gULEbkOuADnsksrVHWWxyF5RkSuBM4Eokrnqepj3kXkDRFZp6r9vY4jmFiLKEiISJSIxAOtRKSFiMS7j07Aad5G5w0ReRG4C0gFvgPuFJEXvI3KGyLyEjAa+AXOtc1uBBI9Dco7i0TkehF/v3lC4LAWUZAQkXuB+4AEYJfPSweBV1T1H54E5iER2QT0VPdDICIhwHpV7eltZI1PRFJUtY/P3xhgnqoO9Tq2xiYih4BooAg4gpOYVVWbeRpYALPbQAQJVX0WeFZEfqGqz3sdj59IA04HMt3pju68YFTg/j0sIgnAAaC9h/F4RlVjq3tdRM5U1fWNFU8wsK654PO6iPxBRKYDiEg3EbnK66A8EgtsFJGlIrIE2AA0E5E5IjLH49ga21wRiQOeBpKBbcC7nkbkv970OoBAY11zQUZE3gfWAhNU9SwRaQp8oar9PA6t0YnIRdW9rqrLGisWr4lIpKoeLX2OM2DhSOk8c5wNZqh/1jUXfLqo6mgRuRlAVQ8H60HZYEo0tfAlMADATT5HRSS5dJ4px3691zNLRMGnUESa4H6YRKQLEJS/ekXkXOB5oCcQAYQC+cF0UFpE2uGMmmwiIv05fjfQZkBTzwIzQcUSUfB5BJgPdBSRt4EhwCRPI/LOP4AxwAfAIGACcIanETW+y3D+/x2Av/rMPwj8jxcBecntHeigqv+tplhhY8UTLOwYUZBxzyUS4Fz371dArKpmeBqYB0RkjaoOKh2y7M4Lyv5/EbleVf/P6zj8gYikqmpvr+MIJjZqLvj8Bzimqh+r6lygtTsvGB0WkQjgGxF5SkR+SfB+JlaKyGsiMg9ARHqJyGSvg/JIsoic7XUQwSRYP3TB7E/Af0QkWkQGAh8C4z2OySu34HwG7gHycc4jut7TiLzzT+BTnBOeAb7HOQE6GJ0DfCkiW0UkRURSRSTF66ACmXXNBSER+SnwG5zzaK5X1e89DskTIhINFKhqiTsdCkSq6mFvI2t8IrJaVc/27ZoUkW+CdFh/pZc2UtXMyuabU2eDFYKEiDxP+WGnzYGtwD0igqpO9SYyTy0CRgB57nQT4DMgGK++nC8iLTk+mvJcINfbkDzTHudST4cARKQZzshKS0QNxBJR8FhTYXqtJ1H4lyhVLU1CqGqee4JvMPoVMAfoLCIrcY4d3uBtSJ75X8qfP5VXyTxTjywRBQlVnel2Pb2hquO8jsdP5IvIAFVNBnCPmRXUsEyg2gDMAg4Dh4CPcI4TBSNRn2MWqloiIvZd2YCscoOIqhaLSKKIRKiqnQvhHIz/QER24Qxlb4dzK4Rg9AbOuUN/cqfH4lxT7UbPIvJOuohMxWkFAfwMSPcwnoBngxWCjIi8gdPfPQdnpBgAqvrXKhcKYCISDnR3Jzer6jGf10aq6gJvImtcIrJBVXvVNC8YiEgb4DngEpxjZouA+1R1n6eBBTBrEQWfre4jBGfUXFBzE893Vbz8JBAUiQjn3JlzVfUrABE5hxOPKwYFN+GM8TqOYGItImOqEAxXWRCRVJxf/aUtw+3udCKwKUhbRE8Bf8Q5Xjgf6AP8UlXf8jSwAGYtoiAjIq1xziE6E+dS/wCo6iWeBeW/guFXWrDei6o6l6rqb0TkWpz7Ml0HLAcsETUQS0TB523gfZwvoLuAicB+TyMynrGTNCtV+r14JfCBquYG6Z1SGo1d4if4tFTV13CuN7dMVW/DOSgbdNwbwFU3b1vjRWP8yFwR2QQMBBa5vQhHPI4poFkiCj6lo8J2i8iV7j1o4r0MyENfVjdPVa9rxFiMn1DVB3GurjHIHcySD1zjbVSBzbrmgs8fRaQ5cD/OTeGaAb/0NqTGZTeDM5URkUtUdbGIXOczz7fIvxs/quBgiSjIuLd+AOc6Yhd7GYuH7GZwpjIXAouBq3EGqkiFv5aIGogN3w4yIpIE/ALohM8PEVUd5VVMXrGbwRlfInI/JyYg3OdBe9J3Y7AWUfD5CHgN52Z4JR7H4rWVIvIakKCqV4hIL+A8dzCHCT4x7t/uwNnAbJxkdDWwyquggoG1iIKMiHytqud4HYc/cO9G+k/g96ra172w5Tq7TXRwE5HlwJU+t4GIBT5W1Qu9jSxw2ai54POsiDwiIueJyIDSh9dBeaSVqv4Lt2WoqkVAsbchGT/QFvC9KHChO880EOuaCz69cW6RfQnHu+aU4DyXyG4GZyrzBrBKRGa50z8FZngXTuCzrrkgIyJpQC+7DQS4LcHngbNwLnzaGrhBVVM8Dcx4zt03hrqTy1V1nZfxBDprEQWf74A4wC5pD9nARTgHpwXYDPTzNCLjF9ybJSZ7HUewsEQUfOKATSKyGjhaOjMYh28DHwKjVHU9gIhcCLyA031pjGkkloiCzyNeB+BH7gI+EpGrgQHAn4GfeBuSMcHHjhGZckTkS1U9z+s4GouInAe8jHNRyytV1a5EbkwjsxaRqSiq5iI/biLyH8rfa6gpzmi510QkWLspjfGMJSJTUTA0kf/idQDGmOMsEZmgo6rLRCQUWKiqwXrhV2P8hl1ZwVQUFLeiVNVioMS9JYYxxkPWIgpCItIW56KOAKtU1fecols8CMkreUCqiCzAufkZAKo61buQjAk+NmouyIjITcDTwFKc1s9Q4AFV/dDLuLwgIhMrm6+qMxs7FmOCmSWiICMi3wIjS1tBItIa51hJX28jM8YEK+uaCz4hFbriDhCkxwpFpBvOSay98Bm2rqqdPQvKmCBkiSj4zBeRT4F33enRwDwP4/HSP3GuNPE3nNum30qQJmVjvGRdc0FIRK4DLnAnP1fVWdWVD1QislZVB4pIaunN8ErneR2bMcHEWkRBRkSeVNXfAv+uZF6wOSoiIcAWEbkH2Mnx20UbYxqJdUMEn5GVzLui0aPwD/fiXN5nKjAQGA9M8DQiY4KQtYiChIjcDfwM6Cwivjd+iwVWehOV5xR4E0gEwt15rwB9PIvImCBkx4iChHsFgRY4o8Qe9HnpkKpm+ZRroarZjR2fF0RkM/AAkMrx26ajqpmeBWVMELJEZMoRkWRVHeB1HI1BRFao6gU1lzTGNCTrmjMVBcW15lyPiMirwCLK363231UvYoypb5aITEXB1ES+FeiBc3yotGtO8RlRaIxpeJaITDA7W1W7ex2EMcHOhm+bioKpa+4LEenldRDGBDtLREFGRCZXMu8Jn8nhjRiO184FvhGRzSKSIiKpFYa2G2MagXXNBZ/rReSIqr4NICIvUP6Cn1lVLhl4Lvc6AGOMJaJgdD0wR0RKcL6Ic1T1hFZSMLDzhYzxD3YeUZAQkXifyVjgI5wrKjwMQdcSMsb4EUtEQUJEMnCGJovP31Jq9+AxxnjFEpExxhhP2TGiICMi4cDdwIXurKXAy6p6zLOgjDFBzVpEQca9pE04MNOddQtQrKq3exeVMSaYWSIKMiLyrar2rWmeMcY0FjuhNfgUi0iX0gkR6QwUexiPMSbI2TGi4PMAsERE0nFGziXiXPzTGGM8YV1zQUhEIoHSi31uVtWj1ZU3xpiGZC2iIOOOmrsTn1FzImKj5owxnrEWUZCxUXPGGH9jiSjI2Kg5Y4y/sVFzwcdGzRlj/IodIwo+vqPmADpho+aMMR6yFlHwWQm8DJQAWe7zLz2NyBgT1OwYUZARkX8BB4G33VljgThVvdG7qIwxwcwSUZARkQ2q2qumecYY01isay74JIvIuaUTInIOsMbDeIwxQc4GKwQJEUnFuSFeOPCFiGx3pxOBTV7GZowJbtY1FyREJLG611U1s7FiMcYYX5aIjDHGeMqOERljjPGUJSJjjDGeskRkjDHGU5aIjDHGeOr/A8VhZyorzfdeAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "##val_df data correlations\n",
        "sns.heatmap(val_df.corr(), cmap=\"crest\", annot=True, linewidth=.5)\n",
        "# There is a negative very weak correlation between beta and discr_r (target). \n",
        "# There is a positive very weak correlation between market_cap and discr_r (target). \n",
        "# There is a negative very weak correlation between book_to_market and discr_r (target)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BO2Ffb6gn-od"
      },
      "outputs": [],
      "source": [
        "#define train,val,test variables and target\n",
        "X_train = train_df[[\"book_to_market\",\"market_cap\",\"beta\"]]\n",
        "y_train = train_df[\"discr_r\"]\n",
        "\n",
        "X_val = val_df[[\"book_to_market\",\"market_cap\",\"beta\"]]\n",
        "y_val = val_df[\"discr_r\"]\n",
        "\n",
        "X_test = test_df[[\"book_to_market\",\"market_cap\",\"beta\"]]\n",
        "y_test = test_df[\"discr_r\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y3RTz-IUd75z"
      },
      "outputs": [],
      "source": [
        "from sklearn.preprocessing import StandardScaler\n",
        "scaler = StandardScaler()\n",
        "scaler.fit(X_train)\n",
        "X_train_s, X_test_s, X_val_s = scaler.transform(X_train), scaler.transform(X_test), scaler.transform(X_val)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8UL0T9NFy6bi"
      },
      "source": [
        "# step 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I1sq9Huhg9tV"
      },
      "outputs": [],
      "source": [
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.metrics import r2_score\n",
        "\n",
        "# Train the model on the training data\n",
        "log_reg = LinearRegression()\n",
        "log_reg.fit(X_train_s, y_train)\n",
        "\n",
        "# Make predictions on the test data\n",
        "y_test_pred = log_reg.predict(X_test_s)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "skfLYJ_DuiIS"
      },
      "source": [
        "### Mean-Absolute-Error"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kYsu_YVF33jk",
        "outputId": "42be8568-ee23-4579-982e-9fba7db77695"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.13710582505046462\n"
          ]
        }
      ],
      "source": [
        "from sklearn.metrics import mean_absolute_error\n",
        "\n",
        "# Calculate the mean squared error\n",
        "lr_mae = mean_absolute_error(y_test_pred, y_test)   \n",
        "\n",
        "print(lr_mae)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a4dtVfVCn7--",
        "outputId": "269d5946-9ad1-4582-87a3-1700db7ccea2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Test R2 score:-0.4750\n"
          ]
        }
      ],
      "source": [
        "lr_r2_score = r2_score(y_test, y_test_pred)\n",
        "print(f'Test R2 score:{lr_r2_score:.4f}')\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "7r9iXDXoqn5I",
        "outputId": "8e5a47f6-8957-45f2-8dfe-e43db4864a79"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'realizations')"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5hU9dXA8e/ZZYAFlAXBwkpTEQRRkBVQ1IglWKIS9bXkNdVIYkmCiUSMRtGYiOFNjFGjwRaNRoklBIOKBayRsjTpgvQFdSkLAgtsOe8f9846DFPu7JQ75XyeZx+m3Jk5DMs999fOT1QVY4wxJpoivwMwxhiT3SxRGGOMickShTHGmJgsURhjjInJEoUxxpiYmvkdQKp16NBBu3Xr5ncYxhiTG/ZUwa71zF7ZsElVO0Y6JO8SRbdu3aioqPA7DGOMyW7bl8PMa+CL2XDIUOSsaWuiHZp3icIYY0wMDXWw7E/w8a+hqDkMfBSOvJpYIxGWKIwxplBs/RhmXA1bKqDsQjjxL9CqLO7LLFEYY0y+q98Di37n/DRvB0Oehy6XgYinl1uiMMaYfLZputOK2LYYul0FJ9wHLTsk9BaWKIwxJh/V7YT5t8Gy+53upa9NhrLzmvRWviUKEekMPA0cAigwXlXvDzvmdODfwCr3oZdV9a5MxmlMLps4t5JxU5axobqGTqUljBrWk+H94/dJmxz32dsw4xrYuQp6XAv9xkLgwCa/nZ8tijrgF6o6R0QOAGaLyJuqujjsuPdV9Rs+xGdMTps4t5JbXl5ATW09AJXVNdzy8gIASxb5am81zL0JPn0cDugBZ70LB5+W9Nv6tjJbVTeq6hz39pfAEsB+e41JkXFTljUmiaCa2nrGTVnmU0Qmrdb/Gyb3hpVPwjG/hHPnpyRJQJaU8BCRbkB/YEaEp08Skfki8pqI9Iny+hEiUiEiFVVVVWmM1JjcsaG6JqHHTY6q+Rw+uBzeGw4tOsLXZ0D/e6FZSco+wvdEISJtgJeAkaq6PezpOUBXVT0eeACYGOk9VHW8qparannHjhFXoBtTcDqVRj5RRHvc5BhVWPV3pxWxfiIcdzecUwEHlaf8o3xNFCISwEkSz6rqy+HPq+p2Vd3h3n4VCIhIYvO6jClQo4b1pCRQvM9jJYFiRg3r6VNEJmV2roV3zoePvgMH9oRz58Gxt0JRIC0f5+esJwEeB5ao6h+jHHMo8LmqqogMxElsmzMYpjE5KzhgbbOe8og2wPJHYN7Nzu0B90OP66GoOP5rk+DnrKchwLeBBSIyz33sV0AXAFV9BLgUuFZE6oAa4Aq1Tb6N8Wx4/zJLDPli+ycw44dQ9T4cehYMHA9tumfko31LFKr6ARBz/biqPgg8mJmIjDEmCzXUwdI/wMd3QHEJDHoCjvie5/IbqWArs40xJlttnQ/TfwBb58Dh34QTH4KSwzIehiUKY4zJNvW7YeHdsPheaHEQnPIidLnEt3AsURhjTDap+q9TxG/7Uuj+XTjhj9Civa8hWaIwxphsULsD5v8KPnkQWnWG01+HTsP8jgqwRGGMMf7b+AbMHAE718DRN8Dxv4PAAX5H1cgShTHG+GXvVpjzc1j5N2fh3Fnvw8Gn+B3VfixRGGOMH9a9DLOuhz1V0PsW6Hs7FLf0O6qILFEYY0wm1XwGFTfAupegXT84/VVo39/vqGKyRGGMMZmgCquecrqa6nY54xDH3JS2+kypZInCmAyx3eYK2I7VMPNH8Nkb0HEIDHwM2vbyOyrPLFEYkwG221yB0gb45CGYf4tzf8ADcPR1IL7v8JCQ3IrWmBxlu80VoG1L4a3TYPZPoeMpcP4i6HlDziUJsBaFMRlhu80VkIZaWDIOFtwJzVrD4Keg+7czWsQv1SxRGJMBnUpLqIyQFGy3uTyzZY5TfmPrPOh8KZQ/CCWH+B1V0nKvDWRMDrLd5vJcXQ3MuwWmDHSmv576Epz6Ql4kCbAWhTEZYbvN5bEvPnBaEV9+Akd8H074AzRv53dUKWWJwpgMsd3m8kztl04rYvlD0LobDH0DDjvb76jSwhKFMcYkasPrzrqIXevg6J/C8b+FQBu/o0obSxTGGOPVns3OyupVT8OBveDsD6DjyX5HlXa+DWaLSGcRmSYii0VkkYj8LMIxIiJ/FpEVIvKxiJzgR6zGmAKnCmtfgMm9YfU/oM9tcO68gkgS4G+Log74harOEZEDgNki8qaqLg455lygh/szCHjY/dMYYzKjZiPMug7WT4T2A5yxiHbH+x1VRvmWKFR1I7DRvf2liCwByoDQRHER8LSqKjBdREpF5DD3tcYYkz6qsPJJp6upYQ/0uxd6/RyKCq/HPiv+xiLSDegPzAh7qgxYF3J/vfuYJQpjTPrsWOXsOPfZW9DxVBj0GBx4tN9R+cb3RCEibYCXgJGqur2J7zECGAHQpUuXFEZnjCkoDfXOntXzfwVSDCf+BY76UU7WZ0olXxOFiARwksSzqvpyhEMqgc4h9w93H9uHqo4HxgOUl5drGkI1xuS7bYthxg9h00dw2Lkw8K/QunP81xUAP2c9CfA4sERV/xjlsEnAd9zZT4OBbTY+YYxJqfq9sPBueK2/s7r6pGfg9MmWJEL42aIYAnwbWCAi89zHfgV0AVDVR4BXgfOAFcAu4Ps+xGmMyVebK5zyG9UfQ5fLofzP0PJgv6PKOn7OevoAiFl3153tdH1mIjLGFIy6GlhwByz9A7Q8BE6bCIdf5HdUWcv3wWxjjMmoz991xiJ2rIAjfwj9x0HzUr+jymqWKIwxhaF2O8y9GVY8Am2OgDPegkPP9DuqnGCJwhiT/ypfhVk/gpoNzqK54+5ydp8znliiMMbkr92bYM5IWP0stO0Np7wIHawKUKIsURhj8o8qrJkAs38Ce6vh2Dugzy1Q3MLvyHKSJQpjTH7ZVekU8aucBO3L4cwnoLSv31HlNEsUxpj8oAqfPgZzb4KGvdD//6DnzwqyiF+q2TdojMl9X34KM6+Bz6fBwafDoEfhgKP8jipvWKIwxuSuhnpYdj98fBsUBZz6TEf+sOCL+KWaJQpjTG6qXuiU39g8Ezp9AwY+DK0O9zuqvGSJwhiTW+r3wuJ7YNFvIdAWTv4HdL0CJGZFIJMESxTGmNyxaabTiti2ELp+Cwb8CVp29DuqvGeJwhiT/ep2wce3w7L7oOVhcNokOPwCv6MqGJYojDHZ7fNpbhG/lc5uc/3uheZt/Y6qoFiiMMZkp73bYN4vYcV4aHMknDkNDjnd76gKkiUKY0z2Wf8KzPox7P4MjrkJ+t4JzVr5HVXBskRhjMkeu6tg9k9hzfNO2Y3TJsJBJ/odVcGzRGGM8Z8qrHnOSRK1250WRO/RUNzc78gMliiMyXsT51YybsoyNlTX0Km0hFHDejK8f5nfYX1l5zqYdS1smAwHDYJBj0NpH7+jMiEsURiTxybOreSWlxdQU1sPQGV1Dbe8vADA/2ShDbDiUZg7CrQeTrgPjv4JFBX7G5fZj68FUUTkCRH5QkQWRnn+dBHZJiLz3J/bMx2jMbls3JRljUkiqKa2nnFTlvkUkWv7cnj7DGfA+qCBcP4C6DXSkkSW8rtF8TfgQeDpGMe8r6rfyEw4xuSXDdU1CT2edg11sPQ+WHA7FLWAQY/BET+w8htZztdEoarviUg3P2MwJp91Ki2hMkJS6FRakvlgtn7slN/YUgGHXwTlf4FWnTIfh0lYLtTiPUlE5ovIayIScYRLREaISIWIVFRVVWU6PmOy1qhhPSkJ7NudUxIoZtSwnpkLon6PU37j9QGwcw0MmQCn/suSRA7xu+spnjlAV1XdISLnAROBHuEHqep4YDxAeXm5ZjZEY7JXcMDat1lPm6a7RfwWQ7dvw4D7oMVBmflskzJZnShUdXvI7VdF5C8i0kFVN/kZlzG5ZHj/sszPcKrbCfNvczYVanU4nP4qdDo3szGYlMnqRCEihwKfq6qKyECcrrLNPodlTN5J6VqLz96GGdfAzlXQ4zrodw8EDkxtwCajfE0UIvIccDrQQUTWA3cAAQBVfQS4FLhWROqAGuAKVbWuJWNSKGVrLfZWw9yb4NPH4YAecNa7cPBp6QjZZJjfs56ujPP8gzjTZ40xaRJrrYXnRLFuIlRcB7u/gN43w7F3QDMfZlaZtMjqridjTPoltdai5nOY/RNY+wKUHg9fewXaD0hxhMZvliiMKXBNWmuhCqufgdkjoW4HHHc39P4lFAXSGKnxSy6sozDGpFHCay12roV3zoePvgMH9oRz58Gxt1qSyGPWojCmwHlea6ENsPwRmHczoDDgz86sJqvPlPcsURhj4q+12L7MmfJa9T4cejYMHA9tumUsPuMvSxTGmOga6mDJ/8GCMVBcAoOfhO7fTWkRv6zfL8NYojDGRLF1Hky/GrbOgcO/CSc+BCWHpfQjsnq/DNPIBrONMfuq3w3zb4XXy6GmEk55EU57OeVJArJ4vwyzD2tRGGO+UvVfp4jf9qVOF9MJf4QW7dP2cVm3X4aJyFoUxhio3QEVP4U3T4G6XXD663DS39KaJCD6Wg1f9sswUVmiMKbQbXwDXj0WPnkQjr4ezl8InYZl5KOzYr8ME5enRCEivxeRA0UkICJvi0iViFyV7uCMMWm0ZwtM/z5MGwbFLeGs96D8AQgckLEQhvcv456L+1JWWoIAZaUl3HNxXxvIzjJexyi+rqq/FJFvAquBi4H3gGfSFZgxJo3WvgQV18OeTdDnV3Dsr51k4QNf9sswCfGaKILHnQ+8oKrbxDZDNyb31HwGFTfAupegXX8Y+jq06+d3VAmztReZ5TVR/EdEluLsCXGtiHQEdqcvLGNMSqnCqqdgzs+dwerj74FjfpGT9Zls7UXmeRqjUNXRwMlAuarWAjuBi9IZmDEmRXashmnnOOMRbfvAefOhz+icTBJgay/8kMg6il5ANxEJfc3TKY7HGJMq2gCfPATzbwEEyh+EHteC5PZkR1t7kXmeEoWI/B04EpgHBFO5YonCmOy0bQnM+CFs+i8cNgwG/hVadwVyv3+/SftnmKR4bVGUA71tv2pjslxDLSwZBwvuhGZtYPBT0P3bjUX88qF/f9Swnvv8HcDWXqSb1zboQuDQdAZijEnSljkwZaBTp+nwi+D8xXDEd/ap9JoP/fu29iLzvLYoOgCLRWQmsCf4oKpemMyHi8gTwDeAL1T12AjPC3A/cB6wC/ieqs5J5jONyTt1NbDwLqcl0aIjnPoydP5mxEPzpX/f1l5kltdEMSZNn/834EGij3WcC/RwfwYBD7t/GmMAvvjAKeL35SdwxA/ghP+D5u2iHm79+6YpvE6PfRdYChzg/ixxH0uKqr4HbIlxyEXA0+qYDpSKSOprHRuTa2q/hFk3wFunQsNeOONNGPx4zCQBVlvJNI3XWk+XATOB/wEuA2aIyKXpDMxVBqwLub/efSw8vhEiUiEiFVVVVRkIyxgfbXgNJveB5X+Bnj+D8xbAoWd5eqn175um8Nr1dCtwoqp+AeCuzH4LeDFdgSVCVccD4wHKy8ttZpbJT3s2w+wbYfXf4cBj4OwPoeNJCb+N9e+bRHlNFEXBJOHaTGZKlFcCnUPuH+4+ZkzhUIV1Lzo1mvZscQr49bkVilv4HZkpEF4TxesiMgV4zr1/OfBqekLaxyTgBhF5HmcQe5uqbszA5xqTHWo2wqzrYP1EaD8Ahr4B7Y73OypTYDwlClUdJSKXAEPch8ar6r+S/XAReQ44HeggIuuBO4CA+5mP4CSj84AVONNjv5/sZxqTE1Rh5ZNOEb+GPdDv99DrRihq+u7Ft01cwHMz1lGvSrEIVw7qzN3D+6YwaJOvPP/WqepLwEup/HBVvTLO8wpcn8rPNCbr7VgJM38En70FB58GAx+FA49O6i1vm7iAZ6avbbxfr9p435KFiSfmOIOIfOD++aWIbA/5+VJEtmcmRGMKREM9LP0TTO4Lm2bAiQ/DmdOSThIAz81Yl9DjxoSK2aJQ1VPcPzO3N6IxhWjbYph+NWyeDp3OgxMfgdad47/Oo/ooZdqiPW5MKK/rKP7u5TFjTILq98KC38Br/WHHcjjpGfjaf5j4SRFDxk6l++jJDBk7lYlzk5vsVxxlR8pojxsTyusU1z6hd9w9KQakPhxjCsjmCphyIiy4HTpf7BTx6/6/TJy3gVteXkBldQ3KVxVek0kWVw6K3DqJ9rgxoeKNUdwiIl8Cx4WOTwCfA//OSITG5Ju6Gpj7S3hjEOzZBKf9G4Y8By0PBtJT4fXu4X25anCXxhZEsQhXDe5iA9nGk3hjFPcA94jIPap6S4ZiMiZ/ff6us6HQjhVw5DXQ//fQvHSfQ9JV4fXu4X2zNjHk+mZK+c7rOopbRKQdThXXliGPv5euwIzJK7XbYe7NsOIRaHMEnPE2HHpGxEMLrcJrPmymlO+8Dmb/EHgPmALc6f45Jn1hGZNHKic7Rfw+HQ+9fu4W8YucJCBzFV4nzq1M6YB5U+XDZkr5zuuCu58BJwLTVXWoiPQCfpe+sIzJA7s3wZyRsPpZaNsHTnkROsTfTiV4FZ3OrphsuorPl82U8pnXRLFbVXeLCCLSQlWXiogVsDcmElVYMwFm/wRqt8Gxd0CfX0Fxc89vke4Kr7Gu4jOdKAqtqy0XeZ0eu15ESoGJwJsi8m9gTfrCMiZH7aqE94bDf6+E1t3hnNlw3JiEkkQmZNNVvG2mlP28DmYHN+AdIyLTgLbAa2mLyphcowqfPgZzb4KGWuj/f9BzJBQVx39tDOmaDRTtKl6BIWOnZnTWUaJdbTZDKvNEPSzhF5HHgQdUdV7IY2NUdUwaY2uS8vJyraio8DsMU0i+/BRmXgOfT4ODT4dBj8IBRyX9thPnVjLqxfnU1n/1fzRQLIy79PiET4zhJ9ehvTry0uzK/bqfgkoCxVm581342Apkb6y5RkRmq2p5pOe8dj0NA54Ske+GPHZh0pEZk8sa6mHJH+HVvrBlNgwcD2dOTUmSALjzlUX7JAmA2nrlzlcWJfQ+wZNr6Ervl2ZXcsmAMsqijANk66wjmyHlD6+D2V8AQ4FnRGQgziwoKxJjClf1QphxNWyeCWUXOJVeW6X2inbrrtqoj3cfPdlzt0u0k+u0pVV8OPoMuo+eTKR+hWyZdRTaGorW/5EtseYrry0KUdVtqnoBUAW8gzNOYUxhqd8LC+6E109w9o04+TmnBEeKk0Q8idSAijdwHW12UTbMOgpvDUWTDbHmM68tiknBG6o6RkRmAzemJyRjstSmmU4rYttC6PotGHA/tOyQ8NtEG4wNf7wkUERNbUPM96qprefOVxbFbFXEm346aljPiP3+2TDrKFJrKFy2xJrPPLUoVPWOsPuvqGr0paXG5JO6XTDnF/DmSbB3K3ztFRjybJOTRKTKsLdNXLDf43UN6uk/6NZdtTFbFfGmnw7vX8Y9F/elrLQEAcpKS7JmcDhWl1K2xZrPYrYoROQDVT3FrRgb2vITnJ1KD0xrdMb47fNpbhG/lXDUj6HfWGieWK9raEuhSGS/zYJqausb97IOVVuvtGsVoFXzZlFfGxRroZyX6aepXOCXyumr0VpDZaUlfDjarlUzxdcd7kTkHOB+oBh4TFXHhj3/PWAcELxcelBVH0tHLMbsY+82mDsKPn0U2hwFZ74Dh3wt4bcJn86Z6E5z1btqmXv71xvfa+SEeRGPizeYm+6V3kHxSoMEk0hldQ3FbuIri5FMsrlbrJDEa1G0j/W8qm5p6geLSDHwEHA2sB6YJSKTVHVx2KETVPWGpn6OMQlb/wrM+jHs/gyOGQV9x0CzVk16Ky997EDjSTNc6CDt8P5ljJm0iOqa/WdDxRrMzeQCtVjTVyvWbOHZ6WsbuyaCf99YdaYyUffKxBdvMHs2TpdTpKmwChyRxGcPBFao6koAEXkeuAgITxTGZMbuKpj9U1jzPJT2dWYzHRRx/ZFnXqZtlgSKuWRA2X4L4CJdOY+5sE9CV9iZLv4X7e9bWV3DM9PXRn1dTW09YyYtitrasG4mf8Xreuqexs8uA9aF3F8PRCqteYmInAZ8AtyoqusiHGNM06nC6n/AnJ85+0b0vQt635yS+kzR+tiLRWhQ3ecKubxr+7hXzvGusMNbD7v21mW0+F+0v68X1TW1ja0lL62Npki0dWXlQhyeSngApHrjIhG5FDhHVX/o3v82MCi0m0lEDgJ2qOoeEfkRcHmk2VYiMgIYAdClS5cBa9ZYvULj0c51MOta2DAZDhoEgx6H0j7xX+dRJktORPqsaARYNfb8lH5+ojEkIhWD14n+WxRauZBYJTw8raNwNy76GXA4MA8YDHwEJPMvVwmE7ux+OF8NWgOgqptD7j4G/D7SG6nqeGA8OLWekojJFAptgBXjnb2rtR5OuA+O/knSRfzCZbKP3et4COw/phHvyjlSrahpS6v2Oz74mmiD7k0VqUsr0av9REurZ1Mpdr/5uXHRLKCHiHTHSRBXAN8KPUBEDlPVje7dC4ElSX6mMbB9uVPE74t34ZAzYdB4Z3vSNMnU1FOvZSzCxzS8zFQKfz50vCH8+OH9yxrHGlIlUmILj2nkhHnc+M95qBJxRlWipdWzqRS733zbuEhV60TkBpxtVYuBJ1R1kYjcBVSo6iTgpyJyIVAHbAG+l8xnmgLXUAdL74MFt0NRC6eb6Yjvg+RG2bJ4J/Ro4wOlJQFat2i2T2tg3JRl3DhhnqdxDC8tldDCfMEkIRCz7IZXoYktdHptJMGe9NAxjpET5jFm0iLalgQSmjFmGyp9xWuiCN+4aCsp2LhIVV8FXg177PaQ27cAtyT7Ocaw9WOn/MaWCjj8Iij/C7Tq5HdUCYnXFRJtzcGYC/vsM9gdnmyiCV45e72CDiau4HsHp0sqTrLaubduv2q48RSLNI4JJDP+UV1TS6BYCBQJtQ1fxRBrxpit4fhKMhsXvZ62qIxJlfo9sOi3sOgeaNEeTvkndL7Ul1ZEsjNoYk09DVaTvWRAWePYQduSACJw44R5jJuyjFHDejZpHMPrTCaB/d5boXF1eXVNbdT1ItEEjx0ydmrSXVnhK93j/RvYGo6vJDLr6RSgh6o+KSIdgTaquiqt0TWBbVxkGlV95LQiti+Bbt+GAfdBi4PS8lFeBoOTnUHj5WQZfE8g4ud5TRKhsaVrJpNXgWJJuCUSTbpme+WDpDcuEpE7gJv5qhsoADyTmvCMSbG6nTB7JLw5BOp2wOmvwslPpzVJRCr0F1qoLxUb7kQq7hcu+J7RPi9aO6okUBS1KODw/mVcMqAs6msFaN08tbPFQqUqSUBhji+kgtcxim8C/YE5AKq6QUTSUv/JmKR89hbMuAZ2roYe10O/eyCQ3l9VL9MoUzGDJrwrpCmb+CjO1WF48fK99RqzW2Xa0qqon1faKhB1k6VMKALaujEEu7ZEvhrYDirU8YVU8Joo9qqqiogCiEjrNMZkTOL2VjulwFc+AQf0gLPeg4NPzchHe0kCqZpBEzrVNlpXlBK9dhTsnyQA6huUkSFjGeEJI1by8TNJgPP3adW8WWPxxCAvY0K28tqbuIlCRAT4j4j8FSgVkWuAHwCPpjs4YzxZNxEqroPdX0Dv0XDs7dAsc10MXpJAOmbQRHrPoEQGjENVVtdw44R5jJwwrzHZFIukZJprOkVKZPHWr2S6DlYui5so3JbE/wA/B7YDPYHbVfXNdAdnTEw1n8Psn8DaF6D0eGdDofYDMh5GpBO2AEN7dWy8n+wMmtsmLmjcs0KAVs2L2bk3+phDMsKruzY16WRSoivNwVZeJ8Jr19McoFpVR6UzGGM8UYVVf4c5I52B6+N/65QDLwr4Es7w/mX7ldBW4Nnpa3lm+tp9VgdXrNnCczPWUVldwy/+OZ+KNVu4e3jfqO/tXPV+vM+WqArs3PvVWgXjtAb63fkGYy506nR5aSnYymvvPE2PFZGlwFE4i+x2Bh9X1ePSF1rT2PTYPLdzLcz8EWx8HTqc7KyubtvL76jiTl0tCRRzQpe2fPhp5C1c2rUKoArbamobr4Bh/ymuhai4SKhv8JYSA0VCm5bNIo6bBNdQhJYxj6RQd89LuiggMCyF8RiTOG2A5Q/DvNGAwoA/w9HXg3ia4Z128a5Ca2rroyYJ2HdAuLK6hlEvzqeuXq3FAJ6TBEBtg0YdXN+6q7bxuWhJwmZGReZ1ZbbV7Tb+2b7M2be66gM49GwYOB7adPM7qn0ksw9DJKlcO2C8EeCSAd4KOBbabKnsuBwzJpKGOlg0Fl49HqoXwuAnYeiUrEsS4G0xnMmM0pJAk/4tFGe9SDxeFljmG69dT3mv0K4Qst7WeTD9atg6BzpfDOUPQcmhfkcVVeisplRWTjWJKcIp41VTW79PqfGde+oiVo4Nt6G6Ju65INpsqVjrUHKd51pPuaIpg9mFtpNVVqvfDQt/A4vvhRYdnATR5RK/o0rYxLmVjJm0yNPJyaRGwM0Sod12sWpfRdKuVYDdtQ0xzwXdR0+OeRGQq+eOpGs95btU1OExKVD1IbzWDxb9Drp/G85fnJNJImhPXaQ10CZdahv2H9upqa3n1n8tYMykRXGTREmgGNX9K+CGnwvirabPx3OHJQpsPrXvandAxU/hzVOdFsXQKc54RIv2fkfWZImU8zbptXNv/X4tu0CxcNXgLvsVQtwWpQUYei7wMh6Vb+cOG6PAdrLy1cY3YOYIZ33E0TfA8b+DQBu/o0paKmdAmdSrrVf+M38j8+7Ytz5UtN3zQs8F4eNRkeTbucNaFES+QrD51Gm2ZwtM/z5MGwbFLeHs96H8zzmfJCbOraTfnW/4HYbxoLqmln53vrHPbCWv54Lh/cv4cPQZ/Onyfllx7pg4t5IhY6fSffRkhoydmvIZWNaiwHayyri1L0HF9bBnE/T5FRz7aydZ5Di/N/gxiauuqW3cUzt0y9hI54JYs6H8PHdkorihzXoymVOzESpugHUvQ7v+MPgJaNfP76iaLPzEsXXnHnbV2gB2roo1W+m2iQv2qeUV7/hMilY+JtFSJFk760lEzhGRZSKyQkRGR3i+hYhMcJ+fISLdMh+lSZoqrPwb/Kc3VE6GfmNh2MycTxLhi64sSeS2mtp67nxl0X6PT5xbuVVnGvIAABgPSURBVF+SCB6fDbObMjEZx7euJxEpBh4CzgbWA7NEZJKqLg457Gpgq6oeJSJXAPcCl2c+WtNkO1Y7g9WfvQkdT4FBj8GBuT/2Y7Oa8tPWXbX7rJMoCRSxu66hSbsJZkomJuP42aIYCKxQ1ZWquhd4Hrgo7JiLgKfc2y8CZ7obKZlspw2w7AF49VjY9JGzcO6sd/MiSUB2nCBMeoQmhZrahv22VA2VDbObMjEZx8/B7DJgXcj99cCgaMeoap2IbAMOAjaFHiQiI4ARAF26dElXvMarbUucIn6b/guHnQMDH4HWXf2OKqVSXQTQ5B6BrJgZmYkB9byY9aSq44Hx4Axm+xxO4WqohSXjYMGd0KwNnPQ0dLvKKb6TZ2JtQ2oKQ6vmxdyYJfWd4m37miw/E0Ul0Dnk/uHuY5GOWS8izYC2wObMhGcSsmUOTP8BVM+HLpc5+0WUHOJ3VGnjZdGVyV+BYmncZbAQ9tr2c4xiFtBDRLqLSHPgCmBS2DGTgO+6ty8Fpmq+zefNdXU1zmZCUwbC7s/h1H/BKRPyOkkEBRddrR57PqUl/mzDavwRqaZUNsyAShffEoWq1gE3AFOAJcA/VXWRiNwlIhe6hz0OHCQiK4CfA/tNoTU++uJ9p4jf4nvhiO/BNxZD5+F+R+WLMRf2sf0oClw+T3CwBXcmcbVfOq2I5X+B1t1g0KNw6Fl+R+W7SIuyTOEI7smdq9UdUrFntjGODa/BzB/BrvXQcyQcfzc0a+13VBkXXJVdWV3TuEFOsYgliQIVKBZ27K5r3JM738YtLFEYb/Zshtk3wuq/Q9vecPaH0PEkv6NKWCp2MgyvrVPvtsrr86x1bqITgU5tSxp/jyLtoBcct7BEYfKfKqx9wanRtHerU8Cvz61Q3MLvyBKWquJptirbqLJPHaXuoydHPC5fxi2szLiJbtcGeP9i+PByaN0FzpkNx92Vk0kCUreTYb785zdNVxa2IjvaCu1sWLmdCtaiMPtThZVPwJxfQMMe6Pd76HUjFOX2r0uqiqd5XZUdnDJr+2bnLoH9xp0CxdK4Ijt0rCr82Hza08ZaFGZfO1bC1LOdEhztjodzP4beo3I+SUDqrvq8bIUJ0LpFM8Zc2IdAUf6tTC8UEUed3AdDKwgHHw7+Swe3Vs2H8QmwFoUJaqiHTx6A+beCFMOJD8NRI0Dy51oiUtmNplz1eV2VvaG6pvHYkRPmxXzPSFeuJjvVNmhjd2V4V6aS+D4QucAShYFti2H61bB5OnQ630kSrTvHf12OSWXxtNDaOtE2jgltqcRKBMUiXDmos63ByCGxuivzcQzLEkUhq9/rrKpe9BsIHAgnPwtdr8zLIn5B6SieFq+lcucri2ImgCsHdaa8a3tenr3eNj/KEcGLgHTvA5EtLFEUqs2zYMbVUL0Aul7hFPFr2dHvqHJSvH2Wg4uwIrlqcBfKu7Z3E40liVwQehGQiq7MXGCJotDU7YIFY2DpH6DloXDav+HwC+O+zMQWraUSa+ptWWkJdw/vy5CxU21dRpYLrr4vi9Bdmc59ILKFJYpC8vm7zmymHSvgyGug/zho3tbvqPJarP7q4JVnPvZp54uSQHHM2Uvp3gciW1iiKAS122HuzbDiEWhzBJzxNhyaX7Mysk1wfn20sYnSkkDjCcZ2y8ssLzPMBPK6hZAoSxT5rnIyzPox1GyAXr9wVlY3a+V3VDmhqXWhwkuFhAsUCSJO2YdOpSUM7dXRZjxlQLDbaNSL8/fbTyL8uHyb3posSxT5ancVzB4Ja/4BbY+FU16CDgP9jipnJFMXKlYtqNKSADv37ltl1JJE+gUHmcdNWRYzSeTrYHSy8mc1lXGowurnYXJvWPcC9B3j1GiyJJGQZOpCRRtzEJzV2uEnKksS6VVaEmgcZ4g1HpRvq6lTyVoU+WRXJcy6FipfgYMGwqDHofRYv6PKScnUhYo25tCptCTm6211dnq0btEs7niQ1+6mVJSpz0XWosgHqrDiUacV8dlb0P8PcPZ/LUkkIZm6UJFqQQW7NGK9Plj+Qdw/bR/u1AhNzrH+beIJre2kfNUdOXFuZapDzjqWKHLdl5/C1DNh5ghoPwDOWwDH/ByKbP/mZCRzQhnev4x7Lu67z0k/2KUxalhPoq17D17Vrhp7Ph+OPsP24U6RIpHGk3nw36Zdq6+ScItm3k6DqSpTn4us6ylXNdTDsj/Bx7+GogAMfBSOvDqvy29kUrJ1oaLNrx/ev4yKNVv2G8COlITCYyhyF31F0ypQRG29UtsQ+ZjSkgB76uoLbgV4vep+ExF2h3wH1TW1niYqpKpMfS4S9WH7RhFpD0wAugGrgctUdWuE4+qBBe7dtaoadwlxeXm5VlRUpC7YbFS9EKb/ALbMgrILnCJ+rfK/nzSfNKWvu/voyTHHMILTP2O9b7z3yBYCNCuWmDOUEhVssUUr4hhvnKKpr8sVIjJbVcsjPedXi2I08LaqjhWR0e79myMcV6Oq/TIbWhar3wuLfgeLfweBUhjyPHS5zFoROcjLit7wZNK2JBBzE6RgWfNY75vOxX0lgaKUtlYuPzG1FXWDV/5NbRmkqkx9LvJrjOIi4Cn39lPAcJ/iyB2bZsLrJ8DCO53kcP5i6Hq5JYk8FWngdOfeupibIDV1oD11hKsGd9mn/7+pOpWWMG1plackURIoQnC61lo3j/53C34/TZ2oEGvsKd/5lSgOUdWN7u3PgEOiHNdSRCpEZLqIRE0mIjLCPa6iqqoq5cH6qm6XsyXpmydB7Tb42n/g5GegZQe/IzNpFGngtLZeadOyWcTZUE0daC8tCdCuVQCBxj+jiXdJUlNbz7SlVcy9/etJz9ga2qtjAn3/wn2X92PeHV+ntFXzqEdVVtcwZOxUhvbqmNREhdAJB4WQJCCNXU8i8hZwaISnbg29o6oqItEuHLqqaqWIHAFMFZEFqvpp+EGqOh4YD84YRZKhZ4/Pp7lF/FbCUT+G/vc6+0aYvBftJFm9q5ZVY8/ntokLeG7GOupVKRbhkgHei9PF6p6aOLeSUS/M329APFAsXH5iZ6YtrYq7qx/AtiT3CZ+2tMpzN1lw5lG8BXXgJIuXZldyyYAypi2tKrj1EE2VtkShqmdFe05EPheRw1R1o4gcBnwR5T0q3T9Xisg7QH9gv0SRd/ZWw9xfwqePQpuj4Mx34JCv+R2VyaBYi/Ymzq3kpdmVjTOg6lV5aXYl5V3bJ32yC75+zKRFjeMh7VoFuOOCPo37a8SqY9W2JMCQsVNjdhmVlgQQcZJetOMqq2u4anAXnpm+1lPcwQThJbkEWz75MACdKX7NehoHbA4ZzG6vqr8MO6YdsEtV94hIB+Aj4CJVXRzrvXN+1tP6Sc7q6t2fQa+bnBIczfJvxywTW6QTcnDldnGUabKZmH0TbeYPOMUOEaLOVAov2T1xbmXMvcQTWake/LvHS2Sh771q7Pke370wZOOsp7HAP0XkamANcBmAiJQDP1bVHwLHAH8VkQacsZSx8ZJETtv9BVT8FNZOgNK+zoZCB0X8NzN5ItYU2dA1FJXVNfucNKOtpcjEfP5YnxFt/QYQdcOfWLwmCeGrvT28rj3Jx+1K08mXRKGqm4EzIzxeAfzQvf1foG+GQ8s8VVj9D5jzM6j9Eo77DRzzSyiOPihncp+X6rTBsYRYV/Gh0nXyC01o8Rb9RSIQsaWTisQmwP8O7rJPAgodg4nUwsi2Ka25UD/KSnj4aec6ePcC+OgqaNMDzp0Lx95mSaIAJFIOwssJNV0nv/BpuokmCUiubla4IthnCmzbkgDlXdtHPT7bp7TmSv0oK+HhB22AFeOdAWuthxP+BEffYPWZCki0k39wCmfoVWW0AdpiERpU03oVGm1vjeBnx0sbsRJYpAVs8TQrFkJ7uLyU38jm7UpjXTBkU8zWosi07cvh7aHOgHWHQXD+Quj1M0sSBSbW1XT4VWW0AoV/uOz4tM/nj5bQgi2L4hgLPtu1CtCiWRE3TpjHkLFT97tKDr3a92pvveZVYb5cqR9liSJTGupg8e/hteNg63xnr4ihb0Cb7n5HZtJg4txKhoydSvfRkyOeJOOtkA49+fnZfRKvLHqkrqiSQDFXDe7C7toGqmtqG7tUbpwwj25h30dwAVsiySKSbDuxepXKbrl0skSRCVvnwxuDYd7NcNg5TvmNI39g5TfylJd+Zy9X0xuqaxoTzo3uNNL7Lu+X0RXBXkt+FIvsk8SmLa3a78o/mFIifR/JlhZpm6N7dyRTzj6TLFGkU/0emP9reL0cdq2DU/4Jp74MrTr5HZlJI68D1fGuptuWBHwf6AxvzUTToLpPN1i8K/zw7yNSqylW3aZwuXrNle2D7UE2mJ0uVR/BjKth+xLo/h044Y/Q4iC/ozIZkGi/c7SqpHsj7B2RyEBnqqZdhg4GR5uqG95V4mWFdPj3ET7o7HXxHDirvHNVNg+2B1mLItXqdsLskfDmEOf26a/BSU9ZkiggifY7R7qqvGRAGbuilOz20h+frmmXXrtKvHQlNaVaa7Rig9nWp59vrEWRSp+9BTOugZ2rocf10O8eCBzgd1Qmw5qyb0H4VeWQsVOjHuvlpJiuaZded/6LtbIcEqvWGq+VkY19+vnGEkUq7N0Kc26ClU/AAUfDWe/Bwaf6HZXxSbLbqELsVoOXk2I6p1167SoJXyGdqm4wSO67NYmzRJGsdf+CWdfBniroPRr63gHFLf2Oyvgs2X7naH387VoFPL1vrOqzfkhlP3wu9OnnGxujaKqaz+GDy+D9i6HkUBg20+lqsiRhUiDaWMAdF/RJ6vXWRWOawloUiVKFVX+HOSOd3eeO/x0ccxMU5eY8bpOdku1isS4ak0q+7EeRTmndj2LnWpj5I9j4OnQ42Vld3bZXej7LGGMyKBv3o8gt2gDLH4Z5owGFAQ/A0deBWM+dMSb/WaKIZ/syZ9/qqg/g0K/DwL9Cm25+R2WMMRljiSKahlpY8gdYMAaatYLBf3NWWOdqrQBjjGkiSxSRbJnrlN/YOhc6XwLlDzozmwpALuy2ZYzJLEsUoep3w8LfwOJ7oUUHOOVF6HKJ31FljJftOY0xhceX0VgR+R8RWSQiDSIScZTdPe4cEVkmIitEZHRag6r6EF7rB4t+53QxfWNJQSUJSGx7TmNM4fBr2s5C4GLgvWgHiEgx8BBwLtAbuFJEeqc8ktodUPFTePNUp0UxdAoMfgKat0v5R2W7XNltyxiTWb50PanqEgCJPTA8EFihqivdY58HLgIWpyyQDVNg5ghnr4ijfwLH/xYCbVL29rkm28o+GGOyQzYvBCgD1oXcX+8+lrw9W+Cj78E75zgzms5+H8rvL+gkAVb2wRgTWdpaFCLyFhBpqtCtqvrvFH/WCGAEQJcuXWIfvPYlqLge9myCPrfCsbdZfSaXlX0wxkSStkShqmcl+RaVQOeQ+4e7j0X6rPHAeHBKeER8t5qNUHEDrHsZ2p0AQ1+Hdv2SDDH/WGVOY0y4bJ4eOwvoISLdcRLEFcC3En4XVVj1FMy+EeproN9Y6PULKMrmv7oxxmQPv6bHflNE1gMnAZNFZIr7eCcReRVAVeuAG4ApwBLgn6q6KKEP2rEapg2D6d+H0r5w3sfQ+2ZLEsYYkwC/Zj39C/hXhMc3AOeF3H8VeDXhD2ioh+UPwfxfAQLlD0GPH1sRP2OMaYL8u7Su3w1vnQab/guHneMU8WsdZ4DbGGNMVPmXKLYthu3t4KSnodtVVsTPGGOSlHcbF4lIFbDG7zhCdAA2+R1EAize9Mq1eCH3YrZ4m6arqnaM9ETeJYpsIyIV0XaNykYWb3rlWryQezFbvKlno7vGGGNiskRhjDEmJksU6Tfe7wASZPGmV67FC7kXs8WbYjZGYYwxJiZrURhjjInJEoUxxpiYLFGkgIi0F5E3RWS5+2fE7fFE5HURqRaR/4Q9/jcRWSUi89yftJa1TUG83UVkhrtF7QQRaZ4l8X7XPWa5iHw35PF33C11g9/vwWmKM+bWvSLSwv2+VrjfX7eQ525xH18mIsPSEV+q4hWRbiJSE/J9PpIl8Z4mInNEpE5ELg17LuLvRpbHXB/yHU/KVMwRqar9JPkD/B4Y7d4eDdwb5bgzgQuA/4Q9/jfg0hyK95/AFe7tR4Br/Y4XaA+sdP9s595u5z73DlCe5hiLgU+BI4DmwHygd9gx1wGPuLevACa4t3u7x7cAurvvU5zF8XYDFmbq9zWBeLsBxwFPh/5/ivW7ka0xu8/tyOR3HOvHWhSpcRHwlHv7KWB4pINU9W3gy0wFFUOT4xVn/9ozgBfjvT6FvMQ7DHhTVbeo6lbgTeCcNMcVqnHrXlXdCwS37g0V+vd4ETjT/T4vAp5X1T2qugpY4b5ftsbrh7jxqupqVf0YaAh7rV+/G8nEnFUsUaTGIaq60b39GXBIE97jtyLysYjcJyItUhhbJMnEexBQrU4ZeEjlFrXReYk33ta5T7pN+F+n6WTnZevexmPc728bzveZvm1/o0smXoDuIjJXRN4VkVPTHOs+sbgS+Y78+H5T8bktRaRCRKaLSLovxmLKv6KAaSIxtnYNvaOqKiKJzjm+BecE2BxnTvXNwF1NiTMozfGmXJrj/V9VrRSRA4CXgG/jNPVN02wEuqjqZhEZAEwUkT6qut3vwPJMV/f39ghgqogsUNVP/QjEEoVHGmNrVxH5XEQOU9WNInIY8EWC7x28Wt4jIk8CNyURavA90xXvZqBURJq5V5lRt6hNRArirQROD7l/OM7YBKpa6f75pYj8A6dLINWJwsvWvcFj1otIM6AtzvfpedvfFGpyvOp0oO8BUNXZIvIpcDRQ4XO8sV57ethr30lJVPE/t8n/riG/tytF5B2gP86YR8ZZ11NqTAKCMym+C/w7kRe7J79g//9wYGFKo9tfk+N1TxLTgOAMjYT/vk3gJd4pwNdFpJ07K+rrwBQRaSYiHQBEJAB8g/R8v41b97qzwK5w447297gUmOp+n5OAK9xZRt2BHsDMNMSYknhFpKOIFAO4V7s9cAaI/Y43moi/G2mKM1STY3ZjbeHe7gAMARanLdJ4/B5Nz4cfnH7bt4HlwFtAe/fxcuCxkOPeB6qAGpz+ymHu41OBBTgnsGeANlke7xE4J7IVwAtAiyyJ9wduTCuA77uPtQZmAx8Di4D7SdOMIpzdGT/Bueq71X3sLuBC93ZL9/ta4X5/R4S89lb3dcuAczP0e9ukeIFL3O9yHjAHuCBL4j3R/T3didNSWxTrdyObYwZOds8J890/r85UzJF+rISHMcaYmKzryRhjTEyWKIwxxsRkicIYY0xMliiMMcbEZInCGGNMTJYojEkhETld3Gq7InJhpIqhIceWish1Ifc7iciL0Y43xi82PdYYD0SkWFXrPRx3OnCTqn7Dw7HdcCrzHpt0gMakkbUoTMFz91dYKiLPisgSEXlRRFqJyGoRuVdE5gD/IyJfF5GP3P0DXhCRNu7rz3FfPwe4OOR9vyciD7q3DxGRf4nIfPfnZGAscKRbrHCcG8dC9/iWIvKkiCxwi+8NDXnPl8XZK2S5iPzefbxYnH1NFrqvuTGz36LJZ1bryRhHT5zVrx+KyBM4ezGAU9voBLeMwsvAWaq6U0RuBn7unqgfxSm9vgKYEOX9/wy8q6rfdMtftMHZW+NYVe0HjS2MoOtxKqb0FZFewBsicrT7XD+cuj97gGUi8gBwMFAWbJ2ISGmyX4gxQdaiMMaxTlU/dG8/A5zi3g6e+AfjbDD0oYjMw6mB1BXoBaxS1eXq9OM+E+X9zwAeBlDVelXdFieeU4LvpapLgTU4hfcA3lbVbaq6G6f+T1ecWktHiMgDInIOYJVcTcpYi8IYR/hgXfD+TvdPwdn85srQgyTN29ZGsSfkdj3QTFW3isjxOJv0/Bi4DKe+kTFJsxaFMY4uInKSe/tbwAdhz08HhojIUQAi0trtCloKdBORI93jriSyt4Fr3dcWi0hbnN0DD4hy/PvA/7rHHw10wSkYGJHbNVakqi8BtwEnRDvWmERZojDGsQy4XkSW4Oyr/HDok6paBXwPeE5EPgY+Anq53T8jgMnuYHa0vT1+BgwVkQU41Wx7q+pmnK6shSIyLuz4vwBF7vETgO+p6h6iKwPecbvFnsHZDMuYlLDpsabg2TRVY2KzFoUxxpiYrEVhjDEmJmtRGGOMickShTHGmJgsURhjjInJEoUxxpiYLFEYY4yJ6f8BYcx/PYqUSWsAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "fig,axs = plt.subplots()\n",
        "axs.scatter(y_test_pred,y_test)\n",
        "axs.plot([0,1],[0,1], transform=axs.transAxes, c=\"orange\")\n",
        "axs.set_xlabel(\"predictions\")\n",
        "axs.set_ylabel(\"realizations\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zhBdSbqX53lp"
      },
      "source": [
        "# step 4"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LHgBFXitbzIH"
      },
      "outputs": [],
      "source": [
        "output_fun = 'linear'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T_dfTz8t6NHa",
        "outputId": "619818e9-b92b-4948-8862-7903e56fdd85"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/25\n",
            "135/135 [==============================] - 3s 8ms/step - loss: 0.0431 - val_loss: 0.0223\n",
            "Epoch 2/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0213 - val_loss: 0.0211\n",
            "Epoch 3/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0210 - val_loss: 0.0259\n",
            "Epoch 4/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0211 - val_loss: 0.0221\n",
            "Epoch 5/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0209 - val_loss: 0.0250\n",
            "Epoch 6/25\n",
            "135/135 [==============================] - 0s 4ms/step - loss: 0.0209 - val_loss: 0.0219\n",
            "Epoch 7/25\n",
            "135/135 [==============================] - 1s 5ms/step - loss: 0.0209 - val_loss: 0.0215\n",
            "Epoch 8/25\n",
            "135/135 [==============================] - 1s 5ms/step - loss: 0.0209 - val_loss: 0.0252\n",
            "Epoch 9/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0209 - val_loss: 0.0227\n",
            "Epoch 10/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0209 - val_loss: 0.0227\n",
            "Epoch 11/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0209 - val_loss: 0.0230\n",
            "Epoch 12/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0208 - val_loss: 0.0229\n",
            "Epoch 13/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0208 - val_loss: 0.0202\n",
            "Epoch 14/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0209 - val_loss: 0.0227\n",
            "Epoch 15/25\n",
            "135/135 [==============================] - 0s 3ms/step - loss: 0.0208 - val_loss: 0.0232\n",
            "Epoch 16/25\n",
            "135/135 [==============================] - 1s 5ms/step - loss: 0.0208 - val_loss: 0.0209\n",
            "Epoch 17/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0209 - val_loss: 0.0243\n",
            "Epoch 18/25\n",
            "135/135 [==============================] - 0s 4ms/step - loss: 0.0208 - val_loss: 0.0215\n",
            "Epoch 19/25\n",
            "135/135 [==============================] - 0s 4ms/step - loss: 0.0208 - val_loss: 0.0229\n",
            "Epoch 20/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0207 - val_loss: 0.0280\n",
            "Epoch 21/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0208 - val_loss: 0.0231\n",
            "Epoch 22/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0207 - val_loss: 0.0236\n",
            "Epoch 23/25\n",
            "135/135 [==============================] - 0s 4ms/step - loss: 0.0208 - val_loss: 0.0213\n",
            "Epoch 24/25\n",
            "135/135 [==============================] - 1s 4ms/step - loss: 0.0207 - val_loss: 0.0241\n",
            "Epoch 25/25\n",
            "135/135 [==============================] - 0s 4ms/step - loss: 0.0208 - val_loss: 0.0212\n"
          ]
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from keras import Sequential\n",
        "from keras.layers import Dense, Input\n",
        "\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.InputLayer(input_shape=(X_train_s.shape[1])),\n",
        "    tf.keras.layers.Dense(20, activation='tanh', name='hidden_layer_1'),\n",
        "    tf.keras.layers.Dense(20, activation='tanh', name='hidden_layer_2'),\n",
        "    tf.keras.layers.Dense(10, activation='tanh', name='hidden_layer_3'),\n",
        "    tf.keras.layers.Dense(5, activation='tanh', name='hidden_layer_4'),\n",
        "    tf.keras.layers.Dense(1, activation=output_fun)\n",
        "])\n",
        "\n",
        "model.compile(loss='mean_squared_error', optimizer= \"RMSprop\")\n",
        "history_nn = model.fit(X_train_s, \n",
        "          y_train, \n",
        "          epochs = 25,\n",
        "          validation_data=(X_val_s, y_val))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 404
        },
        "id": "alQ6y1oC_nbP",
        "outputId": "ab7feb7b-70da-4705-b1a7-7d7f550b0e23"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAGDCAYAAAABCJbEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5ycZb338c9v2u7OpneSAAkESCeNiAIBDGCQ3kEsKIJybOc5R89BHxX16Hn0HAsWLCBYEBUMgqggRQEBEUhIT4CEJEB6b9t353r+uO7Znd3M7k7dmWS/79drXzNzz33fc83u7Pzuq/7MOYeIiEiuQqUugIiIHNoUSEREJC8KJCIikhcFEhERyYsCiYiI5EWBRERE8qJAIpIFM/u5mX01w33Xm9lZRSzLtWb2WKH3FcmWAolICWQTkDrjnLvHOXdOofcVyZYCiUgZMrNIqcsgkikFEjnsBE1KnzGzpWZWY2Z3mtlwM3vEzPab2RNmNjBl/wvNbIWZ7TGzp8xsQspz083s5eC4e4HKDq91vpktDo79h5lNzaB8NwLXAv9hZgfM7I8p5f5PM1sK1JhZxMxuNrPXg9dfaWaXpJznOjN7NuWxM7OPmtnqoDy3mZnlsG/YzL5lZjvMbJ2ZfTzYX8FN0lIgkcPVZcDZwPHABcAjwOeAofjP/ScBzOx44DfAvwbPPQz80cxiZhYDHgTuBgYBvwvOS3DsdOAu4CPAYOAnwENmVtFVwZxztwP3AP/jnOvjnLsg5elrgPOAAc65ZuB14DSgP/Bl4FdmdkQXpz8fOAmYClwJvCuHfW8AzgWmATOAi7t6PyIKJHK4+r5zbqtzbiPwDPCCc26Rc64eeACYHux3FfBn59zjzrkm4JtAFfAO4GQgCtzqnGtyzs0HXkp5jRuBnzjnXnDOtTjnfgE0BMfl6nvOubecc3UAzrnfOec2OecSzrl7gdXA7C6O/7pzbo9z7k3gSXwwyHbfK4HvOuc2OOd2A1/P4/1IL6BAIoerrSn369I87hPcHwm8kXzCOZcA3gJGBc9tdO1XNn0j5f7RwL8HTUN7zGwPcGRwXK7eSn1gZu9PaTrbA0wGhnRx/JaU+7W0vc9s9h3ZoRztyiTSkdo8pbfbBExJPgj6CY4ENgIOGGVmlhJMjsI3N4H/gv2ac+5rObxuZ8tut243s6OBO4C5wPPOuRYzWwxYDq+Xjc3A6JTHRxb59eQQpxqJ9Hb3AeeZ2VwziwL/jm+e+gfwPNAMfNLMomZ2Ke2ble4APmpmbzOv2szOM7O+GbzuVuCYbvapxgeW7QBm9kF8jaTY7gM+ZWajzGwA8J898JpyCFMgkV7NOfcq8F7g+8AOfMf8Bc65RudcI3ApcB2wC9+f8vuUYxfgO6Z/AOwG1gT7ZuJOYGLQZPVgJ2VbCXwLH9C24mtOz2X3DnNyB/AYsBRYhB+A0Ay09MBryyHIlNhKRLpiZucCP3bOHV3qskh5Uo1ERNoxsyoze3cwj2UUcAt+pJtIWqqRiEg7ZhYHngbG40e4/Rn4lHNuX0kLJmVLgURERPKipi0REcmLAomIiOSlV0xIHDJkiBszZkypiyEickhZuHDhDufc0O726xWBZMyYMSxYsKDUxRAROaSY2Rvd76WmLRERyZMCiYiI5EWBRERE8tIr+khE5PDR1NTEhg0bqK+vL3VRDhuVlZWMHj2aaDSa0/EKJCJySNmwYQN9+/ZlzJgxBNmBJQ/OOXbu3MmGDRsYO3ZsTudQ05aIHFLq6+sZPHiwgkiBmBmDBw/Oq4anQCIihxwFkcLK9/epQCIikoU9e/bwwx/+MOvj3v3ud7Nnz54u9/niF7/IE088kWvRSkaBREQkC50Fkubm5i6Pe/jhhxkwYECX+3zlK1/hrLPOyqt8paBAIiKShZtvvpnXX3+dadOmcdJJJ3Haaadx4YUXMnHiRAAuvvhiZs6cyaRJk7j99ttbjxszZgw7duxg/fr1TJgwgRtuuIFJkyZxzjnnUFdXB8B1113H/PnzW/e/5ZZbmDFjBlOmTOGVV14BYPv27Zx99tlMmjSJD3/4wxx99NHs2LGjh38L7WnUlogcsr78xxWs3FTYNCkTR/bjlgsmdfr817/+dZYvX87ixYt56qmnOO+881i+fHnriKe77rqLQYMGUVdXx0knncRll13G4MGD251j9erV/OY3v+GOO+7gyiuv5P777+e9733vQa81ZMgQXn75ZX74wx/yzW9+k5/+9Kd8+ctf5p3vfCef/exn+ctf/sKdd95Z0PefC9VIuvDS+l0s37i31MUQkTI2e/bsdsNmv/e973HiiSdy8skn89Zbb7F69eqDjhk7dizTpk0DYObMmaxfvz7tuS+99NKD9nn22We5+uqrAZg3bx4DBw4s4LvJjWokXfjc75dx3PA+/PDamaUuioik0VXNoadUV1e33n/qqad44okneP7554nH45xxxhlph9VWVFS03g+Hw61NW53tFw6Hu+2DKSXVSLoQj4WpaWgpdTFEpIz07duX/fv3p31u7969DBw4kHg8ziuvvMI///nPgr/+Kaecwn333QfAY489xu7duwv+GtlSjaQL8ViEukYFEhFpM3jwYE455RQmT55MVVUVw4cPb31u3rx5/PjHP2bChAmccMIJnHzyyQV//VtuuYVrrrmGu+++m7e//e2MGDGCvn37Fvx1stErcrbPmjXL5ZKP5Pqfv8SWffX8+ZOnFaFUIpKLVatWMWHChFIXo2QaGhoIh8NEIhGef/55brrpJhYvXpz3edP9Xs1soXNuVnfHqkbShXiFaiQiUl7efPNNrrzyShKJBLFYjDvuuKPURVIg6Uo8GqamsXw7uESk9znuuONYtGhRqYvRjjrbuxCvCFOrGomISJcUSLoQj/lA0hv6kUREcqVA0oV4LEJLwtHYkih1UUREypYCSRfisTAAtZpLIiLSKQWSLlTH/FiE2iYFEhHJTZ8+fQDYtGkTl19+edp9zjjjDLqbonDrrbdSW1vb+jiTZel7igJJF6qCGkmdRm6JSJ5GjhzZurJvLjoGkkyWpe8pCiRdqK7wgUTLpIhI0s0338xtt93W+vhLX/oSX/3qV5k7d27rku9/+MMfDjpu/fr1TJ48GYC6ujquvvpqJkyYwCWXXNJura2bbrqJWbNmMWnSJG655RbALwS5adMmzjzzTM4880ygbVl6gG9/+9tMnjyZyZMnc+utt7a+XmfL1Rea5pF0oSoaNG1pCLBIeXrkZtiyrLDnHDEFzv16p09fddVV/Ou//isf+9jHALjvvvt49NFH+eQnP0m/fv3YsWMHJ598MhdeeGGnKWx/9KMfEY/HWbVqFUuXLmXGjBmtz33ta19j0KBBtLS0MHfuXJYuXconP/lJvv3tb/Pkk08yZMiQdudauHAhP/vZz3jhhRdwzvG2t72N008/nYEDB2a8XH2+VCPpQrJGUqumLREJTJ8+nW3btrFp0yaWLFnCwIEDGTFiBJ/73OeYOnUqZ511Fhs3bmTr1q2dnuPvf/976xf61KlTmTp1autz9913HzNmzGD69OmsWLGClStXdlmeZ599lksuuYTq6mr69OnDpZdeyjPPPANkvlx9vlQj6ULrqC3VSETKUxc1h2K64oormD9/Plu2bOGqq67innvuYfv27SxcuJBoNMqYMWPSLh/fnXXr1vHNb36Tl156iYEDB3LdddfldJ6kTJerz5dqJF2IJ0dtqUYiIimuuuoqfvvb3zJ//nyuuOIK9u7dy7Bhw4hGozz55JO88cYbXR4/Z84cfv3rXwOwfPlyli5dCsC+ffuorq6mf//+bN26lUceeaT1mM6Wrz/ttNN48MEHqa2tpaamhgceeIDTTuvZhWZVI+mCaiQiks6kSZPYv38/o0aN4ogjjuDaa6/lggsuYMqUKcyaNYvx48d3efxNN93EBz/4QSZMmMCECROYOdMnzzvxxBOZPn0648eP58gjj+SUU05pPebGG29k3rx5jBw5kieffLJ1+4wZM7juuuuYPXs2AB/+8IeZPn160Zqx0tEy8l1obE5w/Ocf4TPvOoGPnTmuCCUTkWz19mXkiyWfZeTVtNWFWCREJGRq2hIR6YICSTeUbldEpGsKJN1Qul0Rka4pkHQjXqHkViLlpjf07fakfH+fCiTdiMfCqpGIlJHKykp27typYFIgzjl27txJZWVlzufQ8N9uxKMR1UhEysjo0aPZsGED27dvL3VRDhuVlZWMHj065+MVSLoRrwizu6ax1MUQkUA0GmXs2LGlLoakUNNWN+KxMDVq2hIR6ZQCSTc0aktEpGsKJN3wNRL1kYiIdEaBpBvxWERrbYmIdEGBpBvxWJjG5gTNLYlSF0VEpCwpkHSjdQXgJtVKRETSUSDpRmtOEq23JSKSlgJJN5RuV0Skawok3aiKKrmViEhXFEi6UV2RTLerQCIiko4CSTeqgs52zSUREUmvqIHEzOaZ2atmtsbMbk7zfIWZ3Rs8/4KZjenw/FFmdsDMPp3pOQutOuhs1+x2EZH0ihZIzCwM3AacC0wErjGziR12ux7Y7ZwbB3wH+EaH578NPJLlOQsqOfy3pkE1EhGRdIpZI5kNrHHOrXXONQK/BS7qsM9FwC+C+/OBuWZmAGZ2MbAOWJHlOQsqGUjqNI9ERCStYgaSUcBbKY83BNvS7uOcawb2AoPNrA/wn8CXczhnQSXnkShvu4hIeuXa2f4l4DvOuQO5nsDMbjSzBWa2IJ8EOJXREGZQp852EZG0ipnYaiNwZMrj0cG2dPtsMLMI0B/YCbwNuNzM/gcYACTMrB5YmME5AXDO3Q7cDjBr1qycc3KaGfGocpKIiHSmmIHkJeA4MxuL/7K/GnhPh30eAj4APA9cDvzN+UTMpyV3MLMvAQeccz8Igk135yy4eIVWABYR6UzRAolzrtnMPg48CoSBu5xzK8zsK8AC59xDwJ3A3Wa2BtiFDwxZn7NY7yEpHgtriRQRkU4UNWe7c+5h4OEO276Ycr8euKKbc3ypu3MWm3KSiIh0rlw728tKPBbWhEQRkU4okGRA6XZFRDqnQJIB1UhERDqnQJKB6lhENRIRkU4okGSgSjUSEZFOKZBkoLoioiVSREQ6oUCSgapomLqmFhKJnCfIi4gcthRIMpDM264VgEVEDqZAkoGqmNLtioh0RoEkA9VBThItkyIicjAFkgzEWwOJaiQiIh0pkGSgrWlLNRIRkY4USDJQrRqJiEinFEgyUBUEEs0lERE5mAJJBqqDpq26JjVtiYh0pECSgbhqJCIinVIgyUC8IqiRqI9EROQgCiQZqIoGNRKN2hIROYgCSQbCIaMyGlKNREQkDQWSDMWVk0REJC0FkgzFY2HNIxERSUOBJEPxWJhajdoSETmIAkmG4rEItVpGXkTkIAokGfI1EvWRiIh0pECSoXgsoj4SEZE0FEgy5DvbVSMREelIgSRD1RUatSUiko4CSYaqomraEhFJR4EkQ75G0oxzrtRFEREpKwokGaqKhUk4aGhOlLooIiJlRYEkQ9Wt6XbVvCUikkqBJENtWRI1cktEJJUCSYbasiSqRiIikkqBJEPJLIlq2hIRaU+BJEOtgURNWyIi7SiQZCiuznYRkbQUSDIUr1C6XRGRdBRIMpRs2lK6XRGR9hRIMpRs2qpRIBERaUeBJENtNRI1bYmIpFIgyVA0HCIWDqlGIiLSgQJJFqpiYfWRiIh0oECShepYWEukiIh0oECShapYmFotkSIi0o4CSRbisYhmtouIdKBAkgWft101EhGRVAokWVAgERE5mAJJFuIVEWo1j0REpB0FkizEo6qRiIh0pECSheqKiAKJiEgHCiRZqIqF1bQlItKBAkkWqmNhmlocjc2JUhdFRKRsKJBkoSqZt13NWyIirRRIslCdTLfbpOYtEZEkBZIsVAWBpKZBNRIRkSQFkixUq2lLROQgRQ0kZjbPzF41szVmdnOa5yvM7N7g+RfMbEywfbaZLQ5+lpjZJSnHrDezZcFzC4pZ/o6Sya2Ut11EpE2kWCc2szBwG3A2sAF4ycwecs6tTNntemC3c26cmV0NfAO4ClgOzHLONZvZEcASM/ujcy75DX6mc25HscremXiFaiQiIh0Vs0YyG1jjnFvrnGsEfgtc1GGfi4BfBPfnA3PNzJxztSlBoxJwRSxnxlQjERE5WDEDySjgrZTHG4JtafcJAsdeYDCAmb3NzFYAy4CPpgQWBzxmZgvN7MbOXtzMbjSzBWa2YPv27QV5Q8lAotntIiJtyraz3Tn3gnNuEnAS8FkzqwyeOtU5NwM4F/iYmc3p5PjbnXOznHOzhg4dWpAyxYPOduUkERFpU8xAshE4MuXx6GBb2n3MLAL0B3am7uCcWwUcACYHjzcGt9uAB/BNaD2itUaiLIkiIq2KGUheAo4zs7FmFgOuBh7qsM9DwAeC+5cDf3POueCYCICZHQ2MB9abWbWZ9Q22VwPn4Dvme0RFJETIoFbzSEREWhVt1FYw4urjwKNAGLjLObfCzL4CLHDOPQTcCdxtZmuAXfhgA3AqcLOZNQEJ4F+cczvM7BjgATNLlv3Xzrm/FOs9dGRmVMe0ArCISKqiBRIA59zDwMMdtn0x5X49cEWa4+4G7k6zfS1wYuFLmjmtACwi0l7ZdraXK+UkERFpT4EkS1VR1UhERFIpkGSpukLpdkVEUimQZKlKne0iIu0okGSpWp3tIiLtKJBkyY/aUo1ERCRJgSRLmkciItKeAkmW4mraEhFpR4EkS/FYhPqmBC2JsljZXkSk5BRIspRcuLFOCzeKiAAKJFmLVwQrAGspeRERQIEka0puJSLSngJJlqqifp1LpdsVEfEUSLJUHTRt1alGIiICKJBkLdm0VaNAIiICKJBkLZm3vU5NWyIigAJJ1lprJEq3KyICKJBkLVkjqdU8EhERQIEka63DfzWPREQEUCDJWlVU80hERFIpkGQpFDKl2xURSaFAkgOl2xURaaNAkgMltxIRaaNAkgOf3EpNWyIioECSE9VIRETaKJDkQOl2RUTaKJDkoCoWpkbzSEREAAWSnFTHwsqQKCISUCDJQVUsorW2REQCCiQ5qI6FtfqviEhAgSQH8ViY2qYWnHOlLoqISMllFEjM7FNm1s+8O83sZTM7p9iFK1fxigjOQX1TotRFEREpuUxrJB9yzu0DzgEGAu8Dvl60UpW5tiyJat4SEck0kFhw+27gbufcipRtvU5blkR1uIuIZBpIFprZY/hA8qiZ9QV6bbuOaiQiIm0iGe53PTANWOucqzWzQcAHi1es8taa3Eo1EhGRjGskbwdedc7tMbP3Ap8H9havWOWtNd2u5pKIiGQcSH4E1JrZicC/A68DvyxaqcpcW41ETVsiIpkGkmbnJ01cBPzAOXcb0Ld4xSpvyUCiZVJERDLvI9lvZp/FD/s9zcxCQLR4xSpv1RX+16ZlUkREMq+RXAU04OeTbAFGA/9btFKVuSo1bYmItMookATB4x6gv5mdD9Q753pvH0lUo7ZERJIyXSLlSuBF4ArgSuAFM7u8mAUrZ5FwiFgkpEAiIkLmfST/FzjJObcNwMyGAk8A84tVsHJXHQuraUtEhMz7SELJIBLYmcWxh6W40u2KiACZ10j+YmaPAr8JHl8FPFycIh0aqlQjEREBMgwkzrnPmNllwCnBptudcw8Ur1jlzzdtqUYiIpJpjQTn3P3A/UUsyyGlKhbWEikiInQTSMxsP5AuDaABzjnXryilOgRUxyJs3V9f6mKIiJRcl4HEOddrl0HpjmokIiJerx55lY9qjdoSEQEUSHJWFQsrsZWICAokOauuCFPX2IJfFFlEpPdSIMlRPBahOeFobOm1GYdFRAAFkpy15iRRP4mI9HIKJDlKBpIaBRIR6eWKGkjMbJ6ZvWpma8zs5jTPV5jZvcHzL5jZmGD7bDNbHPwsMbNLMj1nT0nmba9Th7uI9HJFCyRmFgZuA84FJgLXmNnEDrtdD+x2zo0DvgN8I9i+HJjlnJsGzAN+YmaRDM/ZI1prJJpLIiK9XDFrJLOBNc65tc65RuC3+JzvqS4CfhHcnw/MNTNzztU655KX+pW0za7P5Jw9Ilkj0VwSEentihlIRgFvpTzeEGxLu08QOPYCgwHM7G1mtgJYBnw0eD6TcxIcf6OZLTCzBdu3by/A22kvrnS7IiJAGXe2O+decM5NAk4CPmtmlVkef7tzbpZzbtbQoUMLXr7qCqXbFRGB4gaSjcCRKY9HB9vS7mNmEaA/PmlWK+fcKuAAMDnDc/aIqtamLdVIRKR3K2YgeQk4zszGmlkMuBp4qMM+DwEfCO5fDvzNOeeCYyIAZnY0MB5Yn+E5e0R1TDUSERHIIh9JtpxzzWb2ceBRIAzc5ZxbYWZfARY45x4C7gTuNrM1wC58YAA4FbjZzJqABPAvzrkdAOnOWaz30JUqBRIREaCIgQTAOfcwHVLyOue+mHK/HrgizXF3A3dnes5SiIVDREKmpi0R6fXKtrO93JmZXwFY80hEpJdTIMlDdSyitbZEpNdTIMlDXDlJREQUSPIRD3KSiIj0ZgokeYhHI6qRiEivp0CSB9VIREQUSPLi+0gUSESkd1MgyUNco7ZERBRI8hGPhTUhUUR6PQWSPMRjETVtiUivp0CSh3gsTGNzguaWRKmLIiJSMgokeWhNbtWkWomI9F4KJHlIpttVh7uI9GYKJHlI1khqGtThLiK9lwJJHuLKSSIiokCSj3hrul0FEhHpvRRI8hCvSNZI1LQlIr2XAkke1LQlIqJAkpdqNW2JiCiQ5KMqpqYtkbL22mOwdUWpS3HYUyDJg2okImUskYD7r4e//lepS3LYUyDJQ2U0hBnUah6JSPnZvQ4a9sHmxaUuyWFPgSQPZkY8GlaNRKQcbVnqb/dvhn2bS1uWw5wCSZ6qtAKwSHnavCTlvmolxaRAkqfqijB16mwXKT+bl8KgY8BCsEmBpJgipS7Aoa4qqnS7ImXHOV8jOf5dEK6ATYtKXaLDmgJJnqorlG5XpOzs3wy1O+CIE8El4PW/+eBiVuqSHZbUtJWneCxMjZq2RMrL5qCjfcRUGDkdDmz1wUWKQjWSPMVjYbbvbyh1MUQk1ZalgMGIyb6PBHzzVr+RJS3W4Uo1kjz5vO2qkYiUlc1LYPCxUNEXRkxRh3uRKZDkKR4Lq49EpNxsXuqbtQBicRg6QR3uRaRAkqd4LExNgwKJSNmo3QV734QjprZtGznNzyVxrnTlOowpkOQpHotQ19RCIqEPqEhZSM5oP+LEtm0jp0PNdti3sTRlOswpkOQpmZOkrkm1EpGy0Dpiq0MgATVvFYkCSZ7iFVoBWKSsbFkK/UZB9eC2bcMngYXV4V4kCiR5ikeVk0SkrGxe0r5ZCyBaBcMmqkZSJAokeaquULpdkbLRWAM7VreN2Eo18kR1uBeJAkmeqlqTW6lGIlJyW1cArv2IraSR06F2J+x9q8eLdbhTIMlTdUw1EpGykVw6vmPTFqjDvYgUSPKUzNuuuSQiZWDzEqga5DvbOxo2CUIRdbgXgQJJnpJ52+ua1LQlUnJblvraSLpVfqOV6nAvEgWSPMVVIxEpD82NsHVl+v6RJM1wLwoFkjwl55FovS2REtv+CiSa0o/YSho5Hep2w543eq5cvYACSZ6qoupsFykLrUujTOt8H3W4F4UCSZ7CIaMyGtLwX5FS27wEYn18nvbODJsIoag63AtMgaQA4rGIaiQipbZ5KQyfDKEuvtYiFX65FNVICkqBpACqonmk2921Dl74iTr/RPKRSMCWZennj3SkDveCUyApgOqKPJJbvXgHPPIfsGttYQsl0pvseh2aaroesZU0cjrU74Xd64pfrl5CgaQAqmIRanINJMkq9rqnC1cgkd4mOaO9qxFbSepwLzgFkgKojoWpy6VpK9HSNtJk3d8LWyiR3mTLUgjHYOj47vcdOsHvqw73glEgKYCc0+3uXAONByDWF9Y949t5RSR7m5fAsAkQiXW/byTmO+VVIykYBZICSKbbzVrygzzrg1C7A7avKmzBRHoD5/yIrUyatZJGTvPH6OKtIBRICsDXSHJo2tq0GKJxOOnD/vFa9ZOIZG3fRqjbldmIraSR06FBHe6FokBSAPFYJLdRW5sW+auogUf7SVTqJxHJXldLx3dGHe4FpUBSAPGYn0fishmX3tLsOwiTH+ixc+CN5/x2Ecnc5qWA+YmGmRo6HsIVCiQFUtRAYmbzzOxVM1tjZjeneb7CzO4Nnn/BzMYE2882s4Vmtiy4fWfKMU8F51wc/Awr5nvIRLwiTMJBQ3MW7a07XoOm2vaBpGFf29VVb1G7SxPDJD9blsKQ4yFWnfkx4SiMmKKRWwVStEBiZmHgNuBcYCJwjZlN7LDb9cBu59w44DvAN4LtO4ALnHNTgA8Ad3c47lrn3LTgZ1ux3kOm4rks3Lg5+AAnA8mYOf62N80nqd0F35kML/y41CWRQ9nmJZlNROxo5DR/rDrc81bMGslsYI1zbq1zrhH4LXBRh30uAn4R3J8PzDUzc84tcs5tCravAKrMrKKIZc1Lcin5rBZu3LTILzA3eJx/3Geoz+DWmwLJur/72cgv3q5/ZslNzU7f2Z7NiK2kkdOhcb+fFS95KWYgGQW8lfJ4Q7At7T7OuWZgLzC4wz6XAS875xpStv0saNb6glm6VGg9K55L3vZNi3znYOoCc2PnwJv/hOaGzo/L1a61fgJkOVn7lL/dtRbWP1PSosghaksOHe1JrR3uat7KV1l3tpvZJHxz10dSNl8bNHmdFvy8r5NjbzSzBWa2YPv27UUtZzLdbsaBpKXZLzCX/CAnjZ0DzfWw4aXCFnDHGvj+LFj4s8KeN1/rnoZjzoTKAbDw56UujRyKWpdGmZL9sUNOgEiVOtwLoJiBZCNwZMrj0cG2tPuYWQToD+wMHo8GHgDe75xrrXs65zYGt/uBX+Ob0A7inLvdOTfLOTdr6NChBXlDnalK1kgynUuy/RUfMDom4BlzClio8MOAl88H1wLLHyjsefOx501fEzn+XTDtPfDKn6BmR6lLJYeazUuh/1EQH5T9seFI0OGuQJKvYgaSl4DjzGysmcWAq4GHOuzzEFPYkwwAACAASURBVL4zHeBy4G/OOWdmA4A/Azc7555L7mxmETMbEtyPAucDy4v4HjKSdY0k+cHtWCOp7O+3FTKQOAfL7/f33/xH+XxZJydfjj0dZnwAWhphyW9KWyY59GxZmltHe9LIaf4c5dbse4gpWiAJ+jw+DjwKrALuc86tMLOvmNmFwW53AoPNbA3wb0ByiPDHgXHAFzsM860AHjWzpcBifI3mjmK9h0wlayQZ5yTZvBgq+qXP5DZ2jm/aajhQmMJtWeaHGs/6ELgEvPpwYc6br7VPQfUwvz7SsPFw5Mm+eUtDgSVTDfv9enW59I8kjZzu17vbuaZw5eqFitpH4px72Dl3vHPuWOfc14JtX3TOPRTcr3fOXeGcG+ecm+2cWxts/6pzrjpliO8059w251yNc26mc26qc26Sc+5TzrmSX0pUV/hAkvHs9nQd7Ulj50Ci2Xe6F8Ly+yEUgTM/DwOOhlV/LMx58+Fc0D9yOiTHSsy8zv8zv/Fcl4eKtNoSNEbkMmIrSR3uBVHWne2HinjUN21llJOkudH/A4yclv75I0/2S1wXYhiwc7D8975Du3owTLjA1wTq9+V/7nxsWwk12+GYM9q2TbrYN+2p010ylUzBkE+NZMjxfr079ZPkRYGkAJJNWxnlJNm+CloaDu4fSYrFYfTswvSTbHgJ9r4JUy73jydc4PsiVj+W/7nzkdo/khStgqlXw8o/+ImKIt3ZvASqh0LfEbmfIxT2NRoFkrwokBRALBIiGrbMaiSbOsxoT2fsHP9PUrc7v4Itmw+RSjjh3f7x6Nm+X6LUzVvrnoZBx8KAI9tvn5nsdP9tacoluavb42vbPSm5dHy+U8nU4Z43BZICyXgF4E2LfBPOwLGd7zN2DuBg/bO5FyjRAisegOPOgcp+flsoBOPPg9WPQ1N97ufOR0uTf1/HnH7wc8MnweiT1Ol+qEm0wI/eAY9/sedes7nB1+7zadZKGjndr3u347X8z9VLKZAUSMY5STYt8vNHurqKGjXTt9vm07y1/hmo2QaTL2u/fcIFflmStU/mfu58bHzZj5I55oz0z8+8Dna8WrjBBlJ8Gxf6ZUqW3ecvFHrCtpV+UEo+Q3+T1OGeNwWSAonHwtR2lyWxuQG2rui6WQt8KtCj35FfIFl+v1/L6/h3td8+5jSo6A+r/pT7ufOx9inAfDnSmXSJHxqtTvdDx+rH/W3tzp5bK25zATrakwaPg2h1efeTNNXDo/8X9nac010eFEgKJB6LdD+zfdtKSDR1PmIr1dg5fgb8/q3ZF6a5EVY+5JuxolXtn4vE4IR58OqfS5P7ZN3T/iqys5nIsWqYeqVvllOn+6FhzeP+C72if8+tnrBlqb/gGDAm/3OFwr785RxIXvkTPP8DeKnk0+bSUiApkHgs3P3M9s5mtKczNrmsfA61ktf/BvV7YPLl6Z+fcIHvyO/pORuNNfDWi503ayXNvM6PbFt6Xw8USvJyYLv/XJ9wHkw43w/kKMaiox1tXuKXN0k3FysXI6f7ybvlmlguuTrFigfKsv9QgaRAMgski6FqoJ8Y2J0RU32nfC5NBcvn+9c55oz0zx871y9W90oPN2+98byvkY1N09GeasQUGDlDne6Hgtf/6m+POwsmXerzoK/5a3FfM9Him4gL0ayVNHIaNNf5/rlyU7fbNx/2GwW715dl8jsFkgKJV0S6z0eSSUd7Uijs+xGyrZE01sIrD8PEi3wzVjqxOIyb6/tJejIPyNon/WTLo97e/b4zr/Ojcgq9ErIU1urHIT4EjpjuR+JVDYIVvy/ua+5c40dZ5TOjvaNy7nBf9Ud/AXbB98DCsPLBUpfoIAokBRKPdlMjaar3fSSZNGsljT0d9rzhr0Iy9dpf/KisjqO1OppwAezfBJtezvzc+Vr3NBz5Nh/IujP5Mj9YQJ3u5SvR4msk4+b6JqZwFCZe6C9kGmuL97rJK/JCjNhKGnQsxPqWZz/Jsvl+Xb5xc32T94oHy66mrkBSINUVka4DydYVfrhiVoEk2U+SRdKn5fdDnxFw9Cld73f8u/waXD01ObFmh2+DTjd/JJ2KPjDlCr/ES92e4pZNcrPxZd/sMu7stm2TL/MXMqsfLd7rbl7iJ9oOOaFw5wyFcu9w37+leM1N+7f6ofyTL/ctGZMuht3r2paHKRMKJAVSFQt33bS1OYuO9qShJ/iZ6Jn2k9Tv9U0Nky7xTWNdqRroA9WqP/bM1U2yiW7sGZkfM/M632697HfFKJHka83jgMGx72zbdvQp0Ge4vwAoli1LYdhEn0+kkEZOg63LM58LU7cbnvgSfHca3PHO4gzNXfGAX7U7uczR+At889aK8mreUiApkOpYmKYWR1NLJ30OmxZBfDD0H535Sc38l/26v2f2Zf/Kn/1opymdjNbqaPz5Pl/19lcyL1Ou1j3th2tmE0hHTvN9Sgt+VnZVeQHWPOEnz1anZMcOhWHixX49t2IsDuqcv/ovZLNW0sjpPuFcd/8PjbXwzLfhuyfCs7fC8ef4Zr5iNMMunw/Dp/iLSvC/67Gn+X6SMvqfUCApkKruklttWuw/qNmuCzR2DhzYmtnyDcvm+xFho2Zmdu7x5wHWM81ba5+CMadmfxU58wOwbYWfPS3lo2aHb9o67uyDn5t8mf9CfvWRwr/unjd9zbuQI7aSuutwb2mCl34K35sOf/2yHzTy0Wfhyl/C8fN8KutCDn3evd4PNpnSob9z4sU+u+iWZYV7rTwpkBRIPJluN13zVlMdbFuV3dV4UrJPobvRWzU7/Jf15MsyD1Z9R/jO71UdE1cW2O71/ueYM7I/dvLlftaxOt3Ly+t/A1z7/pGk0SdB/yOLM3or2TcwogiBZOBYX2vu2E+SSPiLtB+cBH/+dxg0Fj70KLznXhgx2e8z+wafGmFlAf+XknNH0i1zVGajtxRICqQtkKSpkWxZ7nOmd8zRnomBY2DAUd33k6x80L9Gd6O1Oppwvr+yyWZkWLbSLRufqcp+/ops+f2lz6MibVY/7ptq010chUK+U3jNXwu/OsHmJf5LdPjEwp4XDu5wd86/z5/Mgfuv96suvOd38MFH4KiT2x97zJl+5FchZ54vu99f6A04qv326iG+dl9Go7cUSAoknmzaakgTSLKZ0Z7O2Dl+5FZXy1wv/70fxTJ8UnbnHn++vy3m2lvrnvYjyYbmOMpm5nV+3oA63bO3YzW8eEdhl3hPJPyw32Pndj6zfPJlfu5DoSe9bl7qP0cdl/4plJHT/QjL9c/Cz8+Dey6Hxv1w6U/hI8/4/pB0Nf5QyNdK3nqhMHNRtq3yTbqdrU4x6WLfv7l1ef6vVQAKJAVS3VXT1qZFfvRVv5G5nXzs6X7Jk87aRPduhDf+4TvZs+2DGTTWd+YVq58kkfA1ktS0utkaOcPPdl+oTveMbVwI977XN8c8/GlY8pvCnXvTIr9AY7r+kaQjpvm5D4UevbVlaWEnInY0cpofsPLz83wQfvc34WMvwdQrul+O5cRrfDNsIWoly+aDBTW7dCZc6J8vk9FbCiQFUtVV09bmxf4DmusXaXfrbq14AHDZN2slTbjAX0nlskBkd7athNoduTVrJZn5WsmWZeU5YaxcOOf7Ln5xgR+Ouu7vcNq/+6GyL95euCDcOux3buf7mPklU9Y97dfjKoQD22D/5uKM2Eoae7pPd/3OL8CnFvtaRmcrRHRUNcAvOLpsfn5Nes750VpjT4c+w9LvUz3Er3xRJmtvKZAUSHVFJ6O2Gmv8cMJcm7XAd4oPOaHzQLJ8vr8CHHxsbuefcAHg/IrAhbb2KX+b6UTEzky5wudoKXanu3OwY40fnfPPH8HLd/ur6tce87W+zUtg5+s+6DYcKIt/4tYkZrefDndf4q+kz/kq/J8VMPcL8LaP+iaQN58vzOutfhxGzWg/7DedyZf5ORCF6hQu5NLxnakeAtc/CnM+7ftEsjX7Bj9ibdGvci/Dxpd9n2V3w/jLqHmrwDN6eq+qqK+R1HRs2tqyzP8z5RNIwNdKFv/aD0EMR9u273zdX6Wf/V+5n3vYBN8MsepPMOtD+ZWzo3VP+3wP2cyfSaeyv7/CXX4/vOtrUNG3MOUDaNjvg/Sav/q5EXveyOJg8184sT4+4L/3991/wRZKU71vsvrH9/xw0MHj4MLvw9SrIFLRtt+UK3z2whd+4vPc5KNmp282O/0/u993+EQYOt4Hudk35Pe60DYEfMSU/M9VLMMn+UmZL/0U3v6x7icGp7N8vl+TLtl/2ZnxF/hRZCseLPnvRIGkQJI1koPS7SY73nIZsZVq7Bzf9rpxYfsRI8k26MmX5n5uM18ref42vxxJ1YD8yprU3Ajrn4Np1xTmfDOvg8W/8sFk5nW5n8c5fxW35gkfPN78p+8YjvXxzQmnfMrP1q7s77M5Ntb4n4b9wf0D/qfhQNvjut3+S33ZfXDyTYV5v52p3wcL7oJ//tDPMRo53c9lGH9++i+uWBxmvA+e/6HvT+s/KvfXTg777ap/JNXky+DJ/4Z9m3LvIwRf7udv8805lf1zP09PmH0D/O46X3M7YV52xyZa/P/0ced0/3/YZ6gfvbXyQXjn5/PPXZ8HBZICSQ7/PahGsmmRH7HU74j8XmDMqYD5K+d2geR+PzEq3yv+CRfCc9+F1x6FE6/K71xJGxf6dZfy6R9JNXoWDJsEL/7U16Ailf7KLVLpr8Bbb4P7qV+qtbv86sNr/up/Dmzx24dP8VeO487yQy07tod3loArna0rYMlvixtInvsu/P1bfrn2Y86ES2/3v9/uvkRO+jD84wc+AM39Qu6vv+Zxv8JvpjXsSZfCk1/zV81v/5fcXtM5eOgTwQq4383tHD1p/PnQ9wh/4ZdtIHnjOf/ZzLS/c+LF8Od/85+95JyWElAgKZCKSIiQpauRLMq/WQv8F9oRU30gOf0//LatK/xS6+/+Zv7nHznDf/hf+WPhAkkyre7YTtLqZssMTvqQr87/4oIM9g+3BZf6Pb6JsXKAr22MO8vf5hvgU514NTz6Odj+au5Dnbuy/TXfRHXsO2HuF7P7XA0cAyec6/uY5nwGopXZv34i4YPwuLmZN9kMGedHWS2/P/dAsuAuP9z4vG/l3g/Yk8JRmPlBeOq/fdNzNmVeNj9IkZ1hAJpwoR+Vt/JBBZLDgZkRj0WoSZ1H0nDAL22S62iqjsbO8e3cTXV+HP3y+/2X5cROhghmIxTyV1KLfuXXEspkqffurHvaj1arGpj/uZJmfsgHvcYaP0yzucF3bjY3BrfB45YOj6uH+lFGo2bk1m6dicmXwWOfh6X3+i/6Qlt0t1+x+ZKfdD6apyuzb4RXH/Z9Frk0N25e5EfgpZvN3pXJl8ETt/gO5IFjsjt211p47Au+9jXr+uyOLaWZ18Hf/9f3lcz7f5kd09wIK/8AJ7w78/+/PkN9n8yKB+HM/1uy5i0FkgKKx8LUNaU0bW1ZCrjMcrRnYuzp8I/v+zb9Y87wgeSY0/2HqRAmnO+r46//NRjJlYeGA36doHd8ojBlSwqFfDAoR31H+C+8pb+DMz9fuDSw4AdZLPmNv1LNJYiA/8wMOcEPBc4lkKx+AjBfI8nGpEt8IFnxAJz6fzI/LtECD/6LD54X/aCkfQBZ6zvcJ5dbdI/vv8hkBFgyRXami64mTbrY19K3rcx+QnKBaPhvAcVj4fY1kuSch3w72pOOerv/p1r397YhgoWq7YC/sqkaWJhZ7m/8w+dfKVT/yKFi6lWw983CDbVNeu1Rv5bT9Pfmfg4z3xG86WXYsCD749c87pvTqodkd9zAo/36W8m1ozL1/G3+93juN/LvAyyF2Tf4vqyl92W2f2uK7DOze50ymJyoQNIZ5/wHOYuZufFYh+RWmxb5PMt9hxemTBV9YNQsH0gyHSKYjXDUV6tfeyTznAydWfc0hCsOXpPocDfhfD+7eem9hT3vorv9oI1sm5U6OvFqnwnwhZ9kd1ztLh98Mh2t1dHky/xQ+B2rM9t/2yr423/5z/eJV+f2mqV25Nv8sNwX7+h+vlEmKbI702eYvwgs4dLyCiSdSbT49sqHPuEnqGUg3jG51abFhauNJI2d468ol/3Of6kUaqhu0vjz/TLd67PIypjO2qfgqLcVb02kchWr9s2CKx708zwKYd9mn99j2jX5J3Oq6AvTr/XNTNmsZNDVar+ZmHgxYJldmLU0wQMf8WU9/9ZDq0krlZnvl9q2ovsa6muPBCmys2zWSpp0se+P3bYqt+PzpEDSmXAELv+Zv+q/7/2+g7sb8dR0u/X7YOfqwozYSjV2jh99VLP94DwFhXDsmf6KOp+1tw5s9/M0eluzVtLUK32TRqHSzS75tf+bT39fYc530g1+KG02qwSsDob95to/1e8If9W8fH73V83PfMuvIHD+rYXr/yuVyZf7kYIv3t71fsvu96Mmc50wmmzeKtHS8gokXek/Ci69w19RPPyZbnePR1NqJMkczoUOJKNP8kNao/HMhwhmI1rlmy9e+bMf7pmL5JL32bb1Hi6OOcOnm11SgOYt5/xIuqNPKdzQ1yHj/PDnBXdltipwIuEnbx77zvxGvE2+1F81b13R+T6bFvnRTlOvgokX5v5a5SIW9/1aq/7oa5bp1O3x/U+TLs3995ts3irR2lsKJN057iw47dO+jXpx1yuoxivCbTWSzcGM9kKN2EqKVsK0a32VOZe1gDIx4QI/Y3rDS7kdv+5pqOhf+Pd+qAiF/bIkqx/LPx/HG8/5IbCFqo0kzf6In/iWSVKzzYv9sN9c+0eSJl4U5BvvpHmrqR4e+KhfKfvcb+T3WuXkpOu7TsW76o9+uHq+LQwTLypZ85YCSSbO+CwcfaqfQdrFH8n3kQSBZNMinyUu2xEumTj/23D2lwt/3qTjzvFNerlmTlz7lJ+EWKz5GoeCqVf55qN8swS+fLfP2jfxosKUK2ncWX51gBczWPJ8zRP+tqvVfjNRPcQPV19+f/qr5ie/6hc4vej7hZ17VGqDjvFBeOHP0tcAl8/32RlH5jmsfcKFgJWkeUuBJBPhCFx+p68B3PcBP0cijepYpK1pa9OiQ/eKvLKf799Y9VD2V9S71vm82r21fyRpxBQYOiG/5q36vX7Ax+TLCjNBNFUo5PtK3vpnWzNsZ1YHw34L0V8x+TI/bH3Ty+23v/EPv4TLrA/5IHe4mX2jr+V3vDjbv9WPwswll1BHfYe3TU7sYQokmeo7Ai6701cd//xvaa+oqmJh6psStNTu9s0Rhe4f6Ukz3u8DwrcnwB8+3nlSrY5al40/o0gFO0SY+aVmNrzoPwu5WDYfmuv8govFMO09fmDFC110BNfugo0L8h92nDT+PAhF24/eajgAD97k55vks4p1OTt2rq91vPTT9ttXPugHUuQ6WqujSRfDjld7vHlLgSQbx5zum7mW3gsv//Kgp6uDdLuNbxV4ImIpTLwQbnreZ31bNh9+fCr87Dx/hdySJgtk0rqn/eiTIcf1XFnL1ZQrAMt8QlpHi+72i1Tm2+TRmaoBPtgt+51fHj6d1//mv+jy7R9pfc2Bvsax4oG2wRyPfwF2vwEX/8jPlTochUJ+4cw3n2/LqwL+f2v4ZBg2vjCvk2ze6uFaiQJJtuZ82o9Gevgz7T8QtGVJbNkQVNsP5RoJ+HwSF9wK/7bSXynuedMPhf7uifDsdw5u9kokfDU9k9Voe4P+o/2qzUvvzX4kzZblvnl0xvuK+7ucfaNfs+zlX6R/fs0T/st/1MzCvebky2DfRp+Vc80TfvTYOz6ef66Ucjf9WohUtaXi3f2Gr7EWcnWKZPNWD/eTKJBkKxT2Q4Ljg+B3H/DzRQLVFT6Q2ObFMODo7JYgL2fxQXDKJ33q0at/DYOPgSe+5Ju9HvqE/9IDP3ekdqeatVKdeLVv2sp2SZJFv/IDHqYWaCXmzgybEOS6ufPgmmahhv12dMI8P4R9wV3wh0/45Fdnfr5w5y9XVQP9HKOlv/P5a5JLxhQykIBv3tr+Cmx7pbDn7YICSS76DIXL7/JXFA99ovVqsyrqm7aiW5ccuh3tXQmFfRv3B/4YNHtd7f8pfnyKb/Z67la/X75pdQ8nEy70X5pLf5v5Mc0Nfv/x5/XMxcjsj8C+DX52daotS/zE10L1jyRV9IXj3+WTgB3YCpf8OLdl7Q9Fs2/w/V6L7vGBZPRs3zdUSCUYvaVAkquj3+ETBK18sLUDrboiTH8OENv/5qHfrNWd4RN9kqF/Wwlnf8U3ey2/H4Ycn18mvMNNZT+/ftny32c2+Q/8ZNC63YWfO9KZE86F/kcdvP7W6mDYb7ar/WYi2bk85zOH//9KqhFT/OKrz3zL1+CzXek3E32H+++nHuwnUSDJxzs+Bce9C/7yWdi4kHgszJTQOv9cb/nniA/yqWk/tRiuuddn7JP2pl4Fdbva5mN0Z9Hdfg5ST60MEAr7SXPrn4GtK9u2r3ncDxjJddn6row/3+e3TyZp601m3+A/DxbyS+wXw8SLfdK7HmreUiDJRyjkq+V9hsPvrqOPO8AU84Fkc7wIGfLKWSjs2757SwDNxri5EB+c2YrAe96E15/0qxcUMp9Jd2a83zfBJdeEqt3lVzYo1GitjkKh7DItHk7GX+BHNh5zZnGCNATLy/Rc85YCSb7ig+CKn8O+TRzz3H9wcuV61ieG8/ZbX+ac7zzNfz+8iufW7KChuaXbU8lhKhz1HaqvPuLXVerK4l/72+nXFr9cqeKDfDPL0nt9s9raJ/2w30L3j4hfJv76x4pbe+87wjeh9VDzljIkFsKRJ8HZ/0X00c8yx0LsP/58PnfUeJ5+bTs/e24dt/99LfFYmHccO4TTTxjKGccP5chBmc1Ubm5JsHlvPW/uqm33UxEJMfGIfkwIfgZVZ5nDQHrW1Kv91f6qh/zVfzqJhO+EPeYMGHBUT5bOm/0RP1ps0T1+YcXKATB6Vs+Xozfoib/vpEv8sjMHthWv5hNQICmUk2+CN57DXvkT/caexI2nHMuNc46lpqGZ51/fyVOvbeOpV7fzxCqfA+LYodWcfvwwzjhhKJNG9jsoWLwV3G7cXUdzom0OQiRkjBpYRV1jC79/eWPr9hH9KplwRN/WwDLhiH6MHVJNOKT5HGVh1AwYdKxfMqWzQLLuKZ9d8ewv9WTJ2hwx1V/FvnSHT7RU6GG/0rOmv9fnjs82UVYOFEgKxQwuus1H/kkXt26urohw1sThnDVxOM451u6o4elXt/PUa9v51QtvcNdz6w461cB4lKMGxZkyqj/nTTmCowfHOXJQnKMGxTmif1VrcNhxoIFVm/cFP/tZtXkfz6ze0Rp4KqMhThjRj4lH9OWE4X3pUxklZL6oITPMjFBw358y5XGI4Pm2bcnjwiELznPw8xWRMP0qI/SpjFAVDWN5Tqarb2phT20Tu2sb2V3byJ7aJuoaWxjUJ8awvhUM7VvB4OqK8g+YZn649JNf8/0g6a5IX77bzzUoZNbLbM2+EeZ/0N8vVv+I9IxCr8/WBXMlSs3Yk2bNmuUWLMghR3WR1TW28M91O3l92wFGD6ziyEE+YPSrjOZ8zobmFlZvPdAuuKzaso89tXmmzs1BOGT0qYjQtzJC38oofVvv+0DTtzJKnwq/0OXu2ib21Dayu6aJPXXB/dpG6pu6z4kSDhmDq2MM7VvRGlyG9a1kWL8KhvapYFi/CuKxCM0tjqZEgpaEo6klQXOLozmRoKnFHbStOeGIhkPEY2GqomGqgtt4LEI8FqYyGm69zTiI7VoH35sGc78Ip/17++dqd8G3ToBZH8LN+zp1TS0caGimpqGFmoZm/9PoH1dEQgyIxxgQjzKgKkr/eJSKSIFqDi1NcOtU2L8JPr26XZNIXWNLazD3f5+m4LHfVhkNt/sbJH/iMV2vHqrMbKFzrtv2TQWSXsA5x/b9DTQ0J0g4R8JBwjmc888lH7dt849b0u2TOHj/5DlbEo6G5hb21zezv76ZAw1N/ra+mX31zeyvb+JAQ/I5/7ipxREOGQOqogyIRxmY/IKMxxjYepvc5p+viobZWdPA9v0NbNsf3O5rYNv+erYf8Pd3HGgg0UMf7YpIqDXQhEO+xhY2IxQyIiFrq8WFjK/v+TR9Ewf49LCfEA6HaEk4ahpaOGf/A3yi8adc6v6XRY2jsl5RpSoaZkA8Sv8q/+ODjP+99a2M4Bw0J/zfrCXR9pNu24m7H2NU7Upuj3+kNXDsrm2kobnzoF4VDdPQ3JL2d96nIuKDSp/2AWZInxjhlJFpyXCcWolN3rfg2YpIqN2FQmU0vwDa0NzClr31bNxTx6Y99WzaU8e2/fVUxyIMrG77DA5KuT+gKkokXJ7jlJxz1Dcl2FffxP76JvbWNTPz6NyX5M80kOhSoRcwM4b1K7+Zw845GpoTVERCWTeBjRnSdVKvloRjV01jEGzqqWtsIRIOEQkb0VCIcMiIhs1vCxnR4LlIqG1bY3OCuqYW6hpbqG1sob7J3/ptzSn3225bUr6UO35Btzh4vuksPrTnexzduIY1kXGEzRhSHeWS/X/jrarxTJt4KqdWhIlXRKiuiNCnIkx1zN+vrohQHawwvbeuiT11/kt+b1CDa71f18T6HbXsqdvD7tomGlMCQMgImiaDIBcEvkjKtudDM6mInMTAlgSjB8aZMirKwOpYW6CvCgJ8dVvgr4iE2/3OfUD3gT014K/avI+nX2vgQEMXC39moW9lhGEpgSX1fjLg1De1BIEi+eMDx8Y9dWzf33DQOQfEo9Q2trT7vXXUrzIS/E5iDIpHiVdESCRcULv1NdrU2q2/72huCWrEiQRhMyqjYSqiYSojISqjYSqjwW2k7X5FsD0WDlHX2BIEiWb21Texry5527atqaV9NH/lv+blHXC7oxqJSE9KNmGd9GGY9//8to0vwx1nwnnf8tsLrLE50RpA8u2zKpTaxmZ21TS21rxab2n7Pmrb1qauEXRucQAACDVJREFUsYVt++tTaqL+/rbggmHbvoYua04VkRCjBlQxckBV6+3IAZWt90f0r6QyGsY5R11TC7tq2mpkqfd317Q17e2ubaS2oYVI2AiHQkTDPjgnL1iSFyipz0VCRovzfYD1TS00NCWob24JHidat9c3Jw4KaFXRMH0rI/SritKv9Taasi1KvyrfdNyvMsIp44YQzbEGpRqJSDmKD/IZKJfN9ysqhyN+JnuksnA5KTqIRcqvGcb3NeX29TORfp0+55xjX30z24Ogsv1AAxWRcBAoKhlUHcsomJpZaxlHlzhZYyLhaGxJ0NCUoCoWLsu/pwKJSE+behW88iefBOzod/igMvEinx9E8mJmrf1E44b1LXVxCiIUMipD4aI3T+VDgUSkpx3/Lqjs72eR1+6Ahn09t0CjSBEokIj0tEiFn3W89D7Y9bpPwTrm1FKXSiRn5dfYJtIbTL0ammph40I/A7lMOsFFclHUQGJm88zsVTNbY2Y3p3m+wszuDZ5/wczGBNvPNrOFZrYsuH1nyjEzg+1rzOx7Vi7DUESycdTJfna7hWDae0pdGpG8FC2QmFkYuA04F5gIXGNmEzvsdj2w2zk3DvgO8I1g+w7gAufcFOADwN0px/wIuAE4LviZV6z3IFI0ZnDOV2HuLUoEJoe8YvaRzAbWOOfWApjZb4GLgJTMOVwEfCm4Px/4gZmZc25Ryj4rgCozqwAGAf2cc/8MzvlL4GKgQ45QkUPAxItKXQKRgihm09Yo4K2UxxuCbWn3cc41A3uBwR32uQx42TnXEOy/oZtziohIDyrrUVtmNgnf3HVODsfeCNwIcNRRJcjtICLSSxSzRrIRODLl8ehgW9p9zCwC9Ad2Bo9HAw8A73fOvZ6y/+huzgmAc+5259ws59ysoUOH5vlWRESkM8UMJC8Bx5nZWDOLAVcDD3XY5yF8ZzrA5cDfnHPOzAYAfwZuds49l9zZObcZ2GdmJwejtd4P/KGI70FERLpRtEAS9Hl8HHgUWAXc55xbYWZfMbMLg93uBAab2Rrg34DkEOGPA+OAL5rZ4uAnmRjhX4CfAmuA11FHu4hISWn1XxERSSvT1X81s11ERPKiQCIiInlRIBERkbwokIiISF4USEREJC8KJCIikhcFEhERyUuvmEdiZtuBN3I8fAh+WfveqDe/d+jd7783v3fo3e8/9b0f7Zzrdo2pXhFI8mFmCzKZkHM46s3vHXr3++/N7x169/vP5b2raUtERPKiQCIiInlRIOne7aUuQAn15vcOvfv99+b3Dr37/Wf93tVHIiIieVGNRERE8qJA0gkzm2dmr5rZGjO7ufsjDi9mtt7MlgW5YA77NfjN7C4z22Zmy1O2DTKzx81sdXA7sJRlLJZO3vuXzGxjSj6gd5eyjMViZkea2ZNmttLMVpjZp4Lth/3fvov3nvXfXk1baZhZGHgNOBvYgM/2eI1zbmVJC9aDzGw9MMs51yvG0pvZHOAA8Evn3ORg2/8Au5xzXw8uJgY65/6zlOUshk7e+5eAA865b5aybMVmZkcARzjnXjazvsBC4GLgOg7zv30X7/1Ksvzbq0aS3mxgjXNurXOuEfgtcFGJyyRF5Jz7O7Crw+aLgF8E93+B/yc77HTy3nsF59xm59zLwf39+Gyuo+gFf/su3nvWFEjSGwW8lfJ4Azn+gg9hDnjMzBaa2Y2lLkyJDHfObQ7ubwGGl7IwJfBxM1saNH0ddk07HZnZGGA68AK97G/f4b1Dln97BRLpzKnOuRnAucDHguaPXsv5NuDe1A78I+BYYBqwGfhWaYtTXGbWB7gf+Ffn3L7U5w73v32a9571316BJL2NwJEpj0cH23oN59zG4HYb8AC+ua+32Rq0Iyfbk7eVuDw9xjm31TnX4pxLAHdwGP/9zSyK/yK9xzn3+2Bzr/jbp3vvufztFUjSewk4zszGmlkMuBp4qMRl6jFmVh10vmFm1cA5wPKujzosPQR8ILj/AeAPJSxLj0p+iQYu4TD9+5uZAXcCq5xz30556rD/23f23nP522vUVieCIW+3AmHgLufc10pcpB5jZsfgayEAEeDXh/v7N7PfAGfgVz7dCtwCPAjcBxyFXz36SufcYdcp3cl7PwPftOGA9cBHUvoMDhtmdirwDLAMSASbP4fvKzis//ZdvPdryPJvr0AiIiJ5UdOWiIjkRYFERETyokAiIiJ5USAREZG8KJCIiEheFEhEypiZnWFmfyp1OUS6okAiIiJ5USARKQAze6+ZvRjkb/iJmYXN7ICZfSfI9fBXMxsa7DvNzP4ZLIr3QHJRPDMbZ2ZPmNkSM3vZzI4NTt/HzOab2Stmdk8wI1mkbCiQiOTJzCYAVwGnOOemAS3AtUA1sMA5Nwl4Gj9jHOCXwH8656biZxUnt98D3OacOxF4B37BPPCrsv4rMBE4Bjil6G9KJAuRUhdA5DAwF5gJvBRUFqrwi/wlgHuDfX4F/N7M+gMDnHNPB9t/AfwuWNtslHPuAQDnXD1AcL4XnXMbgseLgTHAs8V/WyKZUSARyZ8Bv3DOfbbdRrMvdNgv1/WIGlLut6D/WykzatoSyd9fgcvNbBi05vs+Gv//dXmwz3uAZ51ze4HdZnZasP19wNNBhroNZnZxcI4KM4v36LsQyZGubETy5JxbaWafx2eUDAFNwMeAGmB28Nw2fD8K+GXJfxwEirXAB4Pt7wN+YmZfCc5xRQ++DZGcafVfkSIxswPOuT6lLodIsalpS0RE8qIaiYiI5EU1EhERyYsCiYiI5EWBRERE8qJAIiIieVEgERGRvCiQiIhIXv4/oJ304js69gsAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "fig = plt.figure(figsize=(6,6))\n",
        "\n",
        "# summarize history for loss\n",
        "plt.plot(history_nn.history['loss'], label=\"training\")\n",
        "plt.plot(history_nn.history['val_loss'], label=\"validation\")\n",
        "plt.title('model training')\n",
        "plt.ylabel('loss')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BlPaL7CtxqAw"
      },
      "source": [
        "### Mean-Absolute-Error"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ciSan15-_-5o",
        "outputId": "4eb1eb10-d459-4e4b-88b2-e689a02ff49e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n",
            "0.13058082382271863\n"
          ]
        }
      ],
      "source": [
        "y_nn_test_pred = model.predict(X_test_s)\n",
        "nn_mae = mean_absolute_error(y_nn_test_pred, y_test)\n",
        "print(nn_mae)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kx6p1z8Cq-EH",
        "outputId": "72f88e68-9bf6-41e8-ba83-9e8e576b4067"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Test R2 score:-0.3843\n"
          ]
        }
      ],
      "source": [
        "nn_r2_score = r2_score(y_test, y_nn_test_pred)\n",
        "print(f'Test R2 score:{nn_r2_score:.4f}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "GdMezpLErJWg",
        "outputId": "b1251f0f-6621-483d-fac1-a2d288892874"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'realizations')"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deXhU5dXAfydhgASUBEGRAIKooIiAIC6oVauiUpWqVbtqa2vrWmxFQa1gaxVLW7Vq9cOl1WqVipaqqLjRWq0gS0B2F0AgIrIFhAQySd7vjzsTJ5N779yZuTN3ZnJ+z5Mns9y599z1vOe8ZxFjDIqiKIriRFHQAiiKoii5jSoKRVEUxRVVFIqiKIorqigURVEUV1RRKIqiKK60CVoAv+nSpYvp3bt30GIoiqLkB3s2Qc165q9q3GyM6Wq3SMEpit69ezNv3rygxVAURcltdnwE7/8EvpgP+52MnDrrU6dFC05RKIqiKC401sPKe+CDX0FRWxj+MPS9DLeZCFUUiqIorYVtH8Ccy2DrPKg4B476M5RWJPyZKgpFUZRCp2EPLL3D+mtbDiOegV4Xgoinn6uiUBRFKWQ2z7asiO3LoPf34Mi7oX2XpFahikJRFKUQqd8Fi26Blfda7qWvzYCKs1JaVWCKQkR6Ak8A+wEGmGKMuTdumZOAfwGrIx89b4z5dTblVIJjemUVk2eu5LPqWrqXlTB2ZD9GD0nsT1WUVs/nb8Kcn8Cu1XDwFTB4EoT2Tnl1QVoU9cAvjTELRGQvYL6IvG6MWRa33H+NMd8IQD4lQKZXVjH++cXUhhsAqKquZfzziwFUWSiKE3XVUHk9fPIo7HUwnPof2PfEtFcbWGa2MWaDMWZB5PWXwHJAnwAKAJNnrmxSElFqww1MnrkyIIkUJcdZ/y+YcRis+gscegOcucgXJQE5UsJDRHoDQ4A5Nl8fKyKLROQVERng8PvLRWSeiMzbtGlTBiVVssVn1bVJfa4orZbajfDORfD2aGjXFU6fA0PugjYlvm0icEUhIh2B54AxxpgdcV8vAA4wxgwC7gOm263DGDPFGDPMGDOsa1fbDHQlz+heZn+RO32uKK0OY2D13ywrYv10OOJ2OGMe7DPM900FqihEJISlJJ4yxjwf/70xZocxZmfk9ctASESSi+tS8pKxI/tREipu9llJqJixI/sFJJGi5BC71sK/R8F7P4C9+8GZC+Hwm6EolJHNBRn1JMCjwHJjzB8dlukGbDTGGBEZjqXYtmRRTCUgohPWGvWkKDGYRvjoIVh4o/V66L1w8FVQVJz4t2kQZNTTCOD7wGIRWRj57CagF4Ax5iHgAuAKEakHaoGLjTb5bjWMHlKhikFRouz4EOb8GDb9F7qdCsOnQMc+Wdl0YIrCGPMO4Jo/boy5H7g/OxIpiqLkII31sOIP8MEEKC6Box+DAy/1XH7DDzQzW1EUJVfZtghm/wi2LYAe34SjHoCS/bMuhioKRVGUXKNhNyy5HZbdBe32geOnQa/zAxNHFYWiKEousel/VhG/HSugzyVw5B+hXedARVJFoSiKkguEd8Kim+DD+6G0J5z0KnQfGbRUgCoKRVGU4NnwGrx/Oez6FA65GgbdAaG9gpaqCVUUiqIoQVG3DRb8Alb91UqcO/W/sO/xQUvVAlUUiqIoQbDueZh7FezZBIeNh4G3QnH7oKWyRRWFoihKNqn9HOZdDeueg/LBcNLL0HlI0FK5oopCURQlGxgDqx+3XE31NdY8xKHXZ6w+k5+ooiggtCOcouQoO9fA+z+Fz1+DriNg+CPQqX/QUnlGFUWBoB3hFCUHMY3w4QOwaLz1fuh9cMiVIIF3eEiK/JJWcUQ7wilKjrF9BbxxIsy/FroeD6OWQr+r805JgFoUBYN2hFOUHKExDMsnw+LboE0HOOZx6PP9rBbx8xtVFAVC97ISqmyUgnaEU5QssnWBVX5j20LoeQEMux9K9gtaqrTJPxtIsUU7wilKgNTXwsLxMHO4Ff56wnNwwrMFoSRALYqCQTvCKUpAfPGOZUV8+SEc+EM48g/QtjxoqXxFFUUBoR3hFCWLhL+0rIiPHoAOveHk12D/04KWKiOoolAURUmWz1618iJq1sEh18Kg30KoY9BSZQxVFIqiKF7Zs8XKrF79BOzdH057B7oeF7RUGSewyWwR6Skis0RkmYgsFZGf2ywjIvInEflYRD4QkSODkFVRlFaOMbD2WZhxGKz5Owy4Bc5c2CqUBARrUdQDvzTGLBCRvYD5IvK6MWZZzDJnAgdH/o4GHoz8VxRFyQ61G2DulbB+OnQeas1FlA8KWqqsEpiiMMZsADZEXn8pIsuBCiBWUZwLPGGMMcBsESkTkf0jv1UURckcxsCqv1iupsY9MPgu6P8LKGp9Hvuc2GMR6Q0MAebEfVUBrIt5vz7ymSoKRVEyx87VVse5z9+ArifA0Y/A3ocELVVgBK4oRKQj8BwwxhizI8V1XA5cDtCrVy8fpVMUpVXR2GD1rF50E0gxHPVnOOineVmfyU8CVRQiEsJSEk8ZY563WaQK6Bnzvkfks2YYY6YAUwCGDRtmMiCqoiiFzvZlMOfHsPk92P9MGP5/0KFn4t+1AoKMehLgUWC5MeaPDou9APwgEv10DLBd5ycURfGVhjpYcju8MsTKrj72SThphiqJGIK0KEYA3wcWi8jCyGc3Ab0AjDEPAS8DZwEfAzXADwOQU1GUQmXLPKv8RvUH0OsiGPYnaL9v0FLlHEFGPb0DuNbdjUQ7XZUdiRRFaTXU18LiCbDiD9B+PzhxOvQ4N2ipcpbAJ7MVRVGyysb/WHMROz+Gvj+GIZOhbVnQUuU0qigURWkdhHdA5Y3w8UPQ8UA45Q3o9vWgpcoLVFEoilL4VL0Mc38KtZ9ZSXNH/NrqPqd4QhWFoiiFy+7NsGAMrHkKOh0Gx0+DLloFKFlUUSiKUngYA59OhfnXQF01HD4BBoyH4nZBS5aXqKJQFKWwqKmyivhVvQCdh8HXH4OygUFLldeoolAUpTAwBj55BCqvh8Y6GPJ76PfzVlnEz2/0CCqKkv98+Qm8/xPYOAv2PQmOfhj2OihoqQoGVRSKouQvjQ2w8l744BYoCln1mfr+uNUX8fMbVRSKouQn1Uus8htb3ofu34DhD0Jpj6ClKkhUUSiKkl801MGyO2HpbyHUCY77OxxwMYhrRSAlDVRRKIqSP2x+37Iiti+BA74DQ++B9l2DlqrgUUWhKEruU18DH9wKK++G9vvDiS9Aj7ODlqrVoIpCUZTcZuOsSBG/VVa3ucF3QdtOQUvVqlBFoShKblK3HRbeAB9PgY594euzYL+TgpaqVaKKQlGU3GP9izD3Z7D7czj0ehh4G7QpDVqqVosqCkVRcofdm2D+tfDpM1bZjROnwz5HBS1Vq0cVhaIowWMMfPq0pSTCOywL4rBxUNw2aMkUVFEoecj0yiomz1zJZ9W1dC8rYezIfoweUhG0WEqq7FoHc6+Az2bAPkfD0Y9C2YCgpVJiUEWh5BXTK6sY//xiasMNAFRV1zL++cUAqizyDdMIHz8MlWPBNMCRd8Mh10BRcdCSKXEEWhBFRB4TkS9EZInD9yeJyHYRWRj5uzXbMiq5xeSZK5uURJTacAOTZ64MSCIlJXZ8BG+eYk1Y7zMcRi2G/mNUSeQoQVsUfwXuB55wWea/xphvZEccJdf5rLo2qc+VHKOxHlbcDYtvhaJ2cPQjcOCPtPxGjhOoojDGvC0ivYOUQckvupeVUGWjFLqXlQQgjZIU2z6wym9snQc9zoVhf4bS7kFLpXggH2rxHisii0TkFRGxneESkctFZJ6IzNu0aVO25VOyyNiR/SgJNXdPlISKGTuyX0ASKQlp2GOV33h1KOz6FEZMhRP+qUoijwja9ZSIBcABxpidInIWMB04OH4hY8wUYArAsGHDTHZFVLJJdMJao57yhM2zI0X8lkHv78PQu6HdPkFLpSRJTisKY8yOmNcvi8ifRaSLMWZzkHIpwTJ6SIUqhlynfhcsusVqKlTaA056GbqfGbRUSorktKIQkW7ARmOMEZHhWK6yLQGLpeQRmnMRAJ+/CXN+ArtWw8FXwuA7IbR30FIpaRCoohCRp4GTgC4ish6YAIQAjDEPARcAV4hIPVALXGyMUdeS4gnNucgyddVQeT188ijsdTCc+h/Y98SgpVJ8IOiop28n+P5+rPBZRUkat5wLVRQ+s246zLsSdn8Bh90Ih0+ANhqJVijktOtJUdJBcy6yQO1GmH8NrH0WygbB116EzkODlkrxGVUUSsGiORcZxBhY8yTMHwP1O+GI2+GwG6AoFLRkSgbIhzwKRUkJzbnIELvWwr9HwXs/gL37wZkL4fCbVUkUMGpRKAWL5lz4jGmEjx6ChTcCBob+yYpq0vpMBY8qCqWg0ZwLn9ix0gp53fRf6HYaDJ8CHXsHLZWSJVRRKEorxVOOSWM9LP89LJ4IxSVwzF+gzyVaxK+VoYpCUVohnnJMti2E2ZfBtgXQ45tw1ANQsn9QIisBopPZitIKce3r0bAbFt0Mrw6D2io4fhqc+LwqiVaMWhSK0gpxyiXpVrcAXrkKdqywXExH/hHadc6ydEquoYpCUVoh8TkmpUW1jO32BJfs8xLU94STXoXuIwOUUMkl1PWkKK2Q2ByTEzou4LVDruKSfV5iTedLYNQSVRJKMzxZFCLyO+B2rMJ8rwJHANcZY57MoGyKomSI0UMqCDVsgwXXM6rjTD4N9+Sd3s9x4ohvBi2akoN4dT2dboy5QUS+CawBzgPeBlRRKEo+svY5Rq29CvbaDIfdxAGH/4oDitsHLZWSo3hVFNHlRgHPGmO2i8ZRK0r+Ufs5zLsa1j0H5UPg5FehfHDQUmUc7UuSHl4VxUsisgLL9XSFiHQFdmdOLEVRfMUYWP04LPgF1NfAoDvh0F+2ivpM2pckfTxNZhtjxgHHAcOMMWFgF3BuJgVTFMUndq6BWWfA7B9CpwFw1iIYMK5VKAlIkDOieCKZ8Nj+QG8Rif3NEz7LoyiKX5hG+PABWDQeEBh2Pxx8BUjrCnbUviTp4zXq6W9AX2AhEFXNBlUUSo6gPug4ti+HOT+Gzf+D/UfC8P+DDgcELVUgaF+S9PFqUQwDDtN+1Uouoj7oGBrDsHwyLL4N2nSEYx6HPt9v1UX8xo7s1+z6AO1LkixebdAlQLdMCqIoqaI+6AhbF8DM4Vadph7nwqhlcOAPWrWSAGuwcOd5A6koK0GAirIS7jxvYOsbRKSBV4uiC7BMRN4H9kQ/NMack87GReQx4BvAF8aYw22+F+Be4CygBrjUGLMgnW0qhUer90HX18KSX1uWRLuucMLz0FMT52LRviTp4VVRTMzQ9v8K3I/zXMeZwMGRv6OBByP/FaWJVu2D/uIdmHMZfPkhHPgjOPL30LY8aKmUAsNreOx/gBXAXpG/5ZHP0sIY8zaw1WWRc4EnjMVsoExEtNax0oxW2Rs7/CXMvRreOAEa6+CU1+GYR1VJKBnBk6IQkQuB94FvARcCc0TkgkwKFqECWBfzfn3ks3j5LheReSIyb9OmTVkQS8klWp0P+rNXYMYA+OjP0O/ncNZi6HZq0FIpBYxX19PNwFHGmC8AIpnZbwDTMiVYMhhjpgBTAIYNG6aRWa2QVuGD3rMF5l8Ha/4Gex8Kp70LXY8NWiqlFeBVURRFlUSELWSnRHkV0DPmfY/IZ4rSejAG1k2zajTt2QqH/woG3AzF7YKWTGkleFUUr4rITODpyPuLgJczI1IzXgCuFpFnsCaxtxtjNmRhu4qSG9RugLlXwvrp0HkonPwalA8KWiqlleFJURhjxorI+cCIyEdTjDH/THfjIvI0cBLQRUTWAxOAUGSbD2Epo7OAj7HCY3+Y7jaVwiCaiV1VXUuxCA3GUFFIGdnGwKq/WEX8GvfA4N9B/+ugSJtSKtnH81VnjHkOeM7PjRtjvp3gewNc5ec2lfwnPhO7IVIwoGAysneugvd/Cp+/AfueCMMfhr0PCVoqpRXjOs8gIu9E/n8pIjti/r4UkR3ZEVFRmmOXiR0lrzOyGxtgxT0wYyBsngNHPQhfn6VKQgkcV4vCGHN85P9e2RFHURKTKOM6LzOyty+D2ZfBltnQ/Sw46iHo0DPx7xQlC3jNo/ibl8+U3GF6ZRUjJr1Fn3EzGDHpLaZXFk6wWKKM67zKyG6og8W/gVeGwM6P4Ngn4WsvqZJQcgqvIa4DYt9EelIM9V8cxQ+iPvyq6loMX/nuC0VZ2GViR8mrjOwt82DmUbD4Vuh5nlXEr893W30RPyX3SDRHMV5EvgSOiJ2fADYC/8qKhErSFHo11dhMbIDiyIM1bzKy62uh8gZ47WjYsxlO/BeMeBra7xu0ZIpiS6I5ijuBO0XkTmPM+CzJpKRJa6immreZ2Bv/YzUU2vkx9P0JDPkdtC0LWipbtBmUEsVrHsV4ESnHquLaPubztzMlmJI6rbqaaq4S3gGVN8LHD0HHA+GUN6HbKUFL5Yg2g1Ji8TqZ/WPgbWAmcFvk/8TMiaWkQ6usphpHTk3mV82wivh9MgX6/yJSxC93lQQUvvtSSQ6vCXc/B44CZhtjThaR/sAdmRNLSYfoiK+1ug1yZjS8ezMsGANrnoJOA+D4adAlP9qptAb3peIdr4pitzFmt4ggIu2MMStEpPUMT/OQvPXh+4DbaDgrx8QY+HQqzL8Gwtvh8Akw4CYobpv5bfuEui+VWLyGx64XkTJgOvC6iPwL+DRzYilK6gQ6Gq6pgrdHw/++DR36wBnz4YiJeaUkQN2XSnO8TmZHG/BOFJFZQCfglYxJpQRGIUS6uI2GM7Z/xsAnj0Dl9dAYhiG/h35joMg+3yPXKQT3ZSFcy7mCGJO4z4+IPArcZ4xZGPPZRGPMxAzKlhLDhg0z8+bNC1qMvGN6ZRUTX1hKdW242ecloeKcyk3wcvPHz1GAtR/nD63guflVLT5Pe/++/ATe/wlsnAX7ngRHPwx7HZT6+pS0cboGculazjVEZL4xZpjdd15dTyOBx0XkkpjPzklbMiUniN5U8UoCcivSxWvGuVNr1FkrNvkbydPYAMv/CC8PhK3zYfgU+PpbqiRyAI3a8hevk9lfACcDT4rIcKwoKK0zUCC4VWOF5r79IM35ZCap7Sbzr5u6EDtSmruoXgJzLoMt70PF2Val11IdqfpFuteZRm35i1eLQowx240xZwObgH9jzVMoBUCimyca6RJ0Dal0b36niJ2kInka6mDxbfDqkVbfiOOetkpwqJLwDT+uM1/OtdKEV0XxQvRFZF7iLmBNBuRR0iRRopnd9243T2yki9OI/rYXl/q/Izake/OnHcmz+X14dSgsngg9vwWjlkPvi7WIn8/44TbSqC1/8aQojDET4t6/aIzJ7dTSVkiikZjT9yf372pbjbW8NNRs8s9p5L6tJpwVqyLdm99p7iKhS6O+Bhb8El4/Fuq2wddehBFPQfsuKe6J4oYfbqOUz7Vii+schYi8Y4w5PlIxNjY8SrA6le6dUekUz0yvrOKX/1jU1BY0SqwP32mkNmvFJu48b2BCn7BT2CmQlWQ2P0I2k05E3DgrUsRvFRz0Mxg8Cdo297pqGKa/+JXs15qTTv0m0A53InIGcC9QDDxijJkU9/2lwGQgOly93xjzSCZkyWeilkK8kogSHYm5jdS83FRjR/ZjjJ8TwimQtZu/bjtUjoVPHoaOB8HX/w37fa3FYtksF5KqQso3RTZ2ZD/b0FZ1GwVHIouis9v3xpitqW5YRIqBB4DTgPXAXBF5wRizLG7RqcaYq1PdTmsgUdRSdCSW7kht9JAK21yLZNaRKll92K1/Eeb+DHZ/DoeOhYEToU2p7aKZLBcSu8+dSkLsqqsn3GANBrwqpJype5UEhZDsV2gkCo+dj+VysputM8CBaWx7OPCxMWYVgIg8A5wLxCsKJQFuo/nYkZgfI7WJ5wzI+mgvaw+73Ztg/rXw6TNQNtCKZtrHNv+oiUT+9HSsgNh9dstxcVuf34osWwo7Gcsx3yymfCSR66lPBrddAayLeb8esCuteb6InAh8CFxnjFlns0yrxslSKBZpNoHnl48/3XUke2NnvMifMbDm77Dg51bfiIG/hsNu9FSfKVG5kGQVXPTYOM0FxZPI5ed1YjiVjPdcsE5yUaZkSEXJBaEYvSbcEVDjoheBp40xe0Tkp8DjQItoKxG5HLgcoFevXhkWKfdwshTsojz88PGns45UbuyMJk/tWgdzr4DPZsA+R8PRj0LZgMS/i+BmpSWr4OzKTiQikcvPi7vR6zkJvCqvDbkok1dSHUgEoRiDbFxUBfSMed+DryatATDGbDHG7Im8fQQYarciY8wUY8wwY8ywrl27pilW/pFPoYCpxMj7lTwVm0Ny/KQ3WPjWJJgxgPoNb3HvtivpO+smRjy0kVumL/bc9Mjt2Cer4BLNNcXjxeXnJaTY6znJxWznZGXKpYZWqdwLQZUmCbJx0VzgYBHpg6UgLga+E7uAiOxvjNkQeXsOsDzNbRYs2YgG8sPkTeVh48fcSuxIrHfbKiZ1vo/Bny/hk6Kj+dlHP+Wjmn0Ba4T25Oy1Tb/zMmJzOvbJBg8keuCGioSO7dtQXRN2dQ/Fn6NEoc9ez0mnklAggQxuJHOM3Ubj4OxOjT+mJ/fvyqwVm9J2/aRyLwSlrANrXGSMqReRq7Gsk2LgMWPMUhH5NTDPGPMCcK2InAPUA1uBS9PZppI6fpm8qUReeZkXSaTEJs9cSV24jsu7TucX+z1FnQkxdt21PF99Og0JCignchc5bTdZBeeWp1Lh4YHkdI7uPG8g745zzo/16p7aVVffYplQkQQatmp3jAVr30dMeqvZMXMajU98YSl76hsdFUj8MXUaSES34VWBpHIvBNVQymuZ8X8CPwTGYM0RbANCxpizMipdCmiZ8cwwYtJbthdoRVmJ60MonkyUf/ayzjMn3s+kHn9iUOlHvLb9GG6puoIv6vfxvA0BVk8alfR2k7HC0j02qZ4jt+0CrpPr5aUhKm89PaFsmSQ2AEBonhkce/z6jJtB4qfdV1REHr5eAgvKS0PsDjcmde5SOd+ZLJ/uVmY8ncZFr6YlleI7mYyG8Ct6xg/rIB7XCc0jusDS3/LiwXewrX4vrvx0HC9vH0E04rtYxDFRMZb4EZuXTPjo/iZriaR6DlN1S0TXf9uLS9lWY7mW2rUpYt6nW1v074inuqalKyodklWsscuWl4aa5I8Sez7cLDY7knHnxG83ftt2pHK+g8oxSSbq6XjgYGPMX0SkK1Z46+qMSaa4Yuc3jb2p/Y6G8DN6xm0+xc+oqP3qFsArV8OO5XxWdj4Xzb2QDbs7NH3v1Mwonnh3UaJM+KrqWvqMm9F0EwNJnSuvD8b4B4TTOepUEnLct1h2hxubXlfXhnlq9tqEI3A/XR7JnHu7ZZ2IXh9OrsD2oSLbB333JCyKRNt2IpW5xSBKk3iNepoA3AiMj3wUAp7MlFCKO3bF/Z6avTaj0RB+Rs+44bSOMVMXOkapxD+sSmQ3t+4/hWl9b6CmZjuc9DK9Rk3jxnNHtIhOun30QM4f6nzTxRdGdJIxnuh5GTttEWOfXZT2ufJSenvsyH6Eilrmxu6qq09YRXjiC0tbyJRIScTOBfgRPZTM9ZNMhFj0+nCKUJtw9gDHa9vuuo+nJFRMmYMyLpSy5l4tim8CQ4AFAMaYz0QkI/WflMTY3SRON7Vf0RBeTF4/IjLclnUaYcaOFEd0XMikHvfRs+1GHt88irs3XcrEg49gdHfnkdisFZsct1na9qtbJNlkOKCp5EYsTueqqrqW6ZVVtjJ6yRcYPaSimfsoVobY5ZIZjTsROxcQPS/zPt2aVjRQMtdPMtfUyf2/Cpl3G40nCoZwi3oCkgpcyDe8Koo6Y4wREQMgIh0S/UDJHMncJH6OaBKZvImylL34VRP5ke38vqOHVBBq2M6u2WO4sPPrrNrTnW99PIm5NYcDcPM/F7uGPrptL2oVePHX+4GTq8XrQ9RpziB2uWRG43aTw3aumtpwQzNXVSquz2QiepKZb3AbCERxu7aTcfWkO3eQq+VIErqeRESAl0Tk/4AyEfkJ8AbwcKaFU+xxevjHOx2yPaJxck+d3L+r545lY0f2I1Ts3gioxUNz3XRGrTuV88rf5M9fXMCZH97XpCQAdtU1NNv22GmLOOxXr9B73AzHarixhBsMT9q4izKBk6vFa9Khl+W8DjQEOK5v5xauGidlFG8pZbLZkBeXUJRsVjZ+d9wprJ40infHnZKSkgiyg6QbCS2KiCXxLeAXwA6gH3CrMeb1TAun2OMUO35c386s2VLr22hkemVVs2qx5aUhRh2xfzOzO94MP39oBbNWbGoa7dWGG5rFnUdx63XtVKE2StNDr3YjzL8G1j4LZYMY/cGNLKk9KOF+hRuMrUsoE4SKBQyEG71vL+r3j1aNFbGiauxG9/EPUadrI9b94jQaLw0VURtubNqGARas3W47R+NXLapYkonoiX5mF3kWT75UNs7lciRe8ygex+oFMTfzIqVHa8mjuGX6YtuolFARRINXyktDTDh7QEoF+1KN9BDgoH078NEXuzwtu3rSqBY3WqJtf+/ontw+aCEsGAP1u2DgBDh0LIN/M8tVwQRBaaiImphoIi/EKwS779wS8OyujdhY++mVVYx9dlEz5RXN+raL/omGEEf/l5eG2Lm7vtnvnWRONs8mWRLlRrjltZSVhjAGttc6Z7onws+8Bqd9scvhyQRp51FgVXX9roh8CjQ9AYwxR/ggn5ICs1Zssr2oYp9J22rCjJm6kNteXNrCEvBSHTQVDHhSEuBcYdXtQdk99AWjtt4Gs+dCl+OsIn6d+gNWCfT4B2DQJKskwD3aKKok3B6+dtdGi5FpvHdP7HMBgKYRe/T/tpowoWKhrCTU9JCND/mF1F2fdpas04AnmWz2+Gstdn9TDSdP1QqIHZDFK2K7fQwar4piZEalUJIm2WQgu7ID8VEqu/bUZ8UPD+4VVu1HVY18b5+XubHb4wiGu7deTZ/+NzC601d1JaM3ppd5h3wm3dLik2eubOF6CzcYigS86thwg6FDuzYsnPBVVvawAzr7MqC5NQMAACAASURBVJkbr+yjA54xUxe2ePgnUzk50SR+Km6eVCL94hVWvCKOJVcip7xmZn+aaUGU5Eg2yzQeuyiVbNI+VOR5uwe2W8+kHn9ieIdlvP3lEG5afzXrw/vBPz5g/D+X0D5U3FQoL9YXn6u4WUxeSLe0uNMxbzTWnIrX+ZtoYmF0HiV6Du6+aHDKPvXJM1e6WoROyYnxCgpomueJfuZlcJXsxHcqtZe8Rp2VlYSYeI676zhbUVKeM7OV3GLsyH5cN3VhWg+cIB0022rCjH9+setDs5gGLu/6PGP2+zu7G9ty/boxTNv2dWL9JrXhRmoj7p34gm25SkkK8xZf/dZbaXG3mH63siX1HuYdYjE0776XjAtnemVVs7yPMofqtPHUhhu47cWlzdrESowrLT6UOSqTU/XbWJJ186RS2dirMurQrk1KRSDB/94UqijylNFDKpj36VZPZRZyFbdR1WHtV3FXj3sZWPoJr2w/jlurrmBTfXkWpcscqSoJwNMkaaLoIbcoodiv2qQQsQVfZdJHI5Ji3UVuwRLJBCJsqwk3KZh4RWV3T9SGG2gfKqIkVOx43aXi5ok91tH5htiw4FRyhaLEKhQ7y8FpfuS2F5eqolC+4vbRVnXPfBhFe6Wd1HHNvs/ws32nsa1+b362Zjyv7hgRtFgZp0PbYmrrGnBTIR3aFnuuSuuWJGZXPM+OcIOhNFREfaNJaTASVUixc2LZSFp0krW6JszdFw1uFvW0J9zQpLjbtfFU0agF0ePsdXRvZ4XYEbVunCwHp99vqwk7ZviniiqKPGZ6ZRXPzQ8+GScdYt0bQ0uX8bsef6Jv+/U8u/VUbt9wGdsbWkelmEYDxUXQ6KIpdtU1NCVfeXkoOSmT3Uk8qGvCjRQXCQ1pRpLVhht4es46T5V6wXLP1aZhedlh51aKnY+prg03O46JlHHs90U27rzacAO//Mcirpu6sEXOUVlpiHZtiqiubZkfA82tGyfLwc2F6Hfuhac8inyiteRRgHP/gXyjtKiWsd2e4JJ9XuKzcFfGr7+a/+48MmixcpJEPRIqytxrD50/tCLnLdBoj4tDf/WK78rCC9Fj6BZN5Ucoefz6nJRSsn00ILXcC7c8ClUUeUwqF1CucULHBdzZ4366hzbx+JZvMPnzH1DTGHzceD5TEipuGq3mI2UlIb4xaP9AFZrTaL1YhEZjbC2IVLfzhwsHuY7+3RpS7dpTb3ueU0l0dFMUqTnllJygtK23Wje5SKfiL5nc4x7+duCt7Glsy7c+uYvbPvupKgkfqA035K2SgK96YQSJkxJoMNZ8jR9KIrq+MVMX0jtS7t2p/ll8XatoiXegRW20TORe6BxFnmL1MM5OcpzfnLH3u/ym4kHK2+zg/o0Xct8XF7PHtA1aLCWHSOUxnG5+StBUVdcyZupCJr6wlInnDABoNvFuN6dRXRsmVCSUl4aa8lgykUuhiiIBuVL2N16Orbv2ZF2GdOnaZiu/rniIMzv9jyW1fblk9a9ZtvvAoMVSCoR8VhKxVNeGGfvsIpCvJtu31YSbGiTFW4vhRkNp2zYZ7V0eqKIQkTOAe4Fi4BFjzKS479sBTwBDgS3ARcaYNZmQxU4hgPeQt0ziR6OZYDFcUP4mv9r/YdoX1TFpw6U8vOmbNJC/rjNFySR2uSu14QbHyfNMPxMCUxQiUgw8AJwGrAfmisgLxphlMYtdBmwzxhwkIhcDdwEX+S2LU5xyuzZFOVH2N5lGM7lGj9BG7uhxPyfuVcn7uw5j3PprWbWnR9BiKUpBUSzxVR79JUiLYjjwsTFmFYCIPAOcC8QqinOBiZHX04D7RUSMz6FaTnHKTg/nbDVCCWp7fiA08oN9ZnBDt8cxCLdUXcFTW87EaPyEoviOX5PrTgSpKCqAdTHv12OVM7ddxhhTLyLbgX2AzbELicjlwOUAvXr1SlqQVAqBZZN0CwBmm77t1nFXjz8xrMNy/r1jKDdXXUVVeN+gxVKUgiXTFkVBDO+MMVOMMcOMMcO6dk2+eqjTg7+8NOS5NWMmSabtY5C0oZ4ru/6Dlw++hr7t1nPd2l9w6ZqJqiQUJcMUskVRBfSMed8j8pndMutFpA3QCWtS21ecsjAnnN08RM2v9qLJri+Zto9BMaDkYyb3uJfDSlbzUvUJTPzscjYXSBE/Rcl1CnmOYi5wsIj0wVIIFwPfiVvmBeAS4D3gAuAtv+cnIHG1Tb8mrtMpC2xXeCwXaCd7GLPf0/yk6/Nsre/E5Wtu5rUdxwYtlqK0KgrWoojMOVwNzMQKj33MGLNURH4NzDPGvAA8CvxNRD4GtmIpk4zgVm3TL9Jtnh5f0jhojipdwqSe99G3XRXPbD2dOzb8iB0NHYMWS1FaHWUloYyuX2s9ZRE/m6dPr6wKrOVnh6Iabuz2OD/oMoN1dfsxbv01vLtzcCCyKIpilaAvK22blovcrdaTZmZnkVTaJjpx24tL/RApaU7aax6/rXiA/UObeXTTufz+8+9Ta9oHIouiKBa76hrYVWc9WzKRGFwQUU/5gl30UipRVNMrqzw1nvGTsuId/KHnH/hrn4nsaizhgk9+x282/ESVhKLkILFd9vxALYoskmjS3In4SKlde+qzIW4Ew6hO73BbxUN0Kt7JvRsv5oEvLqLOZNYnqihKeviZqKuKIsskO2keZJ2nfdts4TcVDzKy02w+qDmI762/nRW7+2Rt+4qipI6ficGqKHKcYOo8GS4sf51buj9KWwlzx4Yf8uim0VrET1FylCJo1m/d78RgVRQ5TrbrPPVs+zl3VtzH8XstYs7Ow7lx/TWsqct+WXVFUbxTXCzs3bYN22sz05NCFUVAeM3Q9qvOU6KmLkU0cGmXl7i+2xM0mCJuXn8lf996hhbxU5Q8INxg6NCuDQsnZKYnhT4FAiA671BVXYvhq3A2r20Qow1MvFJRVsJ3j+mFU5L/Qe3WMq3vDdza/WHe23kEp3/4Z57aepYqCUXJIzLpfVCLIgCSydB2ipQCb+U8iqBp+ZcWbWjWHSskYX7WdRrX7DuVnY2lXLv2el6o/ho4qhRFUXKVTFa1VkURAE6a3+lzt0ip215c6ppT8ceLrIzpeKVyRMmH3NXjTxxasoYXqk9kYtVP2drQyesueCLfexgrSr6Q6arW6lsIACfNn+yIYPSQCkrbOuv6irISRg+paGbBtJfdjOv2GP886HrK2+zgx2t+xbVrb/BdSYSKhe8e04uKshIEq2S7osTS2u3WdCu+Rn9fUVbCnecNzGitOrUoAsCprHkqIwI3v2R0fdFlju6wmEk9/kSfdhv4+5aR3LnhR3zZ2CHhNkJFQsf2bVwtl1ARhCPxeeWlISacPaDZhTti0ltZzybPZcpLQ+zcHW46ZkEShOVXGiqiXaiYbTXhgrc8i4uEIpr3wS4JFXP+0AqenL024e/jj09JqDjjiiEetSgCYPSQCu48b2DTaDudEUEnh0ntspJQ0/oOKm/k9ooHmNp3PEUYvv3Jb7mp6hpPSqKirITJ3xpE5a2ns2bSKCocrJ599y7hnosGU1FWQnVNmMkzVzabnM/Hdq6ZojRUxISzB6StJELF/ozJ24eKfFuXF0b07YxBmgYOhq+si7KSEOWloab7InpN5RKJrOP4ffjDtwYx+VuDWtzvww7ojBejwkR+k+6zIh20emwec8v0xY4jkqZR/b4LqXn3J7QLb+TRzefyx8+/y26P9ZmKRfjDhYOaXZROFXDtEOC7x/Ti9tEDGTHpLdsw3w5ti2k0tJiUL+RRZllJiA7t2qQd9jyib2f+98lWT8cpV45nEdCpNGRrXRaL0GhMi3Dx+OoEflFRVsLJ/bsy44MNTfKUlYSaBXzYcc9Fgx3nBivKSnh33CkJt53MPnldZ7po9dgCZHplFU+5mK2yZzP877tQ/h/C7frx7eXXs6g2OddWgzEtqlAmk9dhgKdmr2XYAZ05uX9XW6XWaAznD+3RIiLLYJnsDY258Hjzl+21YbYneBh5YfaqbUkpiWKRwDskNoKjCzIqW3z100z0YSkvDTU9fG8fPbDZd0N+/VpCN+mEswe0eNALcHJ/b62YvVZcCKL1sh3qespTJs9c6fCQMJzd6T+83u8Kzur0Lnd//h1OWDiZtXK47dKJJtTiq1Am27/bRGSdtWKTw/obeW5+la0JngtKok2R/y6Z7mUlvoQyJnrol5eGCBVJ03WSrpIoCWXvcRF/3Y0eUsG7407hnosGp90/PlQsTW2O45leWcXO3e5FN6NK7PyhFc0m5A3w3Pwq23yoeNxcsUG7mexQiyJPsbvQ9muzmdt7/JnT9n6fhTWHcMO6a/lwT28AytpYo5P4CfTzh1bw3Pwq19FN7Lbi8zq8PHoSzU/Uhht8cytU+JTJHuXi4T15dt569tQnP6Hg5O7ZtaeeAd338nT8OrQtZled/bFxsxCifn27kXGRQCo6uHOHdoB9YcrSUBF76o2vFovddRN//RU5HAOJHPzuEffSrBWbWuQhjZj0VovKCJNnrmw26WxHrBKLX9Jrx0onyzwZN5PX6g5+oIoiT2l+oRku7jyTm/Z/jJA08JvPLuMvm8+hMaaI3/baMHdfNNj2whp2QGdXs75IhD7jZrS4GL26AqKjZ78e4Kk+6FLBS1SKE51KQnxj0P7NfOAA1bVh3v1ka4vlYyPHSkJF1DcaRyUhwLeP7ukon5tyTvXYfVZdy90XDbaN2LsjMvJNZg4rEU5WV6w7qs+4GbbLGAPfi8yPxePWu95r0IXbcl7WkW7ko9s+ZEJZqKLIU6IXWldZz6Qe93Fcxw/4384jGLf+GtbW7d9i+e6RnAq7iyj6udMEW7zveN6nWxNaIVFiL363ybuSUBG1NmFAdlUx3aygoHuJx1oR1bVhnptfRbs23lw2HduHqLzVqtXjNPkfxWD51uOVUBS/lXN0nYl6qqRam8zO2vXy0HTbXnR+LP6ad6uM4FV+t+Prxa0YlSl2UtzrdQLJVXfwA52jyFNGD+rGs1/7H68dcjWHl3zMpC2/4OWuf2NHca8Wy3q96eLDdu3mL2rDDTw9Z52rkigJFbXwsUbXbRdaWBIqpr2D37lTaaiFz/b20QOb5Mw17FwRiaJoomyrCXPL9MUJlQR85VqacPYAx66JdvNJoRTnXGKvoeh8wepJo3h33CnNHkxOtcncwlyj5zUV37zbdR2dH4vHrTKClzm46LFwmrj2OqENsDtmcFRdG3as+WYnazKfp0sgFoWIdAamAr2BNcCFxphtNss1AIsjb9caY87Jlow5TfUSmP0jDt86F3qeTfujHmRcqXVT3f7NI9LyXXox6xP5oTt3aGfrZ421XOLlu27qQvtdrQk3jbLt1uWnqyMZMhVu+tTstQnXG//QBveuibHf1dTVJ534WJHENZRIHr8STWO351bGxu7B6da7PirnL/+xyPY6LxZpUmJOrUadAjfiSccqcNuHTBCU62kc8KYxZpKIjIu8v9FmuVpjzODsipbDNNTB0jtg2R0QKoMRz0CvC4kPGUq2ix7YT4w5XYwilg/YiUSjGjv5nOY7El34fpVhT4aokvCqLMpLQ+wON3py1SVaX1lJiInnNM96dzvf8d85KX8nBJKO4XdzcULiApdV1bVcN3UhY6Yu9KSkJpw9gOumLrQ9dnbXT6L5ATelFlUS0yurHK+7dOc5sjHHkSxBKYpzgZMirx8H/o29olCibH4f5vwIti+F3t+FI++B9l18WbXTxJjdXECoyIoycXugpTKqSfXCd/rdkb06eU5GS0SHtsXU1DVQVhpi5+76pqgYL+suCRU3hWLGPiC37dpDTRKp2cmM6t1wUqxOAQJ+j1DtlMiISW+1UKJRUbxM0o4eUsG8T7e2mNgPFYvt9ePFCnNbJnq/OOH1mKVjFXjZBz8JSlHsZ4zZEHn9ObCfw3LtRWQeUA9MMsZMt1tIRC4HLgfo1auljz6vqa+BD34FK++Bku7wtZegYpSvm3AygWet2MSd5w1MynWR6qgm1Qs/0Q0d+7lT0p8TRQLfOfqryJlk61XFP9xj98UucMDJQvEzM9dOsYaKhYaGllt2etD6jZfwaTd3zPTKKmZ8sKHlFy6a3IvV7bSMW7Jc/PXv5gZO1ypIxXOQKhlTFCLyBtDN5qubY98YY4yIOJ3SA4wxVSJyIPCWiCw2xnwSv5AxZgowBawSHmmKnjtsnAVzfgw7V8FBP4Mhd0Fob98342YCJ+O6sCsGmAypXviJornisVMWI/p2Zs2WWqqqa5vyE/bvVMKwAzo3LePVpeClaJudgju5f9cWFpzf7oT40OZiEcI2SgKgQ9s2WXkQeXEfOn3vVgoj3GgyEgXkdh3EnvdEIazZtgrSIWOKwhhzqtN3IrJRRPY3xmwQkf2BLxzWURX5v0pE/g0MAVooioKjrhoqb4BPHoaOB8HX/w37fS1jm0vGBHa7qXfnQinUBEStg6fnrKPBGIpF+PbRPbl99MCEN7bjnA1QVhqiuia5fsV2iiya05KJB0fs6LYskrXtllzmR5kRL9iNrOMRLPm9hLrGkokoILdkufgggkST1dm0CtIhkKKAIjIZ2BIzmd3ZGHND3DLlQI0xZo+IdAHeA841xixzW3feFwVc/wLMvQJ2fw79r4eBE6FNZsNA7UZl0XwFu4xWt5s6WwXMMoFTWGp0n9xGr0GUfk6GVArrRYtCQuZHvVEl5mZZ2F1biaLeMnE9Ot0v8effSTYBVk/y133sB25FAYPKo5gEnCYiHwGnRt4jIsNE5JHIMocC80RkETALa47CVUnkNbu/gHcuhrfPhXb7wOlzLFdTikpiemUVIya9RZ9xMxgx6S3X2Gy7sufRiez4vt5gmddO5HM5cSfZq6prGTHpLcDad6f8EqdwyVzAaxG6WBqMYeyzixg7bVGz6+C6qQvp7eG6SoZobsYalweoU6irE5mKAvLaJsCvBmW5gJYZDxpjYM3fYcHPIfwlHP4rOPQGKG6b8iq9jnjcSDS6TvR9LuM0wZgo0S16DJ1CMXN1pAjJlYdPhugEvF9RWZD42ovFyVKyCyPONn7ch8lsK12rLxctCgVg1zr4z9nw3veg48FwZiUcfktaSgLcfaNeSRTj7ZSBmwslkd2I3rzxltL0yqqEWbmxZR7syOWRYqZkiw9j9cPC8HptRR+OteGGZm1B77loMAsnnB64G9Cr5ZEubte0X6iiCALTCB89BDMGWJFNR94Dp70DnQ7zZfVuLhSvJHoYZusm8JtEE4yJSoM4lXnIdSVpW86jWCgr+aoTW5lDt0Sv+OV+83JtxT4cwXKTRc9BLl2DbuVO/MKPgWEitChgttnxEbz/Y/jibeh2KgyfAh37+LoJt+gcu8gRO7zEeOdSxIYX09tLNm10n5zcH14K4+UiXmT2o5OcX3NUia6tbBfFy2WyUfdJFUW2aKyHFX+ExROgqB0c/Sgc+MMW5TeSxe4BGa2dFO+TjhZJ86NmTy7hpeTyLdMXu3YEjLegvJR5yMVj4UYimePPuVOvBzey5X5Lt3xGIZGNuk+qKLLBtkUw5zLYOh96jIZhD0Bp97RX6/SAvPO8gY4Tl4lupGw2Q/GLRKPLaNtYp2Ni5zbKJ0XpJ7HKJFkLI1vut+mVVY5Z7Lk8T5QpslH3SRVFJmnYA0tuh2WToF1nOP4f0POCtK2IKG4PSKdOb243UrabofhFItPbuW2shdPcSj5aDX6SqJJqeWmI0rZtsq5Inc6n4F52vFDJxqBGFUWm2PSeZUXsWA59fgBH/tHKj/ARtwfk3RcNZuyzi5pl3oaK3Gv35ILfNxWLJpHpnag/cWtWBolwq6SaTrmWdHA6n4bcHtBkkkwPajTqyW/qd8H8MfD6COv1Sa/AsY/7riTAQ0JPvOGSwJDJdjOUeFIN80sUheR0nFrrCDRZci3Czel85mIjq0JBFYWffP4GzDgcVt4LB18Jo5ZA9zMytjm3B+TkmStbFHsLNxjXkLmg8wNSDfNL9CCzO04CfPeYXq12BJos2Qjz9Eo+hifnO+p68oO6bbDgelj1GOx1CJz6Nux7QsY36+abdOoY52YdZLsZSjzpWDRupndrnZguVPR8Zh9VFOmy7p8w90rYswkOGwcDJ0Bx+6xt3ukBmUrIXNA3YCbD/Fr7xHShoeczu6iiSJXajTD/Glj7LJQPhpNmQOcjg5aqiVStgyBvwKAtGkVR7FFFkSzGwOq/wYIxVve5QXfAoddDUXrlD/wmaOsgFfJRZkVpDWj12GTYtRbe/ylseBW6HGdlV3fqn5ltKYqiZBG36rFqUXjBNMJHD8LCcYCBoffBIVeCaNCYoiiFjyqKROxYafWt3vQOdDsdhv8fdOwdtFSKoihZQxWFE41hWP4HWDwR2pTCMX+1Mqx9Kr+hKIqSL6iisGNrpVV+Y1sl9Dwfht0PJd2ClirvyMcCg4qitEQVRSwNu2HJb2DZXdCuCxw/DXqdH7RUeUm+FhhUFKUlgczGisi3RGSpiDSKiO0se2S5M0RkpYh8LCLjMirUpnfhlcGw9A7LxfSN5aok0iAbXbcURckOQYXtLAHOA952WkBEioEHgDOBw4Bvi4g/vUJjCe+EedfC6ydYFsXJM+GYx6Btue+bak0EXWBQURT/CMT1ZIxZDiDuE8PDgY+NMasiyz4DnAss802Qz2bC+5dDzTo45BoY9FsIdfRt9a2ZbHTdUhQlO+RyIkAFsC7m/frIZ+mzZyu8dyn8+wwroum0/8Kwe1VJ+IhW+FSUwiFjFoWIvAHYhQrdbIz5l8/buhy4HKBXr17uC699DuZdBXs2w4Cb4fBbslrEr7Wg5TgUpXDImKIwxpya5iqqgJ4x73tEPrPb1hRgClglPGzXVrsB5l0N656H8iPh5FetYn5KxtAKn4pSGORyeOxc4GAR6YOlIC4GvpP0WoyB1Y/D/OugoRYGT4L+v4SiXN51RVGU3CGo8Nhvish64FhghojMjHzeXUReBjDG1ANXAzOB5cA/jDFLk9rQzjUwayTM/iGUDYSzPoDDblQloSiKkgRBRT39E/inzeefAWfFvH8ZeDnpDTQ2wEcPwKKbAIFhD8DBP9MifoqiKClQeEPrht3wxomw+X+w/xlWEb8OCSa4FUVRFEcKT1FsXwY7yuHYJ6D397SIn6IoSpoUXOMiEdkEfBq0HBG6AJuDFsJnCnGfQPcrnyjEfYLg9+sAY0xXuy8KTlHkEiIyz6ljVL5SiPsEul/5RCHuE+T2funsrqIoiuKKKgpFURTFFVUUmWVK0AJkgELcJ9D9yicKcZ8gh/dL5ygURVEUV9SiUBRFUVxRRaEoiqK4oooiBRK1aBWRdiIyNfL9HBHpHfPd+MjnK0VkZDblTkSq+yUip4nIfBFZHPl/SrZldyOd8xX5vpeI7BSR67MlcyLSvAaPEJH3Iu2IF4tIztTZT+MaDInI45H9WS4i47Mtuxse9utEEVkgIvUickHcd5eIyEeRv0uyJ3UMxhj9S+IPKAY+AQ4E2gKLgMPilrkSeCjy+mJgauT1YZHl2wF9IuspDnqffNivIUD3yOvDgaqg98eP/Yr5fhrwLHB90Pvjw7lqA3wADIq836dArsHvAM9EXpcCa4DeQe9TEvvVGzgCeAK4IObzzsCqyP/yyOvybO+DWhTJ09Si1RhTB0RbtMZyLvB45PU04Oti9X09F+ti3mOMWQ18HFlfLpDyfhljKo1V0BFgKVAiIu2yInVi0jlfiMhoYDXWfuUK6ezT6cAHxphFAMaYLcaYhizJnYh09ssAHUSkDVAC1AE7siN2QhLulzFmjTHmA6Ax7rcjgdeNMVuNMduA14EzsiF0LKooksdLi9amZYxVLn071sgtc+1d0yed/YrlfGCBMWZPhuRMlpT3S0Q6AjcCt2VBzmRI51wdAhgRmRlxddyQBXm9ks5+TQN2ARuAtcDvjTFbMy2wR9K573PimVF4RQGVwBCRAcBdWKPWQmAicLcxZqcUTnHJNsDxwFFADfCmiMw3xrwZrFhpMxxoALpjuWj+KyJvGGNWBStWYaAWRfJ4adHatEzEFO4EbPH426BIZ78QkR5YPUZ+YIz5JOPSeied/Toa+J2IrAHGADeJyNWZFtgD6ezTeuBtY8xmY0wNVr+XIzMusTfS2a/vAK8aY8LGmC+Ad4FcqZuUzn2fG8+MoCd68u0Pa0S2CmsyOjoxNSBumatoPuH2j8jrATSfzF5F7kwkprNfZZHlzwt6P/zcr7hlJpI7k9npnKtyYAHWhG8b4A1gVND75MN+3Qj8JfK6A7AMOCLoffK6XzHL/pWWk9mrI+etPPK6c9b3IeiDmI9/WF34PsSKZLg58tmvgXMir9tjRcl8DLwPHBjz25sjv1sJnBn0vvixX8AtWP7hhTF/+wa9P36cr5h15Iyi8OEa/B7W5PwS4HdB74tP12DHyOdLI0pibND7kuR+HYVl7e3CspCWxvz2R5H9/Rj4YRDyawkPRVEUxRWdo1AURVFcUUWhKIqiuKKKQlEURXFFFYWiKIriiioKRVEUxRVVFIriIyJykoi8FHl9jl2l0Jhly0Tkypj33UVkWjbkVJRk0PBYRfGAiBQbD8XzROQkrHyLb3hYtjfwkjHm8LQFVJQMohaF0uoRkd4iskJEnor0MpgmIqUiskZE7hKRBcC3ROT0SB+HBSLybKRoYLTXwIrIcufFrPdSEbk/8no/EfmniCyK/B0HTAL6ishCEZkckWNJZPn2IvKXSH+FShE5OWadz4vIq5H+BL+LfF4sIn8VkSWR31yX3aOoFDJaFFBRLPoBlxlj3hWRx7D6HgBsMcYcKSJdgOeBU40xu0TkRuAXkQf1w8ApWJmzUx3W/yfgP8aYb4pIMVYm8TjgcGPMYGiyMKJcBRhjzEAR6Q+8JiKHRL4bjNUDZA+wUkTuA/YFKqLWiYiUpXtAFCWKWhSKYrHOGPNu5PWTWBVW4asH/zFYjafemITj0gAAAW1JREFUFZGFwCXAAUB/YLUx5iNj+XGfdFj/KcCDAMaYBmPM9gTyHB9dlzFmBfApVolwgDeNMduNMbuxylUcgFVL6EARuU9EziB3ejEoBYBaFIpiET9ZF32/K/JfsBrIfDt2IREZnGnBbIjt9dEAtDHGbBORQViNbn4GXIhVI0hR0kYtCkWx6CUix0Zefwd4J+772cAIETkIQEQ6RFxBK4DeItI3sty3sedN4IrIb4tFpBPwJbCXw/L/Bb4bWf4QoBdWIUlbIq6xImPMc1hFGnOldLhSAKiiUBSLlcBVIrIcq5zzg7FfGmM2AZcCT4vIB8B7QP+I++dyYEZkMvsLh/X/HDhZRBYD87F6Jm/BcmUtEZHJccv/GSiKLD8VuNS4dw2sAP4dcYs9CYz3uuOKkggNj1VaPRqmqijuqEWhKIqiuKIWhaIoiuKKWhSKoiiKK6ooFEVRFFdUUSiKoiiuqKJQFEVRXFFFoSiKorjy/2QnGl8iY0AHAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "fig,axs = plt.subplots()\n",
        "axs.scatter(y_nn_test_pred,y_test)\n",
        "axs.plot([0,1],[0,1], transform=axs.transAxes, c=\"orange\")\n",
        "axs.set_xlabel(\"predictions\")\n",
        "axs.set_ylabel(\"realizations\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qqbvqAtBHW9u"
      },
      "source": [
        "Compare **mean absolute error of neural network** and  **mean absolute error of linear regression**. (Closer to 0 is a better result.)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fZj_BzVSHtuM",
        "outputId": "c335678b-527b-4235-a10d-8e8d7796989b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "mean_absolute_error of neural network: 0.13058082382271863\n",
            "\n",
            "mean_absolute_error of linear regression: 0.13710582505046462\n"
          ]
        }
      ],
      "source": [
        "print(\"mean_absolute_error of neural network: \" + str(nn_mae)+ \"\\n\")\n",
        "print(\"mean_absolute_error of linear regression: \" + str(lr_mae))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aXg2T8H9IgeN"
      },
      "source": [
        "####Result of Step 4\n",
        "1.   Both models are well-trained and return results with a low margin of error.\n",
        "2.   The neural network model gives better result than the linear regression model.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k-gqx4Mx4tO7"
      },
      "source": [
        "# Step 5"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xQe6l7QMuWc7"
      },
      "source": [
        "### Feature Importance\n",
        "####Define the feature_analysis"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "55uHhTNwuVZb"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from scipy.stats import mode\n",
        "\n",
        "# a function for feature analysis\n",
        "def feature_analysis(X, nn, feature_names):\n",
        "    # determine first and second partial derivatives\n",
        "    with tf.GradientTape() as snd:\n",
        "        snd.watch(X)\n",
        "        with tf.GradientTape() as fst:\n",
        "            fst.watch(X)\n",
        "            # prediction with the neural network, i.e., f(X)\n",
        "            pred = nn(X)\n",
        "        # gradient\n",
        "        g = fst.gradient(pred, X)\n",
        "    # jacobian which outputs Hessian matrix\n",
        "    h = snd.batch_jacobian(g, X)\n",
        "\n",
        "    # first partial derivatives\n",
        "    g_np = g.numpy()\n",
        "    # average squard partial derivatives\n",
        "    g_mag_sq = (g_np**2).mean(axis = 0)\n",
        "    # square root of average squard partial derivatives\n",
        "    g_mag = np.sqrt(g_mag_sq)\n",
        "    # sign of average partial derivatives\n",
        "    g_dir = np.sign(g_np.mean(axis = 0))\n",
        "\n",
        "    # normalizing constant\n",
        "    C_ = np.sum(g_mag)\n",
        "    # normalized feature importance with sign\n",
        "    fi = (g_mag * g_dir) / C_\n",
        "\n",
        "    # get signs of each sample\n",
        "    fi_signs = np.sign(g_np)\n",
        "    # the mode is the sign which can be observed most often among all samples, the counts is how often this sign is observed\n",
        "    fi_modes, fi_counts = mode(fi_signs)\n",
        "    # dividing the count of the sign which is observed most often by the overall sample size gives us a frequency measure\n",
        "    # which is closer to one, the higher the conformity of the sign\n",
        "    fi_conformity = fi_counts / g_np.shape[0] #fi_modes * \n",
        "\n",
        "    # in analogy to the calculation above, we do the same thing with the second partial derivatives\n",
        "    h_np = h.numpy()\n",
        "    # get the square root of average squared direction of curvature and interactions\n",
        "    h_mag_sq = (h_np**2).mean(axis = 0)\n",
        "    h_mag = np.sqrt(h_mag_sq)\n",
        "\n",
        "    # the the sign of average curvature and interactions\n",
        "    h_dir = np.sign(h_np.mean(axis = 0))\n",
        "\n",
        "    # normalize the values on the diagonal line to compare the degree of non-linearity\n",
        "    C_nonlin = np.sum(h_mag.diagonal())\n",
        "    nonlinearity = (h_dir.diagonal() * h_mag.diagonal()) / C_nonlin\n",
        "\n",
        "    # normlize the interactions\n",
        "    lti = np.tril_indices(h_mag.shape[0], k = -1)\n",
        "    C_ia = np.sum(h_mag[lti])\n",
        "    interactions = (h_mag[lti] * h_dir[lti]) / C_ia\n",
        "\n",
        "    # bring curvature and interaction effects back to matrix format\n",
        "    snd_degree_summary = np.diag(nonlinearity)\n",
        "    a, b = lti\n",
        "    inter_iter = iter(interactions)\n",
        "    for i, j in zip(a, b):\n",
        "        snd_degree_summary[i, j] = next(inter_iter)\n",
        "        snd_degree_summary[j, i] = snd_degree_summary[i, j]\n",
        "\n",
        "    # get the conformity of second order effects\n",
        "    snd_signs = np.sign(h_np)\n",
        "    snd_degree_modes, snd_degreee_counts = mode(snd_signs)\n",
        "    snd_degree_conformity = snd_degreee_counts / h_np.shape[0] #snd_degree_modes * \n",
        "\n",
        "    # finally summarize feature importances and second order effects\n",
        "    summary = pd.DataFrame(data = snd_degree_summary, index = feature_names, columns = feature_names)\n",
        "    summary.loc[:, 'feature_importance'] = fi\n",
        "    # as well as their conformity\n",
        "    summary_conformity = pd.DataFrame(data = snd_degree_conformity.reshape(h_np.shape[1], h_np.shape[2]), index = feature_names, columns = feature_names)\n",
        "    summary_conformity.loc[:, 'feature_importance'] = fi_conformity.flatten()\n",
        "\n",
        "    return summary, summary_conformity"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4ERPmOb7PH1f"
      },
      "outputs": [],
      "source": [
        "X_test_tensor = tf.Variable(X_test_s)\n",
        "sum_test_nn, sum_c_test_nn = feature_analysis(X_test_tensor, model, X_test.columns)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y9pmd07O6Ymq",
        "outputId": "e9c881aa-754d-43b5-c167-63b568067c41"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "book_to_market   -0.300961\n",
              "market_cap        0.516052\n",
              "beta              0.182986\n",
              "Name: feature_importance, dtype: float64"
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sum_test_nn[\"feature_importance\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "tX41_BHe6gZe",
        "outputId": "a66a7abf-9922-447c-f8c2-f5384d1a8502"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-6821d76a-d3e5-40c9-b805-fc5da3cf24e4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>book_to_market</th>\n",
              "      <th>market_cap</th>\n",
              "      <th>beta</th>\n",
              "      <th>feature_importance</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>book_to_market</th>\n",
              "      <td>0.583027</td>\n",
              "      <td>0.619135</td>\n",
              "      <td>0.609476</td>\n",
              "      <td>0.990340</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>market_cap</th>\n",
              "      <td>0.619135</td>\n",
              "      <td>0.637305</td>\n",
              "      <td>0.586707</td>\n",
              "      <td>0.971251</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>beta</th>\n",
              "      <td>0.609476</td>\n",
              "      <td>0.586707</td>\n",
              "      <td>0.548068</td>\n",
              "      <td>0.997470</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6821d76a-d3e5-40c9-b805-fc5da3cf24e4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6821d76a-d3e5-40c9-b805-fc5da3cf24e4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6821d76a-d3e5-40c9-b805-fc5da3cf24e4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                book_to_market  market_cap      beta  feature_importance\n",
              "book_to_market        0.583027    0.619135  0.609476            0.990340\n",
              "market_cap            0.619135    0.637305  0.586707            0.971251\n",
              "beta                  0.609476    0.586707  0.548068            0.997470"
            ]
          },
          "execution_count": 32,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sum_c_test_nn"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "t7V43IaU6tFi",
        "outputId": "14671629-f6f2-4772-82f9-778e62490421"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f06c37388b0>"
            ]
          },
          "execution_count": 33,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAD4CAYAAABmBQicAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAcl0lEQVR4nO3de3yU9Z328c83IRpOoo1CSSkGLWKAQIghlioqQsGzUOyioIV1n4pFl8UqSrXa2ErFw7N9ioos7WNVDsoKUlG6lYNQVLCBYAKyIIiNclqFuFJEoKv57h9zJ4R0gAkkmfzker9e8/Ke+3jNJHLlPszc5u6IiIiELCXZAURERI6VykxERIKnMhMRkeCpzEREJHgqMxERCV6TZAc4Hp166qmelZWV7BgiIkEpLi7e6e6nxZumMkuCrKwsVq5cmewYIiJBMbMPDjVNhxlFRCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeDp5pwiDa2wVbITiCRP4a56Wa32zEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREghd8mZnZRWb2Si3mH2FmmfWZSUREGlbQZWZmR3Nz0RGAykxE5CskKWVmZllmtt7MnjazDWY23cz6mdmbZrbRzAqix3Ize9vMlplZp2jZEWY218xeAxbVWG/PaP4zzewcM/uTmRWb2atm1tbMrgHygelmVmJmTQ+Rr2e0zVIzKzKzllHm181sVfT4TjTvRWa21Mzmmdm7ZjbZzIL+I0FEJDRHs2dTV74FfB+4EVgBDAXOB64C7gZ+APR29y/MrB/wS2BwtGwe0M3dPzGziwCicnkMuBrYDkwFrnb3HWY2BBjv7jea2a3AHe6+Ml4oMzsBmAkMcfcVZnYSsBf4GPiuu+8zs47Ac8SKEaAA6Ax8APwR+B4wq8Z6bwJuAmjfvv1RvmXyVZC1b0ayI4gkTVk9rTeZZfYXd18DYGZrgUXu7ma2BsgCWgHPRMXhQFq1ZRe4+yfVnmcDU4D+7r7NzLoCXYEFZgaQSqzgEtEJ2O7uKwDc/a9RxubA42aWC3wJnFVtmSJ3fz+a7zlipXxQmbn7lCgj+fn5nmAWERFJQDLLbH+14YpqzyuI5foFsNjdB5lZFrCk2vx7aqxrO5AO9AC2AQasdfdedZj3NuAjoDuxw7P7qk2rWU4qKxGRBtSYz+20ArZGwyOOMO+nwOXAg9Fhx3eB08ysF4CZpZlZl2je3UDLw6zrXaCtmfWMlm0ZXWjSitgeWwVwA7G9vUoFZtYhOlc2BHgjsZcoIiJ1oTGX2cPEyultEtiDdPePgCuAJ4jtoV0DPGRmpUAJ8J1o1qeByYe6AMTd/0askB6Lll1AbK9vEjA8Gnc2B+8drgAeB9YBfwHm1PrViojIUTN3HRE7FtGe4B3ufkWiy+Tn5/vKlXGvP5HjQNa4ecmOIJI0ZRMuP+plzazY3fPjTWvMe2YiIiIJSeYFIElnZnOADjVG3+Xurya6DndfwsEXp4iISAM7rsvM3QclO4OIiBw7HWYUEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4B3X3wAikgzH8kWrIhKf9sxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESC1yTZAUSOO4Wtkp1AjkXhrmQnkDhUZiKSkP854WS25N3FvlZnAJbsOMmzbl2yE3zlpaen065dO9LS0hJeRmUmIgnZkncXLc/IJ6t5E8yO4zLLzE52gq80d6e8vJwtW7bQoUOHhJfTOTMRSci+VmeQcbwXmdQ7MyMjI4N9+/bVajmVmYgkyFRk0iCO5vdMZSYiIsHTOTMROSpZE7fV6frKRmcecZ6J//85nnz2BfJyspn++PjE1715G8tWljJ00KXHEvGILrvsMmbMmMHJJ59cr9up9OmnnzJjxgxGjRrVINtrzLRnJiLBmPTMCyx47slaFRnEymzGnP+o9fa+/PLLWs3/hz/8ocGK7IsvvuDTTz9l0qRJDbK9xk5lJiJBuPmu8bz/4RYuveGfGf/r33LjjwspuPwGevS/jpdeXQLESqv3oBvJGzCUvAFDWbaiFIBxv5zI60Ul5H73Wn41ZRpPz5zLrfdMqFr3FT8YzZJlKwFo0fE8br//X+nebwjLi1czbfY8Ci6/gdzvXsvIOx84bMFlZWWxc+dOysrKOPvssxkxYgRnnXUWw4YNY+HChZx33nl07NiRoqIiAAoLC7nhhhvo1asXHTt25De/+Q0Qu6Jv7NixdO3alZycHGbOnAnAkiVL6N27N1dddRWdO3dm3LhxbNq0idzcXMaOHctnn31G3759ycvLIycnh5deein2vpSVkZ2dzQ9/+EO6dOlC//792bt3LwDvvfce/fr1o3v37uTl5bFp0yYAHnnkEXr27Em3bt342c9+Vlc/xnqjMhORIEx+6B4y25zG4hf+jT2f7+Xi83pSNG8qi1+Ywthf/D/2fL6X1qeewoLnnmTVqzOY+eQERt/3MAAT7h5N74JcShY8z203XX/Y7ez5fC/n9uhK6cKZZJzSiplz5/Pm75+iZMHzpKamMn369ITyvvfee9x+++2sX7+e9evXM2PGDN544w0effRRfvnLX1bNt3r1al577TWWL1/Oz3/+c7Zt28aLL75ISUkJpaWlLFy4kLFjx7J9+3YAVq1axa9//Ws2bNjAhAkTOPPMMykpKeGRRx4hPT2dOXPmsGrVKhYvXsztt9+OuwOwceNGbrnlFtauXcvJJ5/M7NmzARg2bBi33HILpaWlLFu2jLZt2zJ//nw2btxIUVERJSUlFBcXs3Tp0lr/zBqSzpmJSHDmL32LuQuW8ujkqQDs2/83Pty6ncw2p3HrPQ9R8p8bSE1JYcP7H9Z63ampqQy+vC8Ai94oonjNOnpedgMAe/ftp3WHzgmtp0OHDuTk5ADQpUsX+vbti5mRk5NDWVlZ1XxXX301TZs2pWnTpvTp04eioiLeeOMNrrvuOlJTU2nTpg0XXnghK1as4KSTTqKgoOCQn79yd+6++26WLl1KSkoKW7du5aOPPqrKk5ubC8A555xDWVkZu3fvZuvWrQwaNAiIfVgZYP78+cyfP58ePXoA8Nlnn7Fx40YuuOCC2ryVDUplJiLBcXdmT3mETt/KOmh84f+dTJvTMihd8DwVFRWkn9Er7vJNmqRSUeFVz/ft/1vVcPqJJ5CamhptB4Z//0oe/Mk/H1g4s0dCGU888cSq4ZSUlKrnKSkpfPHFF1XTal6GfqTL0ps3b37IadOnT2fHjh0UFxeTlpZGVlZW1ee1qudJTU2tOswYj7vzk5/8hJEjRx42S2Oiw4wiEpwBF/bisd89X3UI7e131gOw66+f0bb1qaSkpDB19ryq81stWzRj957Pq5bP+mYmJWvfpaKigs1b/4uikrVxt9P3/AJmvbKQj3d+AsAn/72LDz74oE5fy0svvcS+ffsoLy9nyZIl9OzZk969ezNz5ky+/PJLduzYwdKlSykoKPi7ZVu2bMnu3burnu/atYvWrVuTlpbG4sWLj5i1ZcuWtGvXjt///vcA7N+/n88//5wBAwbw1FNP8dlnnwGwdetWPv744zp81XVPe2YiclQSuZS+vtw75oeM+dmjdOs3hIqKCjp8M5NXnp3IqOH/wOCb7uDZWa9wSZ/v0LxZUwC6ZXckNSWF7v2GMOIfrmTMD4fRof036HzRNWR37EBeztlxt9P5rDN44M5R9L9uFBVeQVqTJjwx5XecfvrpdfZaunXrRp8+fdi5cyf33nsvmZmZDBo0iOXLl9O9e3fMjIcffpivf/3rrF+//qBlMzIyOO+88+jatSuXXnopd911F1deeSU5OTnk5+dz9tnxX1d1U6dOZeTIkdx3332kpaXxwgsv0L9/f9atW0evXrE92xYtWjBt2jRat25dZ6+7rlnlXzbScPLz833lypXJjiHJEui35q8b8O9kn954/zFrMAkeZkxEYWEhLVq04I477qizdX5VrFu3juzsg78H08yK3T0/3vw6zCgiIsE74mFGM8sCXnH3rseyITMrA/LdfecR5jsZGOruSf8koJmNIJb51gTnHwNMcffPjziziATr3HPPZf/+/QeNmzp1atXVi4kqLCysw1THt8Z4zuxkYBSQ1DIzs6N5b8YA0wCVmchX2J///OdkR5AaEv0Hu4mZTQfygLXAD4BewKPROlYAP3L3/WbWN974yhWZWVPgReBFd/9NnG1NAM40sxJgAXAn8DBwKeDAA+4+M15IM7sIuB/4FMgB/h1YA/wL0BQY6O6bzOxK4KfACUA5MMzdPzKzQuBM4AzgQ+DVauu+PFrmyuh9uB84EdgE/CNwI5AJLDazne7ep0a2m4CbANq3bx/3TU5U1rh5x7S8JFfZhEDvVLxune7lJY1WoufMOgGT3D0b+CvwY+BpYIi75xArrh+ZWXq88dXW0wJ4GXjuEEUGMA7Y5O657j4W+B6QC3QH+gGPmFnbw2TtDtwMZAM3AGe5ewHwW6DywyJvAN929x7A88QKs1JnoJ+7X1c5wswGRbkui0b9NJonD1gJ/NjdJwLbgD41iwzA3ae4e76755922mmHiS8iIrWVaJltdvc3o+FpQF/gL+6+IRr3DHABsdKLN77SS8Dv3P3ZWmQ8n1j5fenuHwF/AnoeZv4V7r492hvcBMyPxq8BsqLhdsCrZrYGGAt0qbb8XHev/mnCi4G7gMvd/b+BbxMrvDejvcfhQN1dpysiIrWWaJnVvH7/06Pc3pvAJVa/d/irfla2otrzCg4cVn0MeDzaexwJpFdbZk+N9W0CWgJnRc8NWBDtOea6e2d3/6e6fAEiIlI7iZ4za29mvdx9OTCU2KG1kWb2LXd/j9jhvD8B7wJZccZXui96PEHsIo94dhMrj0qvR9t6BvgasT29sQnmPpRWwNZoePgR5v0g2t6LZvZ94C3gicrXaGbNgW9Ee6OV2Q97xabIV0Jdf16u8MjnEidOnMiTTz5JXl5ewl/4C7FvjV+2bBlDhw49loRHpPuZHaxFixZV3yJS3xLdM3sXuMXM1gGnAL8idtHDC9Ghugpgsrvvize+xrr+BWhqZg/H25C7lxM7hPeOmT0CzAFWA6XAa8Cd7v5ftXmRcRRGGYtJoHjcfT0wDHgBOAkYATxnZquB5UDlx+ynAH80s8XHmE9E4pg0aRILFiyoVZFBrMxmzJhR6+3pfmbhOGKZuXuZu5/t7te7e7a7D3b3z919kbv3cPccd7+x8orFw4zPcvedHvOP7n7nYbY51N27uvvYaP6x0fOcQ13JGC23xN2vqPb8IndfWXOau7/k7me4+znRui+Kxhe6+6PVln+68jNm7v52dEhxk7u/5u493b1b9JgbzfOYu3eKdwGIiBybm2++mffff59LL72U8ePHc+ONN1JQUECPHj0Oum9X7969ycvLIy8vj2XLlgEwbtw4Xn/9dXJzc/nVr37F008/za23Hvj46BVXXMGSJUuA2N7E7bffTvfu3Vm+fDnTpk2joKCA3NxcRo4cqfuZxTFw4EDOOeccunTpwpQpUw6adtttt1XdNWDHjh1AbA+7c+fOdOvWjWuvvfYIP/nE6BtARCQIkydPJjMzk8WLF7Nnzx4uvvhiioqKWLx4MWPHjmXPnj20bt2aBQsWsGrVKmbOnMno0aMBmDBhAr1796akpITbbrvtsNvZs2cP5557LqWlpWRkZDBz5kzefPNNSkpKdD+zQ3jqqacoLi5m5cqVTJw4kfLy8qr3Mj8/n7Vr13LhhRdy//33V/083n77bVavXs3kyTUP3h2dpH1o2swygEVxJvWNDjUebtkcYGqN0fvd/dy6yicijdf8+fOZO3cujz4aO5Cyb98+PvzwQzIzM7n11lurimfDhg1HWNPfS01NZfDgwQAsWrSI4uJievaMXUC9d+/ehL9s93i6n9nEiROZM2cOAJs3b2bjxo1kZGSQkpLCkCFDALj++uv53ve+B8S+XHnYsGEMHDiQgQMHJvR+HknSyiwqrNyjXHbN0S4rIuFzd2bPnk2nTp0OGl9YWEibNm0oLS2N3c8sPT3u8k2aNKGioqLqeeU9vyD2D/qB+5k5w4cP58EHH6x1xuPlfmZLlixh4cKFLF++nGbNmnHRRRcd9H5WV/na5s2bx9KlS3n55ZcZP348a9asoUmTY6sjHWYUkeAMGDCAxx577MD9zN5+G4jdz6tt27ax+5lNnXrgfmY17vuVlZVFSUlJ7H5mmzdXncOqqW/fvsyaNavqXl6ffPKJ7mdWw65duzjllFNo1qwZ69ev56233qqaVlFRwaxZswCYMWMG559/ftV73qdPHx566CF27dpVJ1c8NsbvZhSRECRwKX19uffeexkzZgzdunWL3c+sQwdeeeUVRo0axeDBg3n22We55JJLqvZiunXrRmpqKt27d2fEiBGMGTOGDh060LlzZ7Kzs8nLy4u7nc6dO/PAAw/Qv39/KioqSEtL44knntD9zKq55JJLmDx5MtnZ2XTq1Ilvf/vbVdOaN29OUVERDzzwAK1bt64q6Ouvv55du3bh7owePbpOrgDV/cyS4FjvZ6bvZgxb2YTLkx3hqMS7v5QcG93P7NB0PzMRETnu6DCjiEgtHY/3MysvL6dv375/N37RokVkZGQkIdHBVGYikjB3P+LVdseD4/F+ZhkZGZSUlDTIto7m9JcOM4pIQtLT0ykvLz+qf2hEEuXulJeXH/JjFYeiPTMRSUi7du3YsmVL1VcSidSX9PR02rVrV6tlVGYBCvVqOAlbWlraIb95QiTZdJhRRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeE2SHUDkuFPYKtkJGpfCXclOIF8B2jMTEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCpzITEZHgqcxERCR4KjPAzLLM7J1azD/CzDLrM5OIiCROZXZ0RgAqMxGRRkJldkATM5tuZuvMbJaZNTOzc8zsT2ZWbGavmllbM7sGyAemm1mJmTU1s/vMbIWZvWNmU8zMkv1iRESOJyqzAzoBk9w9G/grcAvwGHCNu58DPAWMd/dZwEpgmLvnuvte4HF37+nuXYGmwBXJeQkiIsenJskO0Ihsdvc3o+FpwN1AV2BBtKOVCmw/xLJ9zOxOoBnwNWAt8HL1GczsJuAmgPbt29d5eAlH1r4ZyY7QqJQlO4B8JajMDvAaz3cDa9291+EWMrN0YBKQ7+6bzawQSP+7lbtPAaYA5Ofn19yWiIgcAx1mPKC9mVUW11DgLeC0ynFmlmZmXaLpu4GW0XBlce00sxbANQ0VWEREYlRmB7wL3GJm64BTiM6XAQ+ZWSlQAnwnmvdpYLKZlQD7gd8A7wCvAisaOLeIyHFPhxkBdy8Dzo4zqQS4IM78s4HZ1Ub9NHqIiEgSaM9MRESCpzITEZHgqcxERCR4KjMREQmeykxERIKnMhMRkeCpzEREJHgqMxERCZ7KTEREgqcyExGR4KnMREQkeCozEREJnspMRESCp2/NF2lgZRMuT3YEka8c7ZmJiEjwVGYiIhI8lZmIiARPZSYiIsFTmYmISPBUZiIiEjyVmYiIBE9lJiIiwVOZiYhI8FRmIiISPJWZiIgET2UmIiLBU5mJiEjwVGYiIhI8lZmIiARPZSYiIsFTmYmISPDM3ZOd4bhjZjuAD45hFacCO+soTl1RpsQ1xlzKlJjGmAkaZ676yHS6u58Wb4LKLEBmttLd85OdozplSlxjzKVMiWmMmaBx5mroTDrMKCIiwVOZiYhI8FRmYZqS7ABxKFPiGmMuZUpMY8wEjTNXg2bSOTMREQme9sxERCR4KjMREQmeyiwAZvY1M1tgZhuj/54SZ57TzWyVmZWY2Vozu7kRZMo1s+VRntVmNiTZmaL5/mhmn5rZK/WY5RIze9fM3jOzcXGmn2hmM6PpfzazrPrKUstcF0S/R1+Y2TWNJNOPzew/o9+hRWZ2eiPIdLOZrYn+f3vDzDonO1O1+QabmZtZg1wWn8B7NcLMdkTvVYmZ/Z96CeLuejTyB/AwMC4aHgc8FGeeE4ATo+EWQBmQmeRMZwEdo+FMYDtwcjIzRdP6AlcCr9RTjlRgE3BG9HMpBTrXmGcUMDkavhaY2QC/R4nkygK6Ac8C1zSSTH2AZtHwj+r7vUow00nVhq8C/pjsTNF8LYGlwFtAfiP5+Y0AHq/vLNozC8PVwDPR8DPAwJozuPvf3H1/9PRE6n+vO5FMG9x9YzS8DfgYiPvp/YbKFGVZBOyuxxwFwHvu/r67/w14PspWXfWss4C+Zmb1mCmhXO5e5u6rgYp6zlKbTIvd/fPo6VtAu0aQ6a/VnjYH6vtKukR+pwB+ATwE7KvnPLXNVe9UZmFo4+7bo+H/AtrEm8nMvmlmq4HNxPZKtiU7U7VsBcT+ctvUWDLVo28Q+xlU2hKNizuPu38B7AIyGkGuhlbbTP8E/Ee9Jkowk5ndYmabiB0RGJ3sTGaWB3zT3efVc5Za5YoMjg4TzzKzb9ZHkCb1sVKpPTNbCHw9zqR7qj9xdzezuH8FuvtmoJuZZQK/N7NZ7v5RMjNF62kLTAWGu/sx/cVfV5kkPGZ2PZAPXJjsLADu/gTwhJkNBX4KDE9WFjNLAf6V2CG9xuZl4Dl3329mI4kdkbi4rjeiMmsk3L3foaaZ2Udm1tbdt0fF8PER1rXNzN4BehM7hJW0TGZ2EjAPuMfd3zraLHWZqQFsBar/9dkuGhdvni1m1gRoBZQ3glwNLaFMZtaP2B8sF1Y7nJ7UTNU8DzxZr4mOnKkl0BVYEh2t/jow18yucveVScyFu1f/vf4tsT3ZOqfDjGGYy4G/+oYDL9WcwczamVnTaPgU4Hzg3SRnOgGYAzzr7kddqnWZqYGsADqaWYfoPbg2ylZd9azXAK95dLY8ybka2hEzmVkP4N+Aq9y9If5ASSRTx2pPLwc2JjOTu+9y91PdPcvds4idW6zvIjtiLqg6MlPpKmBdvSSp7ytM9Dj2B7FzKYuI/Q+zEPhaND4f+G00/F1gNbGriVYDNzWCTNcD/wOUVHvkJjNT9Px1YAewl9gx/gH1kOUyYAOxc4T3RON+TuwfGIB04AXgPaAIOKOBfpeOlKtn9J7sIbanuLYRZFoIfFTtd2huI8j0a2BtlGcx0CXZmWrMu4QGuJoxwffqwei9Ko3eq7PrI4e+zkpERIKnw4wiIhI8lZmIiARPZSYiIsFTmYmISPBUZiIiEjyVmYiIBE9lJiIiwftfH4v+4WfDAFgAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "fi = pd.DataFrame(sum_test_nn[\"feature_importance\"])\n",
        "fi.loc[:, \"feature_importance_abs\"] = fi.feature_importance.abs()\n",
        "fi.sort_values(by=\"feature_importance_abs\", ascending=True, inplace =True)\n",
        "fi.plot.barh()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YNt5yWoF5Be4"
      },
      "source": [
        "####linear regression feature importance"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v3uYjnl8_6BQ",
        "outputId": "d701b5cf-680c-4bee-b7ab-3e78dd3b6aa3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "book_to_market, Score: 0.00118\n",
            "market_cap, Score: -0.00005\n",
            "beta, Score: 0.02529\n"
          ]
        }
      ],
      "source": [
        "importance = log_reg.coef_\n",
        "# summarize feature importance\n",
        "for i,v in enumerate(importance):\n",
        " print('%s, Score: %.5f' % (X_train.columns[i],v))\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "6vXL-oX6wNXF",
        "outputId": "81f54496-81ee-47a0-8599-f46e670a9c65"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f06c36ff6a0>"
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAD4CAYAAABmBQicAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWDklEQVR4nO3de5BW1Znv8e+jYBqF4AS8hoOQjOIFRLAhXolO1IziXaccgxGNVeiMOeWcSbyOGhMzcxJJVRKjxjCpqPGSeBR1NNaImpigjjlcDESUEERxbPR4wQkisRkvz/nj3WjbdmPTF95ezfdTtcv97r322mu9G/vXa7273x2ZiSRJJdus3g2QJKmrDDNJUvEMM0lS8QwzSVLxDDNJUvH61bsBm6KhQ4fmiBEj6t0MSSrK/PnzX83MbdraZ5jVwYgRI5g3b169myFJRYmI59rb5zSjJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4PpyzRJcNrncLJKlzLlvVI9U6MpMkFc8wkyQVzzCTJBXPMJMkFc8wkyQVzzCTJBXPMJMkFc8wkyQVzzCTJBXPMJMkFc8wkyQVzzCTJBXPMJMkFc8wkyQVzzADImJERCzagPKnRcSOPdkmSVLHGWadcxpgmElSL2GYva9fRNwcEYsj4vaI2DIi9o6I30TE/IiYFRE7RMSJQCNwc0QsiIgBEXFpRMyNiEURMSMiot6dkaRNiWH2vlHANZm5G/A6cDbwA+DEzNwb+Anwz5l5OzAPmJKZe2Xmm8BVmTkhM0cDA4Aj69MFSdo09at3A3qR5zPz0Wr9JuAiYDTwQDXQ2hx4sZ1jD46I84AtgU8ATwL3tCwQEdOAaQDDhw/vUkNHNN/SpeMlqV6W91C9htn7stXr1cCTmbnv+g6KiAbgGqAxM5+PiMuAhg9VnjkDmAHQ2NjY+lySpC5wmvF9wyNiXXB9AfgtsM26bRHRPyL2qPavBgZV6+uC69WIGAicuLEaLEmqMczetwQ4OyIWA39B9XkZ8O2IWAgsAParyl4PXBsRC4C1wL8Ci4BZwNyN3G5J2uQ5zQhk5nJg1zZ2LQAmtVF+JjCzxaaLq0WSVAeOzCRJxTPMJEnFM8wkScUzzCRJxTPMJEnFM8wkScUzzCRJxTPMJEnFM8wkScUzzCRJxTPMJEnFM8wkScUzzCRJxfNb8wu0/FuT690ESepVHJlJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSitev3g1QJ1w2uJPHreredkhSL+HITJJUPMNMklQ8w0ySVDzDTJJUPMNMklQ8w0ySVDzDTJJUPMNMklQ8w0ySVDzDTJJUPMNMklQ8w0ySVDzDTJJUPMNMklQ8w0ySVLyPDLOIGBERi7p6oohYHhFDO1Bu64j4+66erztExGkRcdUGlP+HiNiyJ9skSfqw3jgy2xqoe5hFRGceXPoPgGEmSRtZR39g94uIm4HxwJPAqcC+wHeqOuYCf5eZayPic21tX1dRRAwA7gDuyMx/beNc3wI+HRELgAeA84ArgMOBBL6Zmbe21ciIOAj4OvAnYAzwf4AngHOAAcCxmbksIo4CLga2AFYCUzLzpYi4DPg08CngP4FZLeqeXB1zVPU+fB34GLAMOB34ErAj8FBEvJqZB7dq2zRgGsDw4cPbfJM7akTzLZ06bnmXzipJvVdHR2ajgGsyczfgdeAfgeuBkzJzDLXg+ruIaGhre4t6BgL3AD9rJ8gALgCWZeZemXkucDywFzAWOASYHhE7rKetY4GzgN2ALwK7ZOZE4MfA/6zKPALsk5njgJ9TC8x1dgcOycyT122IiOOqdh1Rbbq4KjMemAf8Y2ZeCbwAHNw6yAAyc0ZmNmZm4zbbbLOe5kuSNlRHw+z5zHy0Wr8J+BzwbGb+sdp2AzCJWui1tX2dfwOuy8yfbkAbD6AWfu9k5kvAb4AJ6yk/NzNfrEaDy4D7q+1PACOq9WHArIh4AjgX2KPF8Xdn5pstXv8VcD4wOTP/C9iHWuA9Wo0epwI7bUB/JEndrKNhlq1e/6mT53sU+OuIiE4e3xFrW6y/2+L1u7w/rfoD4Kpq9Hgm0NDimDWt6lsGDAJ2qV4H8EA1ctwrM3fPzDO6swOSpA3T0TAbHhH7VutfoDa1NiIi/rLa9kVqI6Yl7Wxf51Lgv4Cr13Ou1dTCY52HgZMiYvOI2IbaSG9OB9vdnsHAimp96keUfQ44AfhpROwB/BbYf10fI2KriFgXdK3bLknaCDoaZkuAsyNiMfAXwHep3fRwWzVV9y5wbWY2t7W9VV3nAAMi4oq2TpSZK6lN4S2KiOnAncDvgYXAr4DzMvP/bUgn23BZ1cb5wKsfVTgz/wBMAW4DPg6cBvwsIn4PPAbsWhWdAdwXEQ91sX2SpA0Qma1nENXTGhsbc968eZ0+fsQF93bquOXfmtzpc0pSvUXE/MxsbGtfb/w7M0mSNkhn/jC4W0TEEOCXbez6XDXVuL5jxwA3ttq8NjM/013tkySVo25hVgXWXp089onOHitJ6nucZpQkFc8wkyQVzzCTJBXPMJMkFc8wkyQVzzCTJBXPMJMkFc8wkyQVr25/NK3O8zsWJemDHJlJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYSZKKZ5hJkopnmEmSimeYleiywfVugST1KoaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXiGmSSpeIaZJKl4hpkkqXj96t0ASaq3t956i6amJpqbm+vdFAENDQ0MGzaM/v37d/iY4sMsIg4CvpqZR3aw/GnA/Zn5Qk+2S1I5mpqaGDRoECNGjCAi6t2cTVpmsnLlSpqamhg5cmSHjyt6mjEiOhPGpwE7dnNTJBWsubmZIUOGGGS9QEQwZMiQDR4l1yXMImJERPwhIq6PiD9GxM0RcUhEPBoRSyNiYrU8FhG/i4j/iIhR1bGnRcTdEfEr4Jet6p1Qlf90ROwdEb+JiPkRMSsidoiIE4FG4OaIWBARA9pp34TqnAsjYk5EDKra/HBEPF4t+1VlD4qI2RFxb0QsiYhrI6LoXxKkTZFB1nt05lrUc5rxL4G/Ab4EzAW+ABwAHA1cBJwKHJiZb0fEIcC/ACdUx44H9szM16ppRqpw+QFwDPAicCNwTGa+EhEnAf+cmV+KiC9Tm5ac11ajImIL4FbgpMycGxEfB94EXgYOzczmiNgZ+Bm1YASYCOwOPAfcBxwP3N6q3mnANIDhw4d38i2rXLaqa8dLUh9TzzB7NjOfAIiIJ4FfZmZGxBPACGAwcEMVHAm0/CTwgcx8rcXr3YAZwGGZ+UJEjAZGAw9UCb85tYDriFHAi5k5FyAzX6/auBVwVUTsBbwD7NLimDmZ+UxV7mfUQvkDYZaZM6o20tjYmB1si6Q6GHHBvd1a3/JvTe7W+vRh9ZwOW9ti/d0Wr9+lFrKXAw9l5mjgKKChRfk1rep6EWgGxlWvA3gyM/eqljGZeVgX2/u/gJeAsdRGZFu02Nc6nAwrSX3GlVdeyW677caUKVPq3ZR29ea7GQcDK6r10z6i7J+AM6iNxNYA/wFsExH7ZuZjEdEf2CUznwRWA4PWU9cSYIeImFBNMw6iNs04GGjKzHcjYiq10d46EyNiJLVpxpOoRmCS1BmZSWay2Wa94+P3a665hgcffJBhw4bVuynt6h3vVNuuAP53RPyODoRuZr4EHAlcTW2EdiLw7YhYCCwA9quKXg9c294NIJn539QC6QfVsQ9QGxVeA0yttu3KB0eHc4GrgMXAs8CdG9xbSZu05cuXM2rUKE499VRGjx7N5ZdfzoQJE9hzzz352te+9l65yy+/nFGjRnHAAQdw8skn853vfKfdOp9++mkOOeQQxo4dy/jx41m2bBmZybnnnsvo0aMZM2YMt95663vlp0+f/qFznnXWWTzzzDMcfvjhfPe73+25N6CL6jIyy8zl1D7TWvf6tHb2tfxc6uJq//XUAmld+V8Dv67W/xPYo8Uxk9o490xg5ke0by6wT6vNS4E9W7w+v8X66x39OzdJas/SpUu54YYbeP3117n99tuZM2cOmcnRRx/N7NmzGTBgADNnzmThwoW89dZbjB8/nr333rvd+qZMmcIFF1zAcccdR3NzM++++y533HEHCxYsYOHChbz66qtMmDCBSZMm8cQTT7B06dIPnfPaa6/lvvvu46GHHmLo0KEb8d3YML15mlGSNik77bQT++yzD1/96le5//77GTeudhvAG2+8wdKlS1m9ejXHHHMMDQ0NNDQ0cNRRR7Vb1+rVq1mxYgXHHXccUPtWDYBHHnmEk08+mc0335ztttuOz372s8ydO5fZs2e3ec5Jkz40JuiVNukwi4g7gdZ/Yn5+Zs7qaB0tR4aS1BVbbbUVUPvM7MILL+TMM8/8wP7vfe97PXbu9s5ZjHUfNLpsvGXvvfdOSb3HU089Ve8m5LPPPpt77LFHZmbOmjUrJ06cmKtXr87MzKampnzppZdyzpw5OW7cuHzzzTdz9erVufPOO+f06dPbrfMzn/lM3nnnnZmZ2dzcnGvWrMmZM2fmYYcdlm+//Xa+/PLLOXz48HzxxRfbPWdm5k477ZSvvPJKT3b/Q9q6JsC8bOfn6iY9MpOk3uiwww5j8eLF7LvvvgAMHDiQm266iQkTJnD00Uez5557st122zFmzBgGDx7cbj033ngjZ555Jpdeein9+/fntttu47jjjuOxxx5j7NixRARXXHEF22+/Pdtvv32b59x22203Sp+7Kmphp42psbEx581r8wtIJNXB4sWL2W233erdjA554403GDhwIH/+85+ZNGkSM2bMYPz48fVuVrdr65pExPzMbGyrvCMzSSrItGnTeOqpp2hubmbq1Kl9Msg6wzCTpILccsstH9p29tln8+ijj35g2znnnMPpp5++sZpVd4aZJBXu6quvrncT6q43fwOIJEkdYphJkopnmEmSiudnZpLU2mXt/+1W5+rzgbo9zZGZJGm9Ovs8s+uvv54vf/nLPdSqD3JkJkm9zLqvaPJ5Zh3XO94pSdrElfw8szlz5rDvvvsybtw49ttvP5YsWfLevueff56DDjqInXfema9//esArFmzhsmTJzN27FhGjx79gTZ0liMzSeolSn2e2a677srDDz9Mv379ePDBB7nooouYObP22Mg5c+awaNEittxySyZMmMDkyZN57rnn2HHHHbn33nsBWLWq658pGmaS1EuU+jyzVatWMXXqVJYuXUpE8NZbb72379BDD2XIkCEAHH/88TzyyCMcccQRfOUrX+H888/nyCOP5MADD+zcG9aC04yS1Eu0fp7ZggULWLBgAU8//TRnnHFGj567K+e85JJLOPjgg1m0aBH33HMPzc3N7+2LiA+UjQh22WUXHn/8ccaMGcPFF1/MN77xjS6335GZJLVW51vpP//5z3PJJZcwZcoUBg4cyIoVK+jfvz/7778/Z555JhdeeCFvv/02v/jFL5g2bVqbdQwaNIhhw4Zx1113ceyxx7J27VreeecdDjzwQH70ox8xdepUXnvtNWbPns306dNpaGho85wdeQTMqlWr+OQnPwnU7mBs6YEHHuC1115jwIAB3HXXXfzkJz/hhRde4BOf+ASnnHIKW2+9NT/+8Y+7/J4ZZpLUy5T2PLPzzjuPqVOn8s1vfpPJkyd/YN/EiRM54YQTaGpq4pRTTqGxsZFZs2Zx7rnnstlmm9G/f39++MMfduHdqvF5ZnXg88yk3sXnmfU+Ps9Mkvown2fWNsNMkgpSz+eZXXfddXz/+9//wLb999+/VzyCxjCTJGp387W+864UGytMTj/99I3ywM/OfPzlrfmSNnkNDQ2sXLmyUz9E1b0yk5UrV773d3Ed5chM0iZv2LBhNDU18corr9S7KaL2y8WGfg+kYSZpk9e/f39GjhxZ72aoC5xmlCQVzzCTJBXPMJMkFc9vAKmDiHgFeK4LVQwFXu2m5pTA/vZt9rdv687+7pSZ27S1wzArUETMa+8rXfoi+9u32d++bWP112lGSVLxDDNJUvEMszLNqHcDNjL727fZ375to/TXz8wkScVzZCZJKp5hJkkqnmHWC0TEX0fEkoh4OiIuaGP/xyLi1mr//42IES32XVhtXxIRn+9onfXUQ/1dHhFPRMSCiOhVj/HubH8jYkhEPBQRb0TEVa2O2bvq79MRcWX0omeX9FB/f13VuaBatt04vfloXejvoRExv7qO8yPir1oc0xev7/r62/Xrm5kudVyAzYFlwKeALYCFwO6tyvw9cG21/rfArdX67lX5jwEjq3o270idfam/1b7lwNB696+b+7sVcABwFnBVq2PmAPsAAfw7cHi9+9rD/f010Fjv/nVzf8cBO1bro4EVffz6rq+/Xb6+jszqbyLwdGY+k5n/DfwcOKZVmWOAG6r124HPVb+pHQP8PDPXZuazwNNVfR2ps156or+9Waf7m5lrMvMRoLll4YjYAfh4Zv42az8Jfgoc26O96Lhu728v15X+/i4zX6i2PwkMqEY1ffX6ttnf7mqYYVZ/nwSeb/G6qdrWZpnMfBtYBQxZz7EdqbNeeqK/AAncX01fTOuBdndWV/q7vjqbPqLOeumJ/q5zXTUFdUkvmnbrrv6eADyemWvZNK5vy/6u06Xr6/PM1FcckJkrqrn2ByLiD5k5u96NUreZUl3fQcBM4IvURizFi4g9gG8Dh9W7LRtDO/3t8vV1ZFZ/K4D/0eL1sGpbm2Uioh8wGFi5nmM7Ume99ER/ycx1/30ZuJPeM/3Ylf6ur86Wj+HtK9e3XS2u72rgFvrI9Y2IYdT+vZ6amctalO+T17ed/nbL9TXM6m8usHNEjIyILah9YHp3qzJ3A1Or9ROBX1Vz6XcDf1vNs48Edqb2wXFH6qyXbu9vRGxV/UZHRGxF7Te+RRuhLx3Rlf62KTNfBF6PiH2q6ZhTgX/r/qZ3Srf3NyL6RcTQar0/cCR94PpGxNbAvcAFmfnousJ99fq2199uu771vjvGJQGOAP5I7S6hf6q2fQM4ulpvAG6jdsPDHOBTLY79p+q4JbS446mtOnvL0t39pXZn1cJqebKP9Xc58BrwBrXPJ3avtjdW/8MvA66i+jaf3rB0d3+p3eU4H/h9dX2/T3UXa29YOttf4GJgDbCgxbJtX72+7fW3u66vX2clSSqe04ySpOIZZpKk4hlmkqTiGWaSpOIZZpKk4hlmkqTiGWaSpOL9f6U9m6c1RGegAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "log_reg_coef = pd.DataFrame(data = log_reg.coef_.flatten(), index = X_train.columns, columns=[\"reg_coef\"])\n",
        "log_reg_coef.loc[:, 'reg_coef_abs'] = log_reg_coef.reg_coef.abs()\n",
        "log_reg_coef.sort_values(by=\"reg_coef_abs\", ascending= True, inplace = True)\n",
        "log_reg_coef.plot.barh()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mZEfTr7nQWt_"
      },
      "source": [
        "# Step 6"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TofsYIGIFAQc"
      },
      "source": [
        "###Install optuna"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "92zGZg7KQZ5h",
        "outputId": "0161e7ea-04b3-4a82-cee1-95c5b29edee0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting optuna\n",
            "  Downloading optuna-3.0.5-py3-none-any.whl (348 kB)\n",
            "\u001b[K     |████████████████████████████████| 348 kB 5.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from optuna) (4.64.1)\n",
            "Requirement already satisfied: scipy<1.9.0,>=1.7.0 in /usr/local/lib/python3.8/dist-packages (from optuna) (1.7.3)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from optuna) (21.3)\n",
            "Collecting importlib-metadata<5.0.0\n",
            "  Downloading importlib_metadata-4.13.0-py3-none-any.whl (23 kB)\n",
            "Requirement already satisfied: sqlalchemy>=1.3.0 in /usr/local/lib/python3.8/dist-packages (from optuna) (1.4.45)\n",
            "Collecting cliff\n",
            "  Downloading cliff-4.1.0-py3-none-any.whl (81 kB)\n",
            "\u001b[K     |████████████████████████████████| 81 kB 9.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from optuna) (1.21.6)\n",
            "Requirement already satisfied: PyYAML in /usr/local/lib/python3.8/dist-packages (from optuna) (6.0)\n",
            "Collecting alembic>=1.5.0\n",
            "  Downloading alembic-1.9.0-py3-none-any.whl (210 kB)\n",
            "\u001b[K     |████████████████████████████████| 210 kB 55.3 MB/s \n",
            "\u001b[?25hCollecting cmaes>=0.8.2\n",
            "  Downloading cmaes-0.9.0-py3-none-any.whl (23 kB)\n",
            "Collecting colorlog\n",
            "  Downloading colorlog-6.7.0-py2.py3-none-any.whl (11 kB)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.8/dist-packages (from alembic>=1.5.0->optuna) (5.10.1)\n",
            "Collecting Mako\n",
            "  Downloading Mako-1.2.4-py3-none-any.whl (78 kB)\n",
            "\u001b[K     |████████████████████████████████| 78 kB 8.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata<5.0.0->optuna) (3.11.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->optuna) (3.0.9)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.8/dist-packages (from sqlalchemy>=1.3.0->optuna) (2.0.1)\n",
            "Collecting cmd2>=1.0.0\n",
            "  Downloading cmd2-2.4.2-py3-none-any.whl (147 kB)\n",
            "\u001b[K     |████████████████████████████████| 147 kB 78.3 MB/s \n",
            "\u001b[?25hCollecting autopage>=0.4.0\n",
            "  Downloading autopage-0.5.1-py3-none-any.whl (29 kB)\n",
            "Requirement already satisfied: PrettyTable>=0.7.2 in /usr/local/lib/python3.8/dist-packages (from cliff->optuna) (3.5.0)\n",
            "Collecting stevedore>=2.0.1\n",
            "  Downloading stevedore-4.1.1-py3-none-any.whl (50 kB)\n",
            "\u001b[K     |████████████████████████████████| 50 kB 6.8 MB/s \n",
            "\u001b[?25hCollecting pyperclip>=1.6\n",
            "  Downloading pyperclip-1.8.2.tar.gz (20 kB)\n",
            "Requirement already satisfied: attrs>=16.3.0 in /usr/local/lib/python3.8/dist-packages (from cmd2>=1.0.0->cliff->optuna) (22.1.0)\n",
            "Requirement already satisfied: wcwidth>=0.1.7 in /usr/local/lib/python3.8/dist-packages (from cmd2>=1.0.0->cliff->optuna) (0.2.5)\n",
            "Collecting pbr!=2.1.0,>=2.0.0\n",
            "  Downloading pbr-5.11.0-py2.py3-none-any.whl (112 kB)\n",
            "\u001b[K     |████████████████████████████████| 112 kB 31.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.8/dist-packages (from Mako->alembic>=1.5.0->optuna) (2.0.1)\n",
            "Building wheels for collected packages: pyperclip\n",
            "  Building wheel for pyperclip (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11136 sha256=dafecea2299549477cc6d5dee3687a1fb93bc8ac301fedf14885bdbe1441dc64\n",
            "  Stored in directory: /root/.cache/pip/wheels/7f/1a/65/84ff8c386bec21fca6d220ea1f5498a0367883a78dd5ba6122\n",
            "Successfully built pyperclip\n",
            "Installing collected packages: pyperclip, pbr, stevedore, Mako, importlib-metadata, cmd2, autopage, colorlog, cmaes, cliff, alembic, optuna\n",
            "  Attempting uninstall: importlib-metadata\n",
            "    Found existing installation: importlib-metadata 5.1.0\n",
            "    Uninstalling importlib-metadata-5.1.0:\n",
            "      Successfully uninstalled importlib-metadata-5.1.0\n",
            "Successfully installed Mako-1.2.4 alembic-1.9.0 autopage-0.5.1 cliff-4.1.0 cmaes-0.9.0 cmd2-2.4.2 colorlog-6.7.0 importlib-metadata-4.13.0 optuna-3.0.5 pbr-5.11.0 pyperclip-1.8.2 stevedore-4.1.1\n"
          ]
        }
      ],
      "source": [
        "!pip install optuna"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FGbL32REFFqD"
      },
      "source": [
        "###Define method \"objective\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q7-430OQbYDH",
        "outputId": "29641b8a-c8b4-4c3f-8fc0-97516e964270"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:04,225]\u001b[0m A new study created in memory with name: no-name-c3d51710-321a-45e2-b889-ad18e0a6434d\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 2ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:15,779]\u001b[0m Trial 0 finished with value: -0.4069961263015558 and parameters: {'n_layers': 1, 'n_units_l0': 45, 'a_fun_l0': 'sigmoid', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 3, 'training_epochs': 59}. Best is trial 0 with value: -0.4069961263015558.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 1s 3ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:22,169]\u001b[0m Trial 1 finished with value: -0.4624729894250661 and parameters: {'n_layers': 3, 'n_units_l0': 25, 'a_fun_l0': 'tanh', 'n_units_l1': 17, 'a_fun_l1': 'tanh', 'n_units_l2': 11, 'a_fun_l2': 'relu', 'loss_fun': 'mean_squared_error', 'optimizer': 'Adam', 'callback_patience': 4, 'training_epochs': 77}. Best is trial 0 with value: -0.4069961263015558.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 1s 3ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:29,277]\u001b[0m Trial 2 finished with value: -0.37012038329694175 and parameters: {'n_layers': 2, 'n_units_l0': 38, 'a_fun_l0': 'tanh', 'n_units_l1': 10, 'a_fun_l1': 'relu', 'loss_fun': 'mean_squared_error', 'optimizer': 'RMSprop', 'callback_patience': 2, 'training_epochs': 12}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 3ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:41,143]\u001b[0m Trial 3 finished with value: -0.527973061010137 and parameters: {'n_layers': 2, 'n_units_l0': 7, 'a_fun_l0': 'tanh', 'n_units_l1': 11, 'a_fun_l1': 'relu', 'loss_fun': 'mean_squared_error', 'optimizer': 'Adam', 'callback_patience': 3, 'training_epochs': 96}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:44,998]\u001b[0m Trial 4 finished with value: -0.4772370959614305 and parameters: {'n_layers': 3, 'n_units_l0': 35, 'a_fun_l0': 'sigmoid', 'n_units_l1': 23, 'a_fun_l1': 'sigmoid', 'n_units_l2': 10, 'a_fun_l2': 'sigmoid', 'loss_fun': 'mean_squared_error', 'optimizer': 'Adam', 'callback_patience': 3, 'training_epochs': 78}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:49,276]\u001b[0m Trial 5 finished with value: -0.4332240167493917 and parameters: {'n_layers': 2, 'n_units_l0': 8, 'a_fun_l0': 'relu', 'n_units_l1': 14, 'a_fun_l1': 'tanh', 'loss_fun': 'mean_squared_error', 'optimizer': 'Adam', 'callback_patience': 3, 'training_epochs': 62}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 2ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:43:55,740]\u001b[0m Trial 6 finished with value: -0.4885293094145702 and parameters: {'n_layers': 2, 'n_units_l0': 6, 'a_fun_l0': 'relu', 'n_units_l1': 49, 'a_fun_l1': 'tanh', 'loss_fun': 'huber_loss', 'optimizer': 'Adam', 'callback_patience': 3, 'training_epochs': 32}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:02,200]\u001b[0m Trial 7 finished with value: -0.5931244404062272 and parameters: {'n_layers': 2, 'n_units_l0': 8, 'a_fun_l0': 'relu', 'n_units_l1': 68, 'a_fun_l1': 'tanh', 'loss_fun': 'mean_absolute_error', 'optimizer': 'RMSprop', 'callback_patience': 5, 'training_epochs': 66}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:04,602]\u001b[0m Trial 8 finished with value: -0.7429512404817156 and parameters: {'n_layers': 2, 'n_units_l0': 42, 'a_fun_l0': 'sigmoid', 'n_units_l1': 14, 'a_fun_l1': 'sigmoid', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 1, 'training_epochs': 92}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:10,665]\u001b[0m Trial 9 finished with value: -0.521456076071924 and parameters: {'n_layers': 3, 'n_units_l0': 4, 'a_fun_l0': 'tanh', 'n_units_l1': 19, 'a_fun_l1': 'relu', 'n_units_l2': 42, 'a_fun_l2': 'relu', 'loss_fun': 'mean_squared_error', 'optimizer': 'Adam', 'callback_patience': 3, 'training_epochs': 22}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:12,772]\u001b[0m Trial 10 finished with value: -0.8146879959659172 and parameters: {'n_layers': 1, 'n_units_l0': 124, 'a_fun_l0': 'tanh', 'loss_fun': 'mean_absolute_error', 'optimizer': 'RMSprop', 'callback_patience': 1, 'training_epochs': 37}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:16,359]\u001b[0m Trial 11 finished with value: -0.5080772090129448 and parameters: {'n_layers': 1, 'n_units_l0': 63, 'a_fun_l0': 'sigmoid', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 2, 'training_epochs': 12}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:19,925]\u001b[0m Trial 12 finished with value: -0.418587256832025 and parameters: {'n_layers': 1, 'n_units_l0': 16, 'a_fun_l0': 'sigmoid', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 2, 'training_epochs': 44}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:22,102]\u001b[0m Trial 13 finished with value: -0.4383494290095271 and parameters: {'n_layers': 1, 'n_units_l0': 84, 'a_fun_l0': 'tanh', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 2, 'training_epochs': 51}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:27,147]\u001b[0m Trial 14 finished with value: -0.46702871282009406 and parameters: {'n_layers': 1, 'n_units_l0': 17, 'a_fun_l0': 'sigmoid', 'loss_fun': 'mean_absolute_error', 'optimizer': 'RMSprop', 'callback_patience': 4, 'training_epochs': 11}. Best is trial 2 with value: -0.37012038329694175.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 1ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:33,884]\u001b[0m Trial 15 finished with value: -0.35906717195610716 and parameters: {'n_layers': 1, 'n_units_l0': 49, 'a_fun_l0': 'tanh', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 4, 'training_epochs': 56}. Best is trial 15 with value: -0.35906717195610716.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 2ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:41,353]\u001b[0m Trial 16 finished with value: -0.539117359737707 and parameters: {'n_layers': 3, 'n_units_l0': 26, 'a_fun_l0': 'tanh', 'n_units_l1': 4, 'a_fun_l1': 'relu', 'n_units_l2': 87, 'a_fun_l2': 'tanh', 'loss_fun': 'mean_squared_error', 'optimizer': 'RMSprop', 'callback_patience': 5, 'training_epochs': 28}. Best is trial 15 with value: -0.35906717195610716.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 0s 2ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:44,493]\u001b[0m Trial 17 finished with value: -0.4166321821991561 and parameters: {'n_layers': 2, 'n_units_l0': 68, 'a_fun_l0': 'tanh', 'n_units_l1': 5, 'a_fun_l1': 'relu', 'loss_fun': 'mean_squared_error', 'optimizer': 'RMSprop', 'callback_patience': 4, 'training_epochs': 48}. Best is trial 15 with value: -0.35906717195610716.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 1s 4ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:44:58,947]\u001b[0m Trial 18 finished with value: -0.5354295907311524 and parameters: {'n_layers': 2, 'n_units_l0': 12, 'a_fun_l0': 'tanh', 'n_units_l1': 7, 'a_fun_l1': 'relu', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 2, 'training_epochs': 75}. Best is trial 15 with value: -0.35906717195610716.\u001b[0m\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 1s 5ms/step\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m[I 2022-12-21 05:45:09,875]\u001b[0m Trial 19 finished with value: -0.4637415499371549 and parameters: {'n_layers': 1, 'n_units_l0': 118, 'a_fun_l0': 'tanh', 'loss_fun': 'mean_absolute_error', 'optimizer': 'RMSprop', 'callback_patience': 4, 'training_epochs': 21}. Best is trial 15 with value: -0.35906717195610716.\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "import optuna\n",
        "\n",
        "def objective(trial):\n",
        "  n_layers = trial.suggest_int('n_layers',1,3)\n",
        "  my_model = tf.keras.Sequential()\n",
        "  my_model.add(tf.keras.layers.InputLayer(input_shape = X_train_s.shape[1]))\n",
        "\n",
        "  for i in range(n_layers):\n",
        "    num_hidden = trial.suggest_int(f'n_units_l{i}', 4, 128, log=True)\n",
        "    a_fun = trial.suggest_categorical(f'a_fun_l{i}', ['tanh','sigmoid','relu'])\n",
        "    my_model.add(tf.keras.layers.Dense(num_hidden, activation=a_fun))\n",
        "\n",
        "  my_model.add(tf.keras.layers.Dense(1, activation = output_fun))\n",
        "  loss_fun = trial.suggest_categorical(f'loss_fun', [\"mean_squared_error\",\"mean_absolute_error\",\"huber_loss\"])\n",
        "  optim= trial.suggest_categorical(f'optimizer', [\"Adam\",\"RMSprop\"])\n",
        "\n",
        "  my_model.compile(loss = loss_fun, optimizer=optim)\n",
        "\n",
        "  call_back_patience = trial.suggest_int(f'callback_patience',1,5)\n",
        "  my_callback = tf.keras.callbacks.EarlyStopping(monitor=\"val_loss\", patience = call_back_patience)\n",
        "\n",
        "  n_epochs = trial.suggest_int(f'training_epochs',10,100)\n",
        "  my_model.fit(X_train_s, y_train, epochs = n_epochs, validation_data=(X_test_s, y_test), callbacks = [my_callback], verbose=0)\n",
        "\n",
        "  y_test_pred = my_model.predict(X_test_s).flatten()\n",
        "  r2_test = r2_score(y_test,y_test_pred)\n",
        "  return r2_test\n",
        "\n",
        "study = optuna.create_study(direction=\"maximize\")\n",
        "study.optimize(objective, n_trials = 20 )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7qnEke0XFRaD"
      },
      "source": [
        "### Find best hparams and predict"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EEuFl-Wcj-29",
        "outputId": "c8b63194-f210-4228-df85-93fb9f1e8e1f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'n_layers': 1, 'n_units_l0': 49, 'a_fun_l0': 'tanh', 'loss_fun': 'huber_loss', 'optimizer': 'RMSprop', 'callback_patience': 4, 'training_epochs': 56}\n"
          ]
        }
      ],
      "source": [
        "best_hparams = study.best_trial.params\n",
        "print(best_hparams)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xdxO3wUujKcC",
        "outputId": "01984881-81ff-4926-8eee-fcb69368c552"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "136/136 [==============================] - 1s 6ms/step\n"
          ]
        }
      ],
      "source": [
        "best_hparams_iterator = iter(study.best_trial.params)\n",
        "\n",
        "n_layers = best_hparams[next(best_hparams_iterator)]\n",
        "\n",
        "my_model = tf.keras.Sequential()\n",
        "my_model.add(tf.keras.layers.InputLayer(input_shape=X_train.shape[1]))\n",
        "\n",
        "for i in range(n_layers):\n",
        "  num_hidden = best_hparams[next(best_hparams_iterator)]\n",
        "  a_fun = best_hparams[next(best_hparams_iterator)]\n",
        "  my_model.add(tf.keras.layers.Dense(num_hidden, activation = a_fun))\n",
        "\n",
        "my_model.add(tf.keras.layers.Dense(1, activation = output_fun))\n",
        "\n",
        "my_model.compile(loss = best_hparams[\"loss_fun\"], optimizer = best_hparams[\"optimizer\"])\n",
        "my_callback = tf.keras.callbacks.EarlyStopping(monitor=\"val_loss\", patience = best_hparams['callback_patience'])\n",
        "my_model.fit(X_train_s, y_train, epochs = best_hparams['training_epochs'], validation_data = (X_val_s, y_val), callbacks=[my_callback], verbose=0)\n",
        "\n",
        "y_test_pred_with_best_hparams =  my_model.predict(X_test_s).flatten()\n",
        "optuna_nn_r2_score = r2_score(y_test, y_test_pred_with_best_hparams)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T30k6MmGxiJo"
      },
      "source": [
        "### Mean-Absolute-Error"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7XbmypqqxkOo",
        "outputId": "91fb41ea-2286-4277-89a3-e0d845866a8e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.1474903333140406\n"
          ]
        }
      ],
      "source": [
        "best_hparams_mae = mean_absolute_error(y_test_pred_with_best_hparams, y_test)\n",
        "print(best_hparams_mae)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mf2wQO_m2EqJ"
      },
      "source": [
        "#Step 7"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-RyhNh_V8QOn"
      },
      "outputs": [],
      "source": [
        "#Optuna Model\n",
        "sum_test_optuna_model, sum_c_test_optuna_model = feature_analysis(X_test_tensor, my_model, X_test.columns)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mWDG01H18WqA",
        "outputId": "70662f3a-38c7-4e5a-f7a3-966c68514dc2"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "book_to_market    0.141830\n",
              "market_cap       -0.405634\n",
              "beta              0.452537\n",
              "Name: feature_importance, dtype: float64"
            ]
          },
          "execution_count": 42,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sum_test_optuna_model[\"feature_importance\"] \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "id": "p0CWsU0T93lY",
        "outputId": "38421243-ab12-415a-8832-9d200247c55d"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-aa16f3a4-10fd-4a9e-bf6d-f58c103ff1e1\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>book_to_market</th>\n",
              "      <th>market_cap</th>\n",
              "      <th>beta</th>\n",
              "      <th>feature_importance</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>book_to_market</th>\n",
              "      <td>0.625345</td>\n",
              "      <td>0.545998</td>\n",
              "      <td>0.529209</td>\n",
              "      <td>0.790248</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>market_cap</th>\n",
              "      <td>0.545998</td>\n",
              "      <td>0.657544</td>\n",
              "      <td>0.515639</td>\n",
              "      <td>0.999080</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>beta</th>\n",
              "      <td>0.529209</td>\n",
              "      <td>0.515639</td>\n",
              "      <td>0.526449</td>\n",
              "      <td>0.983901</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-aa16f3a4-10fd-4a9e-bf6d-f58c103ff1e1')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-aa16f3a4-10fd-4a9e-bf6d-f58c103ff1e1 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-aa16f3a4-10fd-4a9e-bf6d-f58c103ff1e1');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                book_to_market  market_cap      beta  feature_importance\n",
              "book_to_market        0.625345    0.545998  0.529209            0.790248\n",
              "market_cap            0.545998    0.657544  0.515639            0.999080\n",
              "beta                  0.529209    0.515639  0.526449            0.983901"
            ]
          },
          "execution_count": 43,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sum_c_test_optuna_model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AfaPy4-RyGRV"
      },
      "source": [
        "# STEP 8"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i9uF_LqDJuz1",
        "outputId": "736a68e4-41da-4c31-f85f-0cdf1481eaa2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Linear regression R2 score:-0.4750\n",
            "Neural network R2 score:-0.3843\n",
            "Optuna neural network R2 score:-0.5995\n"
          ]
        }
      ],
      "source": [
        "print(f'Linear regression R2 score:{lr_r2_score:.4f}')\n",
        "print(f'Neural network R2 score:{nn_r2_score:.4f}')\n",
        "print(f'Optuna neural network R2 score:{optuna_nn_r2_score:.4f}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2dopGMVZyNJe",
        "outputId": "7543f5f4-5d81-43a9-fc89-11eb08626a8f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "mean_absolute_error of linear regression: 0.13710582505046462\n",
            "\n",
            "mean_absolute_error of neural network: 0.13058082382271863\n",
            "\n",
            "mean_absolute_error of neural network with optuna: 0.1474903333140406\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(\"mean_absolute_error of linear regression: \" + str(lr_mae)+ \"\\n\")\n",
        "\n",
        "print(\"mean_absolute_error of neural network: \" + str(nn_mae)+ \"\\n\")\n",
        "\n",
        "print(\"mean_absolute_error of neural network with optuna: \" + str(best_hparams_mae)+ \"\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r5b2_ortFtLA"
      },
      "source": [
        "###Result\n",
        "\n",
        "\n",
        "\n",
        "*   Both models are well-trained and return results with a low margin of error.\n",
        "\n",
        "*   The neural network model is the model that gives the lowest mean absolute error value.\n",
        "\n",
        "*   Optuma NN trained would be expected to perform better than baseline NN.\n",
        "\n",
        "> The reasons may be too many hidden layers, too many epochs. This may change if the amount of hidden layer is reduced or the epoch value is reduced\n",
        "\n",
        "\n",
        "*   In general, all 3 models are close to each other and 0. This means that the 3 model can be used to analyze new data.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "3tzT1eTOy1WG",
        "8UL0T9NFy6bi",
        "k-gqx4Mx4tO7",
        "Mf2wQO_m2EqJ"
      ],
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}