main.another.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming Homework 1\n",
    "## Instructions\n",
    "- Do not import other libraries. You are only allowed to use Math,\n",
    "Numpy, Scipy packages which are already imported in the file.\n",
    "- Please follow the type annotations. There are some type annotations of the parameters of function calls and return values. Please use\n",
    "Python 3.5 or 3.6 (for full support for typing annotations). You can\n",
    "use Numpy/Scipy inside the function.  You have to make\n",
    "the functions’ return values converted to the required type.\n",
    "- In this programming assignment you will to implement **Linear Regression**, **K-Nearest Neighbours* and *Perceptron algorithm**. We provide the bootstrap code and you are expected to complete the **classes** and **functions**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 1: Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1.1 Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Implement** the classes in file *hw1_lr.py*.\n",
    "\n",
    "    - LinearRegression\n",
    "    - LinearRegressionWithL2Loss\n",
    "   \n",
    "and the function in file *utils.py*:\n",
    "    \n",
    "    - mean_squared_error\n",
    "\n",
    "For linear regression with l2 loss (a.k.a. Ridge loss), here are two useful links: \n",
    "\n",
    "- [a tutorial blog](https://goo.gl/iTX39z)\n",
    "- [a lecture slide](http://www.stat.cmu.edu/~ryantibs/datamining/lectures/16-modr1.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "from hw1_lr import LinearRegression, LinearRegressionWithL2Loss\n",
    "from utils import mean_squared_error\n",
    "\n",
    "\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1.2 Sanity test\n",
    "\n",
    "Do the following steps, as a simple test to check your model works correctly.\n",
    "\n",
    "- Load data (features and values) from function `generate_data_part_1`.\n",
    "- Create a LinearRegression model.\n",
    "- Train the model using the loaded data.\n",
    "- Calculate the MSE metric by your implementation of the mean squared error function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from data import generate_data_part_1\n",
    "features, values = generate_data_part_1()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.2]\tmse: 0.00175\n"
     ]
    }
   ],
   "source": [
    "model = LinearRegression(nb_features=1)\n",
    "model.train(features, values)\n",
    "\n",
    "mse = mean_squared_error(values, model.predict(features))\n",
    "print('[part 1.2]\\tmse: {mse:.5f}'.format(mse=mse))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x9276a90>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VOX1x/HPCQQS9iUgSwhB2SFhC5uAiMoigiDFKiqKrVK17a+2Sgu4UEGpLZZqKxZTd6vYWsKiIFtBEBQUZElICIZNkrAHQkISyHJ+f8xAA2YZYDKTmTnv1ysvZu59ZnJySb65uXPmeURVMcYYEziCvF2AMcYYz7LgN8aYAGPBb4wxAcaC3xhjAowFvzHGBBgLfmOMCTAW/MYYE2As+I0xJsBY8BtjTICp6u0CShIWFqaRkZHeLsMYY3zGli1bjqtqI1fGVsrgj4yMZPPmzd4uwxhjfIaIHHB1rF3qMcaYAGPBb4wxAcaC3xhjAkylvMZfkvz8fFJTU8nLy/N2KT4tJCSE8PBwgoODvV2KMcZLfCb4U1NTqV27NpGRkYiIt8vxSarKiRMnSE1NpVWrVt4uxxjjJT5zqScvL4+GDRta6F8FEaFhw4b2V5MxAc5ngh+w0HcDO4bGmHIv9YhIC+A9oAlQBMSq6iuXjJkE3FvsOTsAjVQ1Q0T2A1lAIVCgqjHuK98YY3zTwq1pzFqeTPqpXJrVC2XS0HaM7tbcI5/blTP+AuAJVe0A9AF+LiIdiw9Q1Vmq2lVVuwJTgLWqmlFsyCDnfgt9p1q1agGQnp7O2LFjyxz78ssvk5OTc1nP//nnnzNixIgrrs8YU3EWbk1jSlw8aadyUSDtVC5T4uJZuDXNI5+/3OBX1UOq+q3zdhaQBJT1a2kcMM895fmWwsLCy35Ms2bN+M9//lPmmCsJfmNM5TVreTK5+RfnRW5+IbOWJ3vk81/WNX4RiQS6AZtK2V8DGAbML7ZZgRUiskVEJl5Zmd63f/9+2rdvzwMPPEB0dDRjx44lJyeHyMhIpk+fTv/+/fn444/Zs2cPw4YNo0ePHgwYMIBdu3YBsG/fPvr27UvPnj155plnLnrezp07A45fHE8++SRRUVFER0fzt7/9jb/+9a+kp6czaNAgBg0aBMCKFSvo27cv3bt358477yQ7OxuAZcuW0b59e/r3709cXJyHj5AxxlXpp3Iva7u7udzOKSK1cAT646p6upRhI4ENl1zm6aeq6SLSGFgpIrtUdV0Jzz8RmAgQERFRZi3PfbKTxPTSSrgyHZvVYdrITmWOSU5O5s0336Rfv3785Cc/4bXXXgMcvfHr168H4Oabb2bu3Lm0adOGTZs28dhjj7F69Wp+9atf8eijj3L//fczZ86cEp8/NjaWffv2sXXrVqpWrUpGRgYNGjRg9uzZrFmzhrCwMI4fP87zzz/PqlWrqFmzJn/84x+ZPXs2v/3tb3n44YdZvXo1rVu35q677nLr8THGuE+zeqGklRDyzeqFeuTzu3TGLyLBOEL/A1Ut61Tybi65zKOq6c5/jwILgF4lPVBVY1U1RlVjGjVyaYI5j2vRogX9+vUD4L777rsQ9udDNjs7my+//JI777yTrl278rOf/YxDhw4BsGHDBsaNGwfA+PHjS3z+VatW8cgjj1C1quP3cYMGDX4wZuPGjSQmJtKvXz+6du3Ku+++y4EDB9i1axetWrWiTZs2iAj33Xefe794Y4zbTBrajtDgKhdtCw2uwqSh7Tzy+V3p6hHgTSBJVWeXMa4uMBC4r9i2mkCQqmY5bw8Bpl9t0eWdmVeUS1shz9+vWbMmAEVFRdSrV49t27a59PhLqapLYwYPHsy8eRe/jLJt2zZr1TTGR5zv3qnMXT39gPHATSKyzfkxXEQeEZFHio27A1ihqmeKbbsGWC8i24GvgSWqusxt1XvY999/z1dffQXAvHnz6N+//0X769SpQ6tWrfj4448BR0hv374dgH79+vHRRx8B8MEHH5T4/EOGDGHu3LkUFBQAkJHhuGJWu3ZtsrKyAOjTpw8bNmwgJSUFgJycHHbv3k379u3Zt28fe/bsuVCfMabyGt2tORsm30T8c0PZMPkmj4U+uNbVs15VRVWjz7dsqupSVZ2rqnOLjXtHVe++5LF7VbWL86OTqr5QEV+Ep3To0IF3332X6OhoMjIyePTRR38w5oMPPuDNN9+kS5cudOrUiUWLFgHwyiuvMGfOHHr27ElmZmaJz//QQw8RERFBdHQ0Xbp04cMPPwRg4sSJ3HrrrQwaNIhGjRrxzjvvMG7cOKKjo+nTpw+7du0iJCSE2NhYbrvtNvr370/Lli0r7kAYY65axplz/OZf2xj16nry8i+/I/BqiKp69BO6IiYmRi9diCUpKYkOHTp4qSJH982IESNISEjwWg3u4u1jaUwgU1UWbktjxqdJnM7N55GB1/HLm1tTvWqV8h9cBhHZ4up7pXxmkjZjjPF1BzNyeGphAut2H6Nri3q8+KMo2jep4/E6LPhdFBkZ6Rdn+8YYzysoLOLtDfuZvXI3QQK/H9mR8X0jqRLknYYMC35jjKlACWmZTImLJz4tk5vbN2bG6M4e69cvjQW/McZUgNxzhby8ajdvrN9H/RrVePWebtwW1bRStF1b8BtjjJut/+44UxfE831GDnfFtGDq8A7UrVF5Vr2z4DfGGDc5eeYczy9JYv63qbQKq8m8h/vQ97qG3i7rB3xqIRZfMHz4cE6dOlXmmGeffZZVq1Z5qCJjTEVTVRZtS+OW2WtZtC2Nnw+6js9+NaBShj7YGb/bqCqqytKlS8sdO336Vc9aYYypJA5m5PD0wgTW7j5Glxb1+OeYKDo09XyL5uXw2zP+hVvT6PfialpNXkK/F1e7ZYGD2bNn07lzZzp37szLL7/M/v376dChA4899hjdu3fn4MGDREZGcvz4cQBmzJhB+/btGTx4MOPGjeOll14CYMKECRfm4I+MjGTatGl0796dqKioC9M4G2Mqt8Ii5Y0v9jLkL+v4Zn8G00Z2JO7R6yt96IOfnvGfX93m/EIH51e3Aa54PowtW7bw9ttvs2nTJlSV3r17M3DgQJKTk3n77bcvTNF83ubNm5k/fz5bt26loKCA7t2706NHjxKfOywsjG+//ZbXXnuNl156iTfeeOOKajTGeEZi+mkmx+1gR2omNzlbNJt7uUXzcvhl8Je1us2VBv/69eu54447LszEOWbMGL744gtatmxJnz59Shw/atQoQkMd3wwjR44s9bnHjBkDQI8ePWwBFWMqsbz8Ql5e9R3/+GIv9WsE87dx3RgRXTlaNC+HXwZ/RaxuU9qcRud/Ebg6viTVq1cHoEqVKhdm5jTGVC4bUhwtmgdO5PDjmHCmDu9AvRrVvF3WFfHLa/ylvSvuat4td8MNN7Bw4UJycnI4c+YMCxYsYMCAAaWO79+/P5988gl5eXlkZ2ezZMmSK/7cxhjvOXnmHE9+vJ1739iEAB8+3Js/je3is6EPfnrGP2lou4uu8cPVr27TvXt3JkyYQK9ejgXEHnroIerXr1/q+J49e3L77bfTpUsXWrZsSUxMDHXr1r3iz2+M8SxV5ZMdh3hu8U4yc/N57Mbr+L+b2xASfHWzaFYGfjst88KtaV5b3ea87OxsatWqRU5ODjfccAOxsbF0797dozWUxKZlNoGuvHxIO5XL0wviWZN8jC7hdfnDmGg6Nqvc3To2LTOO7h1PB/2lJk6cSGJiInl5eTzwwAOVIvSNCXRldf2N7NKMd7/cz0srkgF4ZkRHJlzvvVk0K4rfBn9lcH4FLWNM5VFa19/MpUm8/eV+th88xY3tGvH86M6E16/hpSorVrkv7opICxFZIyJJIrJTRH5VwpgbRSSz2Jq8zxbbN0xEkkUkRUQmX02xlfGylK+xY2gCXWndfUezzpKakcMrd3fl7Qk9/Tb0wbUz/gLgCVX9VkRqA1tEZKWqJl4y7gtVHVF8g4hUAeYAg4FU4BsRWVzCY8sVEhLCiRMnaNiwoc/1zFYWqsqJEycICQnxdinGeE2zeqGklRD+NapVYdVvBlK/pu9267iq3OBX1UPAIeftLBFJApoDroR3LyBFVfcCiMhHwCgXH3uR8PBwUlNTOXbs2OU+1BQTEhJCeHi4t8swxmsmDW3H5Pk7yCsourCtWpUgZt4RFRChD5d5jV9EIoFuwKYSdvcVke1AOvCkqu7E8QviYLExqUDvKyk0ODiYVq1aXclDjTEGcPzVWyVIqFolCJzB37RuCL8b1t7rzSCe5HLwi0gtYD7wuKqevmT3t0BLVc0WkeHAQqANUNI1mRIvMovIRGAiQEREhKtlGWOMS9JO5fLMwgRW7zpKdHhd/jAmik7NAvO9NS4Fv4gE4wj9D1T1B5PJFP9FoKpLReQ1EQnDcYbfotjQcBx/EfyAqsYCseDo43f5KzDGmDIUFinvf7WfWcuTKVJ4+rYOTLg+0nHWH6DKDX5xvJL6JpCkqrNLGdMEOKKqKiK9cHQLnQBOAW1EpBWQBtwN3OOu4o0xpiy7Dp9m8vx4th08xcC2jhbNFg38t1vHVa6c8fcDxgPxIrLNuW0qEAGgqnOBscCjIlIA5AJ3q6NvsEBEfgEsB6oAbzmv/RtjTIXJyy/k1dUpzF27hzqhwbxyd1du79LMOgKdfGbKBmOMccXGvSeYGhfP3uNnGNO9OU/f1pEGAdCtY1M2GGMCTmZOPn/4LImPvjlIiwahvP/TXgxo08jbZVVKFvzGGJ+mqiyNP8y0xTs5mXOOn91wLY/f0pbQar4/i2ZFseA3xvis9FO5PLsogVVJR+ncvA7vPNiTzs0Ds0XzcljwG2N8TmGR8s+NB/jTsl0UqvLU8A482C+wWzQvhwW/Mcan7D6Sxe/m72Dr96cY0CaMmXdEWYvmZbLgN8b4hLz8Ql5bk8Lf1+6hVvWq/OWuLozu2txaNK+ABb8xptL7el8Gk+N2sPfYGcZ0a87TIwKjRbOiWPAbYyqtzNx8XvxsF/O+/p7w+qG895Ne3NDWWjSvlgW/MabSUVWWJThaNI9nn+XhAa349eC21KhmkeUOdhSNMZXK4cw8nlmUwMrEI3RqVoc3H+hJVLi1aLqTBb8xplIoKlI+2HSAPy5LpqCoiCm3tuen/VtZi2YFsOA3xnjdd0eymBwXz5YDJ+nfOowX7uhMy4Y1vV2W37LgN8Z41MKtacxankz6qVya1g0hKrwuq3cdpWb1qvz5zi6M6W4tmhXNgt8Y4zELt6YxJS6e3PxCANIz80jPzKNHy/rEju9Bw1rVvVxhYLCLZ8YYj5m1PPlC6Bd3ODPPQt+DLPiNMR6Tdiq3xO3ppWw3FcOC3xhT4Y6czuNn75e+uFKzeqEerMZY8BtjKkyRcxbNW/68ls+TjzEiuikhVS+OndDgKkwa2s5LFQYmVxZbbwG8BzQBioBYVX3lkjH3Ar9z3s0GHlXV7c59+4EsoBAocHVpMGOMb0s5msWUuHi+2X+S669ryMw7oogMq3lRV0+zeqFMGtqO0d2ae7vcgOJKV08B8ISqfisitYEtIrJSVROLjdkHDFTVkyJyKxAL9C62f5CqHndf2caYyupcQRF//3wPc9akEFqtCrPGRjO2R/iFFs3R3Zpb0HtZucGvqoeAQ87bWSKSBDQHEouN+bLYQzYC4W6u0xjjA7YcyGDy/Hi+O5rN7V2a8ezIjoRZt06lc1l9/CISCXQDNpUx7KfAZ8XuK7BCRBR4XVVjS3nuicBEgIiIiMspyxjjZVl5+fxpWTL/3HSAZnVDeXtCTwa1b+ztskwpXA5+EakFzAceV9XTpYwZhCP4+xfb3E9V00WkMbBSRHap6rpLH+v8hRALEBMTo5fxNRhjvGjFzsM8u2gnR7PyePD6VjwxpC01q9t7Qyszl/53RCQYR+h/oKpxpYyJBt4AblXVE+e3q2q689+jIrIA6AX8IPiNMb7l6Ok8pi3eyWcJh2nfpDavj+9Blxb1vF2WcYErXT0CvAkkqersUsZEAHHAeFXdXWx7TSDI+dpATWAIMN0tlRtjvKKoSPnom4P84bMkzhUU8dth7Xh4wLUE2yyaPsOVM/5+wHggXkS2ObdNBSIAVHUu8CzQEHjN+cr9+bbNa4AFzm1VgQ9VdZlbvwJjjMfsOZbNlPnxfL0/g77XNmTmmChahdksmr7Gla6e9UCZU+Wp6kPAQyVs3wt0ueLqjDGVwrmCIuau3cOrqx0tmn8aG82dxVo0jW+xV2CMMWXacuAkU+J2sPtINiOimzJtZCca1bYWTV9mwW+MKVH22QJmLdvFexsP0LROCG9NiOGm9td4uyzjBhb8xpgfWJV4hGcWJXD4dB4P9I3kyaHtqGUtmn7D/ieNMRcczcrjucWJLIk/RPsmtXnt3u50i6jv7bKMm1nwG2NQVf71zUFmLk0ir6CISUPbMfEGa9H0Vxb8xgS4vceymRIXz6Z9GfS5tgEz74ji2ka1vF2WqUAW/MYEqHMFRcSu28NfV6cQUjWIP/4oih/HtLAWzQBgwW9MANr6/Ukmz48n+UgWt0U3ZdrIjjSuHeLtsoyHWPAbE0Cyzxbw0vJk3v1qP03qhPDG/THc0tFaNAONBb8xAeK/SUd44uPtnMrJv7At+2yBFysy3mLBb4yfO5Z1luc+2cmnOw5dNPfKocw8psTFA9iKWAHGerWM8VOqyr+/Ocgts9eyYucRaodU5dKFLnLzC5m1PNkr9RnvseA3xg/tO36Ge/6xid/O30G7JrX57PEBZOeVfFkn/VSuh6sz3maXeozxI/mFRcSu28tf//sd1aoG8YcxUdwV04KgIKFZvVDSSgj5ZvVCvVCp8SYLfmP8xLaDp5g8fwe7DmcxPKoJvx/ZicZ1/teiOWloO6bExZObX3hhW2hwFSYNbeeNco0XWfAb4+POnC3gpRXJvPvlfhrXDuEf98cwuIQWzfMv4M5ankz6qVya1Qtl0tB29sJuALLgN8aHrdl1lKcXJpCemcv4Pi2ZNLQdtUOCSx0/ultzC3pT/ou7ItJCRNaISJKI7BSRX5UwRkTkryKSIiI7RKR7sX0PiMh3zo8H3P0FGBOIjmef5ZfztvLgO99Qo1oV/vNIX6aP6lxm6Btznitn/AXAE6r6rYjUBraIyEpVTSw25lagjfOjN/B3oLeINACmATGAOh+7WFVPuvWrMCZAqCofb0nlhSVJ5J4r5DeD2/LIwOuoVtUa9IzrXFlz9xBwyHk7S0SSgOZA8eAfBbynqgpsFJF6ItIUuBFYqaoZACKyEhgGzHPrV2FMADhw4gxTF8SzIeUEPSPr84cx0bRubLNomst3Wdf4RSQS6AZsumRXc+Bgsfupzm2lbTfGuCi/sIg3vtjHy6t2U61KEC/c0ZlxPSMICrJZNM2VcTn4RaQWMB94XFVPX7q7hIdoGdtLev6JwESAiIgIV8syxq/tSD3F7+bHk3ToNMM6NeG5UZ24po7NommujkvBLyLBOEL/A1WNK2FIKtCi2P1wIN25/cZLtn9e0udQ1VggFiAmJqbEXw7GBIqccwX8ecVu3t6wj0a1q/P6+B4M7dTE22UZP1Fu8ItjVYY3gSRVnV3KsMXAL0TkIxwv7maq6iERWQ7MFJHzi3YOAaa4oW5j/NbnyUd5akECaadyua9PBL8d1p461q1j3MiVM/5+wHggXkS2ObdNBSIAVHUusBQYDqQAOcCDzn0ZIjID+Mb5uOnnX+g1xlzsRPZZpn+ayKJt6bRuXIv/PNKXmMgG3i7L+CFXunrWU/K1+uJjFPh5KfveAt66ouqMCQCqyvxv03h+SSJnzhbw+C1tePTG66hetYq3SzN+yt65a4wXHThxhqcWJLA+5TgxLevz4o+iaN24trfLMn7Ogt8YLygoLOLN9fv4y6rdVA0K4vnRnbmnl7VoGs+w4DfGwxLSMvnd/B3sTD/NkI7XMH1UZ5rUtRZN4zkW/MZ4SM65Av6ycjdvrt9HWK3qzL2vO8M6N/V2WSYAWfAb4wHrdh/jqYXxHMzI5Z7eEfxuWHvqhlqLpvEOm9nJmAqUceYcv/7XNu5/62vO5hcRVqsa8zZ9z/BXvmDh1jRvl2cClJ3xG1MBVJUFW9OY8Wki2WcLGNrpGtYmHyOvoAiAtFO5TImLB7D58Y3H2Rm/MW52MCOH+9/6mt/8ezutwmqy5P8GkJB2+kLon5ebX8is5cleqtIEMjvjN8ZNCgqLeGvDPmavdLRozhjViXt7tyQoSEgvYZFzoNTtxlQkC35j3CAhLZPJcTtISDvNLR2uYcboTjStG3phf7N6oaSVEPLN6oX+YJsxFc0u9RhzFXLPFTJzaRKj5mzgyOmzvHZvd/5xf4+LQh9g0tB2hAZfPAVDaHAVJg1t58lyjQHsjN+YK/bFd8eYusDRojmuVwsmD+tA3Rolt2iefwF31vJk0k/l0qxeKJOGtrMXdo1XWPAbc5kyzpzj+SWJxH2bxrVhNfloYh/6XNuw3MeN7tbcgt5UChb8xrhIVVm0LZ3pnyZyOjefX97Ump8Pak1IsM2iaXyLBb8xLjiYkcPTCxNYu/sY3SLq8eKYaNo1sVk0jW+y4DemDAWFRbzz5X7+vGI3QQLP3d6J+/q0pIrNoml8mAW/MaXYmZ7J5PnxxKdlckuHxkwf1dnaL41fsOA35hK55wp5+b+7eeOLfdSvUY0593RneFQTHMtPG+P7XFls/S1gBHBUVTuXsH8ScG+x5+sANHKut7sfyAIKgQJVjXFX4cZUhA0px5m6IJ4DJ3K4K6YFU4eX3qJpjK9y5Yz/HeBV4L2SdqrqLGAWgIiMBH59yYLqg1T1+FXWaUyFOnnmHM8vSWL+t6m0CqvJvIf70Pe68ls0jfFFriy2vk5EIl18vnHAvKspyBhPUlUWb09n+ieJZObm8/NB1/HLm9pYi6bxa267xi8iNYBhwC+KbVZghYgo8Lqqxpbx+InARICIiAh3lWVMqVJPOlo0P08+RpcW9fjnmCg6NK3j7bKMqXDufHF3JLDhkss8/VQ1XUQaAytFZJeqrivpwc5fCrEAMTEx6sa6jLlIYZE6WzQdUyJPG9mR+/tGWoumCRjuDP67ueQyj6qmO/89KiILgF5AicFvjCckpp9mStwOtqdmclP7xswY3Znm1qJpAoxbgl9E6gIDgfuKbasJBKlqlvP2EGC6Oz6fMZcrL7+QV/77HbHr9lK/RjB/G9eNEdFNrUXTBCRX2jnnATcCYSKSCkwDggFUda5z2B3AClU9U+yh1wALnD9YVYEPVXWZ+0o3xjVfOls095/I4ccx4Uwd3oF6Nap5uyxjvMaVrp5xLox5B0fbZ/Fte4EuV1qYMVfrVM45XliSxMdbUmnZsAYfPtSb61uHebssY7zO3rlr/I6q8smOQ0z/ZCcnc/K5uUNjktJPc+8bm2wefGOw4Dd+Ju1ULs8sTGD1rqN0Ca/Lg9e34tU1KeTmF17YPyUuHsDC3wQsW3rR+IXCIuWt9fsYPHstG/ee4JkRHYl7rB8ffv39hdA/Lze/kFnLk71UqTHeZ2f8xuftOnya382PZ/vBU9zYrhHPj+5MeP0aAKSXsMB5WduNCQQW/MZn5eUX8rfV3/H62r3UDQ3mlbu7cnuXZhe1aDarF0paCSFv0yubQGbBb3zSV3tOMHVBPPuOn2Fsj3CeGt6B+jV/2KI5aWg7psTFX3S5JzS4CpOGtvNkucZUKhb8xqdk5uQzc2kS/9p8kJYNa/DBQ73pV0aL5vkXcGctTyb9VK519RiDBb/xEarKkvhD/H5xIidzzvHIwOv41c1tCK1W/iyao7s1t6A3phgLflPppTtbNP+76yjR4XV59yc96dSsrrfLMsZnWfCbSquwSHn/q/3MWp5MkcLTt3VgwvWRVK1iXcjGXA0LflMpJR/OYnLcDrZ+f4ob2jbihdGdadGghrfLMsYvWPCbSiUvv5BXV6cwd+0e6oQG8/JdXRnVtZnNommMG1nwm0pj494TTI2LZ+/xM4zp3pynb+tIgxJaNI0xV8eC33hdZm4+L36WxLyvD9KiQSjv/7QXA9o08nZZxvgtC37jNarKZwmHmbZ4JxlnzvGzG67l8VvautSiaYy5chb8xisOZebyzMKdrEo6QufmdXh7Qk86N7cWTWM8wYLfeFRRkfLPTQf407JkCoqKeGp4Bx7sZy2axnhSuT9tIvKWiBwVkYRS9t8oIpkiss358WyxfcNEJFlEUkRksjsLN75n95Esxs79kmcX7aRbRD1W/nogD99wrYW+MR7myhn/O8CrwHtljPlCVUcU3yAiVYA5wGAgFfhGRBarauIV1mp81NmCQuas2cPfP0+hVvWq/OWuLozu2txaNI3xElfW3F0nIpFX8Ny9gBTn2ruIyEfAKMCCP4B8vS+DyXE72HvsDGO6NefpEdaiaYy3uesaf18R2Q6kA0+q6k6gOXCw2JhUoLebPp+p5BwtmruY9/X3hNcP5d2f9GJgW2vRNKYycEfwfwu0VNVsERkOLATaACX9Ha+lPYmITAQmAkRERLihLOMtyxIO8eyinRzPPsugdo3YdTiLCW99bVMiG1NJXPWraqp6WlWznbeXAsEiEobjDL9FsaHhOP4iKO15YlU1RlVjGjWyM0NfdDgzj4nvbeaRf35Lo9rV+fUtbdm4N4NDmXko/1vofOHWNG+XakxAu+rgF5Em4nyVTkR6OZ/zBPAN0EZEWolINeBuYPHVfj5T+RQVKe9vPMDg2WtZ990xptzankU/78dH3xy0hc6NqYTKvdQjIvOAG4EwEUkFpgHBAKo6FxgLPCoiBUAucLeqKlAgIr8AlgNVgLec1/6NH/nuSBZT4uLZfOAk/VuH8cIdnWnZsCZgC50bU1m50tUzrpz9r+Jo9yxp31Jg6ZWVZiqzswWFvLZmD699nkLN6lX5851dGNO9uS10bowPsHfumsu2eX8Gk+PiSTmazeiuzXhmREca1qr+g3G20LkxlZMFv3HZ6bx8/vjZLj7Y9D3N64XyzoM9ubFd41LH20LnxlROFvzGJcsSDjNtcQLHss7yUP9W/GZIW2pUK//bxxY6N6byseA3ZTpyOo9nFyWwfOcROjStwz/ujyE6vJ63yzLGXAULflOioiJlC18kAAAMc0lEQVTlw6+/54+f7eJcYRG/G9aehwa0ItgmVDPG51nwmx9IOZrNlLgdfLP/JNdf15CZd0QRGVbT22UZY9zEgt9ccK6giL9/voc5a1IIrVaFWWOjGdsj3GbRNMbPWPAbALYcyGDy/Hi+O5rN7V2a8ezIjoSV0KJpjPF9FvwBLisvnz8tS+afmw7QrG4ob0/oyaD2pbdoGmN8nwV/AFux8zDPLtrJ0aw8Hry+FU8MaUvN6vYtYYy/s5/yAHT0dB7TFu/ks4TDtG9Sm9fH96BLC2vRNCZQWPAHkKIi5aNvDvKHz5I4W1DEb4e14+EB11qLpjEBxoI/QOw5ls2UuHi+3pdB32sbMnNMFK2sRdOYgGTB7+fOFRQxd+0eXl3taNH804+iuTPGWjSNCWQW/H7s5ZW7eXVNCgVFSmhwFZ4c0pYf92xR/gONMX7Ngt8PZZ8t4NH3t/BFyvEL23LzC5m5dBe1Q4Jt0jRjApy9qudnViUeYfDstReF/nm27KExBuyM328czcrjucWJLIk/RPsmtTmUmVfiOFv20BhT7hm/iLwlIkdFJKGU/feKyA7nx5ci0qXYvv0iEi8i20RkszsLNw6qykdff88tf17LyqQjTBrajk9+2Z/mpSxvaMseGmNcudTzDjCsjP37gIGqGg3MAGIv2T9IVbuqasyVlWhKs/dYNnfHbmRyXDwdmtZh2a8G8PNBrQmuEsSkoe0IDa5y0Xhb9tAYA64ttr5ORCLL2P9lsbsbgfCrL8uU5VxBEbHr9vDX1SmEVA3ixTFR/DimBUFB/2vRtGUPjTGlcfc1/p8CnxW7r8AKEVHgdVW99K+BC0RkIjARICIiws1l+Y+t359k8vx4ko9kcVtUU6bd3pHGtUNKHGvLHhpjSuK24BeRQTiCv3+xzf1UNV1EGgMrRWSXqq4r6fHOXwqxADExMequuvxF9tkCXlqezLtf7adJnRDeuD+GWzpe4+2yjDE+yC3BLyLRwBvArap64vx2VU13/ntURBYAvYASg9+U7r9JR3hmYQKHTudxf5+WPDm0HbVDgr1dljHGR1118ItIBBAHjFfV3cW21wSCVDXLeXsIMP1qP18gOZZ1luc+2cmnOw7R9ppa/Oee6+nRsr63yzLG+Lhyg19E5gE3AmEikgpMA4IBVHUu8CzQEHjNOf9LgbOD5xpggXNbVeBDVV1WAV+D31FVPt6cygtLk8g9V8gTg9vys4HXUa2qvd/OGHP1XOnqGVfO/oeAh0rYvhfo8sNHmLLsO36GqXHxfLX3BL0iGzBzTBStG9fydlnGGD9i79ytJPILi4hdt5dX/vsd1asGMfOOKO7ueXGLpjHGuIMFfyWw7eApJs/fwa7DWdzauQnP3d6JxnVKbtE0xpirZcHvRWfOFvDSimTe/XI/jWuHEDu+B0M6NfF2WcYYP2fB7yVrdh3l6YUJpGfmcl/vlvx2mLVoGmM8w4Lfw45nn+W5TxL5ZHs6rRvX4j+P9KVHywbeLssYE0As+CvIwq1pF82T8+SQtuQXKS8scbRoPn5LGx698TqqV61S/pMZY4wbWfBXgIVb05gSF09ufiEAaadyeeLj7RQpxLSsz4s/iqJ149pertIYE6gs+CvArOXJF0L/vCKFeqHB/Ptnfa1F0xjjVfZW0ApQ2ipXmbn5FvrGGK+z4HeznHMF1Khe8nV7W/3KGFMZWPC70efJRxk8ex1nzhZS5ZIze1v9yhhTWdg1fjc4kX2W6Z8msmhbOtc1qsnHj/Ql7WSurX5ljKmULPivgqoy/9s0nl+SyJmzBfzfzW34+SBHi2bPSCzojTGVkgX/FTpw4gxPLUhgfcpxerSszx/GRNH2GmvRNMZUfhb8l6mgsIg31u/j5VW7qRoUxIxRnbi3d0vr1jHG+AwL/ssQn5rJ5Lgd7Ew/zeCO1zB9VCea1rVOHWOMb7Hgd0HOuQL+snI3b67fR8Na1fn7vd0Z1rkJztXFjDHGp7jUzikib4nIURFJKGW/iMhfRSRFRHaISPdi+x4Qke+cHw+4q3BPWbf7GEP+so5/fLGPu3pGsOo3A7k1qqmFvjHGZ7l6xv8O8CrwXin7bwXaOD96A38HeotIAxxr9MYACmwRkcWqevJqii7PpROkXUkrZcaZc8z4NJEFW9O4tlFN/jWxD72vbVhBFRtjjOe4FPyquk5EIssYMgp4T1UV2Cgi9USkKY5F2leqagaAiKwEhgHzrqbospQ0QdqUuHjAtfZKVWXB1jRmfJpI9tkC/u+m1jw2qDUhwTaLpjHGP7jrGn9z4GCx+6nObaVtrzAlTZCWm1/IrOXJ5Qb/wYwcpi6I54vvjtMtoh5//FG0tWgaY/yOu4K/pAveWsb2Hz6ByERgIkBERMQVF1LaBGmlbQdHi+ZbG/Yxe+Vuqojw3O2duK9Pyx9Mu2CMMf7AXcGfCrQodj8cSHduv/GS7Z+X9ASqGgvEAsTExJT4y8EVzeqFklZCyJc2QVpCmqNFMyHtNLd0aMz0UZ1tMjVjjF9z1yRti4H7nd09fYBMVT0ELAeGiEh9EakPDHFuqzCThrYj9JLr8SVNkJZ7rpCZS5MYNWcDhzPPMuee7vzj/hgLfWOM33PpjF9E5uE4cw8TkVQcnTrBAKo6F1gKDAdSgBzgQee+DBGZAXzjfKrp51/orSjnr+OX1dXzxXfHmLognoMZudzdswVTbu1A3Rq20LkxJjCIoxGncomJidHNmze7/Xkzzpzj+SWJxH2bxrVhNZk5Joo+1qJpjPEDIrJFVWNcGRsQ79xVVRZtS2f6p4mczs3nF4Na84ubrEXTGBOY/D74D2bk8PTCBNbuPkbXFvV48UdRtG9Sx9tlGWOM1/ht8BcUFvHOl/v584rdBAn8fmRHxveNtBZNY0zA88vgzzhzjglvf82O1Exubt+YGaOtRdMYY87zy+CvFxpMiwY1eHjAtYyItgnVjDGmOL8M/qAgYc493csfaIwxAchdb+AyxhjjIyz4jTEmwFjwG2NMgLHgN8aYAGPBb4wxAcaC3xhjAowFvzHGBBgLfmOMCTCVclpmETkGHHDDU4UBx93wPL7OjoODHYf/sWPh4E/HoaWqNnJlYKUMfncRkc2uzk/tz+w4ONhx+B87Fg6BehzsUo8xxgQYC35jjAkw/h78sd4uoJKw4+Bgx+F/7Fg4BORx8Otr/MYYY37I38/4jTHGXMIvgl9EholIsoikiMjkEvZXF5F/OfdvEpFIz1dZ8Vw4Dr8RkUQR2SEi/xWRlt6os6KVdxyKjRsrIioiftnV4cpxEJEfO78ndorIh56u0VNc+NmIEJE1IrLV+fMx3Bt1eoyq+vQHUAXYA1wLVAO2Ax0vGfMYMNd5+27gX96u20vHYRBQw3n70UA9Ds5xtYF1wEYgxtt1e+n7oQ2wFajvvN/Y23V78VjEAo86b3cE9nu77or88Icz/l5AiqruVdVzwEfAqEvGjALedd7+D3Cz+N96jOUeB1Vdo6o5zrsbgXAP1+gJrnw/AMwA/gTkebI4D3LlODwMzFHVkwCqetTDNXqKK8dCgTrO23WBdA/W53H+EPzNgYPF7qc6t5U4RlULgEygoUeq8xxXjkNxPwU+q9CKvKPc4yAi3YAWqvqpJwvzMFe+H9oCbUVkg4hsFJFhHqvOs1w5Fr8H7hORVGAp8EvPlOYd/rDmbkln7pe2Krkyxte5/DWKyH1ADDCwQivyjjKPg4gEAX8BJniqIC9x5fuhKo7LPTfi+OvvCxHprKqnKrg2T3PlWIwD3lHVP4tIX+B957EoqvjyPM8fzvhTgRbF7ofzwz/TLowRkao4/pTL8Eh1nuPKcUBEbgGeAm5X1bMeqs2TyjsOtYHOwOcish/oAyz2wxd4Xf25WKSq+aq6D0jG8YvA37hyLH4K/BtAVb8CQnDM4+OX/CH4vwHaiEgrEamG48XbxZeMWQw84Lw9Flitzldx/Ei5x8F5ieN1HKHvr9dzyzwOqpqpqmGqGqmqkThe67hdVTd7p9wK48rPxUIcL/gjImE4Lv3s9WiVnuHKsfgeuBlARDrgCP5jHq3Sg3w++J3X7H8BLAeSgH+r6k4RmS4itzuHvQk0FJEU4DdAqS1+vsrF4zALqAV8LCLbROTSb36f5+Jx8HsuHoflwAkRSQTWAJNU9YR3Kq44Lh6LJ4CHRWQ7MA+Y4IcnhxfYO3eNMSbA+PwZvzHGmMtjwW+MMQHGgt8YYwKMBb8xxgQYC35jjAkwFvzGGBNgLPiNMSbAWPAbY0yA+X/uOCRuvwyI7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8a55f10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter([x[0] for x in features], values, label='origin');\n",
    "plt.plot([x[0] for x in features], model.predict(features), label='predicted');\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1.3 Feature Engineering\n",
    "In this part, we are following the same procedure as that in part \n",
    "\n",
    "a). First we will try the same process as above.\n",
    "We will try two cases : \n",
    "- No extra features\n",
    "- Adding polynomial features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1.3.1 No extra features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from data import generate_data_part_2\n",
    "features, values = generate_data_part_2()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.3.1]\tmse: 0.39997\n"
     ]
    }
   ],
   "source": [
    "model = LinearRegression(nb_features=1)\n",
    "model.train(features, values)\n",
    "\n",
    "mse = mean_squared_error(values, model.predict(features))\n",
    "print('[part 1.3.1]\\tmse: {mse:.5f}'.format(mse=mse))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x92ff790>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VdW99/HPygCBEAgQZggBEuaAhKAgEUFCtYoTxVZaB25tUbEVejtc79PndrD3trW9D6Moxql6S22LKNZqB0JAZFQQLJYhA2FIAIEAIfNwsp4/knAZErJPkjPlfN+vFy8yrLP3b58D3+ys/TtrG2stIiISOEJ8XYCIiLhHwS0iEmAU3CIiAUbBLSISYBTcIiIBRsEtIhJgFNwiIgGmyeA2xgwzxuy55M8FY8xCbxQnIiJXM+68AccYEwrkAzdYa494rCoREWlUmJvjpwM5TYV2TEyMjYuLa3ZRIiLBZteuXWestT2cjHU3uO8H3mhqUFxcHDt37nRz0yIiwcsY43gWw/HFSWNMO+AuYHUj359njNlpjNl5+vRpp5sVERE3udNV8kXgE2vt5w1901qbZq1NttYm9+jh6GxfRESawZ3gnoODaRIREfEsR3PcxpiOwAzg0ebuqKqqiry8PMrLy5u7CQEiIiLo378/4eHhvi5FRHzEUXBba0uB7i3ZUV5eHlFRUcTFxWGMacmmgpa1loKCAvLy8hg0aJCvyxERH/HaOyfLy8vp3r27QrsFjDF0795dv7WIBDl32wFbRKHdcnoORfzL2t35/PpvBzl+voy+0R34/q3DuGdcP4/u06vBLSLSlqzdnc+/v7WXsioXAPnny/j3t/YCeDS8tchUM3Xq1AmA48ePM3v27GuOXbJkCaWlpW5tf+PGjcycObPZ9YmI5/36bwcvhna9sioXv/7bQY/uV8F9CZfL1fSgK/Tt25c333zzmmOaE9wi4v+Ony9z6+utJWiC+/DhwwwfPpyHH36YMWPGMHv2bEpLS4mLi+Ppp58mJSWF1atXk5OTw2233cb48eO56aabOHDgAAC5ublMmjSJCRMm8B//8R+XbXf06NFAbfB/73vfIzExkTFjxrB8+XKWLVvG8ePHmTZtGtOmTQPg73//O5MmTSIpKYn77ruP4uJiAP76178yfPhwUlJSeOutt7z8DImIu/pGd3Dr663FJ3PcP333n+w7fqFVtzmyb2d+fOeoa445ePAgL7/8MpMnT+brX/86zz33HFDbG71582YApk+fzsqVK0lISGDHjh3Mnz+fjIwMFixYwOOPP85DDz3EihUrGtx+Wloaubm57N69m7CwMM6ePUu3bt1YtGgRGzZsICYmhjNnzvCf//mfpKenExkZyTPPPMOiRYv4wQ9+wDe/+U0yMjKIj4/nK1/5Sqs+PyLS+r5/67DL5rgBOoSH8v1bh3l0v0F1cXLAgAFMnjwZgAceeIBly5YBXAzJ4uJitm7dyn333XfxMRUVFQBs2bKFNWvWAPDggw/yb//2b1dtPz09nccee4ywsNqntVu3bleN2b59O/v27btYR2VlJZMmTeLAgQMMGjSIhISEi/WlpaW1ynGLiGfUX4AMiq6Sps6MPeXKVrr6zyMjIwGoqakhOjqaPXv2OHr8lay1jsbMmDGDN964fPWAPXv2qNVPJADdM66fx4P6SkEzxw1w9OhRtm3bBsAbb7xBSkrKZd/v3LkzgwYNYvXq2gUQrbV8+umnAEyePJnf//73AKxatarB7X/hC19g5cqVVFdXA3D27FkAoqKiKCoqAmDixIls2bKF7OxsAEpLS8nMzGT48OHk5uaSk5NzsT4RkYYEVXCPGDGC1157jTFjxnD27Fkef/zxq8asWrWKl19+mbFjxzJq1CjeeecdAJYuXcqKFSuYMGEChYWFDW7/G9/4BrGxsYwZM4axY8fyu9/9DoB58+bxxS9+kWnTptGjRw9+85vfMGfOHMaMGcPEiRM5cOAAERERpKWlcccdd5CSksLAgQM990SISEBz69ZlTiUnJ9srb6Swf/9+RowY0er7curw4cPMnDmTzz77zGc1tBZfP5ci0vqMMbustclOxgbVGbeISFsQNMEdFxfXJs62RUSCJrhFRNoKBbeISIBRcIuIBBgFt4hIgFFwX+H222/n/Pnz1xzzox/9iPT0dC9VJCK+sHZ3PpN/mcGgp95j8i8zWLs739clXeT0ZsHRwEvAaMACX7fWbvNkYd5mrcVay/vvv9/k2KefftoLFYmIr/jqBglOOT3jXgr81Vo7HBgL7PdcSbU88dNu0aJFjB49mtGjR7NkyRIOHz7MiBEjmD9/PklJSRw7doy4uDjOnDkDwM9+9jOGDx/OjBkzmDNnDv/93/8NwNy5cy+uwR0XF8ePf/xjkpKSSExMvLgMrIgELl/dIMGpJoPbGNMZmAK8DGCtrbTWXnsuoYXqf9rlny/D8r8/7VoS3rt27eLVV19lx44dbN++nRdffJFz585x8OBBHnroIXbv3n3Z28x37tzJmjVr2L17N2+99RZXvhP0UjExMXzyySc8/vjjF8NdRAKXr26Q4JSTM+7BwGngVWPMbmPMS8aYyCsHGWPmGWN2GmN2nj59ukVFeeKn3ebNm7n33nuJjIykU6dOzJo1iw8//JCBAwcyceLEBsfffffddOjQgaioKO68885Gtz1r1iwAxo8fz+HDh5tdo4j4B1/dIMEpJ8EdBiQBz1trxwElwFNXDrLWpllrk621yT169GhRUZ74adfYmiz1S7o6Hd+Q9u3bAxAaGnpxZUARCVzfv3UYHcJDL/uaN26Q4JST4M4D8qy1O+o+f5PaIPcYT/y0mzJlCmvXrqW0tJSSkhLefvttbrrppkbHp6Sk8O6771JeXk5xcTHvvfdes/ctIoHlnnH9+MWsRPpFd8AA/aI78ItZiX5xYRIcdJVYa08aY44ZY4ZZaw8C04F9nizKE7cDSkpKYu7cuVx//fVA7RKsXbt2bXT8hAkTuOuuuxg7diwDBw4kOTmZLl26NHv/IhJYfHGDBKccLetqjLmO2nbAdsAh4F+stecaG98ay7qu3Z3v9dsBXam4uJhOnTpRWlrKlClTSEtLIynJo79sOKJlXUXaHneWdXXUx22t3QM42mBr8YefdvPmzWPfvn2Ul5fz8MMP+0Voi4gE1c2C3VV/BxsREX/i1be8e+JuO8FGz6GIeC24IyIiKCgoUPC0gLWWgoICIiIifF2KiPiQ16ZK+vfvT15eHi19c06wi4iIoH///r4uQ0R8yGvBHR4ezqBBg7y1OxGRNksXJ0UkaPhDm3FrUHCLSFDw96Va3aEbKYhIUPD3pVrdoeAWkaDg70u1ukPBLSJBwd+XanWHgltEgoK/L9XqDl2cFJGgUH8BUl0lIiIBxB8Wr2sNmioREQkwCm4RkQCj4BYRCTAKbhGRAKPgFhEJMI66Sowxh4EiwAVUO70vmohIsCivchFxRZ+4p7jTDjjNWnvGY5WIiASg3DMlLFufxe6j51j3rzcTHur5iQz1cYtIwPPFcq1HCkpYtj6btXvyCQ81PDQpjsrqGr8Kbgv83RhjgRestWlXDjDGzAPmAcTGxrZehSIi1+Dt5VqPnS1leUYWaz7JJyzEMPfGOB69eTA9o7x3S0GnwT3ZWnvcGNMTWGeMOWCt3XTpgLowTwNITk7WjSVFxCuutVxrawZ33rlSVmzIZvXOPEJCDA9OHMj8qUPo2dn794B1FNzW2uN1f58yxrwNXA9suvajREQ8z9PLtZ4oLGPFhmz+8PExDIav3hDL/Knx9O7iu5t2NxncxphIIMRaW1T38ReApz1emYiIA32jO5DfQEi3dLnWzy+U89yGbN746BgWy33JA3hiWjz9/GAZWCdn3L2At40x9eN/Z639q0erEhFx6Pu3DrtsjhtatlzrqaJynt+Yw6odR6mpscwe358npsUzoFvH1iq5xZoMbmvtIWCsF2oREXFbay3Xerqoghc+yOG3O45Q5bLMGtePb9+SQGx3/wnsemoHFJGA15LlWguKK0jbdIjXtx2hotrFPeP68eQtCcTFRLZyla1HwS0iQelcSSVpHx7ita2HKatycffYvjw5PYHBPTr5urQmKbhFJKgUllbx0uZDvLI5l9IqFzPH9GXB9Hjie0b5ujTHFNwiEhQKy6p4eXMur27OpaiimjsS+7AgNYGhvQInsOspuEWkTSsqr+LVLYd58cNDFJVXc9uo3ixITWBEn86+Lq3ZFNwi4pdauv5IcUU1r209TNqmQxSWVTFjZC8WpiYwqm8XD1btHQpuEfE7LVl/pKSimte3HSFtUw7nSquYPrwnC1OHktg/8AO7noJbRPxOc9YfKat08T/bD7Pyg0OcLalk6rAeLEwdynUDor1RslcpuEXE77iz/kh5lYvfbj/Cyg8Ocaa4gpsSYvjOjKEkxXb1dJk+o+AWEb/jZP2R8ioXb3x0lOc25nC6qILJ8d1ZmZpEclw3b5bqEwpuEfE711p/pKLaxR8/PsaKDTmcvFDODYO68eyccdwwuLsPK/YuBbeI+J2G1h/5TmoCJZXVTPv1Ro4XljMhriuLvjKWG4fE+Lha71Nwi4hfql9/pMpVw5pdeSxOzyL/fBlJsdH8avZYJsd3p27V0qCj4BYRv1TtquGt3fksz8ji2Nkyxg6I5uezEpmSEBO0gV1PwS0ifqXaVcM7e46zLCOLIwWlJPbrwk/njmLasJ5BH9j1FNwi4hdcNZZ3Pz3O0vVZ5J4pYWSfzrz4UDKpIxTYV1Jwi4hPuWos7+09wdL0THJOlzC8dxQrHxjPraN6KbAboeAWEa+qX4Mk/3wZ3Tq2o11YCCcvlDO0Vyee+1oSt43qTUiIAvtaHAe3MSYU2AnkW2tneq4kEWmr1u7O56k1/6C8ugaAs6WVGOChSQP5yZ2jFNgOhbgxdgGw31OFiEjbZq3l6T/vuxjaF78OrN9/SqHtBkfBbYzpD9wBvOTZckSkrbHWsn7/59z57GbOllQ2OKaxtUmkYU6nSpYAPwAavVWEMWYeMA8gNja25ZWJSECz1rIx8zRL1mXyaV4hsd06Et0hnPNlVVeNvXQNEmlak2fcxpiZwClr7a5rjbPWpllrk621yT169Gi1AkUksFhr2ZR5mnuf28q/vPoxBSWVPPOlRNZ/92Z+ctcoOoSHXja+fg0Scc7JGfdk4C5jzO1ABNDZGPNba+0Dni1NRAKJtZatOQUsWpfJriPn6Nslgp/fm8js8f1pF1Z7jtjQGiTu3tlGwFhrnQ82Zirwvaa6SpKTk+3OnTtbWJqIBIptOQUsTs/ko9yz9O4cwRO3xPPl5P60Dwtt+sECgDFml7U22clY9XGLSLN9lHuWxesy2XaogJ5R7fnJnSO5//pYIsIV2J7kVnBbazcCGz1SiYgEjF1HzrJ4XRabs88Q06k9P5o5kq/eoMD2Fp1xi4hju4+eY3F6FpsyT9M9sh0/vH0ED0wcSId2oS2+K7s4p+AWkSb9I+88i9dlsuHgabp2DOepLw7noUkD6diuNkJacld2cZ+CW0Qa9Vl+IUvSs0jf/znRHcP5/q3DePjGODq1vzw6mnNXdmk+BbeIXGX/iQssSc/kb//8nM4RYXx3xlDmTo4jKiK8wfHu3JVdWk7BLSIXHTxZxNL1mby/9yRREWEsTE3g6ymD6NxIYNdzcld2aT0KbhEh+1QRS9KzeG/vCSLbhfHkLfE8kjKYLh2vHdj1rnVXdml9Cm6RIJZzuphl67P406fH6RAeyvypQ/jmTYOJ7tjOre3oHZHepeAWCUK5Z0pYvj6LtXvyaR8WyqNThjBvymC6RboX2Jeqvyu7eJ6CWySIHC0oZVlGFm/vzic81PBIyiAevXkIMZ3a+7o0cYOCWyQIHDtbyrMZ2bz5SR5hIYa5N8bx6M2D6RkV4evSpBkU3CJtWP75Mp7NyGb1zmOEhBgenDiQ+VOH0LOzs8DWuyH9k4JbpA06UVjGig3Z/OHjYxgMX70hlvlT4+ndxfkZtt4N6b8U3CJtyOcXynl+Yw6/23EUi+W+5AE8MS2efs3op9a7If2XglukDThVVBvYq3YcpabGMnt8f56YFs+Abh2bvU29G9J/KbhFAtjpogpe+CCH3+44QpXLMmtcP759SwKx3Zsf2PX0bkj/peAWCUAFxRWkbTrE69uOUFHt4p5x/XjylgTiYiJbbR96N6T/UnCLBJBzJZWkfXiI17YepqzKxd1j+/Lt6QkM6dGp1feld0P6LwW3SAA4X1rJSx/m8uqWXEqrXMwc05cF0+OJ7xnl0f3q3ZD+qcngNsZEAJuA9nXj37TW/tjThYkIFJZV8fLmXF7dnEtRRTV3JPZhQWoCQ3u1LLDVnx3YnJxxVwC3WGuLjTHhwGZjzF+stds9XJtI0Coqr+LVLYd58cNDFJVXc9uo3ixITWBEn84t3rb6swNfk8FtrbVAcd2n4XV/rCeLEglWxRXVvLb1MGmbDlFYVsWMkb1YmJrAqL5dWm0f6s8OfI7muI0xocAuIB5YYa3d0cCYecA8gNjY2NasUaTNK6mo5vVtR0jblMO50iqmD+/JwtShJPZvvcCup/7swOcouK21LuA6Y0w08LYxZrS19rMrxqQBaQDJyck6IxdxoKzSxf9sP8wLHxyioKSSqcN6sDB1KNcNiPbYPtWfHfjc6iqx1p43xmwEbgM+a2K4iDSivMrFqh1HeX5jDmeKK7gpIYbvzBhKUmxXj+9b/dmBz0lXSQ+gqi60OwCpwDMer0ykDSqvcvHGR7WBfaqogsnx3VmZmkRyXDev1aD+7MDn5Iy7D/Ba3Tx3CPBHa+2fPVuWSNtSUe3ijx8fY8WGHE5eKOeGQd1YPmccNwzu7pN61J8d2Jx0lfwDGOeFWkTanMrqGlbvOsaKjGyOF5YzIa4ri74ylhuHxLT6vtSbHTz0zkkRD6hy1bBmVx7LM7LJP19GUmw0v5o9lsnx3THGtPr+1JsdXBTcIq2o2lXDW7vzWZ6RxbGzZYwdEM1/3Tuam4f28Ehg11NvdnBRcIu0gmpXDe/sOc6yjCyOFJSS2K8LP507imnDeno0sOupNzu4KLhFWsBVY3n30+MsXZ9F7pkSRvbpzIsPJZM6wjuBXU+92cFFwS3SDK4ay3t7T7A0PZOc0yUM7x3FygfGc+uoXl4N7HrqzQ4uCm4RN9TUWP7y2UmWpGeSdaqYob068dzXkrhtVG9CQrwf2PXUmx1cFNwiDtTUWP6+7yRL0rM4cLKIIT0iWT5nHHck9vF4YDtt81NvdvBQcItcg7WWdfs+Z3F6FvtPXGBwTCRL77+OmWP6EuqFM2y1+UlDFNwiDbDWknHgFIvTM/ks/wIDu3fk/903lruv60tYaIjX6lCbnzREwS1yCWstGzNPs2RdJp/mFTKgWwd+PXsM947r59XArqc2P2mIgluE2sD+MOsMi9ZlsufYefp37cAzX0pkVlJ/wn0Q2PXU5icNUXBLULPWsjWngEXrMtl15Bx9u0Tw83sTmT2+P+3CfBfY9dTmJw1RcEvQ2pZTwOL0TD7KPUvvzhH87J7RfDm5P+3DQj2+b3c6RUBtfnI5BbcEnY9yz7J4XSbbDhXQM6o9P7lzJPdfH0tEuOcDG9zvFFGbn1xJwS1BY9eRsyxel8Xm7DPEdGrPj2aO5Ks3eC+w66lTRFpKwS1t3u6j51icnsWmzNN0j2zH/71jBF+7YSAd2nk3sOupU0RaSsEtbdbevEIWp2eSceAU3SLb8e9fHM6DkwbSsZ1v/9mrU0RaSsEtbc5n+YUsSc8iff/nRHcM5we3DePhSXFEtvePf+7qFJGWcnKz4AHA60BvoAZIs9Yu9XRhIu7af+ICS9Iz+ds/P6dzRBjfnTGUuZPjiIoI91oNTrpF1CkiLWWstdceYEwfoI+19hNjTBSwC7jHWruvscckJyfbnTt3tm6lIo04eLKIpeszeX/vSaIiwngkZRBfTxlEZy8GNlzdLQK1Z9K/mJWoUJYmGWN2WWuTnYx1crPgE8CJuo+LjDH7gX5Ao8Et4g3Zp4pYkp7Fe3tPENkujCdvieeRlMF06ejdwK6nbhHxFrcm/YwxcdTe8X1HA9+bB8wDiI2NbYXSRBqWc7qYZeuz+NOnx+kQHsr8qUP45k2Die7Yzqd1qVtEvMVxcBtjOgFrgIXW2gtXft9amwakQe1USatVKFIn90wJy9dnsXZPPu3DQnl0yhDmTRlMt0jfBnY9dYuItzgKbmNMOLWhvcpa+5ZnSxK53NGCUpZlZPH27nzCQw2PpAzi0ZuHENOpvddqcHLRUd0i4i1OukoM8DKw31q7yPMlidQ6draUZzOyefOTPMJCDA9PiuOxqYPpGRXh1TqcvkVd3SLiLU66SlKAD4G91LYDAvwfa+37jT1GXSXSEvnny3g2I5vVO48RYgxfvSGWx6cOoVdn7wZ2vcm/zGhwCqRfdAe2PHWLDyqStqi1u0o2A767C6oEjROFZazYkM0fPj4GwJzrY5k/bQh9uvh2jlgXHcXf+MdbySSofX6hnOc2ZPPGR8eosZYvTxjAE9Pi6ecnF/V00VH8jYJbfOZUUTnPb8xh1Y6juGos943vzxPT4hnQraNX9u90TWxddBR/o+AWrztdVMELH+Tw2x1HqHJZZo3rx7dvSSC2u3cCG9xbE1sXHcXfKLjFawqKK0jbdIjXtx2hotrFPeP68eQtCcTFRHq9Fnff5aibGYg/UXCLx50rqSTtw0O8tvUwZVUu7h7bl29PT2BIj04+q0kXHCWQKbjFY86XVvLSh7m8uiWX0ioXM8f0ZcH0eOJ7Rnl0v07mrnXBUQKZgltaXWFZFa9szuWVzbkUVVRzR2IfFqQmMLSXZwMbnM9d64KjBDIFt7SaovIqXt1ymJc+PMSF8mpuG9WbBakJjOjT2Ws1OJ271gVHCWQKbmmx4opqXtt6mLRNhygsq2LGyF4sTE1gVN8uXq/FnblrXXCUQKXglmYrqajm9W1HSNuUw7nSKqYP78nC1KEk9m/9wHbac625awkGCm5xW1mli99uP8LKD3IoKKlk6rAeLEwdynUDoj2yP3d6rjV3LcFAwS2OlVe5WLXjKM9vzOFMcQU3JcSwMHUo4wd29eh+3em51ty1BAMFtzSpvMrF7z86ynMbczhVVMGNQ7rz/ANJTIjr5pX9u9tzrblraesU3NKoimoXf/z4GCs25HDyQjk3DOrG8jnjuGFw91bbh3quRdyn4JarVFbXsHrXMVZkZHO8sJwJcV1Z9JWx3DgkplX3o55rkeZRcMtFVa4a1uzKY3lGNvnny0iKjeaZ2WNIiY+h9kZIrUs91yLNo+AWql01vLU7n+UZWRw7W8bYAdH8172juXloj2YFttPWPfVcizSPgjuIVbtqeGfPcZZlZHGkoJTR/Trz07mjmDasZ7PPsN1p3dPctUjzhDQ1wBjzijHmlDHmM28UJJ7nqrGs3Z3PjMWb+O7qT4lsF0bag+N591sp3DK8V4umRa41/XGl7986jA7hoZd9TXPXIk1zcsb9G+BZ4HXPliKe5qqxvLf3BEvTM8k5XcLw3lGsfGA8XxjZi5CQpsPayRSIu9MfoLlrEXc5uVnwJmNMnOdLEU+pqbH85bOTLEnPJOtUMUN7deK5ryVx26jejgIbnE+BuDv9oblrEfdpjrsNq6mx/H3fSZakZ3HgZBFDekSyfM447kjsc1lgOzmTdtoBotY9Ec9rteA2xswD5gHExsa21malGay1rNv3OUvSs9h34gKDYyJZev91zBzTl9ArzrCdnkk7nQLR9IeI57VacFtr04A0gOTkZNta2xXnrLVkHDjF4vRMPsu/cDGky6tcWMtVoQ3Oz6TdmQLR9IeIZzXZVSL+z1rLhoOnuGfFFh55bScnzpcTHmpw1dT+/DxeWM6/v7WXtbvzr3qs0zNpdYCI+A8n7YBvANuAYcaYPGPMI54vS5yw1rIp8zSznt/Kv7z6MWeKK3nmS4m0DwuhynX5Lz2NteQ1dtHwyq/fM64fv5iVSL/oDhigX3QHfjErUWfWIj7gpKtkjjcKEWfW7s7nV389wPHCctqFhlDpqqFvlwh+fm8is8f3p11YCE+t2dvgYxs6u3bnYqKmQET8g7pK/IDTt4iv3Z3PD978B5WuGgAqXTWEhxj+dcZQZicPuDjO3flo0MVEkUCi4PYxp10dH+We5Qdr/je061XVWBanZ10W3O625OlMWiSwKLg9qDX6o3cdOcvidVlszj7T6H7UkicSXBTcHtLS/uj882U89MpHbMo8TffIdvzw9hG8siWXE4XlV41VS55IcFFwu8npfHRL+6MB9uad56kvDuehSQPp2C6MHlHt9a5EEVFw13MSyO4sWepOf/SVYQxwR2Ifnpk9hk7t//cl0hSIiEAQBHdrBrI7dxt32tkxrHcUCT078Y/8QgCiIsL44e0juP/6hpcN0BSIiLTp4G7tQHZnydKmOjsOnixi6fpM3t97kqiIMBamJvD1lEF0jghv5tGKSLAIyOBu7Xlmp4HcGv3Ro/t15lu/+4T39p4gsl0YT94SzyMpg+nSUYEtIs74VXD7ap7ZaSC3pD8653Qxy9Zn8Z0/7qFDeCjzpw7hGymD6RrZrsHHiog0xm+C25fzzE4DuTkXBw+fKWHZ+izW7smnfVgoj04Zwrwpg+mmwBaRZvKb4PbFPHM9dwLZ6cXBowWlLMvI4u3d+YSHGr5x02DmTRlMTKf2TT5WRORa/Ca4vTnP3JJAbsqxs6U8m5HNm5/kERZieHhSHI9NHUzPqIgWb1tEBPwouL0xz+xJ+efLeDYjm9U7jxESYnhw4kDmTx1Cz84KbBFpXX4T3J6cZ/akE4VlPLchh99/fBSAOdfHMn/aEPp0aXidaxGRlvKb4PbFtEZLfH6hnOc35vC7HUepsZYvTxjAE9Pi6dfIjQlERFqL3wQ3+EcgN+VUUTkrNx5i1Y4jVNdY7hvfnyemxTOgW0dflyYiQcKvgtufnSmu4IUPcvif7UeoclnuHdePJ29JILa7AltEvMtRcBtjbgOWAqHAS9baX3q0Kj9ytqSSFzbl8PrWI1RUu7inLrDjYiJ9XZqIBKkmg9sYEwqsAGYAecDc0xbTAAAH00lEQVTHxpg/WWv3ebo4XzpXUsmLHx7ita2HKa1ycffYvnx7egJDenTydWkiEuScnHFfD2Rbaw8BGGN+D9wNtMngLiyt4uXNh3hly2FKKquZOaYvC6bHE98zyteliYgAzoK7H3Dsks/zgBs8U47vXCiv4pXNuby8OZei8mruSOzDgtQEhvZSYIuIf3ES3KaBr9mrBhkzD5gHEBvb8FrS/qiovIrfbDnMix8e4kJ5NbeO6sXC1KGM6NPZ16WJiDTISXDnAQMu+bw/cPzKQdbaNCANIDk5+apg9zfFFdW8trU2sM+XVjFjZC8WpiYwqm8XX5cmInJNToL7YyDBGDMIyAfuB77q0ao8qLSymte3HeGFD3I4V1rF9OE9WZg6lMT+CmwRCQxNBre1ttoY8y3gb9S2A75irf2nxytrZWWVLn67/QgrP8ihoKSSqcN6sDB1KNcNiPZ1aSIibnHUx22tfR9438O1eER5lYtVO47y/MYczhRXcFNCDAtThzJ+YFdflyYi0ixt9p2T5VUu3vioNrBPFVUwOb47K1OTSI7r5uvSRERapM0Fd0W1iz9+fIwVG3I4eaGc6wd1Y9mccUwc3N3XpYmItIo2E9yV1TWs3nWMFRnZHC8sZ0JcVxZ9eSyThnTHmIY6GkVEAlPAB3eVq4Y1u/JYnpFN/vkykmKjeWb2GFLiYxTYItImBWxwV7tqeGt3Psszsjh2toyxA6L5r3tHc/PQHgpsEWnTAi64q101vLPnOMszsjhcUEpivy78dO4opg3rqcAWkaAQMMHtqrG8++lxlq3P4tCZEkb26Uzag+OZMbKXAltEgorfB3dNjeXPe0+wND2TnNMlDO8dxcoHxvOFkb0ICVFgi0jw8dvgrqmx/OWzkyxdn0nm58UM7dWJ576WxG2jeiuwRSSo+V1w19RY/r7vJEvSszhwsoghPSJZPmccdyT2UWCLiOBnwX3qQjlzX/2YfScuMDgmkiVfuY47x/YlVIEtInKRXwV3TKf2DOjWgUdSBnH3dX0JCw3xdUkiIn7Hr4I7JMTwwoPJvi5DRMSv6ZRWRCTAKLhFRAKMgltEJMAouEVEAoyCW0QkwCi4RUQCjIJbRCTAKLhFRAKMsda2/kaNOQ0cacEmYoAzrVSOP9Dx+Dcdj39ra8cDDR/TQGttDycP9khwt5QxZqe1ts28hVLH4990PP6trR0PtPyYNFUiIhJgFNwiIgHGX4M7zdcFtDIdj3/T8fi3tnY80MJj8ss5bhERaZy/nnGLiEgj/CK4jTH3GWP+aYypMcY0eqXVGHObMeagMSbbGPOUN2t0hzGmmzFmnTEmq+7vro2Mcxlj9tT9+ZO362xKU8+3Maa9MeYPdd/fYYyJ836Vzjk4nrnGmNOXvCbf8EWdThljXjHGnDLGfNbI940xZlnd8f7DGJPk7Rrd4eB4phpjCi95fX7k7RqdMsYMMMZsMMbsr8u2BQ2Maf7rY631+R9gBDAM2AgkNzImFMgBBgPtgE+Bkb6uvZFafwU8VffxU8AzjYwr9nWt1ziGJp9vYD6wsu7j+4E/+LruFh7PXOBZX9fqxjFNAZKAzxr5/u3AXwADTAR2+LrmFh7PVODPvq7T4bH0AZLqPo4CMhv499bs18cvzrittfuttQebGHY9kG2tPWStrQR+D9zt+eqa5W7gtbqPXwPu8WEtzeXk+b70ON8Ephtj/PUGoYH078cRa+0m4Ow1htwNvG5rbQeijTF9vFOd+xwcT8Cw1p6w1n5S93ERsB/od8WwZr8+fhHcDvUDjl3yeR5XPxH+ope19gTUvoBAz0bGRRhjdhpjthtj/C3cnTzfF8dYa6uBQqC7V6pzn9N/P1+q+7X1TWPMAO+U5jGB9H/GqUnGmE+NMX8xxozydTFO1E0hjgN2XPGtZr8+XrvnpDEmHejdwLd+aK19x8kmGviaz1pirnU8bmwm1lp73BgzGMgwxuy11ua0ToUt5uT59qvXpAlOan0XeMNaW2GMeYza3yZu8XhlnhNIr48Tn1D7tvBiY8ztwFogwcc1XZMxphOwBlhorb1w5bcbeIij18drwW2tTW3hJvKAS8+A+gPHW7jNZrvW8RhjPjfG9LHWnqj71edUI9s4Xvf3IWPMRmp/KvtLcDt5vuvH5BljwoAu+O+vuk0ej7W24JJPXwSe8UJdnuRX/2da6tLgs9a+b4x5zhgTY631y3VMjDHh1Ib2KmvtWw0MafbrE0hTJR8DCcaYQcaYdtReDPO7Tow6fwIervv4YeCq3yiMMV2NMe3rPo4BJgP7vFZh05w835ce52wgw9ZddfFDTR7PFfOLd1E7LxnI/gQ8VNe9MBEorJ/CC0TGmN7111CMMddTm18F136Ub9TV+TKw31q7qJFhzX99fH31te7/+b3U/vSpAD4H/lb39b7A+1dchc2k9qz0h76u+xrH0x1YD2TV/d2t7uvJwEt1H98I7KW2u2Ev8Iiv627gOK56voGngbvqPo4AVgPZwEfAYF/X3MLj+QXwz7rXZAMw3Nc1N3E8bwAngKq6/z+PAI8Bj9V93wAr6o53L410bPnLHwfH861LXp/twI2+rvkax5JC7bTHP4A9dX9ub63XR++cFBEJMIE0VSIiIii4RUQCjoJbRCTAKLhFRAKMgltEJMAouEVEAoyCW0QkwCi4RUQCzP8HXQxCuDXnMfQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x925eef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter([x[0] for x in features], values, label='origin');\n",
    "plt.plot([x[0] for x in features], model.predict(features), label='predicted');\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Part 1.3.2 Adding polynomial features\n",
    "\n",
    "Note that in that synthetic dataset, the value is actually not linear with the feature (it is exponential). Inspired by Taylor expansion, we are going to add some polynomial features based on existing features.\n",
    "More specifically, support \n",
    "$${\\vec {x}} = [x_1, ..., x_n]$$\n",
    "is the feature vector of one sample (whose value is $y$). Instead of modeling the relationship between $\\vec{x}$ and $y$, we modeling the relationship of $y$ and $\\vec{x'}$, where (suppose we are adding up to $k$-th degree polynomials)\n",
    "\n",
    "$${\\vec {x'}}_k = [x_1 , x_2 , ..., x_n , x_{21} , ..., x_{2n}, ..., x_{k1}, ..., x_{kn} ]. $$\n",
    "\n",
    "Repeat the 5 steps, then report the MSE value on training set and model weights for the following three cases: $k = 2, k = 4, k = 8$ (for all numbers, keep 6 digits after the decimal point). \n",
    "\n",
    "### !!! Make sure that features’s polynomials follow the order in the above equation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.3.2]\tk: 2\tmse: 0.02315\n",
      "[part 1.3.2]\tk: 4\tmse: 0.00002\n",
      "[part 1.3.2]\tk: 8\tmse: 0.00000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x9354ef0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VFX+x/H3mUnvpDeSEHoJJUSKICqKsOqKsuoqKuqiWNb2W/uuZW2rrq6urm1BFrHArkJEQRFBQQEpJgQpgVADaYQkpGcmmXJ+fyRElJIJJJmZ5Pt6Hp5MZs7c+d4Z8snJueeeq7TWCCGEcB8GZxcghBCidSS4hRDCzUhwCyGEm5HgFkIINyPBLYQQbkaCWwgh3EyLwa2U6quU2nzMvyql1H0dUZwQQojjqdbM41ZKGYECYKTW+kC7VSWEEOKkWjtUcgGwV0JbCCGcx6OV7a8B5rfUKDw8XCclJZ1WQUII0RVlZmaWaq0jHGnr8FCJUsoLKAQGaq2LT/D4DGAGQEJCwvADB6RTLoQQjlJKZWqt0xxp25qhkt8Am04U2gBa65la6zStdVpEhEO/NIQQQpyG1gT3tTgwTCKEEKJ9ORTcSik/YAKQ3r7lCCGEaIlDBye11nVA2Jm8kMViIT8/H7PZfCabcUs+Pj7Ex8fj6enp7FKEEJ1Aa2eVnLb8/HwCAwNJSkpCKdVRL+t0WmvKysrIz8+nR48ezi5HCNEJdNgp72azmbCwsC4V2gBKKcLCwrrkXxpCiPbRYT1uoMuF9lFddb+F6AoWZRXw0rIcCitMxIb48uDEvlw+LK5dX7NDg1sIITqTRVkFPJq+FbOtCox2Cirg0fStAO0a3l1mdcDc3FwGDRrUqufU1dVxySWX0K9fPwYOHMgjjzzSTtUJIdzRS8tyMFlsDI74gIiezxFoKMFksfHSspx2fd0uE9yn64EHHmDnzp1kZWWxdu1ali5d6uyShBAuorDChLdHKfkh+xlWZ6DaHt58f3vqksG9b98+hg0bxo8//njKdn5+fpx//vkAeHl5kZqaSn5+fkeUKIRwA7EhvgyI+BC7gsOHLwdU8/3tySlj3E8t3k52YVWbbnNAbBBP/nZgi+1ycnK45pprmDNnDr6+vgwdOvSE7VatWkVISEjz9xUVFSxevJh77723zWoWQri328bAK3uLGF3pw9KGEQD4ehp5cGLfdn3dLnVwsqSkhMmTJ7Nw4UIGDmwM+c2bN7f4PKvVyrXXXss999xDcnJye5cphHAT24texlNrKk3TUNC5Z5U40jNuD8HBwXTv3p21a9cycOBAcnJy+P3vf3/Ctsf2uGfMmEHv3r257z658I8QolFO3lqW1h3kD97x3PfIPR362l2qx+3l5cWiRYuYOHEiAQEBTJ06tcUe92OPPUZlZSXvvvtuB1UphHAHb6x+jACtuXn8yx3+2l3u4KS/vz9Llizh1Vdf5bPPPjtl2/z8fJ577jmys7NJTU1l6NChEuBCCDbvWswqSyk3B/QlODqlw1+/y/S4k5KS2LZtGwAhISEtzigBiI+PpzXX5BRCdH5aa17b8DdCbXaum/CqU2rocj1uIYQ4E+u2vk+GvYYZ4WfhF5LolBokuIUQwkFaa17Lep1Ym52rLuj4se2jJLiFEMJByze+SjYN3Bl3IV7+4c33a60xW2wdVocEtxBCOMBqbeCN7Lkk2+DS85//xWNz1uZy+ZtrKa9t6JBaJLiFEMIBi9c8zX6DnbuTp2D08mu+f0t+Bc8v3UH3UD9C/DrmKlddZlaJEEKcroaGWt7e/xmDtIELznm8+f4qs4W75mUREeDNS1cO7rC197tMj/t0lnU91mWXXXZGzxdCuK9Pvn2YIgPcM2g6ytjY39Va82j6VgoqTPxr6jBC/Lw6rJ4uE9xnIj09nYCAAGeXIYRwgrraEmYWrWKE9mJU2l3N98/fmMcXW4q4/6I+DE8M7dCaumRwO7qsK0BNTQ2vvPIKjz32WAdUJoRwFYuyChjzwrfc9s40jhgU4yL/gDI0RuaOoiqeWrydc3qHc/u4nh1em0Nj3EqpEOBdYBCggT9orded9qsufQQObT3tp59QdAr85oUWm7V2WdfHH3+c+++/Hz8/vxO2E0J0PkcvSeZlK2J374MMqvHhhT09CY4p4KKBUdw1bxNBvp68+vuhGAwdf01ZRw9OvgZ8pbW+UinlBbhlirV2WdfNmzezZ88eXn31VXJzczuoSiGEsx29JNmIqPfJUopDh6c0X5JszZ5S9pXW8tH0kYQHeDulvhaDWykVBIwDbgLQWjcAZzZZ0YGecXto7bKu69atIzMzk6SkJKxWK4cPH+a8885j1apVHVu4EKJDFVaYiPPczfaQElKqA1hb3/iXeUGFiQWZ+dxzQW/O7hXewlbajyM97mSgBJijlBoCZAL3aq1r27WydtDaZV3vuOMO7rjjDqBxVsqll14qoS1EFxAb4ktSwIfkALuKb2i+XwFn9QjlnvG9nFYbOHZw0gNIBd7WWg8DaoHjLneulJqhlMpQSmWUlJS0cZltpzXLugohuqbpAzLYElhP3yMJlFiTgMbQ9vMy8vo1w/AwOndehyM97nwgX2u9oen7BZwguLXWM4GZAGlpaS63FurpLOt6sucLITovq8XEF6UfEo0mv+HO5sCubbDxxtRUooN9nF1iyz1urfUhIE8pdfTqlxcA2e1alRBCOMnCFfezy6h5sN80vnv0Ut6Ymkptg40Z45I5v1+ks8sDHJ9VcjfwUdOMkn3Aze1XkhBCOEflkT3869B3nGXwZ8KoBzhYVscjC7cwtHsID1zUvldubw2HgltrvRlIa+dahBDCqd5c9keqleLhc56jwa65e/4mUPCva4fh5eE65yu6TiVCCOFEu3d+xsf1BVzln0zf5Ak8u2QHP+VX8tKVg+ke6lqnrsjqgEKILk/bbLy47in8gbsuepNPs/L5YP0BZoxLZtKgGGeXdxzpcQshurxvVj/FBoOFP3afSLEphEfTtzIiKZSHJrrOuPaxukxwn+6yrvPnzyclJYXBgwczadIkSktL26E6IYSzmGsO8/K+hfTSHvxm9NPc8WEmAd6evDHV+fO1T8Y1q3IRVquVe++9l5UrV7JlyxYGDx7MG2+84eyyhBBtaO6yP1JgNPDw8Pv586c7OXCkjjenDiMyyPnztU+mSwa3o8u6aq3RWlNbW4vWmqqqKmJjYzuoSiFEezt0cB2zq7K50DOcbRVns3TbIR6a2JeRyWHOLu2UnHJw8sWNL7LzyM423Wa/0H48POLhFtu1dlnXt99+m5SUFPz9/enduzdvvvlmm9YthHASrXl15Z+wKcWk/k9z5yc7uWhAFDPGJTu7shZ1qVklrV3W1WKx8Pbbb5OVlUVycjJ33303zz//vFxUQYhOICvjbb6khptCUnn8y3q6d/Pl5auHdNh1I8+EU4LbkZ5xe2jtsq67d+8GoGfPxitcXH311bzwgnOWpBVCnLlFWQW8tCyH0opyBiS/SaSHgYziaVSZLcz9wwiCfDrmKu1nqkv1uFu7rGtcXBzZ2dmUlJQQERHB8uXL6d+/fwdWLIRoK0evamOy2JjQ7X3WextILBrDuop6/nHVEPrHBDm7RId1qeCGn5d1nTBhAv7+/kyePPmkbWNjY3nyyScZN24cnp6eJCYm8t5773VcsUKINnP0qjZRhgJyIvaQZPJma8Wl+HkZ+d3weGeX1ypdJrhPd1nX22+/ndtvv709SxNCdIDCChOgGRo1i3UGheXQ1YCirsHm7NJarUtOBxRCdD2xIb6M8f+SH0LM9K6IpcicAkBciK+TK2s9CW4hRJfw4LndqIheRbgFNhffCoCvp5EHXfS09lOR4BZCdAl5Bx7joJcRDl2CTfsRF+LL81NSuHxYnLNLa7UuM8YthOi6dm35iP+YD9CnOpSIxKv45rpUDAbXn699MtLjFkJ0ajZzFU9ufAE/O9RZ/8Q/rh7i1qENEtxCiE7ugyW3ss0TwsrO551pE/D3dv+BBgnuX7n44oupqKg4ZZsnnniCFStWdFBFQojTlbf7K96s3kbPWj/uv+xxEsP8nV1Sm3D/Xz1t5OhKgF9++WWLbZ9++ukOqEgIcSa0pZ4nVj6E8lSc3+s5zu3rGldobwsu2+NelFXAmBe+pccjXzDmhW9ZlFVwxtt85ZVXGDRoEIMGDeKf//wnubm59O/fnzvvvJPU1FTy8vJISkpqvljCM888Q79+/ZgwYQLXXnstL7/8MgA33XQTCxYsABpP7HnyySdJTU0lJSWFnTvbdtVDIcTpmb3wNjK8NSPsY7lnwgXOLqdNuWRwH11ToKDChAYKKkw8mr71jMI7MzOTOXPmsGHDBtavX8+sWbMoLy8nJyeHadOmkZWVRWJiYnP7jIwMFi5cSFZWFunp6WRkZJx02+Hh4WzatIk77rijOdyFEM6zZdt3vFf3I8n13rx4/b/cYsW/1nDJ4D66psCxTBYbLy3LOe1trlmzhiuuuAJ/f38CAgKYMmUKq1evJjExkVGjRp2w/eTJk/H19SUwMJDf/va3J932lClTABg+fDi5ubmnXaMQ4sxVmep54/t7qTMoHj33n/h7u8eKf63h0Bi3UioXqAZsgFVrndaeRTWuKeD4/Y7QWp/wfn//Ex+sOFn7E/H29gbAaDRitVpbX5wQok3Y7ZpX5vyRdf42rgkcxai+Y51dUrtoTY/7fK310PYObWhcU6A19zti3LhxLFq0iLq6Ompra/n0008555xzTtp+7NixLF68GLPZTE1NDV988cVpv7YQomO8teRrVnv/QJLNg4cue8vZ5bQbl5xV8uDEvs3r5h51pmsKpKamctNNNzFixAgAbrnlFrp163bS9meddRaXXXYZQ4YMITExkbS0NIKDg0/79YUQ7WdRVgFPf76dkSF/oSTYwF3xD+Hp4eXsstqNcmRIQCm1HygHNPBvrfXME7SZAcwASEhIGH7gwIFfPL5jx45WXYTg6JUqCitMxIb48uDEvh2+pkBNTQ0BAQHU1dUxbtw4Zs6cSWpq6mltq7X7L4RwzKKsAh5euIWR3p+xufsaelXEs6PsPrdbh0QplenoiIajPe4xWutCpVQksFwptVNr/f2xDZrCfCZAWlqa4wPEJ3H5sDinv+kzZswgOzsbs9nMjTfeeNqhLYRoPy8s3UmQrYjS6O8JsxrZXHwLdt04mcHZGdJeHApurXVh09fDSqlPgRHA96d+lvubN2+es0sQQpyCqcFGcVUdk2LfZK2ngaC8K7DrxmNhZzKZwdW1eHBSKeWvlAo8ehu4CNh2Oi/WmpkanUlX3W8h2pPdrvnTx5u5KOgj1gQ3kFzeg/zakc2Pn8lkBlfnSI87Cvi0aQK7BzBPa/1Va1/Ix8eHsrIywsLCOt1k+FPRWlNWVoaPj4+zSxGiU/nH8hz27FyJOXkr8fVebC6e3vyYu14gwVEtBrfWeh8w5ExfKD4+nvz8fEpKSs50U27Hx8eH+Hj3uhipEK5sYWY+767czujk99luUEzv+zyzjng5dTJDR+qw6YCenp706NGjo15OCNFJbdx/hEfSt3B1/Lt87q15PPkqrj5nEjed6+zKOo5LnvIuhBAncqCslts+yODCbj+wNCCP87wiuWrsE84uq8NJcAsh3EKlycL0uRkE6VIOhXxKkDbw1KUfdqljZkdJcAshXJ7VZueueZvILa3h/O5vs8fTwLNpDxEaGOPs0pxCglsI4dK01vx18XZW7y7l0SErWGis4PqQFMamXO/s0pxGglsI4dLm/pDLh+sPcu9wEx/Ufk0vvLjv4tnOLsupXHKRKSGEAPhmRzFPL8lmYv9u7Kl+nGqDYub4f+Ht2XlPrnGEBLcQwiW9tmIX/1yxGw34H3mO5aEWHk64lD7dz3Z2aU4nQyVCCJcz6/t9vNoU2kO917E6JJeedX74Bf/R2aW5BAluIYRLKaup54WvGi+6HagqIC4dL63YVXAH//h6t5Orcw0S3EIIl2FqsDF9bgY2u0ZhZ0zMq+z1NuJTNJE6a0SnXvGvNWSMWwjhEqw2O3fP38RP+RWE+nkx0vtfrAmup9eRBLKqxwOde8W/1pDgFkI4ndaaJz/fzoodh3l68kC8Shfxculekup8ySqeAXT+Ff9aQ4JbCOF0b63ay0cbDnLbuclcllzFNdmzCFZGqqseRuHRJVb8aw0JbiGEU6VvyuelZTlMHhrLA+fFc/f8cRw2Kuae83dSek5ydnkuSYJbCOE0a3aX8tCCLYxODuPvv0vhnYWTWWu08niP30lon4LMKhFCOEV2YRW3f5hJr8gA/j1tOD+s+Ssz6w9yuX8Prjrnr06uzrVJcAshOlxBhYmb39tIoI8Hc24+i/K8r/lz7qf0x5u/XDa/Sy7V2hoyVCKE6FBHahuYNnsDdQ02Ftx+NsGUcd33D2M0GHj14rn4ePk7u0SXJ8EthOgw1WYLN83ZSH65iff/MII+Eb48/OEE9hrhneEPEhcx0NklugUZKhFCdAizxcat72eQXVjF29enMjI5jA8X38RSark7ehxnp0xzdoluQ3rcQoh2Z7HZuWteFhv2H+H6kYk8vmg7My3PsLP7ZkYSyvSJbzi7RLficI9bKWVUSmUppZa0Z0FCiM7Fbtc8vGALK3YUM2VYHAsy8zHWbqEodiURFsXG/Xfx+eYiZ5fpVlozVHIvsKO9ChFCdD5aa55ekk16VgH3T+jD+n1H8LCWEh7/LiaDojrvRqrq/XlpWY6zS3UrDgW3UioeuAR4t33LEUJ0Jq99s5v3fshl+tge3DW+FyUVlQyNf5k93hBWOJ5DDf0BZNW/VnK0x/1P4CHAfrIGSqkZSqkMpVRGSUlJmxQnhHBfc9bu558rdnPl8Hj+cnF/FHBB3D/YHGCj1+GB7Kie2NxWVv1rnRaDWyl1KXBYa515qnZa65la6zStdVpERESbFSiEcD/pm/J5anE2Fw2I4oUpKRgMio+W/IE1QZUMqohi05GfZ5DIqn+t50iPewxwmVIqF/gvMF4p9WG7ViWEcFvLs4t5cMEWzu4ZxuvXDsPDaODb75/hxbIfOd8jlKvGziYuxBcFxIX48vyUFFn1r5WU1trxxkqdBzygtb70VO3S0tJ0RkbGGZYmhHA36/aWceOcjfSPDuSjW0cR4O3Bti0fcXPm8/RS3sy+5lv8fIKdXaZLUkplaq3THGkrJ+AIIdpE1sFybn0/g8RQP967eQQB3h4UHFzLXRl/IwzFvyZ/LKHdRlp1Ao7WehWwql0qEUK4rS35FUz7z0bCArz4YPpIuvl7UVm+nztX3E6DUvxn/JuEd+vp7DI7DTlzUghxRrYVVHLD7I0E+3oy79ZRRAf7YDFX83+LruSgQfPv1IdIThzn7DI7FQluIcRp23moihtmb8Dfy8j8W0cRF+KLttl4csFv+dHQwN96XMmIwbIGSVuT4BZCnJbdxdVcN2sD3h5G5s8YReaBcq6ZuZ7eXn8jM6yMqzwG8ttxf3V2mZ2SHJwUQrTa3pIarp21AaNBMe/WkWQdrODR9K0kqVlkhhWTUhXI/J3TWJRV4OxSOyUJbiFEq+SW1jJ11npAM+/WkSRHBPDSshwG+HxGdtRW+tZ5sKHgAUwWLWuQtBMJbiGEww6W1XHtrPVYbJqPbhlFr8hAAMKtn5Mbu5ru9QZ25v0JG96ArEHSXiS4hRAOyS9vDG2TxcaH00fSN7oxtDf99D6FcSuIsihyD/4fNfbQ5ufIGiTtQ4JbCNGiokoTU2dtoNps4cPpIxkQGwTAtuwF3Lnp70TaFRWF91Jli2x+jqxB0n4kuIUQp1RcZWbqrA2U1zbwwfSRDIprPPsxZ89SbtvwV0K0YvYlH/DQ5N/IGiQdRKYDCiFOqqDCxHWz1lNSXc/700cypHsIAPsOfMeM1Q/io+Hdi2YSHT2Uy6ORoO4gEtxCiBM6eiCyymTh/ekjGZ7YDYC8gg3c+s0fAXj3/NeJjx/tzDK7JAluIcRx9pbUcN2sDZitNubdOoqU+MbhkUPFW7j161uoV5r/jHmRHj3GO7nSrkmCWwjxCzmHqrnu3Q1orZl/6yj6xzQeiCwt28WtX15PJXbeHfEkffqccnVn0Y4kuIUQzRoXjNqAp9HAvBmj2FZQxS1zM6iqzqN70suUeGhmDnuQgQOvdnapXZrMKhFCAI3raU+dtR4/Lw8+vm002wqqeDR9K0eqikhK+AfFHpqeRRdwQE9wdqldngS3EIKN+49ww+yNhPh58b/bRpEU7s9Ly3Iw2ovplfh38r3t9Cwcw/qqi+Q0dhcgQyVCdHFr95Ryy9wMYkJ8mHdL43raALU1e+me9DqFHpp+BaNYV30ZIKexuwIJbiG6sJU7D3Pbh5n0CPPnw1tGEhHYuMZI/qHNhCe9zhGjnR7541lXO6n5OXIau/PJUIkQXdSy7YeY8UEGvSMDmD9jVHNo78tby41Lb8BksJNQcAkZx4S2nMbuGqTHLUQX9HFGHo+mbyUlLpi5fxhBsK8nANn7vub27/6EQWvmjH6GbPNICpflUFhhIjbElwcn9pWzI12ABLcQXYjWmn9/v48Xlu5kbK9w3rlhOAHejTGQtTOdO9c9QaDWzBr3Kom9LqIPchq7K5LgFqKLsNs1f/tyB++u2c+lg2N45eqheHk0jpb+sGUu92W+RKSGWRfOJCZhjJOrFafSYnArpXyA7wHvpvYLtNZPtndhQoi2Y7HZeWjBFj7NKuDG0YkM6R7C+S+vorDCxFmR37An9Gt62BX/vngu4TGpzi5XtMCRHnc9MF5rXaOU8gTWKKWWaq3Xt3NtQog2UNdg5c6PNrEqp4T7J/Qhvpsvf/50GyaLjUEhi9gVuo5e9TB1wFsS2m6ixVklulFN07eeTf90u1YlhGgTFXUNXPfuBr7fVcLfrkjh7gt68/LXuzBZbAwOnc+BmPX0Myn25j7IP9ZanF2ucJBDY9xKKSOQCfQC3tRabzhBmxnADICEhIS2rFEIcRqKKk1Mm72RA0fqeOu6VCYNigGgsKKW4VFvsys0j6G1BjbnPUy1DqZCTqxxGw7N49Za27TWQ4F4YIRSatAJ2szUWqdprdMiIiLauk4hRCvsOVzN7976gUOVZubePKI5tE0NNaQlvsCu0DzOrvQh4+BfqNaNS7bKiTXuo1Un4GitK4BVwKQWmgohnCTrYDlXvrOOBpvmv7eNYnTPMABKKnK5ef755PhWcl5pDMsLH8eEPyAn1ribFoNbKRWhlAppuu0LXAjsbO/ChBCtt3LnYabO2kCwryfpd5zNwNjG3vSug2uY+ull7LPX8XrCZVwwfg4xIQFyfUg35cgYdwwwt2mc2wB8rLVe0r5lCSFaa+4PuTy1eDsDYoOYc9OI5lPYV295nwcz/46/1swd9iD9h90EyIk17qzF4NZabwGGdUAtQojTYLNrnlmSzXs/5DJhQBSvXTMUP6/GH+353z3OC/s/pY8d3rjwHaISz3FytaItyJmTQrix2nor98zP4pudhzmvTwTbCyoZ+MQyYkK8GdP9Hb6y5nCe3YsXpyzEr1sPZ5cr2ogEtxBu6lClmelzf2TnoWquHB7PF1uKMFlsGJSJiMBn+cpawxXWEJ687guMPkHOLle0IVnWVQg3tL2wksvfXMuBsjpm35jGur1lmCw2/D0O0afHs+T6V3Pu4VhWHn5CQrsTkh63EG7mmx3F3D0/ixBfTz65fTT9Y4IorDDR3X8d9rhFVCjNiIJUllRfg6LB2eWKdiDBLYQbmbN2P88syWZgbDCzb0wjMsgHu7YzIvYDdgRtI8lix5j/O5bXjwLkpJrOSoJbCDdgtdl5Zkk2c9cd4KIBUfyzaeZIZW0Jj37+e7KDSxhd7UFWwT2U6ChATqrpzCS4hXBxlSYL9/43i1U5Jdx6Tg8e+U1/jAbF9gOruH/lvRRj47GgwXgPfI7tK/aj5Go1nZ4EtxAubHdxNTM+yCTvSB3PXTGI60YmorVmwdrn+Nvu+YTZ7Lw/6A5SRt4NwOVpMuWvK5DgFsJFfbXtEPd/vBmDQRHi58ljn27jrZXbGRb3Jt/ZD3K2zcgLk96jW1yas0sVHUyCWwgXY7dr/rliF69/u4eEUD8OV5mpNlsJ9DxIQOi/+d5mZao9ioempmP0DXZ2ucIJZB63EC6kymzh1vczeP3bPVw1PB6rzY7ZaqdH4Er8erxJrYeFUQVDWFL8ZwntLkx63EK4iD2Hq5nxfiYHj9Tx9OSB3DAqkV5/XsjQ2HfYG1xMH7OdhoJr+LphOAqzs8sVTiTBLYQL+Hr7If708U/4eBr46JaRjEwOY/O+ZST1fJp9HnbOL/dn9aG7KacbIPOzuzoJbiGcyG7XvPbNbl77ZjeD44N55/rhRAQaeeOrO5h1aDVRys7owjF8XjUZUIDMzxYS3EI4TaXJwv0f/8SKHcX8LjWe564YRNGRrUz7/Da26Tou0348+tv3WFEQzLZlORTK/GzRRIJbCCfYkl/BH+dtorDCzF9/O4BpoxP5ZO0zvLz3E7ztmldiLmDCxFfBYOTySLnogfglCW4hOpDWmjlrc3nuyx2gwaY1c9b8yMo9N7KJcsbYjDw9/jUie5zn7FKFC5PgFqKDVNZZeHDBT3ydXYxBgV1Dj8BvqY9Yxna75lZDH+6+4SOUl5+zSxUuToJbiA6QdbCcu+ZlUVxlJtjHE7OlmP4xs9gbWE7vejuGwsn8z2cC90hoCwfICThCtCOtNe+u3sdV76wD4OPbRhHr+wnder7IwYAjnFcWzJ59j5FhHkthhcnJ1Qp3IT1uIdpJRV0DD3zyEyt2HOaiAVHcPU7zyurfkh9dxUCTpuHQZBabxzS3l7nZwlES3EK0g8wD5dw9bxMlNfU8dkkvrLWvceOq7/Gxa+7wGsy/C35PpcWzub3MzRat0WJwK6W6A+8D0YAdmKm1fq29CxPCHaVn5vPXxdupMlsxGhTTzypgyb7H2IuFSdqXhy98jfCEs4nNKuAlmZstTpMjPW4rcL/WepNSKhDIVEot11pnt3NtQriVd1fv429f7sCuwdNQzeCoWcyvOky01cabPaYw7rynwdB4WOnyYXES1OK0tRjcWusioKjpdrVSagcQB0hwC0GQO8SQAAAUp0lEQVTjAcj//ZjHc1/sQKPpG/QVpqhV7DHC2PIA8sz3MG76VGeXKTqRVo1xK6WSgGHAhvYoRgh3U1Jdz6PpW1ix4zAxPlsJj/6YXF8LPevtxORPZKnpwqYVRoRoOw4Ht1IqAFgI3Ke1rjrB4zOAGQAJCQltVqAQrmrZ9kP8OX0rNmsBl/Sbx2oK8bRrxhQn892RaZhonJMts0VEW3MouJVSnjSG9kda6/QTtdFazwRmAqSlpek2q1AIF1NttvDU4mzSN+3l3Pj/st0vmx+AKfYovj0wla/qo5vbymwR0R4cmVWigNnADq31K+1fkhCuZ1HTLJCCChNGBf0Cv6Rn7+/JMMJ4uy/3j3mKhN4XM1Rmi4gO4EiPewxwA7BVKbW56b4/a62/bL+yhHAdi7IKeGThFsxWO/E+PxEWvYB9vhZ6WTTv9ruJkaPvB9U4ki2zRURHcGRWyRqQ4yui63p2STY+HGRI7IfsCCrHYNeMLO7FHtttjDz7YmeXJ7ogOXNSiJOoqGvguc9W0DvwZXK6H2YXMKIihM2Hb2aFPRqFHMoRziHBLcSvaK1ZsCGLxRv/zI7AfCwhMKzKj9ySq1hhGdDcTmaLCGeR4BbiGLsLD/Da5/eS4b2buiDFhQQzIu5untobgslia24ns0WEM0lwiy5r0TEzQBKCraSFz2O9cRtVvgbOtvjzf2P/Qr9+kwHwCZfZIsJ1SHCLLmlRVgGPpm/Faq1ldNh8CkJ38LWHgRSTF3ek/h/njJj2i/YyW0S4Eglu0SX9a9l6hgfPJb/bXrZ6GOhjMhJcOIFcj4s5Z8QFzi5PiFOS4BZdSllpDh999ximyGx+MhroW+dJSNG5ZNZcCBhQmJ1dohAtkuAWnc6iE5y9mBZ+gLlrn+ZTUz4NCgaYfCkvnUSGafQvniszRYQ7kOAWncrRseujM0ACzKtJX/swfw0woYAB1cFU1l1Jcp8LSS/Kp/HaII1kpohwFxLcolN5aVkOZksDqf7f4hO2iq3+NsrtmoEVkewt/z0po87lrvG9CPD2YESPUJkpItySBLfoNMzVxfRRbxKWvJPd3gaCbZpBpUlsKfs9a+2hrHrgPJLC/Zvby0wR4a4kuIXbK9i/kv9tfJn02lwqow3E1xvpeyiFnyouJ183rokdF+L7i9AWwp1JcAu3cNwBxwt7ENWwgHm7PuY7TBiAczwjqK6cwKr8IRy7LpqMXYvORoJbuLxjDzhGGA4x0JjOrKz9HPQy0k3DDcFDKGv4PZ9s8cDLaGDCgHC25VdyqMosY9eiU5LgFi7vH19tZ7DnVwRF/MDWwFrWGQx0N3sxoGwkA3vfxqwNxWituWFUIn88vxcRgd7OLlmIdiXBLVzWkfyNfPHjq4SG/US2txFvuyapJpTCI5PINg0FYGNJEVOGxXPfhb3pHurn5IqF6BgS3MKpfj12/dD5UXSr+y+LcpeyUpmxKkUiPvQ4NJjsyt+QYQ9ofq6Pp4HP7xpLn6hAJ+6BEB1Pgls4zdGxa7PFwjDvDUR7r+S1beWUeBgJBaZ2G8LlafeSWRDNY4u2YbX/fOECb6OBF6YMltAWXZIEt3AOrflk2ULGB31JWeBBtvsa2as1PWqDCK4fx8f3PEulWTNn7X7eX7cDq13j7WGg3monTg44ii5Oglt0qOrCLL7NfIulhzewPdqOTSmi671ILunNzopL2WwNB+DZL3bxv4w86q12Jg2M5o7zejI4PsTJ1QvhGiS4RZv79bj1o2O88Kz9mKWFq1mtLDQYFHFGD/pVJ7Oz9Hx21/c6bhsfbTjIFcPiuO3cnvSKDDjBqwjRdUlwizZ1dNw6yFLERUHL0QHbeWaPlTqDgXCDgatDhzBp8HQGJ57PZ5sLeWThFo5d6AlgXO9wXvjdYFmpT4iTaDG4lVL/AS4FDmutB7V/ScItaU1NYRZrvn2esVE5bPO38YPBgL8NYqujMTdcyJI/PYTRYATAbLHRYLMTEehNXrkJgEBvDx7+TT+uH5XozD0RwuU50uN+D3gDeL99SxGu7rjTzi/qzdjgHFZtncs3pT+xwcOOJVwRaIPo6mhKqkaQXzOKQ3igAKPByMGyOj7ccICPM/KoqLPQOzKAZyYP5IrUeAK85Q9AIRzR4k+K1vp7pVRS+5ciXFnzNRot9YzwXk+kx3r+9+MhnvA2opUiztODa0OHsmLXEHJK+lCI4RfPD/X34uY5G1m1qwSDUkwcGMUNo5IYlRyKUuokryqEOBHp4ogWNRzZx6oVf2N82BYO+leT7eVBNhBb70P/qt48fdVD9IkejlKK3r+6kAE0LvdUVtvAtsIq7h7fm6kjEogO9nHa/gjh7tosuJVSM4AZAAkJCW21WdEBjjt7cUIPhvtmsSZnIWuObGejwYIp0oCH1sTXBZFQ0Zs9VePJsUShgL4xac3bumRwDNsLK/lw/cHm8E6O8Of/JvRh4sBoPI2Gk1QhhHBUmwW31nomMBMgLS1Nt9BcuIjmWSDWAsYHfo+X705mbqrioFfjf414Dw8mBw9i/b6+bC0ZSrn+5QJOsSG+aK3ZWlBJ+qYCFv9USFltA938PLkqLZ6pIxPoFx3kjF0TotOSoZJO6kQXzD32TENrzWG2Z3/MqnXppMYWk+MDGw0GvOyaeFMQA+r68eLUh0gMH4BS6rhrOQJ4exgY2j2EC1/5jr0ltXh5GLiwfyRXDIvn3D4ReHlI71qI9uDIdMD5wHlAuFIqH3hSaz27vQsTp+/XIVtQYeKp9A3UF+RhbljPxoocMlU9tQYDhEBsvScJlVGU1qRysPYsyrQ3CkiKGNi8zaOh/+LSnRRVmfEyNp5+/sXWIkYkhXLLOclcnBJDsK+nM3ZZiC7FkVkl13ZEIaLtvLQsB7ulliHem4j030yDfwF7fS08W9bYA04yeHBJYF9GJI7n2W9iyCk//kDhsSe/FFSYWL79EMt3FFNSUw9AfDdfrmi6ZqMspypEx5KhEjdzsiEQc/Uhtu1MJyvvO5KCdqIjrewzGtgHhFkgpiaMkLq+zL79z0SF9Gjensl4/BCIj4eBa0d059Xlu1ieXUx2URUAPSP8ueWcZCYNimZIfLBM4xPCSSS43cixQyCRxny62zew+Pu9/DeznBwPO9amII3zVHSvCcdc15P9tSPItcSTiyIuxPcXoQ0/D4H8/audFFaa8fcy4ulh4OWvd6EUDE/oxqO/6ceEAVEkR8iaIUK4AgluF9DSgcT6+ip27v6SL777hLTIPA771FLoZWA74Kk1ifUeTAsbwLDu5zC07xRW7bEf14v+9QVzbXbNjqIq1u0tY/2+MqrN1sb7tebspFAmDIjign6RhAXIZcCEcDUS3E52/IHEGl5b/F9y9+ZRY9vN1rpCdilLY2+6G4Ra7USbfQmvjOVwXQr7TKls1j58etslzdu8fFjj12N/Gdx/UR/6RgfynzX7WbevjA37yqhqCuse4f5cOiSW8f0iGdsrHF8vY4e/D0IIx0lwt6MWp+TZLPx7xSL6+G/Cz/sAZp8yCnwslBoUs2vB325nkPLlxsBkUqLTeGVNJNvLoznwq9eJO8EqehenxNArMoDMA+Ws21vGM0uyKa+zAJAQ6sdvBsUwumcYo5LD5CxGIdyMSwW31pqr/72OxDB/BsQE0T8miAExQQT7ud8Us+N60pUVvPLFB+TsLcbMPnJqC9mlTdRHNo5Le9k1CQ0wsCoIqzmBQ3UpLHv8QQweXs3brPY7/kCir6eR/7uwN1vyK9haUMm2gkq2FlSSc6gai63xPKi4EF/G94tidM8wRvcMO2HQCyHch0sFd3W9FR9PI6tyDrMgM7/5/rgQX/rHBDYHef+YIBJC/TAYOn5WQ0u9aLu2U1iZy3srZ9MveCfKp5hq72oOe9ooV4oPayHQZqcfnlztG8+Bom7kVvZmf/0gsvD/xT4fG9rQeCCxtt7Kqyt2UVrTgJ+XkVA/Lx79dGtzSAf5eJASH8wfxvZgUGwwQ7uHEN/NV2aACNGJKK3b/uz0tLQ0nZGRcUbbOFxtJruwih1F1ewoqiK7qIp9JTUcvV6sv5eRXlGBJIT60b2bb+PXUD8SQv2ICfbBo5VrYrQUyEfb/Nzj1RiMVcT57+GiPqVYVB576grZY6vDdExGxlisRDd44GfuRkN9d4pN/Vny2J9QXn4n2GYjHw8Dd5zXk6Rwf/aX1pJbWsv+sjpyS2upNFma24X4eZISF8yguGAGxQaTEhdM91AJaSHckVIqU2ud1nJLFw7uEzFbbOwqrm4K9Cr2lNSQd8REQYUJ2zFXADcaFDHBPk2h7kf3UF/CArzp5udJNz8vuvl7EdJ029NoOGF4+noaeX5KCpcOiaKw6iD7Czfyr+VLsah8LF5VlHuZqTnmGF6ozUZvu5Fe3mH0Ckpi5a5gdlT04qAtAesxf9hEBXrz1vWplFQ3UFpTT2lNPT/uP0LmwXLMFjsGBcfsCkpBbLAvSeF+JIX50yPcn8Qwf/pFB0pPWohOpNMG98lYbXaKKs3kldeRd6SOvCMm8srrOHikjj2Ha5qnup1IoLcHdRYbNmrx9yokwnsvgd55KK9S6rxrOOJpxXJMNobabMQ3aIItPniYQzHVd6fY3IdhQy6g2u5NvdWO2WIjr7zxte0OvL0hfp6EB3gTGehNYpg/PcL9mr76kxDqh4+nzPIQorNrTXC71Bi3o042rNE91A96/rLdo+lbm77TKGMtkT4HGZt0BE9jIaX1RZTayzmkzNQ2ZWMZUKE18RYriRYjKXUBGBrCqGmIp6iuJ7m2JA7wyxNRFFCYU423Ry0+ngZ8PI0E+njSKzKAA2V11FvtBHh7ML5fJOP6RBAe4EV4gDcRgd508/OSxZiEEK3iUsHd+nHmxnU0jobzpIHdKCzZSn7JNgrL97B42xb6RZVj8jRR7mmlzgh1wNd2MNg0MTYbCXiQYPLCYA7E2hBBZUM8BfV9yNGxlIcEsvaR8b947UcWbgHrzxe3PTqk8us6hRCivbhMcJ8qkC8fFgcNddRU5LJg+UJSfArwCixFeVZg9aymztPMK1k2Ht/yy/FerwBNtNVGpNVIj2pfjJZgGhoiqa6PZ+6dN+EV1hM8vE86xn3smYbNdUCLv1yEEKI9uUxwN65oV8PvAj7B4FmB3bOaeg8Tn61vYG6mptigqDYaIOrn5xi1JsKqCbF4kFzjz+j4JOIC44nv1pu4iIFcP6+M7ZUeNA5m/CwuxBevqAHN37cmkC9vWhFPCCGcxWWCu7DChEEpvu6e3XxfgA2CLN709Q/lLJ9QYvyiWb7dRnFNKJUNsZRbo6mgca5zXIgv700Z/4tt3jHJsZ40SCALIdyHywR3bIgvBRVgOXAz9dZQtCWYau2FIcSX12/7OZC7RTaFsdWxMAYZ2hBCdC4uE9wPTuzbGMh1PwdwW4wzS09aCNHZuExwyzizEEI4xmWCGySQhRDCEXLmhxBCuBkJbiGEcDMS3EII4WYkuIUQws1IcAshhJuR4BZCCDfTLutxK6VK4Lhr2rZGOFDaRuW4Atkf1yb749o62/7AifcpUWsd4ciT2yW4z5RSKsPRBcXdgeyPa5P9cW2dbX/gzPdJhkqEEMLNSHALIYSbcdXgnunsAtqY7I9rk/1xbZ1tf+AM98klx7iFEEKcnKv2uIUQQpyESwS3UuoqpdR2pZRdKXXSI61KqUlKqRyl1B6l1CMdWWNrKKVClVLLlVK7m752O0k7m1Jqc9O/zzu6zpa09H4rpbyVUv9renyDUiqp46t0nAP7c5NSquSYz+QWZ9TpKKXUf5RSh5VS207yuFJKvd60v1uUUqkdXWNrOLA/5ymlKo/5fJ7o6BodpZTqrpRaqZTa0ZRt956gzel/Plprp/8D+gN9gVVA2knaGIG9QDLgBfwEDHB27Sep9e/AI023HwFePEm7GmfXeop9aPH9Bu4E3mm6fQ3wP2fXfYb7cxPwhrNrbcU+jQNSgW0nefxiYCmNF10dBWxwds1nuD/nAUucXaeD+xIDpDbdDgR2neD/22l/Pi7R49Za79Ba57TQbASwR2u9T2vdAPwXmNz+1Z2WycDcpttzgcudWMvpcuT9PnY/FwAXKKUUrsmd/v84RGv9PXDkFE0mA+/rRuuBEKVUTMdU13oO7I/b0FoXaa03Nd2uBnYAv77YwGl/Pi4R3A6KA/KO+T6f498IVxGltS6Cxg8QiDxJOx+lVIZSar1SytXC3ZH3u7mN1toKVAJhHVJd6zn6/+d3TX+2LlBKde+Y0tqNO/3MOGq0UuonpdRSpdRAZxfjiKYhxGHAhl89dNqfT4ddAUcptQKIPsFDf9Faf+bIJk5wn9OmxJxqf1qxmQStdaFSKhn4Vim1VWu9t20qPGOOvN8u9Zm0wJFaFwPztdb1SqnbafxrYvzxT3Mb7vT5OGITjaeF1yilLgYWAb2dXNMpKaUCgIXAfVrrql8/fIKnOPT5dFhwa60vPMNN5APH9oDigcIz3OZpO9X+KKWKlVIxWuuipj99Dp9kG4VNX/cppVbR+FvZVYLbkff7aJt8pZQHEIzr/qnb4v5orcuO+XYW8GIH1NWeXOpn5kwdG3xa6y+VUm8ppcK11i65jolSypPG0P5Ia51+gian/fm401DJj0BvpVQPpZQXjQfDXG4mRpPPgRubbt8IHPcXhVKqm1LKu+l2ODAGyO6wClvmyPt97H5eCXyrm466uKAW9+dX44uX0Tgu6c4+B6Y1zV4YBVQeHcJzR0qp6KPHUJRSI2jMr7JTP8s5muqcDezQWr9ykman//k4++hr08/5FTT+9qkHioFlTffHAl/+6ijsLhp7pX9xdt2n2J8w4Btgd9PX0Kb704B3m26fDWylcXbDVmC6s+s+wX4c934DTwOXNd32AT4B9gAbgWRn13yG+/M8sL3pM1kJ9HN2zS3sz3ygCLA0/fxMB24Hbm96XAFvNu3vVk4yY8tV/jmwP3cd8/msB852ds2n2JexNA57bAE2N/27uK0+HzlzUggh3Iw7DZUIIYRAglsIIdyOBLcQQrgZCW4hhHAzEtxCCOFmJLiFEMLNSHALIYSbkeAWQgg38/9KOj5VQUSbRQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x9313c70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from utils import polynomial_features\n",
    "features, values = generate_data_part_2()\n",
    "plt.scatter([x[0] for x in features], values, label='origin');\n",
    "\n",
    "for k in [2, 4, 8]:\n",
    "    features_extended = polynomial_features(features, k)\n",
    "    model = LinearRegression(nb_features=k)\n",
    "    model.train(features_extended, values)\n",
    "    mse = mean_squared_error(values, model.predict(features_extended))\n",
    "    print('[part 1.3.2]\\tk: {k:d}\\tmse: {mse:.5f}'.format(k=k, mse=mse))\n",
    "    plt.plot([x[0] for x in features], model.predict(features_extended), label='k={k}'.format(k=k));\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1.4 Train, validation, test\n",
    "\n",
    "### Data processing \n",
    "\n",
    "Do the following steps:\n",
    "\n",
    "- Load data (features and values) from function generate data part 3. It’s a classification dataset, but we just use it as a regression dataset in this assignment.\n",
    "- Check that there are 150 data samples and each sample have a feature vector of length 4.\n",
    "- Split the whole data set into three parts:\n",
    "    - the train set contains first 100 samples (0th - 99th samples),\n",
    "    - the validation set contains the next 20 samples (100th - 119th samples),\n",
    "    - the test set contains the rest 30 samples (120th - 149th samples)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from data import generate_data_part_3\n",
    "features, values = generate_data_part_3()\n",
    "\n",
    "train_features, train_values = features[:100], values[:100]\n",
    "valid_features, valid_values = features[100:120], values[100:120]\n",
    "test_features, test_values = features[120:], values[120:]\n",
    "\n",
    "assert len(train_features) == len(train_values) == 100\n",
    "assert len(valid_features) == len(valid_values) == 20\n",
    "assert len(test_features) == len(test_values) == 30"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1.4.1 LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyper-parameter and model selection \n",
    "\n",
    "For linear regression model with extra polynomial features, $k$ is a hyper-parameter. \n",
    "To choose the best one, we have to \n",
    "\n",
    "    - train a model with that hyper-parameter based on the train set, \n",
    "    - calculate its performance on the validation set\n",
    "    - select the best hyper-parameter (the trained model has the best performance on validation set). \n",
    "    \n",
    "In this task, we only search $k$ among the set {1, 3, 10}. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.4.1]\tk: 1\ttrain mse: 0.00909\tvalid mse: 0.33937\n",
      "[part 1.4.1]\tk: 3\ttrain mse: 0.00261\tvalid mse: 3.28984\n",
      "[part 1.4.1]\tk: 10\ttrain mse: 0.49996\tvalid mse: 129696.11598\n"
     ]
    }
   ],
   "source": [
    "best_mse, best_k = 1e10, -1\n",
    "for k in [1, 3, 10]:\n",
    "    train_features_extended = polynomial_features(train_features, k)\n",
    "    model = LinearRegression(nb_features=k)\n",
    "    model.train(train_features_extended, train_values)\n",
    "    train_mse = mean_squared_error(train_values, model.predict(train_features_extended))\n",
    "\n",
    "    valid_features_extended = polynomial_features(valid_features, k)\n",
    "    valid_mse = mean_squared_error(valid_values, model.predict(valid_features_extended))\n",
    "    print('[part 1.4.1]\\tk: {k:d}\\t'.format(k=k) + \n",
    "          'train mse: {train_mse:.5f}\\tvalid mse: {valid_mse:.5f}'.format(\n",
    "              train_mse=train_mse, valid_mse=valid_mse))\n",
    "\n",
    "    if valid_mse < best_mse:\n",
    "        best_mse, best_k = valid_mse, k"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.4.1 Linear Regression]\tbest_k: 1\ttest mse: 0.08778\n"
     ]
    }
   ],
   "source": [
    "combined_features_extended = polynomial_features(train_features + test_features, best_k)\n",
    "model = LinearRegression(nb_features=best_k)\n",
    "model.train(combined_features_extended, train_values + test_values)\n",
    "\n",
    "test_features_extended = polynomial_features(test_features, best_k)\n",
    "test_mse = mean_squared_error(test_values, model.predict(test_features_extended))\n",
    "print('[part 1.4.1 Linear Regression]\\tbest_k: {best_k:d}\\ttest mse: {test_mse:.5f}'.format(\n",
    "    best_k=best_k, test_mse=test_mse))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1.4.2 Linear Regression With L2 Loss (Ridge Regression)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For Ridge (linear regression with l2 loss) Regression, we still need to search $k$ among {1, 3, 10}.\n",
    "$\\alpha$ is also a hyper-parameter, in this task, we search $\\alpha$ among the set {0.01, 0.1, 1, 10}."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.4.2]\tk: 1\talpha: 0.01\ttrain mse: 0.00909\tvalid mse: 0.33964\n",
      "[part 1.4.2]\tk: 1\talpha: 0.1\ttrain mse: 0.00911\tvalid mse: 0.34196\n",
      "[part 1.4.2]\tk: 1\talpha: 1\ttrain mse: 0.00955\tvalid mse: 0.35604\n",
      "[part 1.4.2]\tk: 1\talpha: 10\ttrain mse: 0.01058\tvalid mse: 0.40436\n",
      "[part 1.4.2]\tk: 3\talpha: 0.01\ttrain mse: 0.00270\tvalid mse: 3.33796\n",
      "[part 1.4.2]\tk: 3\talpha: 0.1\ttrain mse: 0.00288\tvalid mse: 2.69893\n",
      "[part 1.4.2]\tk: 3\talpha: 1\ttrain mse: 0.00336\tvalid mse: 2.12091\n",
      "[part 1.4.2]\tk: 3\talpha: 10\ttrain mse: 0.00412\tvalid mse: 1.56060\n",
      "[part 1.4.2]\tk: 10\talpha: 0.01\ttrain mse: 0.00019\tvalid mse: 15779.19892\n",
      "[part 1.4.2]\tk: 10\talpha: 0.1\ttrain mse: 0.00027\tvalid mse: 9166.65667\n",
      "[part 1.4.2]\tk: 10\talpha: 1\ttrain mse: 0.00038\tvalid mse: 2407.88710\n",
      "[part 1.4.2]\tk: 10\talpha: 10\ttrain mse: 0.00084\tvalid mse: 1519.89814\n"
     ]
    }
   ],
   "source": [
    "from itertools import product\n",
    "\n",
    "best_mse, best_k, best_alpha = 1e10, -1, -1\n",
    "for k, alpha in product([1, 3, 10], [0.01, 0.1, 1, 10]):\n",
    "    train_features_extended = polynomial_features(train_features, k)\n",
    "    model = LinearRegressionWithL2Loss(nb_features=k, alpha=alpha)\n",
    "    model.train(train_features_extended, train_values)\n",
    "    train_mse = mean_squared_error(train_values, model.predict(train_features_extended))\n",
    "\n",
    "    valid_features_extended = polynomial_features(valid_features, k)\n",
    "    valid_mse = mean_squared_error(valid_values, model.predict(valid_features_extended))\n",
    "    print('[part 1.4.2]\\tk: {k:d}\\talpha: {alpha}\\t'.format(k=k, alpha=alpha) +\n",
    "          'train mse: {train_mse:.5f}\\tvalid mse: {valid_mse:.5f}'.format(\n",
    "              train_mse=train_mse, valid_mse=valid_mse))\n",
    "\n",
    "    if valid_mse < best_mse:\n",
    "        best_mse, best_k, best_alpha = valid_mse, k, alpha"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 1.4.2]\tbest_k: 1\tbest_alpha: 0.010000\ttest mse: 0.08777\n"
     ]
    }
   ],
   "source": [
    "combined_features_extended = polynomial_features(train_features + test_features, best_k)\n",
    "model = LinearRegressionWithL2Loss(nb_features=best_k, alpha=best_alpha)\n",
    "model.train(combined_features_extended, train_values + test_values)\n",
    "\n",
    "test_features_extended = polynomial_features(test_features, best_k)\n",
    "test_mse = mean_squared_error(test_values, model.predict(test_features_extended))\n",
    "print('[part 1.4.2]\\tbest_k: {best_k:d}\\tbest_alpha: {best_alpha:f}\\t'.format(\n",
    "    best_k=best_k, best_alpha=best_alpha) +\n",
    "      'test mse: {test_mse:.5f}'.format(test_mse=test_mse))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 2: K-nearest neighbor (KNN) for binary classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Some notes\n",
    "\n",
    "In this task, we will use three distance functions: (we removed the vector symbol for simplicity)\n",
    "\n",
    "- Euclidean distance:  $$d(x, y) = \\sqrt{\\langle x - y, x - y \\rangle}$$\n",
    "- Inner product distance: $$d(x, y ) = \\langle x, y \\rangle$$\n",
    "- Gaussian kernel distance: \n",
    "    $$d(x, y ) = \\exp({−\\frac 12 \\sqrt{\\langle x - y, x - y \\rangle}}) $$\n",
    "\n",
    "\n",
    "F1-score is a important metric for binary classification, as sometimes the accuracy metric has the false positive (a good example is in MLAPP book 2.2.3.1 “Example: medical diagnosis”, Page 29)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2.1 Distance Functions\n",
    "\n",
    "Implement the class in file *hw1_knn.py*\n",
    "    - KNN\n",
    "    \n",
    "and the functions in *utils.py*    \n",
    "    - f1_score\n",
    "    - euclidean_distance\n",
    "    - inner_product_distance\n",
    "    - gaussian_kernel_distance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "from hw1_knn import KNN\n",
    "from utils import euclidean_distance, gaussian_kernel_distance, inner_product_distance\n",
    "from utils import f1_score\n",
    "\n",
    "distance_funcs = {\n",
    "    'euclidean': euclidean_distance,\n",
    "    'gaussian': gaussian_kernel_distance,\n",
    "    'inner_prod': inner_product_distance,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data processing \n",
    "\n",
    "Do the following steps:\n",
    "\n",
    "- Load data (features and values) from function generate data cancer\n",
    "- Check that there are 569 data samples and each sample have a feature vector of length 30.\n",
    "- Split the whole data set into three parts:\n",
    "     - the train set contains first 400 samples (0th - 399th samples),\n",
    "     - the validation set contains the next 60 samples (400th - 459th samples),\n",
    "     - the test set contains the rest 109 samples (460th - 568th samples)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from data import generate_data_cancer\n",
    "features, labels = generate_data_cancer()\n",
    "\n",
    "train_features, train_labels = features[:400], labels[:400]\n",
    "valid_features, valid_labels = features[400:460], labels[400:460]\n",
    "test_features, test_labels = features[460:], labels[460:]\n",
    "\n",
    "assert len(train_features) == len(train_labels) == 400\n",
    "assert len(valid_features) == len(valid_labels) == 60\n",
    "assert len(test_features) == len(test_labels) == 109"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model selection \n",
    "In kNN model, the parameter k is a hyper-parameter. In this task, we only search k among {1, 3, 10, 20, 50}."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 2.1] euclidean\tk: 1\ttrain: 1.00000\tvalid: 0.95000\n",
      "[part 2.1] euclidean\tk: 3\ttrain: 0.95250\tvalid: 0.96667\n",
      "[part 2.1] euclidean\tk: 10\ttrain: 0.94250\tvalid: 0.96667\n",
      "[part 2.1] euclidean\tk: 20\ttrain: 0.94000\tvalid: 0.96667\n",
      "[part 2.1] euclidean\tk: 50\ttrain: 0.90500\tvalid: 0.95000\n",
      "\n",
      "[part 2.1] euclidean\tbest_k: 3\ttest f1 score: 0.92661\n",
      "\n",
      "[part 2.1] gaussian\tk: 1\ttrain: 1.00000\tvalid: 0.93333\n",
      "[part 2.1] gaussian\tk: 3\ttrain: 0.94500\tvalid: 0.95000\n",
      "[part 2.1] gaussian\tk: 10\ttrain: 0.91250\tvalid: 0.93333\n",
      "[part 2.1] gaussian\tk: 20\ttrain: 0.85000\tvalid: 0.73333\n",
      "[part 2.1] gaussian\tk: 50\ttrain: 0.64750\tvalid: 0.51667\n",
      "\n",
      "[part 2.1] gaussian\tbest_k: 3\ttest f1 score: 0.88991\n",
      "\n",
      "[part 2.1] inner_prod\tk: 1\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.1] inner_prod\tk: 3\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.1] inner_prod\tk: 10\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.1] inner_prod\tk: 20\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.1] inner_prod\tk: 50\ttrain: 0.56750\tvalid: 0.78333\n",
      "\n",
      "[part 2.1] inner_prod\tbest_k: 1\ttest f1 score: 0.76147\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for name, func in distance_funcs.items():\n",
    "    best_f1_score, best_k = -1, 0\n",
    "    for k in [1, 3, 10, 20, 50]:\n",
    "        model = KNN(k=k, distance_function=func)\n",
    "        model.train(train_features, train_labels)\n",
    "        train_f1_score = f1_score(\n",
    "            train_labels, model.predict(train_features))\n",
    "\n",
    "        valid_f1_score = f1_score(\n",
    "            valid_labels, model.predict(valid_features))\n",
    "        print('[part 2.1] {name}\\tk: {k:d}\\t'.format(name=name, k=k) + \n",
    "              'train: {train_f1_score:.5f}\\t'.format(train_f1_score=train_f1_score) +\n",
    "              'valid: {valid_f1_score:.5f}'.format(valid_f1_score=valid_f1_score))\n",
    "\n",
    "        if valid_f1_score > best_f1_score:\n",
    "            best_f1_score, best_k = valid_f1_score, k\n",
    "\n",
    "    model = KNN(k=best_k, distance_function=func)\n",
    "    model.train(train_features + valid_features,\n",
    "                train_labels + valid_labels)\n",
    "    test_f1_score = f1_score(test_labels, model.predict(test_features))\n",
    "    print()\n",
    "    print('[part 2.1] {name}\\tbest_k: {best_k:d}\\t'.format(name=name, best_k=best_k) +\n",
    "          'test f1 score: {test_f1_score:.5f}'.format(test_f1_score=test_f1_score))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2.2 Data transformation\n",
    "\n",
    "We are going to add one more step (data transformation) in the data processing part and see how it works. \n",
    "Sometimes, normalization plays an important role to make a machine learning model work (check term “Feature scaling” in wiki).\n",
    "\n",
    "Here, we take two different data transformation approaches.\n",
    "\n",
    "#### Normalizing the feature vector \n",
    "\n",
    "This one is simple but some times may work well. Given a feature vector $x$, the normalized feature vector is given by \n",
    "\n",
    "$$ x' = \\frac x {\\sqrt{\\langle x, x \\rangle}} $$\n",
    "If a vector is a all-zero vector, we let the normalized vector also be a all-zero vector.\n",
    "\n",
    "\n",
    "#### Min-max scaling the feature matrix\n",
    "\n",
    "The above normalization is data independent, that is to say, the output of the normalization function doesn’t depend on the rest training data. However, sometimes it would be helpful to do data dependent normalization. One thing to note is that, when doing data dependent normalization, we can only use training data, as the test data is assumed to be unknown during training (at least for most classification tasks).\n",
    "\n",
    "The min-max scaling works as follows: after min-max scaling, all values of training data’s feature vectors are in the given range.\n",
    "Note that this doesn’t mean the values of the validation/test data’s fea- tures are all in that range, because the validation/test data may have dif- ferent distribution as the training data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Implement** the functions in *utils.py*    \n",
    "    - normalize\n",
    "    - min_max_scale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import NormalizationScaler, MinMaxScaler\n",
    "\n",
    "scaling_functions = {\n",
    "    'min_max_scale': MinMaxScaler,\n",
    "    'normalize': NormalizationScaler,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model selection\n",
    "\n",
    "Repeat the model selection part in part 2.2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[part 2.2] euclidean\tmin_max_scale\tk: 1\ttrain: 1.00000\tvalid: 0.93333\n",
      "[part 2.2] euclidean\tmin_max_scale\tk: 3\ttrain: 0.98000\tvalid: 0.98333\n",
      "[part 2.2] euclidean\tmin_max_scale\tk: 10\ttrain: 0.97000\tvalid: 0.96667\n",
      "[part 2.2] euclidean\tmin_max_scale\tk: 20\ttrain: 0.97250\tvalid: 0.96667\n",
      "[part 2.2] euclidean\tmin_max_scale\tk: 50\ttrain: 0.95000\tvalid: 0.98333\n",
      "\n",
      "[part 2.2] euclidean\tmin_max_scale\tbest_k: 3\ttest: 0.94495\n",
      "\n",
      "[part 2.2] gaussian\tmin_max_scale\tk: 1\ttrain: 1.00000\tvalid: 0.93333\n",
      "[part 2.2] gaussian\tmin_max_scale\tk: 3\ttrain: 0.98000\tvalid: 0.98333\n",
      "[part 2.2] gaussian\tmin_max_scale\tk: 10\ttrain: 0.97000\tvalid: 0.96667\n",
      "[part 2.2] gaussian\tmin_max_scale\tk: 20\ttrain: 0.97250\tvalid: 0.96667\n",
      "[part 2.2] gaussian\tmin_max_scale\tk: 50\ttrain: 0.95000\tvalid: 0.98333\n",
      "\n",
      "[part 2.2] gaussian\tmin_max_scale\tbest_k: 3\ttest: 0.94495\n",
      "\n",
      "[part 2.2] inner_prod\tmin_max_scale\tk: 1\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.2] inner_prod\tmin_max_scale\tk: 3\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.2] inner_prod\tmin_max_scale\tk: 10\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.2] inner_prod\tmin_max_scale\tk: 20\ttrain: 0.56750\tvalid: 0.78333\n",
      "[part 2.2] inner_prod\tmin_max_scale\tk: 50\ttrain: 0.56750\tvalid: 0.78333\n",
      "\n",
      "[part 2.2] inner_prod\tmin_max_scale\tbest_k: 1\ttest: 0.76147\n",
      "\n",
      "[part 2.2] euclidean\tnormalize\tk: 1\ttrain: 1.00000\tvalid: 0.90000\n",
      "[part 2.2] euclidean\tnormalize\tk: 3\ttrain: 0.94750\tvalid: 0.91667\n",
      "[part 2.2] euclidean\tnormalize\tk: 10\ttrain: 0.92750\tvalid: 0.98333\n",
      "[part 2.2] euclidean\tnormalize\tk: 20\ttrain: 0.91750\tvalid: 0.98333\n",
      "[part 2.2] euclidean\tnormalize\tk: 50\ttrain: 0.91000\tvalid: 0.98333\n",
      "\n",
      "[part 2.2] euclidean\tnormalize\tbest_k: 10\ttest: 0.87156\n",
      "\n",
      "[part 2.2] gaussian\tnormalize\tk: 1\ttrain: 1.00000\tvalid: 0.90000\n",
      "[part 2.2] gaussian\tnormalize\tk: 3\ttrain: 0.94750\tvalid: 0.91667\n",
      "[part 2.2] gaussian\tnormalize\tk: 10\ttrain: 0.92750\tvalid: 0.98333\n",
      "[part 2.2] gaussian\tnormalize\tk: 20\ttrain: 0.91750\tvalid: 0.98333\n",
      "[part 2.2] gaussian\tnormalize\tk: 50\ttrain: 0.91000\tvalid: 0.98333\n",
      "\n",
      "[part 2.2] gaussian\tnormalize\tbest_k: 10\ttest: 0.87156\n",
      "\n",
      "[part 2.2] inner_prod\tnormalize\tk: 1\ttrain: 0.39750\tvalid: 0.18333\n",
      "[part 2.2] inner_prod\tnormalize\tk: 3\ttrain: 0.36500\tvalid: 0.20000\n",
      "[part 2.2] inner_prod\tnormalize\tk: 10\ttrain: 0.20500\tvalid: 0.15000\n",
      "[part 2.2] inner_prod\tnormalize\tk: 20\ttrain: 0.19000\tvalid: 0.11667\n",
      "[part 2.2] inner_prod\tnormalize\tk: 50\ttrain: 0.11250\tvalid: 0.06667\n",
      "\n",
      "[part 2.2] inner_prod\tnormalize\tbest_k: 3\ttest: 0.19266\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for scaling_name, scaling_class in scaling_functions.items():\n",
    "    for name, func in distance_funcs.items():\n",
    "        scaler = scaling_class()\n",
    "        train_features_scaled = scaler(train_features)\n",
    "        valid_features_scaled = scaler(valid_features)\n",
    "\n",
    "        best_f1_score, best_k = 0, -1\n",
    "        for k in [1, 3, 10, 20, 50]:\n",
    "            model = KNN(k=k, distance_function=func)\n",
    "            model.train(train_features_scaled, train_labels)\n",
    "            train_f1_score = f1_score(\n",
    "                train_labels, model.predict(train_features_scaled))\n",
    "            \n",
    "            valid_f1_score = f1_score(\n",
    "                valid_labels, model.predict(valid_features_scaled))\n",
    "            print('[part 2.2] {name}\\t{scaling_name}\\tk: {k:d}\\t'.format(name=name, scaling_name=scaling_name, k=k) +\n",
    "                  'train: {train_f1_score:.5f}\\t'.format(train_f1_score=train_f1_score) + \n",
    "                  'valid: {valid_f1_score:.5f}'.format(valid_f1_score=valid_f1_score))\n",
    "\n",
    "            if valid_f1_score > best_f1_score:\n",
    "                best_f1_score, best_k = valid_f1_score, k\n",
    "    \n",
    "\n",
    "        # now change it to new scaler, since the training set changes\n",
    "        scaler = scaling_class()\n",
    "        combined_features_scaled = scaler(train_features + valid_features)\n",
    "        test_features_scaled = scaler(test_features)\n",
    "\n",
    "        model = KNN(k=best_k, distance_function=func)\n",
    "        model.train(combined_features_scaled, train_labels + valid_labels)\n",
    "        test_f1_score = f1_score(test_labels, model.predict(test_features_scaled))\n",
    "        print()\n",
    "        print('[part 2.2] {name}\\t{scaling_name}\\t'.format(name=name, scaling_name=scaling_name) +\n",
    "              'best_k: {best_k:d}\\ttest: {test_f1_score:.5f}'.format(best_k=best_k, test_f1_score=test_f1_score))\n",
    "        print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 3: Perceptron Problem \n",
    "In this problem we will implement perceptron algorithm. Recall that perceptron algorithm can converge only when the data is linearly seperable. \n",
    "\n",
    "### Objective \n",
    "Implement the class `Perceptron` in file `hw1_perceptron.py`. \n",
    "\n",
    "#### Some notes\n",
    "- Perceptron update rule is whenever algorithm makes a mistake update\n",
    "weights as $$w ← w + \\frac{y_i x_i}{\\|x\\|}$$\n",
    "- Perceptron algorithm as discussed only works for linearly seperable\n",
    "data. In this problem you will see that it is indeed the case.\n",
    "- For data which is not linearly seperable there is class of model/classifiers\n",
    "called maximum margin classifiers which will be discussed later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "import matplotlib.pyplot as plt \n",
    "from data import generate_data_perceptron \n",
    "from hw1_perceptron import Perceptron\n",
    "import numpy as np\n",
    "\n",
    "## To clearly visualize the problem, we just use 2 features for now\n",
    "## y = f(x1,x2)\n",
    "\n",
    "nb_features=2\n",
    "model = Perceptron(nb_features=nb_features)\n",
    "x, y = generate_data_perceptron(nb_features=nb_features, seperation=2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VGX2wPHvuVPTCC2ABUTUtWKNvWNjLdjW3nXFxq5rXeu66ro/Xeta17qua69rxV6wKyqiqCgqKk1KKCnT7/n9cQMkzEwIyWQmmTmf58kDM/fOvWcgOXnnveeeV1QVY4wxxcMpdADGGGNyyxK7McYUGUvsxhhTZCyxG2NMkbHEbowxRcYSuzHGFBlL7MYYU2QssRtjTJGxxG6MMUXGX4iT9u/fX4cOHVqIUxtjTI/1ySefzFXVmuXtV5DEPnToUMaPH1+IUxtjTI8lIj+1Zz+bijHGmCJjid0YY4qMJXZjjCkyltiNMabIWGI3xpgiY4m9i7z/7HhO2vhsRlUfzZitzuez178odEjGmBJhib0LvP7wO1xx2PX8MPEnIvURJn80hYv3uZJPX51Y6NCMMSXAEnuOqSp3nvtfYk3xVs/HInHuOPe/BYrKGFNKLLHnWCKeZN6M+Rm3/fLN9DxHY4wpRZbYcywQ9FNRXZ5xW1XfyjxHY4wpRZbYc0xE2P2YnTJua5jfSCwSy29AxpiSk7PELiI+EflMRJ7L1TF7IlXlwxc+zbjN5/fx0dgJeY7IGFNqcjliPx34OofH65E+f3MSs374NeO2ZCpFpD6S54iMMaUmJ4ldRFYF9gLuysXxerIPnhtPKuVm3JaKJ9l01+F5jsgYU2pyNWK/ATgXyJzRSkh5r3L8fl/GbVuP2pz+q/TLc0TGmFLT6cQuInsDs1X1k+XsN1pExovI+Dlz5nT2tN3WrkfugJMhsQfLgpzz79MKEJExptTkYsS+LTBKRKYCDwMjROT+ZXdS1TtUtVZVa2tqlrsASI+18hqDOOOOkwiWBSnvVUZZZZhgWZDDz98ffyDzSN4YY3Kp04ldVc9X1VVVdShwKPC6qh7Z6ch6sF2P2IFHZ97J1qNqiUcTOI7w6NVPc/BKJ/LV+5MLHZ4xpshZHXsXmfrlL7zz5EekkimijTGa6qM0Lmziwr3/j0Q8UejwjDFFLKeJXVXfVNW9c3nMnurFu18jHomnPe+mXCa8/mUBIjLGlAobsXeRpoYoqppxW7QpPeEbY0yuWGLvIjsetDXhilDa88l4ik1GbFCAiIwxpcISexfZdv8tGL79uoQrveTu+BxCZUFOueFYKntXFDg6Y0wx8xc6gGLl8/m4/Nnz+OiFz3jnyQ+p6F3Ob48fwerDVyt0aMaYImeJvQv5fD623qeWrfepLXQoxpgSYlMxxhhTZCyxG2NMkbGpmC42++c5vPXo+8SjCbbce1PW3Hj1QodkWtDEN5D6CfxrIf5hhQ7HmJywxN6FXr3/La4ffTvqKqmUy0NXPsmev9+VU284rtChlTx1G9D5oyHxJYgfNIGGtkF634RIsNDhGdMpNhXTRRbNq+f60bcTjyZIxJO4KZdYU5yxd7/Gl+9+U+jwSp7WXw6JiUAUtAGIQew9tOGmQodmTKdZYu8iH784AV+G9r2xpjhvPPROASIyi6mmIPI8sOwdwDFoeqQQIRmTU5bYu4gIIJk2gDiZNpj8SQHJzJs0mtdIjOkKlti7yBZ7bkoqmb6gVDAcZMTh2xcgIrOYSBD862fY4kBo27zHY0yuWWLvIpW9Kzj33jEEy4KEyoL4g36CZUH2/+NvWW+r3xQ6vJIn1ZeDVACLL5SGQKqQqvMLGZYxOWFVMV1ox4O2Zvj26/D2Ex965Y57bcqQdVZh2rcz+OSViVRUl7PNvptTXlVW6FBLjgTWg/5j0aYHITkZAhsi5YchTt9Ch2ZMp0m21rJdqba2VsePH5/38xaaqnLbGffy/J2v4qZSpJIu6iqD116ZU64/ls1HblLoEI0x3ZiIfKKqy+1RkovFrMMi8pGIfC4ik0Tk0s4es1iNf2kCY5sX4EjGU6jr/VL9ZfIMLj3wGt5+8sMCR2iMKQa5mGOPASNUdSNgY2CkiGyVg+MWnRfveZ1oYyzjtlgkzu1n/SfPERljilGn59jVm8tpaH4YaP7K//xOD5CIZymxazb757kkE0n8Abv00ROouwht+CdExwIOlO2PVJ6KiF0zMYWVk6oYEfGJyARgNvCKqqbNKYjIaBEZLyLj58yZk4vT9ji7HL59xlWVFqvoXZ7xpibTtTQ5FY2NQ1Oz2v8aTaDzDvZuaHLngjsbGu9F647JuiSiMfmSk8SuqilV3RhYFdhCRNLWflPVO1S1VlVra2pqcnHaHme7A7dkox0z1U8DApvtthEidvNSvqjbiFt3LDp3FLrgDHTObrgLzvXuTF2e2GvgzqL13asxSH4LiY+7KmRj2iWndeyqugB4ExiZy+MWC5/PR2XfChxfhuSt8P6z4xl792v5D6xE6aJLIT4er19MPRCD6Ito413Lf218ImhThg0JSEzKeazGrIhOT+aKSA2QUNUF4k0u7gpc1enIipCqMu6x93FTmT+qxyNx7rngwSV17bV7bERFta2P2hVU4xB9gfR+MVFouh8qT2rz9eIfglIGRJbZEATfKrkM1ZgVlourdCsB/xERH94ngEdV9bkcHLfoqCrJRNsf8xfMWcS1J96GICSTKc6551R2OsRuc885TeD1jMm0rSHz8y2F94b6a5t7yyz+Re2AVEJo5xwFaUzHdHoqRlUnquomqrqhqm6gqpflIrBi5DgOw7dfd7nz6JH6KE31EeKROFcfdytzp8/LU4SlQ5wK8A3NtAWcAbizt8edvTXuwstQd0GG11ci/R5q7jnj974CmyL9HkYk0MXRG9M26xWTZ6ffNpqK6nL8wfZ9WFJV3nr0/S6OqjRJ9d9Aylj6wTUAOJCaDu6v4M6DyCPovIO8qZtlX+9fE6f/k8iA95EBH+L0exCxaRjTDVhiz7Mh66zCvd/eyLGXHcJ62/yGyt7eHHpZZRjHl/7fkUqmst7UZDpHgpsh/Z6GskMgsAWE98RL7i2TeALcORB9KftxnGrEqerqcNOo24DGPkAT3+b93KZ7szthCqC6fy8OOXc/Djl3PwBc1+WLt7/mvN0vx11m2jcQ8rPl3psWIMrSIP6hSPUlAGjjf9Hoi+k7aRMan4CU7ZPn6LJzG++G+htAAqAp1D8E6XMn4hvUqeNq8keIvQ4EILx7p49nCsNG7N3AS/e+wQW/vQJ3mRtbwhUhRh43whbAzhffqt76p2nC4B+a72iy0ti7UH8jEGu+0BuB5BR0/smdOq7bcLNX019/HVp/tVfX3/RUTmI2+WUj9gKb/fMcbh5zN/FootXzjiOcddcp7HjwNgWKrASFtgfp1Vzp0uKjkwSQslEFC2tZ2ngvaWWWpCD5A5r8AfEPW/FjJr6ChjvwWj+1sOgvaGgHxNevg9GaQrARe4G9/cSHZLoD3fE7PHnD8+zb+2j2rjiCyw+5jrkz6vIfYAkR8SP9HoFALd6YJwD+dZG+DyJOdaHDW8rNUiUlfshQwdMemrGmHxCneWrG9CQ2Yi+wZCKFuulL6CXjKSaP/x435W1758kP+eq9yfx78o2Ey7P3mzGdI75BSL//om4DkESc3oUOKV14BDR8R9roGhcC63bwoFn622gb20y3ZSP2Att6VC1OlsZfi5P64r83LGjkrUffy1doJU2cym6X1NVtRN0FSPnR4KsBFv+CFyAMVed3uLOkhH/L0mUCW3LthqseyBJ7gQ1ZZxUOOXdfQmVBHJ+D43PwB3wZSx+jjTGmfPZjAaI0haSpubh1J6CzN0dnb4vOOwx6XQmVYyCwKYT2QPreg1N+SIfPIYENoOJYIAz48Mo+Q9DrIsRXmk37ejKbiukGjr7kYLbdbwveevQ9VJWawf2567z7idRHW+0XrggxdP3BBYrSFIKqonVHQeonoLmff+o7WHAS0v9lZDk9bTIeM/EFWn8zpKaAfx2k8jQksB5O1ZloeB+IveqVUYZGIv5Vc/uGTF5YYu8m1thoKGtsNBTw6tqfuvEFZv7wK6nm3jKOI4TLQ4w4fLsCRmnyLjEe3JksSeqLaQKNPIpUnpb2Ek3NQiNPQOpXJLQthHZBmss4NfY+Ov8kvPl5hdQ0NPYO9L3Hu2ErsBYE1uryt2W6lk3FdEOO43DD25ez7X5b4PN70zKb7DKcGz/4O2WVtjpPSUn9kmVDHJLp03Iaexedswc03AaRh9GFf0bnHYqq9+lPF10OtGxcpkAEXXRFu0NSjXi/PNrTt94UhI3Yu6nq/r24+JEzcV0XVcXns5WVSpJ/fdD0qiko8+bXW1BNoQvOpFWNuzZB8lu08UFvDj01JfN5kt9kfFrdJjTyJMTfBWcguPObyx8FJIRWnYdTfmBH3pnpQpbYuznHsQ9VpUwCa6OhrSH2Pt5IG8APTjVStm/rnZOTyViLThSizyKVx6NS1byoyDIy1Om7yZlQdzC4C5vPLbQqfdQoLLoM9fVDQjt14N2ZrmJZw5huTnrf5C384awE0gfKDkD6Pem1Hm61YzDL6B6Q5tLI8mPwKl9aKoPy41s94zbcAXN39rpcLvmFkqmePYI23Lpib8h0uVysoDQYuA8YBLjAHar6z84e1xjjEQl6F0kzXChtxbeGV9+e+oVWSVjKkPJDvb9WnobqQmh61LtTVZNQfjhSccKS3TX6KjTegvfj3A6pGSv2hkyXy8VUTBI4S1U/FZEq4BMReUVVv8rBsY0x7SQi0Odf6Lwj8RqENV/cDI2E8KjmfXxIr4vRyjMgNRN8KyFOZavjaOPdoMv2osl6VghsnPasuo0Qf8/bHtw6/dOF6VKdTuyqOhOY2fz3ehH5GlgFsMRuTJ6Jf00YMA5ib3k9ZYK13nPL7udUgpOlrNGd296zeZ8GKk9f8oxqCq2/AZruBvwgPsCF6uuQ8C4r/H5Mx+T04qmIDAU2AT7M5XGNMe0nEoTwbh0/QHAHiDxEWu08gFPjzfPrPAhsjFSe4dW+A+ouROceBO7U5p2TS2aEdMEZUPM64uvf8bhMu+UssYtIJfAE8CdVXZRh+2hgNMCQIUNydVpjTI5J5Ulo9HnQRbRO7n7wr430uQ2R9EZ0Wn8tuD9nP3D0Rag4MufxmnQ5qYoRb/XeJ4AHVPXJTPuo6h2qWquqtTU11nvCmO5KfAOQ/s+Bb9kBWBLiH6MNN2Z+YfQFsl9wTXg19T2Axt7HnX8a7rzDcRvvRd2eEXdLnU7sIiLA3cDXqnpd50MyxhSc0zvLXa8xaHo8y4vaau8bgNAOOQgsO03NQBNfo5pY/s5ZuA13oQtOhtgrXjuH+uvQuoPQdl9M7h5yMWLfFjgKGCEiE5q/9szBcY0xBePSahWpVrIsrr5kMfBlOV7tfWCdNs+oqbm48/+IO2sD72vBmai7dHEZTf6Cu+jvuHXH4dbfiKbmNr9uDu68Q9A5e6B1h6Ozt8KNPL/cd5h2fnchNPxzmYqgKCSnoU0ZJyK6rVxUxbyDd0ua6UIzvp/Fnefez6evTaSssoz9xozkoLNH4cvSy92YzhAJov71IfnFMlsc7+JqptdUnY3GP4LUr0ATXvtfP1RfgYTbXghcNY7O+x24s1kyrx99EU18Af3HQmISOv9or+6ehDcl1PRf6PcEuuAPkPwWSIE2/9JZeL63wHdgePvfdGJC8+Lgy/7iikD9NbhND0DZ3kjFcR3ue58v1lKgm5v98xzuuehhXn/wbdT1Puo2LYpw/98e56evpvHn+/5Q4AhNsZLqK9C6w0ETeKP0MEg50uu8zPs71dD/eYi94bU38K0G4d0zX2h1F4C7CHyrIOKD6CvNy/q1vFibhNQsiL2JNtywzEg6DppEF/4FklNJ/3QRRxvvQ3pfvQJvuDfZrxE0en12Gm5Doy9Dv8fwLi12T5bYu7Fff5rDyZueQ8OCxrTpy1hTnLcef5/j/nYoA4bYxWiTexJYB/q/jEYehsS3ENgIKT8IcXplf434m0stM5dbqluPLjwHYu8APpAw2usSSE7BG+UvK4ZGxkLy+wzbXEh82nwHbYZtK3pHbGBDcPpBKkr2BB+D1FSIvQbhkSt2/DyyXjHd2P1/e5ymRZGs16QCAT9TJ03Lb1CmpIivP07lGJw+N+JUntBmUm8PXfCH5qQeByKg82HheaBtpKLEZ2RNVU5F8yeKZYVW+GKtiCB9/g2+wSDlLF16cNk30YTGPlihY+ebJfZu7PM3JrVa93RZTfURKvvYrdqmZ9DUdIh/QnoHyhgkJ7bxwvmQcU3WMJQfBRW/B1rOeQfB6YOUH7bCMYp/iLcyVd8HoeJkoDzDXiHwrbTCx84nS+zdWL9V+rS5XUQY/9KEPEVjTCelZnsXJ9No87bKDNvw6umrLgFnZbw6DQEcCG2LVIzGqTrdm0sP1HqN0CqOQ/o/3eFPFyKCBNZDKk8Ep4y02hDxIWUHdOjY+WKJvRs79Nz9CJVn+TiItx7mZ68tW7VgTDflXzPLtEkAQlt7i3OzbLVJGKk6ExpvbG4hrM1fAvHPmu+OBQnvjtPvQZyasThVZyFO24Oi9hAJIn0f8H5ZEPZicwYife7s9gt8W2LvxrbcazNOvOoIQuXLfgT1iCMMHDogz1EZ0zHiVEHlaGhVKugDqfBKCMuPg6pzwGnuJ+MbAtXXgn8DaHqIVitDkQJtQBsf6NqY/cNwal5A+j+H9H8SqRmHBDfv0nPmgiX2bm7f037Lk3P/zdANhqTVrPt8DkPXH0w8mmnVHGO6H6kYg/S6wlvyz1kJyvb3pk18AxERnIojcQa8hwycjFPzKk7ZbpD82ltEJE0cYq/mJ27/EMS/htcauQcQ1bZuA+4atbW1On78+LyftydbOHcRfzv0eia9Oxk3lSKVdPH5HYJlQfwBP9e+8VdWH75aocM0Juc0OQWduw+Z74StwBn0Wb5DKhgR+URVa5e3n43Ye4jq/r24+tVL+P2VR+ALeCP3VNIlUh+lvq6Bv+z3DwrxS9qYrub1k8+WqhpxEz/mM5wewRJ7D/PWo+8Rj6RfgFoweyE/fWU17aZIOW1cS2q6J39x9BCW2HuYRDzD4gd4JVrJLNuM6fHKfpd9W/Tl/MXRQ1hi72F2O2qHjFUysUicl//zJnOmzStAVMZ0sYojsV6D7WeJvYfZa/RuDNtwKOK0/iZ3Uy5P3/IiJw4/k2nfzSxQdMZ0DcephsBmpKesIIT3LkRI3Zol9h4mGA6y29E74PjS/+vclEvjwibuOu/+AkRmTNeS6ivB6dPcxwXvT98QpOpPhQ2sG8pJd0cRuQfYG5itqhvk4pgmu3GPf0AqkW0RBPj01Tb6bhjTQ4l/CNS84a2dmpoG/nUgtJPXUdK0kqt/kXuBm4H7cnQ804ZQWeY7UReLNsZwXRfHsQ9kpriIhKFsv0KH0e3l5CdfVccBdcvd0eTELkds3/YOCl+8/XV+gjHGdDs2pOuBdjpkW8qrsy/NpapMmTA1fwEZY7qVvCV2ERktIuNFZPycOXPyddqiJCJc/MhZbVZ/9bI+7caUrLwldlW9Q1VrVbW2pqZ7t7zsCWp334i9Ttwl4zbH7zB8h/XyHJExpruwqZge7NjLD6Oqb2WrkXsg5Geng7dlkLXzNaZk5SSxi8hDwPvA2iIyTUROyMVxTdt611Rz6/ir2PGgbajsXUHNqv04+pKDOffe0wodmjGmgKxtrzHG9BDWttcYY0qUJXZjjCkyltiNMabIWGIvIqrK5PHf89nrXxBpjBY6HGNMgVj3nCLx8zfTuWDPK1g0tx5xHFLJFGNuOp6Rx40odGjGmDyzEXsRSKVSnLvbZcz+aQ6RhihNi5qINcW4eczdTPnM1oM0ptRYYi8CX77zDU2Lmli2cjURS/Dsv2zZMGNKjSX2IlBf14BIeuMY11UWzF5YgIiMMYVkib0IrL/tOhkXsg5XhNhm380LEJExppAssReBPgOqOfyCAwhXhJY8FyoPMmC1Gmb8MIt7LnyQbz76roARGmPyyVoKFJFPX53IM7e+RP38BvoM6s17T3+Mm3RxXZdgOMjI43ZmzE3WxseYnqq9LQUssRehulnzOWrYacSjiVbPh8pDXPXyxay/zdoFiswY0xnWK6aEffTCZzi+9P/aeCTOW4+9V4CIjDH5ZIm9CPn8voxVMuIIgYDdk2ZMsbPEXoS22mcz3JSb9nwg6Gfnw7crQETGmHzK1UIbI0VksohMEZHzcnFM03FVfSr5831/IFgWJFweIlgWJBgOcMRFB7LmxqsXOjxjTBfr9OdyEfEBtwC7AdOAj0XkGVX9qrPHNh23/YFbseGO6/Hu/z4mEUuw5V6b2nJ5xpSIXEy4bgFMUdUfAETkYWBfwBJ7gVX378Wev8+84LUxpnjlIrGvAvzS4vE0YMscHNcYY4qCxj9Gmx4Ctx4Jj4SyfRAJdtn5cpHY08svIK04XkRGA6MBhgwZkoPTGmNM9+c23AENtwBRQNHERxB5BPre32XJPRcXT6cBg1s8XhWYsexOqnqHqtaqam1NTU0OTmuMMd2bunXQcCMQYcl4VyOQmAzRF7vsvLkYsX8MrCUiqwPTgUOBw3NwXJND8ViCtx59j/EvTaBm1X7seeKurLzGoEKHZUxxi38MEgSNL7MhgkZfRspGdclpO53YVTUpImOAlwAfcI+qTup0ZCZnIo1R/rTtRcz4fhbRxhj+gI//3TyWix85ky332qzQ4RlTvKSKDDPTgANOny47bU7q2FX1BVX9jaquoapX5OKYJneeueVFpn03g2hjDIBkIkWsKc5VR99MKpkqcHTGFLHgFiBlmTYg5Yd02WntztMS8OYj7xGPJNKeT8QTjL37depmzS9AVMYUPxE/0uff4NSAVIBUAiGoOh8JbNBl57XEXgJa9mlvKdoY445z7uPI1U/jutH/IpWy0bsxuSaBtZGacUifO5De1yED3sepOKxLz2mJvQSMOnVk1uQeaYiSiCV4/cG3efzaZ/McmTGlQcSHBDdHQjshTmWXn88SewnY6ZBt2O3oHQmGA5RVhjPuE2uK89SNY/McmTGmK1hiLwEiwh9vOZG7Jl3PydcfiziZ7imDxoWNeY7MGNMVLLGXkJVWH8ieJ+zC4LVXTtsmAhvuuF4BojLG5Jol9hJ0+m2jCZWHlqyy5A/4Kasq46RrjilwZMaYXLDldErQhjusxy0fX8nj1z7L1Em/sO6Wa/G7M/dmwBBr9WBMMbDEXqJWW3dVzrrrlEKHYYzpAjYVY4wxRcYSuzGmpGjqVzT2FpqcUuhQuoxNxRhjSoKqiy76K0SeBAmBJtDA+kif2xGnV6HDyykbsRtjejxNTsNdcBbur1vhztkdt/FBVFt3VdSmByHyNBAHrQeikJiILjy/IDF3JRuxG2N6NE3NRuft35ysXUjVQf1VaOp7pNfFS3ds+g/eghctJSD2Juo25OVW/3yxEbsxpkfTpv+ANgFui2cj0PQImprXYseGLEcQb1WjImKJ3XQZVeXtJz/kzB3/wuiNzuI/f32EhgXWtsDkWPxjIL0tNRKC5LdLHwe3J2PKc2rA6d9V0RVEpxK7iBwkIpNExBWR2lwFZYrDPRc+yD+OuYkv3v6aH7/4mUeueppTa/9MpKG4RkemwPyrkzGVaQJ8qyx5KFVngFQDixeQ9gFlSPUViGTun9RTdXbE/iVwADAuB7GYIjJ/9kKeuOH5Jas2ASRiCebNrOOBK54gHl12DUhjOkbKj2dpsl4sCMFNEP+Qpfv5VkJqxkLFaAhsAWW/Q/o/gYS2yWu8+dCpxK6qX6vq5FwFY4rHNx9+RyCYfm0+Hknw2DXPcuCAE3jyn88VIDJTbCSwNtLnFnBWwkvwAQiNQHrfkr6v0xen6o84/e7Hqb4c8a+Z93jzIW9VMSIyGhgNMGTIkOXsbXq6voN647qZFvEFN+USbYhyz4UPM2BIDdvtv2WeozPFRkLbQ82b4M4FKUecikKHVFDLHbGLyKsi8mWGr31X5ESqeoeq1qpqbU2NNZsqdr+pXYOaVfsu6SCZSawpxkP/91QeozLFTEQQX03JJ3Vox4hdVXfNRyCm56qbNZ+H/v4UH77wKb36V3HQmfuww0Fbc9XLf+GS/a5i6qRpJGIZqhaAupm2kLYxuWY3KBW5ZCLJ+89+wvcTfmSlYQPZ4aCtKavIvDxeRyycu4iTNzmH+roGkokUM3/4lWtOuJUfvvyZ4y47lFvH/4NfJk9nzJbn07SodTWM4wjDd1g3Z7EYYzydLXfcX0SmAVsDz4vIS7kJy+RCw4JGRm90NlcfezMP/O0Jbv7DPRw17DRmfD8rZ+f4381jaVzYRDKRWvJctDHG49c8w6K6egAGr70Kp/7zOELlrRfUdnwOux21Y85iMcZ4OlsV85SqrqqqIVUdqKp75Cow03n3XPggM3/4lUhDFIBoY5RF8+q5+vj0aoGO+vSVicSj6dMsgVCA7ydMXfJ4j2N25qJHzqSsMrykZth1lUsPvIZxj7+fs3iMMXbnaY+iqnzx9tc8ft2zvPXoe8utBX/r0fdJxpOtj+EqX7//HZHGaE5iGjh0QMbFsZPxJP1X6dvquRlTZuKm3CXNmdyUSywS59rf30Y8yxy8MdlofALugjNx5x2B23A36mZrGVB6bI69h4jHElyw5xVM/mgKyXiSQDhA6A9Brn/7clb9Tfri1ECbd9Pl6k67A8/Ym/f+9xGxyNJfMv6Aj2EbrcbgtVdpte+bj7zbar+Wpnz6A+ttvXZOYjLFz216HBZdBsQAhcQXaORB6Pc/xKkqdHgFZyP2HuKJ657lmw++I9oYI5lIEamPsnBuPX875Pqsr9nliO0JhFr/7nZ8DsN3WJfwMvPdHbV27Rqcc+8YevWrJFwZJhAKMHyHdbn8mfPS9i2vKst4DDflEs7hBV1T3FSjUH85EAUW3ysRhdRstOm/BYys+7ARew/x0r1vpI12VZWfv5nOvJnz6bdSn7TXHHPZIXz+1iRmTJlFPBonWBakvFc559xzak5j2/Ggrdlu/y2YPmUWVX0q6DOwd8b9Rp06kknvTW7VZkAE+g7qw+rD7abzeuOKAAAYXElEQVQ1006Jr/H6vCwrBtFXoTK33989kSX2HsJNuRmfF8m+rbyqjFvHX8Unr0zkh89/YtDqA9hm31oCwUDO4/P5fQxZZ5U299l6VC17n7w7T9/8Iv6gD0Eoqwrzt+fOK7omTKYLOb1Ak1m29c38fImRZVcZyYfa2lodP3583s/bk917ySM8dvXTaRUog9dZmXu++meBouqY2b/MZdK7k6nuX8VGO6+Pz5dp9GVMdu7cUZD8Dki1eLYM6XOz116gSInIJ6q63E66NmLvIQ45d1/efepDpk2eQTKRwvE5hMqCXPDgnzp1XNd1eeHO13jm1heJNsbY/sAtOfS8/anq03WryQwY3J8BhxZX/2vTtTT2IRp5HEgg4b2g920w/0RwpwM+r0Vv5WlFndRXhCX2HmLhnEXMnV63pFRQXZdUKsX8WQuyviaZSPLBc58w8/tfGbbRamyyy3Acp/X18muOv5Vxj39ArMmb937qxhd458kPuf3za3N2gdWYznAXXQNN/2XxsnYafRNC20G/55DUN+DOh8DwoluQujMssfcQ/77oIZoWNi3pmKjqtcC9/qTbeWDqbWlz1HOnz+P0bS+ifn4DiWiCQCjAymsO4to3L6WiVzkA06fMbK6HXzq9k4glqZu1gNfuH8deo3fL3xs0JgNN/ty8VmmsxbNNEH8HSXyIhLYuVGjdmpU79hCfvjoxYxvcBXMWMf/X9FH7tSfcxtzpdUTqo155ZEOUn7+exr8vemjJPpM/moLPnz6/HW2M8dnrX+b2DRjTEfF3gQwX1rUJjb2e93B6Chux9xCVvStYMHtR+gZXKasM8+r947j/sseYO2M+q62/KlM+/TGtWiYRS/LsbS8RqY9y2AUH0G+Vvhl/ZvxBHysNG9BF78T0JJqa7pUQAoR3RXxtVz5lPIZGQGOIk7kMtk1SAeIsLVdfwg9iNyJlYyP2PKubNZ+7zrufP25zIVcdezM/TPypXa878Iy905poBUJ+thpVy0v3vsENJ9/B9CmziDXF+Pbj77OWQLop5dX7x3HqZudS2aeCPgOq03qm+/1+m4YxuI33o3NGovXXeF9zRuI23gd491Bocgqa+BbVzN9r6i7AnX8q+mstOntb3Dkj0finKxZEaESWDT6kbL8VO1YJsXLHPJg3cz4fvziBSH0T9136GLHGGIl4EnEEf9DPRQ+fwTajNm/zGKrKbWfey/O3v0IgFCAZT7LeNmtz0SNncMyaf6BhQeMKxSQCtXtswhl3nMTlB1/LlM+m4viEqj6VnPufMWwyYnhn3rLp4TT5Czp3T1rPbQOEoPdN3u387jzvG0kqkd43IsFNl75eFZ13ICQnAy1KdKUc6f/8kpG/puaB1oNvMCKZy1419iG64JSWwUGvy3HKV2itn6LQ3nJHS+xd7KmbXuDOP9+Pz+eQiCVIJTOMbgT+eOuJ7HPS7ss93oI5C/nxi58ZuFoNK68xiLpZ8zlq2GkZOywuT3mvMp5e4I3A5s2cT6wpxkrDBtrNQgZtvAetvw5YtrdPAO+D/jIJXyqQmtcRx7sDWhOT0LrDQSPpry8/Gqk8EV1wBsQ/AXzglCG9rkDCu2SOR+MQf98rawxuhThdV47bnVkdezcwddIv3HXeAySiCdpMuwr/OvM/rLLGIDbddcM2j9m7prrVaLqqb2Wby8+1pbr/0jnKTC0JTClTMkxsA5mnXdAkRJ6FiqO9x6lpZL7tPwGpH9C6EyH5NdB8B6kb8RJ9v0eRwDpprxIJQsh697dXZxfauFpEvhGRiSLylIh04OpI8XrtgXFpbXOziUfiPHr10yt8jkAwwAF/2itt/n15QuUhDj6n9D7KmnYK7Urm9CBkTvgx1J299KF/XW90nX5gcFaD5BSWJPUl4mjTvR2L17TS2YunrwAbqOqGwLfA+Z0PqXjEInHUzTLCyWDu9LoOneeYSw/hkHNHUV5Vhs/vZOyPvlhZVZhgOMD+f/ytXSA1rWhqHu6Cc3F/3cSbH/evBwTxPtj7gRCEf0fmD/phJLjFkkfiHwLhXYCWXTt94FRCcFPIOJ/uQnJa7t5QCevUVIyqvtzi4QfA7zoXTnHZ/oCtGHvXa626GWbjD/jYdLcNcV2XhXPrqaguJxhqX7Mux3E46i8Hc8RFv/MWrjjhVt5+/IO0uvd+K/fh8mfOY+U1By25SckYANUYWvc7SP3KkpF08kvwrQbhfYAoJL6C6BOQcWIxjkpNq+pZqb4G9d8DTQ+CNkFoJ6TqTCDgzZmnCUFomxy/s9KUy3LH44Gx2TaKyGgRGS8i4+fMmZPD03ZfG2y3DjsdvA3hihAi3uLNvoCPgavX4AssHbH4Az4qqstZ9Tcrc8jKozlitVPYv++x3PSHu0jE239R1HEcyirCnPD3IyivLscf9Dc/L4TKg5x55ymstekwS+omXfRF79b8VtMjCXBnQHAjiI2D+HtkTuoALjT8vdUzIn6cytE4A97EGfgRTu9/IL5BiK8flB8JtOzP7wenF1J+eE7fVqlablWMiLwKDMqw6UJVfbp5nwuBWuAAbUeZTSlVxagqE8d9xUN/f5LPXv8SEUglXYLhAOXV5VT2rqB2941Yb5u1uea4W1pVtwTLgux65A4cf8VhRBqiDBjSP63XSzZzZ9Tx+HXP8sW4r1n1Nytz0Nn7sObGq3fV2zQ9nLvo/6Dp3xm2BKH8cIg86o262+THGfRVu86nqhB9Bm38N7gLITQCqTwF8VlzuLbkrCpGVXddzomOAfYGdmlPUi81IsKam6zOl+980+qmoXg0geOLcvZdp7DlXptx3Lqnp5UsxiNxxt79Gi//5w18fh+VvSs46+5T2XyPjds8Z1N9hI+e/xS/38fB5+zbZT3YTfEQ/zCUMhY32lq6YXF5YztKYKX9nwRFBMr2RcrsAn5X6GxVzEjgz8Ao1eX+Oi9ZE17/stXUy2LRxhivPfgO0aYY076dkfG16irJeIpYU5x5M+Zz6YHX8PM307Oe65fJ0zlq2Kncdua9PPKPp7n2hFsZvdHZ1M+3hX5NG8J7gYRpnRL83sIV4b1BU9le2SwA5Yd1YYBmRXR2jv1moAp4RUQmiMi/chBT0cnUaAu8m/b8AR/ffPhdu2vRE7EE/7vphazb/3HMzdTXNS65YBtpiPLrj7O59y+PrHjgpmSIU4n0ewwCtXhpwQeh7ZG+D+MEN/AqWdr6gB/cBqn8w5KH6tah8fFo6teuDt1k0NmqmDVzFUgx22SXDcg0SxUqD7HHsTt7i0AH/MSSmSoFWnNTLjO+z/zD0rioiSmf/Zh2rkQ8yVuPvscfbjqhY2/AlATxD0H63d9csSKILJ2+kz63o/XXN7fQbTl6D0J4L5zeVwGgmkIXXQKR/4GEQGNoaATS+xrvJiOTF9YELA9CZSH++sQ5hCtChCu8OvJgOMCoU/dgo53W5zebDaP3wOp2HStYFmSTERtk3Oa0Ub/e0btTTekRCbZK6t5zIZxe5yEDP4XKP4N/Awhs4SXs6iuX7KeNd3p3oBL3esAQh9ibaP1V+X0TJc5aCuTJprtuyMPTbuedpz4i0hBl85Ebs8qaKwHehaS/v3Ah5+56KZH6KOCNslcfPoSfv5pGtHl1I3/Au4C654mZr2eXVZax3jZrp12oDYQC7HrkDl38Dk0pEClDKk+Ayiyf/lqsdLRUFJoeQ6suRMQGGPlgTcC6kVQqxcS3vqK+roENtluHPgN7M/bu1/nfTS/QuLCJrfap5ciLDqTPwOydG2b/PIfTt7uIxoVNS1ZOGrLeqlz92iWUVYSzvs6UFk18gzbe4S0IHdgAqTgZ8a/W6eO6s4aT3hESwEEGTrTpmE6y7o4lLJlI8v6zn/Dr1NmssfFQNt55A+vYaJbQ2Ifo/NF4CdgFfCAhpO9DSGDdTh3brTvG68K4LP9aOP2f79SxjXV3zLtUMsWHz3/KDxN/YpW1VmLb/bdod0uAXPMH/Gx/wJYFObfp/nTRX2k9XZLylpqr/z+k732dOrZUXYDWHQIax7uL1QFCSK+/duq4ZsVYYs+BRXX1/Gm7i5k7fR7RhhjhihB3nHMf/3zvCgYMtjvpTG5obBxa/09I/Qz+3yBVZ7Va3KJdx9AYpH7MvDH+WadjlMDa0O8ZtPFuSEz04qw8EfFbAV0+2ZWMHLjzz/cz8/tZROqjqCqRhih1sxZw/YlW1m9yw428gM4fA8kvQBdC4mO07lg0/vEKHimA17ExA6d9lVnLI/4hONWX4vR/Cqf3VZbUC8ASew6Me+x9konWd+a5KZdPX/uCZKJ9/diNyUZVof5KILrMlugKlxGKOFB+MK3b6eI9Lj+240GabsUSexdyUy73XPBgocMwPV4U3CwdURPfrvDRpOpcCO8KhECqvD/LDkQqju9UlKb7sDn2TkglU7z7v4/oM7CaSEMUddMrjJ7918tssstwNh+5SQEiNMUhBFIGmqHfj2/ACh9NJIj0vg5NzfWWsPOvtmStUlMcij6xu66Lm3LxB3L7VuOxBOfucinffz61zYU0oo0xnrv9FUvspsNEHLTiBGi4g9bVLGVQcVrHj+vrD9YmtygV7VRMpDHKtb+/jb0rjmTPssMZs9X5TJmQpRqgA175z5tMmdB2Ul8SS8Oyc6PGrBipOAUqjmtujds8hVJ1Jk75/oUOzXRDRTtiv3jUlXz13rckYl6P88kfTeHMHf7CXZOuz0kJ4hsPv0usaflJPVwRYsRh23X6fKa0iThI1Z/QylO9hSmcPoi0/eOrqbkQHQsagdAOSGCdPEVrCq1HjdgjjVF++uoXGhe13fp96qRf+OaD75Yk9cUS8STP3PJiTmIJV4SyblvccCtcGWbNTVZnlyO3z8k5TWnS5I+4C87Anb09Wnc4JD5fblJ3I6+gc0ag9f9AG65H5x2Mu/CyjF1GTfHpESN2VeXfFz3Ekzc8j+P3kUok2f3YnRlz4/EZe51P/26mt7DFMr2IkvEkP0z8KScx7X3S7nz+5qS0qZg+g3qzyxHbsXBuPVvvXcs2+26etR+7McujyanovAO8UTcuuL+iC85Cq87GqTgq82vcRlh4Nq3LI1PeQtTh3SC0dT5CNwXUqcQuIpcD++I1nJgNHKuqmZcC6oSnb3mRp/75ArHI0n7lr9z3JpXV5Zzwf0ek7b/a+oNJxtPrx4PhAGtvkZubJbbca1P2Omk3nrn1Jfx+HyKCP+TnyhcvYtiGnW+mZAyANty8NKkvEYGG69DyQzI31Yq/C+KDZQfnGkGjTyOW2IteZ6dirlbVDVV1Y+A54C85iCnNo/94eknr2sViTXGevuVFXNdN23/VtVZis903Ili29JteRAiGg4w6ZY+cxCQinHzNMdzz1Q2MuekEzn/gdB6edrsldZNb8fG0TuqLKaQyL5GomgTNdmOcTcWUgs6uoLSoxcMKuui7ZuHc+ozPx5piJBMpgqH0308XPXIm9/31UV6481ViTTE2HrEBp1x/XJstbzti0NABDBq64rXExrSLb2VwM3wI1qS3HumyT2sMGu8i/S5VQMqQsC0eXQo6PccuIlcARwMLgZ07HVEGa202jEnvfpP2/KBhA7N2UAyGAvz+/47g9xmmaozpKaTyZK9HTKtEHYLwbkiG3i7a9BAkJ2c+WGgUBG0aphQsdypGRF4VkS8zfO0LoKoXqupg4AFgTBvHGS0i40Vk/Jw5WW6PzuLka48mVB5q1VM8VBZkzI12C7QpbhLaAXpdDNILKMNbY3QkUv33zC9oehRIZNggSMUh1pe/RORsoQ0RWQ14XlUzL8jZQkcW2vhh4k/897LHmPLZjwxeZxWOvPh3rLfVbzoarjE9imoCUjPB6Ys4lVn3c3/dCrQu88be/8IJj+iiCE0+5GWhDRFZS1W/a344CkifL8mRYRuuxiWPn91VhzemWxMJgH/I8nd0ekEqU2IX0js6mmLV2aqYK5unZSYCuwOn5yAmY0xHhX9L5h9rPxK0O09LRWerYg7MVSDGmNZUU94qRBqD4CaIZL/beTGpOAZtesRbjIPFawSEofxIJEMVjSlOPeLOU2OKkUbfQOv/Aamp4AyEyj/ilB/gbUt86S04rRG8aRRFe12JU9b2fRji9IX+/0MbboHYOHCqkYrjILxfl78f031YYjemADQ2Dl1wOkvKGN0ZsOhSXI0h5Qegdcc1j7pbWHgOGlgXWc5cu/gGIdWXd03gpkfoUU3AjCkWWn8t6TcRRaDhBjT6BpDpztEkGnmi64MzPZ6N2I3JE1UXUr94PdWTU7PsVO8tg6eZ2ggkwc1SymhMCzZiNyYPNDYOnbMdOncUOmdnsnbfkAoIbk/G/jBSjoR26sIoTbGwxG5MF9PkD15bAHcuXi/pePPXsneBlkHlaTiBoVB+qPe45Tb/cLDEbtrBpmKM6WLaeD/pt/m7QBCcam/qxekHFaci5UcCIFUXQHAbNPIwaBQJj4KyUYhYb3+zfJbYjelqqV9YWlPeggSRXn+F0C6ItP7wLCIQ3hkJd0lfPVPkbCrGmK4mWW4M0jgEhqcldWM6y76jjOlC6jZAbGzmjcEdEN+g/AZkSoIldmO6Uvxtb5m6NAK+fnkPx5QGS+zGdKWM9ehLNuYtDFNa7OKpMTmkGvP6tEQeB01AaAfvzzRhJLxP3uMzpcESuzE5pPNHQ/xToHnx9ehYkMoWg/MkEISy/SG4RWGCNEXPErsxOaKJLyAxgSVJHYCk13a38o9eDbpGIbQzElivUGGaEpCTxC4iZwNXAzWqOjcXxzSmx0l8nWXaPAKpqdnXKTUmxzp98VREBgO7AT93PhxjejDfYMhYkx4G37C8h2NKVy6qYq4HzsUu8ZtSF9wSnAGkfRAWP9K8gIYx+dCpxC4io4Dpqvp5juIxpscScZC+D0JwO7zk7gf/Bkjfh2xZOpNXy51jF5FXgUy3x10IXIC3iPVyichoYDTAkCHtWG3dmB5IfP2QvnegGgVNIk5loUMyJUhUOzaDIiLDgdeApuanVgVmAFuo6qy2XltbW6vjx4/v0HmNMaZUicgnqlq7vP06XBWjql8AA1qccCpQa1UxxhhTWNZSwBhjikzOblBS1aG5OpYxxpiOsxG7McYUGUvsxhhTZDpcFdOpk4rMAX7K+4kz6w/0hAu+FmduWZy5ZXHmVrY4V1PVmuW9uCCJvTsRkfHtKR8qNIsztyzO3LI4c6uzcdpUjDHGFBlL7MYYU2QsscMdhQ6gnSzO3LI4c8vizK1OxVnyc+zGGFNsbMRujDFFpuQTu4hcLiITRWSCiLwsIisXOqZMRORqEfmmOdanRKR3oWPKREQOEpFJIuKKSLerPhCRkSIyWUSmiMh5hY4nGxG5R0Rmi8iXhY4lGxEZLCJviMjXzf/npxc6pmxEJCwiH4nI582xXlromLIREZ+IfCYiz3X0GCWf2IGrVXVDVd0YeA74S6EDyuIVYANV3RD4Fji/wPFk8yVwADCu0IEsS0R8wC3Ab4H1gMNEpLsuPnovMLLQQSxHEjhLVdcFtgJO68b/njFghKpuBGwMjBSRrQocUzanA1935gAln9hVdVGLhxV005WgVPVlVU02P/wAr01yt6OqX6vq5ELHkcUWwBRV/UFV48DDwL4FjikjVR0H1BU6jrao6kxV/bT57/V4yWiVwkaVmXoamh8Gmr+63c+6iKwK7AXc1ZnjlHxiBxCRK0TkF+AIuu+IvaXjgbGFDqIHWgX4pcXjaXTTRNTTiMhQYBPgw8JGkl3zFMcEYDbwiqp2x1hvwFtq1O3MQUoisYvIqyLyZYavfQFU9UJVHQw8AIzprnE273Mh3kfgB7pznN2UZHiu243aehoRqQSeAP60zCfgbkVVU81TrqsCW4jIBoWOqSUR2RuYraqfdPZYOWvb252p6q7t3PVB4Hngki4MJ6vlxSkixwB7A7toAetUV+Dfs7uZBgxu8Xjxql+mg0QkgJfUH1DVJwsdT3uo6gIReRPvGkZ3uji9LTBKRPYEwkAvEblfVY9c0QOVxIi9LSKyVouHo4BvChVLW0RkJPBnYJSqNi1vf5PRx8BaIrK6iASBQ4FnChxTjyUiAtwNfK2q1xU6nraISM3iSjIRKQN2pZv9rKvq+aq6avPaFocCr3ckqYMldoArm6cRJuItzN1dS7ZuBqqAV5pLM/9V6IAyEZH9RWQasDXwvIi8VOiYFmu++DwGeAnvQt+jqjqpsFFlJiIPAe8Da4vINBE5odAxZbAtcBQwovl7ckLzaLM7Wgl4o/nn/GO8OfYOlxN2d3bnqTHGFBkbsRtjTJGxxG6MMUXGErsxxhQZS+zGGFNkLLEbY0yRscRujDFFxhK7McYUGUvsxhhTZP4fg8lsZSoGzFgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x9319610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot and satisfy your self that data is linearly seperable\n",
    "x1 = []\n",
    "x2 = []\n",
    "for i in x:\n",
    "    x1.append(i[1])\n",
    "    x2.append(i[2])\n",
    "plt.scatter(x1, x2, c=y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now complete the train & predict function in Perceptron class. You algorithm should find the seperating hyperplane and model.predict should give all the labels correct.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training data is 100.0\n",
      "100\n"
     ]
    }
   ],
   "source": [
    "converged = model.train(x, y)\n",
    "y_hat = model.predict(x) \n",
    "correct = 0 \n",
    "for i, y_real in enumerate(y):\n",
    "    if (y_hat[i]==y_real):\n",
    "        correct = correct + 1\n",
    "    \n",
    "print (\"Accuracy on training data is {}\".format(correct*100/len(y)))\n",
    "print (correct)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the data points and seperating hyperplane to see your perceptron has actually learnt correct seperating plane\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VGXax/HvPUlm0gOBUKRLRxCkY1d0xXUVZcVCb4Jr76Kuumsvu+7aVwSkimLFVV9Z7A0Seu+9B0hIzyQz87x/zAAJTBqZ5Ewy9+e6uMjMnDnnnpTfnHnOc+4jxhiUUkrVfjarC1BKKVU9NPCVUipEaOArpVSI0MBXSqkQoYGvlFIhQgNfKaVChAa+UkqFCA18pZQKERr4SikVIsKtLqCo+vXrm5YtW1pdhlJK1ShLly49bIxJKmu5oAr8li1bsmTJEqvLUEqpGkVEdpZnOR3SUUqpEKGBr5RSIUIDXymlQoQGvlJKhQgNfKWUChEa+EopFSI08JVSKkRo4CulqsTSnelM+nmr1WWoIjTwlVIB9+vmwwyfksz7ybvIcbqsLkf5aOArpQJq/toDjJm2mOaJ0cy9tR8xjqA6oT+k6U9CKRUwny7bw4Mfr6JLkwSmje5FnWi71SWpIjTwlVIBMWPhDp6Yt5ZzW9fj3RE9dc8+COlPRClVKcYY3vpxKy/P38hlHRvyxpBziIwIs7os5YcGvlLqtBljeOGbDbzz0zau7XYGLw/uSkSYHhoMVhr4SqnT4vYYHp+3hveTdzGsb3OeuqYzNptYXZYqhQa+UqrCCt0e7p+7ki9W7uMvF7fmoSvaI6JhH+w08JVSFZJf6Ob22cv4bkMqDw1oz20Xt7G6JFVOGvhKqXLLdroYN30xydvTePrazgzv28LqklQFVProiog0E5EfRGS9iKwVkbt99yeKyAIR2ez7v27ly1VKWSU9p4Ch7y5i8Y50/nVDNw37GigQh9NdwP3GmI5AX+B2EekETAS+M8a0Bb7z3VZK1UCpmfncNGkR6w9k8Z9hPbj2nCZWl6ROQ6UD3xiz3xizzPd1FrAeaAIMBKb7FpsOXFvZbSmlqt/utFyu/89CdqfnMm1ULy7v1NDqktRpCugYvoi0BM4BkoGGxpj94H1TEJEGgdyWUqrqbUnNYtjkFPIK3cwe14dzmuvIbE0WsDMkRCQW+AS4xxiTWYHnjReRJSKy5NChQ4EqRylVSav3ZDD4PwtxeQwfTuirYV8LBCTwRSQCb9jPNsZ86rv7oIg09j3eGEj191xjzCRjTE9jTM+kpKRAlKOUqqSU7WkMeXcR0fZwPr61Hx0axVtdkgqAQMzSEWAKsN4Y80qRh74ARvq+HgnMq+y2lFJV78eNqYyYmkxSvIOPbu1Hy/oxVpekAiQQY/jnAcOB1SKywnffo8ALwFwRGQvsAgYHYFtKqSr01ar93PPhcto1jGPGmN7Ui3VYXZIKoEoHvjHmV6Ckc6r7V3b9SqnqMXfxbiZ+uoruzesyZVQvEqIirC5JBZieaauUYvIv23jmq/Vc2C6Jd4b1IMqu7Y1rIw18pUKYMYZ/f7uZV7/bzJWdG/Hvm7rhCNewr6008JUKUR6P4Zmv1jP1t+1c36MpLwzqQrj2sq/VNPCVCkFuj2HiJ6v4aOkeRp/Xksev6qS97EOABr5SIcbpcnPvhyv4evUB7u7flnsua6u97EOEBr5SISSvwM2EWUv5edMh/npVR8ZdcKbVJalqpIGvVIjIzC9k7LTFLN2Zzot/7sKNvZpbXZKqZhr4SoWAI9lORkxNYdPBLF6/uTtXnd3Y6pKUBTTwlarl9mfkMWxyMnvS85g0oieXtNfGtaFKA1+pWmzH4RyGTk4mI6+QGWN60+fMelaXpCykga9ULbXhQCbDp6TgcnuYc0tfujRNsLokZTENfKVqoRW7jzJyagqRETbmTuhH24ZxVpekgoAGvlK1zO9bD3PL9CXUi3Uwe1wfmiVGW12SChIa+ErVIt+uO8ht7y+jZb1oZo7tQ8P4SKtLUkFEA1+pWmLeir3cN3clnc+IZ9ro3tSNsVtdkgoyGvhK1QKzFu3k8Xlr6N0ykSmjehHr0D9tdSr9rVCqhnv7x628+M0G+ndowJtDuxMZoe2NlX8a+ErVUMYYXpq/kbd/3MrVXc/glRu6EqHtjVUpNPCVqoE8HsOTX6xl5qKd3Ny7Oc9c25kwbW+syqCBr1QNU+j28NDHq/hs+V4mXHgmE6/soO2NVblo4CtVg+QXurlzznIWrDvIg1e057aLW2vYq3LTwFeqhshxurhlxhJ+33qEv19zFiPPbWl1SaqG0cBXqgY4mlvAqPcWs3pvBq/c0JVB3ZtaXZKqgTTwlQpyqVn5jJiSwrZDObw5pDsDOjeyuiRVQ2ngKxXE9qTnMmxyMgcznUwd1Yvz29a3uiRVg2ngKxWkth7KZtjkZHKcLmaN60OPFnWtLknVcBr4SgWhNXszGDk1BRH4YHw/Op0Rb3VJqhbQwFcqyCzZkcboaYuJc4Qza1wfzkyKtbokVUto4CsVRH7edIgJM5fSOCGSmeP60KROlNUlqVpEA1+pIPHNmv3cNWcFrRvEMmNMb5LiHFaXpGoZ7bSkVBD4eOkebpu9jM5N4vnglr4a9qpKBCTwRWSqiKSKyJoi9yWKyAIR2ez7X6cYKOXHe79t54GPVnJu6/rMHNuHhOgIq0tStVSg9vCnAQNOum8i8J0xpi3wne+2UsrHGMNr323m7/9dxxVnNWTKqJ7E6IVLVBUKSOAbY34G0k66eyAw3ff1dODaQGxLqdrAGMNzX6/nlQWbGNS9CW8O6Y4jXC9coqpWVe5ONDTG7AcwxuwXkQZVuC2lagy3x/DYZ6v5YPFuRvZrwZNXn4VNe9mramD550cRGQ+MB2jevLnF1ShVtQpcHu6bu4IvV+3njkvacP8f2ml7Y1VtqnKWzkERaQzg+z/V30LGmEnGmJ7GmJ5JSUlVWI5S1sorcDNh5hK+XLWfR67swANXtNewV9WqKgP/C2Ck7+uRwLwq3JZSQS0rv5CR76Xw46ZDPHddFyZc1NrqklQICsiQjojMAS4G6ovIHuBJ4AVgroiMBXYBgwOxLaVqmrScAkZOTWH9/kxevekcrul6htUlqRAVkMA3xtxcwkP9A7F+pWqqAxn5DJ+SzK60XCaN6MGlHRpaXZIKYZYftFWqttp1JJehUxaRll3AtNG96de6ntUlqRCnga9UFdh0MIthk5MpcHt4/5a+dG1Wx+qSlNLAVyrQVu4+ysj3UrCH2fhwfD/aN4qzuiSlAA18pQJq0bYjjJu+hLoxEcwa24cW9WKsLkmp4zTwlQqQ7zcc5C+zltEsMZpZY/vQKCHS6pKUKkYDX6kA+GLlPu77cAUdG8czfUxvEmPsVpek1Ck08JWqpPeTd/HY56vp1TKRKSN7Ehep7Y1VcNLAV6oSJv28lee+3sDF7ZN4e2gPouza8VIFLw18pU6DMYZXFmzi9e+3cNXZjfnXDd2wh+sF5FRw08BXqoI8HsNTX65j2u87uKlXM569rgth2t5Y1QAa+EpVgMvt4aFPVvHpsr2MO78Vj13VUTteqhpDA1+pcnK63Nw1Zznz1x7kvsvbceelbTTsVY2iga9UOeQWuJgwcym/bD7ME3/qxJjzW1ldklIVpoGvVBky8goZM20xy3el8/L1ZzO4ZzOrS1LqtGjgV7PMI1l8+trXLPrvYuo0qMPQv/6ZLud3tLosVYLD2U6GT0lhS2oWbw7pzpVdGltdklKnTQO/GqUdTGdk27vIz8733bOTpf9byQ0PXsMtLw63tDZ1qr1H8xg+OZl9GXlMHtmLi9rpJThVzaYTh6vRw394pkjYnzD35S9Yn7LZgopUSbYdymbw279zKMvJrLF9NOxVraCBX00O7TnCjtW7Snx84uVPU5BfUI0VqZKs25fJDe8sxOnyMGd8H3q2TLS6JKUCQgO/mmxZvh0p5eScvJx8fpq7sBorUv4s3ZnOTZN+JYI0PvjzU3QKvxhP5j8wxmV1aUpVmo7hV5OGLZIICw/DVeA/OIzHMOvpj5jy6Gyyj+YQ4YigSdvG3PjgQM4f1Efne1eDXzcfZvzMFJKiDzFr0Fs0iU8HA+TOwJh0JOHZcq/LGAOFK8BkQMQ5iC2h6gpXqpzEGGN1Dcf17NnTLFmyxOoyqswdfR5h4+ItFX5ex75tmTjrbn6a+xtpBzLo3r8Lvf94DmFh2qgrUOavPcCd7y+nVWIGM679B0kxmSctEQb1f0bMAXCnQkQXJKyB33UZ1w5M2mgw6YANTCHE3YstZkyVvw4VmkRkqTGmZ5nLaeBXn8y0LP7+53+w6qd1p/V8W5gNj9tDVGwkjVo14KaJ19L5vA40aK4HFCvj02V7ePDjVXRpksDUq1+ijn29/wUlBu8uvw1MAUQPQeIeKfbpyxiDOdwf3Ht9yx4ThSS+i9h7V+ErAePJweS8CXlf+DY7EIm5DbFV7spbJv8HTO4M8GRA5OVI9HDEFhuAilUgaOAHqYL8Aq6rN5qCvMofoI1whCMiXDH6Eu58Y5wO+5yGGQt38MS8tZzbuh7vjuhJVN5DkP9lOZ8dhSQ8jURdc/weU7gKkzYCTO6pizsGYKv7WkDqPr69/G8x2W+C5wCEnw3uneDeAxz7/bJDeFuk3ifemwW/g2sThLUCx4WInPiUaEweJvdzKPgJbI2Q6CFIRDs8Wa9B7hQwecdeCIQ1Rup9DiYTCEfC6mM82eDeBrbGSJjuhFSn8ga+juFXM3uknaGPDWLO85+Rn+Os1LoKnd7jAQtm/ETHvu24fPhFgSgxJBhjeOvHrbw8fyOXdWzIG0POITIiDBN2Gyb/K4rvnZckD5M7vVjg48mixLkQnvQAVF5kdTkfQNbzgC+IC37ws1QBuLdj8udDzhveTx6mEMQOtvqQ+AESVg/jycYcuR7c+33rs2HyPsXEPQ45kzjxBgLgBPcOTGo3vK/VhrHV9e79ix1MAcZxMVLnZUSijj/LeLIxBYvBtREkFnH0RcLbBPR7okqngW+Bmx8ZRJ0GCcx5/jPSD2bQultLtizfftp7/fk5TmY+9RER9nB6/KErcXX1o3ZpjDG88M0G3vlpG9d2O4OXB3clIswb0hLeBhPRGwqTy7cyT1bx2xFdvYF6ikiIvKJyhRdhjAuyX+Z42Je6cD7kvAOuHYCvNlMIbicm829I3dcxuTN9w1DHdkI8QD5kPXM8xP3zeP95DvnW61vO+RMm429InRe9S+XM9r05nViPyXJgov6ExD93/NOpMW4wWSBxxT99uA9icj8G13aIvACJvBoRm+85BgoWQeFysDWAyAHHh5uMJxOT9xm4NkN4ZyTq6tMe3jIFS7yfplzbIKIzxNyBkO07ptMVCW96WuutTjqkEyRW/7qOBy7+Gx5PJX4eAoJw82PXMfqpmwNXXC3i9hgen7eG95N3Maxvc566pjO2k6bLmsKNmCM3UHaY2iFmJLa4B4vd6w23l4B8vJ8UoiC8OVJvbrE93sow7r2YQ38sR42ARINxUXwv/ZhwpOFazJHrwOXv2NKxC7GfesJg2exIw8VQuME7zIW/T7RRSMJzEPlHTO40yH7TO3QkkRBzGxIzBgp+w6SPB4rMcJMEqD8fscVijowE12rf64sAHEi9WSDRmCODfW9CeUAU2GKRep8gYY0q9EpM/g+Yo3cX+T4c+51xAGHe2qKuQeKfPv5GVJ10DL8G2r56J2/d/R6rf98AHnC73Ke9rrCIMHoN6Mb1915Nsw5nkNiobgArrZkK3R7un7uSL1bu4y8Xt+ahK9qXeNzDFKzEZL0AhWvAVhccF/oOhBbg3auNBFsiUv8zxHbq99YULMfkzgbPEXD8AYm+DpHIU5Y7XcaTi0ntg/8QLUpA6vqOKfgL7TBv4KeP8u4ln8L7OvEcwPu6K8KOJP2IyXoe8v9bymL9wDEAsl6g+BtYFMTeC9n/wu8bW3h7sF8Cuf/xs9K6EN4aXEspPjwXBo4/YKv7arlfhTEGc+hS8OwtY8koJP5xJPr6cq87UDTwa4HD+9J48tqX2LluN7YwG4VOFx16t2H9ok24XRX746vfNJE3kl+gXuPQDP78Qje3z17GdxtSeWhAe267uOJjx8a1BZMz03tQ1HE+EjXY0pkqnozHIW8exYPcAbZG4NnnvRnREUn4Bybr3+D8H8X2krGB/XxsiZMx+f/DZDxY5MCs7/Gw1kjiJEz6BO9wCv6Gq0pga4Ak/YJJ9+6llyiih/d76jl46mOS4B3i8ftmI4Cdst/0ThaJrdGqci9tTB7mYHegHDtg4R2x1Z9XwXoqTwO/FtmzaR8Zh7M4s2sLomIieWH4q3w3+9cKryc8Iox5mTOwO+xVUGXwyna6GDd9Mcnb03hqYGeG921hdUkBYUwBJvNpyPsc7568A+IexBZ9A8ZzFJDjJ3wZ9yHvQVmT4dvbjwZbNFLvIySsiXcvNusfkDsdJMK7AVtdpO50JNzbDtpTsArSRgB+ZiAVI4ADqfMvJLI/npz3fXvv/j5hRELco5D1N0r+BCH4P4he0v1llReHreHSci9ujBuT2sP/zKuThbXAlrSg4jVVkgZ+Lfbzxwv5x5i3yPPTiK0sXS7syN8+fZD4xLgqqCz4pOcUMOq9FNbsy+Sfg7ty7TlNrC4p4IzJ886QsdVHpOR5GMY4If8bTOEGJLwtRF15yjEF4071HfysBxHdTxmPNoWbMUfvAffJzf7CIawdUAjhLZCYCYi9q2+7+d6xdNc2in9CiPCehZw4FXP4Gu+UzpOFNQf3QfzvxUdS+rEFm+9f0U81doi6EVvC46U871SezJcgd1YZ27NDzChscQ9UaN2BoIFfi7kKXdzZ91F2rd9b8YZrArF1Yuh2SWeOHsqke/8uDLx9APH1at8bQGpmPsOmJLPjSC5vDenOZZ0aWl1SreEpWAXZb3jbR9jqQcx4JOrako+JmDxM7ife4yAmB8LPRKIGgeMiRGwY54+Y9LsoHqiRkPBPkDA4eutJawyDuGchdxq4N5RQZQSENfMefzAeEIHw9kjd9yo8U8cYFybzGcj7BCTcNxPr2CcSFxAFYfWRep9a0kYjaAJfRAYAr+I9lD3ZGPNCSctq4JdfXk4+n732Nd/N+pm0A+nkZORhTmOGjz0ygpg6Mbyz/GXqNqxTBZVaY3daLkMnJ3M428nkET05t019q0tSZTDO3zBZr4B7O4S1QOLuQxwXAOAp3AFZT3nn8Ie1QOLuQey98bj2wuH+nDocZAPH5Uid16AgGdw7ILyd9xNFJU5QNJ5s8KRC2BngTsXkvu89/mA/1/uGZ4uu+Drde71TYsPbILbT68waFIEv3om0m4DLgT3AYuBmY4zf3gIa+KfPmedky4odPHrls+RmlmOqXhFiE+o2SKBJ28ZcObY/5w3qzf6tB6nTIKFGHuTdkprFsMkp5BW6mTa6F+c0r3mvQZWfx50GaWPAvQ7vuL4NHP2RhJdOK4Cri/HkYo7e5X1DOnauQ/QNSNxjFZ7aGSyB3w/4mzHmCt/tRwCMMc/7W14Dv/JchS5ev30y30z9AY+notPoINwejsftITLGQaHTRbdLOvPYnLuJSahcL5bqsnpPBiOmJhNmszFrXG86NIq3uiRVTYwpAPcu73TZ09xTrk6eow9A/jcUPz8iCuIewhYztELrKm/gV/UZAk2A3UVu7/Hdp6pIeEQ49066lfmuD/nk8FRadmmOLaz8P2ZXgQuP20NuZh6FzkJW/LCa54aWf86ylVK2pzHk3UVE28P5+NZ+GvYhRsSO+BkWMcaJcf6Ccf7kPcDthzEGU5CCJ/NFPFmvY1w7q7RWY/L9hD1AHuROrbLtVnVrBX+DZcU+UojIeGA8QPPmzau4nNASnxjHKz/+nUcGPMPOdXuwhdnIy873nopezg92hU4Xy79bQ/rBowEd41/7+0ZmP/Mxezbtp13P1lx8w7nkZObSqGUDulzYEZutYvsiP2xM5daZS2laN4pZ4/rQOCEwZ7Sqms04f8McvbPIPR5IeAWJvPTEMsZgMh4A57feNhSEYXLexcT/HVv0dVVUWB4l/hF6Tm7NHThVHfh7gGZFbjcF9hVdwBgzCZgE3iGdKq6nVktPzSDCHk5snRPDL3F1Y3kj+QU2L9vGge2pJDWrx5t3TWXb6l14XG5chWWfTBJuD+NoakaFA7/AWchnr37F/Pd+wOMxXD7iQv5879Ws/GENT9/wCk5f76D92w7y09zfiXBEgDFE14nmuS8foV3P8p0c9dWq/dzz4XLaNYxjxpje1It1VKhOVfsYk4cp3ATpf+HkqZTm6D2Q9O2J6xkU/ATO74qcdOby/st8AhPZH7FVwSdFqePt+3PK2bs2sPcN/PaObbaKx/DD8R607Q/sxXvQdogxZq2/5XUM//RsXLKVF0e8zv5tB8EYzjqvAxNn3kn9JvX8Lu92ubnn/L+ydeWO4x03SxMZ7WD27v8QX4GmbMYYHuz/dzYkbz4e7PaoCFp1aUH6gaOk7jpc6vPFJtz+6mgG3n5lqcvNXbybiZ+uonvzukwZ1YuEqIhy16hqB+PJ8HYiDWsChGGy/gm5M/HO3PE3bdmOxN2PxIwGwHP0Qcj3c3asxCAJzyORA6qmbuevmPTbONGuIwIk0ju1M7xiJwcGRXtkY4xLRO4A5uOdljm1pLAPVVnp2eTnOKnfJJGs9GxW/rAWe2QEZ3ZtSd2GCYRHlP4jSj94lPsveuJ4qAKs/HEt95z/ODO2voHH7WHhF0tY+/sGGrZsQP+hFzDvjW/YuGSr32mcIoLYBI/7xAFfj8fDkGa3MuTR67j5kUHlmta26ud1bFy8pVhdBXmF7Fy7m4K8sk/PNx7DpAdn0vuP3Wncyv/8+cm/bOOZr9ZzYbsk3hnWgyi7XgGsuhhPNuT/D0wa2HsjEWefeMy1BdxHIKITYvN/focxHihYCO7dEN4RIs4+6UIyhb6umQnFumYWryEXkzERnN9758YT5u3L4/yF0k+QKsR4sk6MN4sd/2ftClUZkeI4H+p9hMmZ7G1bYe+OxIypcGO3iqjy9sjGmK+Br6t6OzXN0UMZvDDsNVb9tA6xCRGOCPJzndhsNgqd3kB0RNm58eFrGfb49bgKXWxI3kxkTCRtzml1/I9j7j++KBaqx6TuOszCLxYz/cm5HNieSl52Po5oO5Mnzio1cMMdYfS7uhdL5q/wjvd7DAX53uXnPP8ZjVo15NKbzy/z9W1I3uL300N+jhNbuK1cfbg8HsOvnyQz+IFrit1vjOHf327m1e82c2XnRvz7pm44wjXsq5Jx78XkfuZtzWBrCtn/xhuQBUA4JrwTOC6G3A/AHMQ7H8RgYm7BFnfvSes6hEkb6m2pbNyAeNtKJ74L2L1X7MqZ4uvbH4mJvQtbzIhTa8p4GJw/ems43pb5f+V4NZGI48LjtyTqOkzel5zaoM0DjvPK8+05bRLRHqnzcpVuoyjth28BYwwTr3iGnWt3Hx9DPxaq7iINmpx5BXzw4ucs/r/lrE/Z7P37Eu+4/E0Tr+ObKd+xZ9P+ErfxwQvz2LNp//E3EGdu2WflFua72LR0K4VO1ymfAPJznHzwwmflCvz6TRKxR0aQl138GEFkjIOOfdux7veNft+oTnoRp7SL9ngMT3+1jvd+28HgHk15flAXwiswC0lVnCdvAWTcj7d5mL+dBRe4lnn/Hef7uee8jUdisMWOB3ytp4/e7Z0+WfRdv3A5JvttkCjIfpfj4WsKIOufeCSu2AFU40kD5w/4H7IphUSD4zLvG8yxu+w9MTGjIGcqx+fxi0HqvBmwdtbBQlsrVDFjDGt/38jyb1cTWzeGi248lyN707jvoicqfcWrssTWjSE7PafCz7NHRmAMx98oikpIiufjg1PKXEd+rpOhLf5CVloWRX/FYhKimb75daY8MptvZ/1CWHgY+Tn+P37bIyN4Z+U/adq2MeDtZT/xk1V8tHQPo89ryeNXdTqll32oMoWboHAZ2JJ8ly48vWMZxrXVe7ZrwRJvy4ToUZD9vLcdwmmLgAarIOuvkPclJXa3tDXwHjg1Wac+FtYMW9J3Rerc4ut1X866JBbs5yFR14LjUr/Dksa1Cwp+9b7pOC4rcTgqGAXFiVcVVV2B73a5Sf56GQe2pdK6W0vOvqhTQK8Ha4zh10+T+b8p37Fl+Xayj+YUG96ISYjGVeAqew+3EsQmNGhWn4M7D1X4uRGOcKJio8g8UvwPT2zCedf25smPy9ccaue63Txz47/Yt/UAiNCgeX0em3MPbbq1AiD7aA5H9qURXz+OV299l4X/XYIxBpvNRlh4GMOfvJ6bHvbu1Tldbu79cAVfrz7A3f3bcs9lbfUavvg6OR69G5w/e++QMJAoJHE2Et6q5Od50sB9AMJaHj8b1bh2YY4M9HWFPJYLxzqrVuZ3VSD2Ich5vfSOk1IH7zVy/Y33ObA1Wn2ifuPEpPb1E/g27/aKtTKOROq+gzj6nfYrCHYhE/jb1+zirbunsvrXDUTFRHLVhMsZ+fcbiLD738M5vPcId5//V7LSsnE5XYTbw2neqSkvf/sEUbGB+fj20qg3+OWTRVW+B1+SsIgwLh1yPs3bN2HW0x9X6I1FBJp3bMrwJwbz8pg3jw8D2cJsOKIdvJnyPM3aV+zcudTdhzEeQ4Pm9UsN6d0b9/LLJ8mA4fxBfWnewbudvAI3E2Yt5edNh/jrVR0Zd8GZFdp+bWUKN2LSbgFz4KRHBMJaY0s69dCZx5kCRx/wjbP7pq/G3orE/AWT+aiv1fLpX3jHv3AIPwdci0tfJvI67x62x88wZXgnbPU/L3aXJ2cGZP2TE2PvNpAYiH8B8j70XdawLRJ75/HOnbVVUMzSqWqpuw9zz3l/JTfL+wPPPprD5699zb6tB3hi7v1+n/OPMW9xeE/a8VkohQUutq3cyfS/zeXWf4ysdE1blm/n548Xlmu8XAQC+X4bHhFGi7Oacc1tAxgw5hJchW6Wf7+GdQs34sx1lrotW7gNR6SdCEc4j8+9jxadmlG3UR3mPP8p+7cd5KxzOzDksUE0adO4wnU1aFa+xmXN2jdhyKODit2XmV/I2GmLWboznRf/3IX3c9ndAAAVO0lEQVQbe4XGyXnGk+VtUyzxvhksJ45TGGMwzp/g6MndJY8vAe49GNcuJPzE98tz9P6Trjzle27OOxDWFAqWUbGwj6BcF0SxJXp79ZfIAbZ6SPz9GOf5kPEwJ3fNlLiHT11tzAhMWFNMzjveFsr2Pkjs7d7XHHV5BV5H6KjRgf/pq19R4CwerM68ApK/XMqBHak0atngpMecrPhhbbEph+Adq/5u5s8BCfzl360u96UJxWYjsWECR/aln/KYI9pBi05N6XxBBz579etTDqCKTYrd54h28PD0O7jgzydO2rA7bLww/6+sT97M/Knfs2DmT8VnzgiceXYLbvv3aDambKF+k0TOu643jijvH+fZF3bi7As7VeTlB9SRbCcjpqaw6WAWr9/cnavOrvibTU3gyf8F8j4A40GirsZ4DkPWy74LkRjvVZ8SpyLhZ3ov5p02wtuhsbSwFaHoMIzHuazkywyaPG9ohjX3dqo8RfiJwDZub01RV0NYB3BtgvwvfLX4+b2XGKTu2+DaiclcetIVtXzrjn8MiRqISBQSdSXGFu29Qpd7t7eDZNz9iL2X/5cZeWmxs2ZV6Wp04G9eug1Xwam/ZBGOCHZv2HtK4Jd2gXC3u+KNxvyJrRtDeES437pO1qRtY6au+zcrfljD7Gc/4cC2VNr3bsPwJwfTomPT48u1PrsF7zwwg8wj2TiiHQy8/Qpi68bw4UvzyM92El8/jrHPDykW9seICJ36tqNT33a06d6KKY+8j9vtwV3o5vxBfbh/8q04ohx0veisgLz+QNmfkcewycnsPZrHpBE9uaR9g7KfFOSMMeBa7x07j+gEtiTMkZvAtfLEMgU/4R3D9oDxDQmaXEzaGEj63nfxkV2UuScucRBWZOgrd3Lpy7uPIPFPYdKSOeWSiY5LkYTnvfPdTQbY+yHhJ9ZtPPdhcj/1XhPX7TtmFN7ce/A48o+ILcY7bTP/Kyj43Rf6dsCG1H0NcVxcvHTHRYjjotLrVaelRo/hv3XPe/z37fmntAewR0Ywec2/aHzmqSfs3HXuY2xI3kzR1x0eEcYfRl3Cve9MOP3ifXIycri52a1lXo3KEWXn0ffv4dyB/vdcysPj8eDMdRIZE1nuA5iFBYUc3HmYOknxxVowBJMdh3MYOjmZjLxCpo7qRe9Wwd/5sCzGfcR7bVf3DiDMO90wrKn/qzz5IzHei4EcvYuyD6A6kMQpiL338Xs8aeOh4MdSnvIHbHXfwJM339t3/lg/l6irkPi/BeQC7MYYKFyMcf4CUgeJ+hMSphelCYRg6ZZZpQbdc5W3/0oR9sgIul92tt+wB3ho2u3EJcYQGeP9iBoVG0mD5vUZ+9yQgNQUkxDDs189Sly9WKLjo4iOjyIyNpIuF3Qgtm4MCJzRphEPz7izUmEPYLPZiIqNqtBslQh7BE3bNg7asN9wIJPB7ywkt8DFnFv61oqwBzAZ93oPIpo8MNlAQfnDHgABz2FK/5O1ga0Z1F9QLOwBiC7t99uOxN3nXUPUFUjSL0jS90jDFGwJLwQk7MF3Fre9N7a4+7HFjtWwt0CN3sMH2LxsG6/fMZkNyVuwR9m5cswl3PLScOyRJV+oOzcrjx/m/Mqezftp16M1513XG7sjsD1Y3C436xZuwuP20OncdiXOGlInLN+Vzqj3FhMZYWPW2D60bVhz5kGXxnjSMKkXUrmpjQ5I+hWO/Ak8B/08HgUxo5HYvyAlHCD1HLkZCk+6eLfUhcS52CJqx4XdQ1XITMs8xuPxePcgdG52jfT71sOMm76E+rEOZo/rQ7PE4L1SUUUZ127M4asovb/LyRx4T1ASIBLi7sUWMwrjXIhJvxVvR8dCvNdSTfRdS7X0K3sZYzB5n0PudMAD0SOQqEEVvrqSCj4hMS2zqIr2T1fBY8G6g9z+/jJa1otm1tg+NIgPzBBC0AhrArY48Jwc+Md+Z0+aMBBzJ9jqe/vC2Oog0UMRu/dvWRz9oP5/MblzvLNY7H2RqOvKdVFuEUGir4Oq6vGugl6tCXxVM81bsZf75q6k8xnxTBvdm7oxJQ/F1VQiNkh4HpN+J95hHTcQCbZ4iJ4AeTO888jDm0Hc49gcvtlWMTf7X194cyT+1HnpSpVFA19ZZtainTw+bw19WiUyeWQvYh2199dRHBdC/U8xOTO90yrt/ZDoG70X14gdbnV5KkTU3r8wFdTe/nErL36zgf4dGvDm0O5ERtT+9sYS3hpJ+JvVZagQpoGvqpUxhpfmb+TtH7dyddczeOWGrkRoe2OlqoUGvqo2Ho/hiS/WMGvRLob0ac7TAzsTpu2Nlao2GviqWhS6PTz40Uo+X7GPCRedycQBHXQKrVLVTANfVbn8Qjd3vL+cb9cf5MEr2nPbxa017JWygAa+qlI5ThfjZy7hty1HeGrgWYzo19LqkpQKWRr4qsoczS1g1HuLWb03g38O7sqfezQt+0lKqSqjga+qRGpWPiOmpLDtUA5vDe3OFWc1srokpUKeBr4KuD3puQybnMzBTCdTR/Xi/Lblu+KVUqpqaeCrgNqSms3wKcnkOF3MGteHHi1Kb+illKo+GvgqYNbszWDk1BRE4IPx/eh0RrzVJSmlitDAVwGxZEcao6ctJs4RzqxxfTgzKdbqkpRSJ9HAV5X286ZDTJi5lMYJkcwc14cmdaKsLkkp5YcGvqqU/1u9n7s+WE6bBnHMGNObpDj/V1tSSllPA1+dto+X7uGhj1fSrVkd3hvVm4RovYyjUsFMA1+dlvd+287f/7uOC9rW553hPYi266+SUsFO/0pVhRhjeP37LbyyYBNXnNWQ124+B0d47e9lr1RtUKlG5CIyWETWiohHRHqe9NgjIrJFRDaKyBWVK1MFA2MMz329nlcWbGJQ9ya8OaS7hr1SNUhl9/DXAIOAd4reKSKdgJuAs4AzgG9FpJ0xxl3J7SmLuD2Gxz5bzQeLdzOyXwuevPosbNrLXqkapVKBb4xZD/hrdTsQ+MAY4wS2i8gWoDewsDLbU9YocHm4b+4Kvly1nzsvbcN9l7fT9sZK1UBVNYbfBFhU5PYe332qhskrcHPb7KX8sPEQj/6xA+MvbG11SUqp01Rm4IvIt4C/VoePGWPmlfQ0P/eZEtY/HhgP0Lx587LKUdUoK7+QsdOXsHhHGs9d14UhffTno1RNVmbgG2MuO4317gGaFbndFNhXwvonAZMAevbs6fdNQVW/tJwCRk5NYf3+TF696Ryu6XqG1SUppSqpUrN0SvEFcJOIOESkFdAWSKmibakAO5CRz43vLGTTwSwmjeihYa9ULVGpMXwRuQ54HUgCvhKRFcaYK4wxa0VkLrAOcAG36wydmmHXkVyGTllEWnYB00b3pl/relaXpJQKkMrO0vkM+KyEx54Fnq3M+lX12nQwi2GTkylwe3j/lr50bVbH6pKUUgGkZ9oqAFbuPsrI91Kwh9n4cHw/2jeKs7okpVSAaeArFm07wrjpS6gbE8GssX1oUS/G6pKUUlVAAz/Efb/hIH+ZtYxmidHMGtuHRgmRVpeklKoiGvgh7L8r93Hvhyvo2Die6WN6kxhjt7okpVQV0sAPUXNSdvHoZ6vp1SKRyaN6Eh+pveyVqu008EPQpJ+38tzXG7i4fRJvD+1BlF07XioVCjTwQ4gxhlcWbOL177dw1dmN+dcN3bCHV9W5d0qpYKOBHyI8HsNTX65j2u87uKlXM569rgth2t5YqZCigR8CXG4PD3+ymk+W7WHc+a147KqO2t5YqRCkgV/LOV1u7pqznPlrD3Lf5e2489I2GvZKhSgN/Fost8DFhJlL+WXzYZ74UyfGnN/K6pKUUhbSwK+lMvIKGTNtMct3pfPy9WczuGezsp+klKrVNPBrocPZTkZMSWFzahZvDunOlV0aW12SUioIaODXMvuO5jFscjL7MvKYPLIXF7VLsrokpVSQ0MCvRbYdymb4lBQy8wqZObYPvVomWl2SUiqIaODXEuv3ZzJ8SjIeA3PG96VzkwSrS1JKBRkN/Fpg2a50Rk1NIdoezqxxfWjTINbqkpRSQUgDv4b7dfNhxs9cQoM4BzPH9qFZYrTVJSmlgpQGfg32v7UHuOP95bSqH8PMcb1pEKe97JVSJdPAr6E+XbaHBz9eRecmCUwf3Ys60drLXilVOg38Gmjmwh08Pm8t57aux6QRPYl16I9RKVU2TYoa5s0ftvDy/I1c1rEhbww5h8gI7WWvlCofDfwawhjDi99s5D8/beXabmfw8uCuRIRpL3ulVPlp4NcAbo/h8XlreD95F0P7NOfpgZ2xaS97pVQFaeAHuUK3hwc+Wsm8Ffv4y8WteeiK9treWCl1WjTwg1h+oZvbZy/juw2pPDSgPbdd3MbqkpRSNZgGfpDKdrq4ZfoSFm0/wtPXdmZ43xZWl6SUquE08INQek4Bo95LYc2+TP51QzeuPaeJ1SUppWoBDfwgk5qZz/ApKWw/ksN/hvXg8k4NrS5JKVVLaOAHkd1puQydnMzhbCfTRvXi3Db1rS5JKVWLaOAHiS2pWQybnEJeoZvZ4/pwTvO6VpeklKplKnXmjoi8LCIbRGSViHwmInWKPPaIiGwRkY0ickXlS6291uzN4IZ3FuHyGD4Y31fDXilVJSp7quYCoLMx5mxgE/AIgIh0Am4CzgIGAG+JiPYA8CNlexo3T1pEVEQYH9/aj46N460uSSlVS1Uq8I0x/zPGuHw3FwFNfV8PBD4wxjiNMduBLUDvymyrNvpxYyojpiaTFO/go1v70bJ+jNUlKaVqsUA2YxkD/J/v6ybA7iKP7fHdp3y+WrWfW2YsoXVSLB9N6McZdaKsLkkpVcuVedBWRL4FGvl56DFjzDzfMo8BLmD2saf5Wd6UsP7xwHiA5s2bl6Pkmm/u4t1M/HQV3ZvXZcqoXiRERVhdklIqBJQZ+MaYy0p7XERGAn8C+htjjoX6HqBZkcWaAvtKWP8kYBJAz549/b4p1CaTf9nGM1+t58J2SbwzrAdRdj20oZSqHpWdpTMAeBi4xhiTW+ShL4CbRMQhIq2AtkBKZbZV0xlj+NeCTTzz1Xqu7NyId0do2Culqldl5+G/ATiABb4OjouMMbcaY9aKyFxgHd6hntuNMe5KbqvG8ngMz3y1nqm/bef6Hk15YVAXwrWXvVKqmlUq8I0xJbZvNMY8CzxbmfXXBm6PYeInq/ho6R5Gn9eSx6/qpL3slVKW0DNtq5DT5ebeD1fw9eoD3N2/Lfdc1lZ72SulLKOBX0XyCtxMmLWUnzcd4q9XdWTcBWdaXZJSKsRp4FeBzPxCxk5bzNKd6bz45y7c2Cs0ppsqpYKbBn6AHcl2MmJqCpsOZvH6zd256uzGVpeklFKABn5A7c/IY9jkZPak5zFpRE8uad/A6pKUUuo4DfwA2XE4h6GTk8nIK2TGmN70ObOe1SUppVQxGvgBsOFAJsOnpOBye5hzS1+6NE2wuiSllDqFBn4lLd+Vzqj3FhMZYWPuhH60bRhndUlKKeWXBn4l/L71MOOmL6F+rIPZ4/rQLDHa6pKUUqpEGvinacG6g9z+/jJa1otm1tg+NIiPtLokpZQqlQb+aZi3Yi/3zV1J5zPimTa6N3Vj7FaXpJRSZdLAr6BZi3by+Lw19G6ZyJRRvYh16LdQKVUzaFpVwNs/buXFbzbQv0MD3hzancgIbW+slKo5NPDLwRjDS/M38vaPW7m66xm8ckNXIrS9sVKqhtHAL4PHY3jiizXMWrSLIX2a8/TAzoRpe2OlVA2kgV+KQreHBz9ayecr9jHhojOZOKCDtjdWStVYGvglyC90c8f7y/l2/UEevKI9t13cWsNeKVWjaeD7keN0ccuMJfy+9QhPDTyLEf1aWl2SUkpVmgb+SbLyCxk+JYXVezN45YauDOre1OqSlFIqIDTwTxJtD+fM+jHcelFrBnRuZHU5SikVMBr4JwmzCa/c2M3qMpRSKuB0MrlSSoUIDXyllAoRGvhKKRUiNPCVUipEaOArpVSI0MBXSqkQoYGvlFIhQgNfKaVChBhjrK7hOBE5BOy0ug6f+sBhq4soB60zsLTOwNI6A6ukOlsYY5LKenJQBX4wEZElxpieVtdRFq0zsLTOwNI6A6uydeqQjlJKhQgNfKWUChEa+CWbZHUB5aR1BpbWGVhaZ2BVqk4dw1dKqRChe/hKKRUiNPBLISJPi8gqEVkhIv8TkTOsrskfEXlZRDb4av1MROpYXZM/IjJYRNaKiEdEgm5GhIgMEJGNIrJFRCZaXY8/IjJVRFJFZI3VtZRGRJqJyA8ist73M7/b6pr8EZFIEUkRkZW+Ov9udU2lEZEwEVkuIl+ezvM18Ev3sjHmbGNMN+BL4AmrCyrBAqCzMeZsYBPwiMX1lGQNMAj42epCTiYiYcCbwJVAJ+BmEelkbVV+TQMGWF1EObiA+40xHYG+wO1B+v10ApcaY7oC3YABItLX4ppKczew/nSfrIFfCmNMZpGbMUBQHvAwxvzPGOPy3VwEBOWFeI0x640xG62uowS9gS3GmG3GmALgA2CgxTWdwhjzM5BmdR1lMcbsN8Ys832dhTekmlhb1amMV7bvZoTvX1D+nYtIU+AqYPLprkMDvwwi8qyI7AaGErx7+EWNAf7P6iJqoCbA7iK39xCEAVUTiUhL4Bwg2dpK/PMNk6wAUoEFxpigrBP4N/AQ4DndFYR84IvItyKyxs+/gQDGmMeMMc2A2cAdwVqnb5nH8H6Unh3MdQYp8XNfUO7p1SQiEgt8Atxz0ifmoGGMcfuGbZsCvUWks9U1nUxE/gSkGmOWVmY9IX8Rc2PMZeVc9H3gK+DJKiynRGXVKSIjgT8B/Y2Fc20r8P0MNnuAZkVuNwX2WVRLrSAiEXjDfrYx5lOr6ymLMeaoiPyI9xhJsB0UPw+4RkT+CEQC8SIyyxgzrCIrCfk9/NKISNsiN68BNlhVS2lEZADwMHCNMSbX6npqqMVAWxFpJSJ24CbgC4trqrFERIApwHpjzCtW11MSEUk6NqtNRKKAywjCv3NjzCPGmKbGmJZ4fze/r2jYgwZ+WV7wDUesAv6A9wh5MHoDiAMW+KaQ/sfqgvwRketEZA/QD/hKROZbXdMxvoPedwDz8R5gnGuMWWttVacSkTnAQqC9iOwRkbFW11SC84DhwKW+38kVvr3TYNMY+MH3N74Y7xj+aU15rAn0TFullAoRuoevlFIhQgNfKaVChAa+UkqFCA18pZQKERr4SikVIjTwlVIqRGjgK6VUiNDAV0qpEPH/HnMEydZEhn4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xd8dcd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w = model.get_weights()\n",
    "x1 = []\n",
    "x2 = []\n",
    "for i in x:\n",
    "    x1.append(i[1])\n",
    "    x2.append(i[2])\n",
    "plt.scatter(x1, x2, c=y)\n",
    "plt.plot(np.arange(-3, 3, 1), -(w[1] * np.arange(-3, 3, 1) + w[0]) / w[2])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will try a 80-20 split of data into train and test and see what happens\n",
    "Try shuffling too. The results will change on shuffling\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 0, 0]\n",
      "Accuracy on testing data is 100.0\n"
     ]
    }
   ],
   "source": [
    "## TODO : Try shuffling the data\n",
    "\n",
    "model.reset()\n",
    "print (model.get_weights())\n",
    "test_x, train_x = x[80:], x[:80]\n",
    "test_y, train_y = y[80:], y[:80]\n",
    "converged = model.train(train_x, train_y)\n",
    "y_hat = model.predict(test_x) \n",
    "correct = 0 \n",
    "for i, y_real in enumerate(test_y):\n",
    "    if (y_hat[i]==y_real):\n",
    "        correct = correct + 1\n",
    "    \n",
    "print (\"Accuracy on testing data is {}\".format(correct*100/len(test_y)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VGXax/HvPUlm0gOBUKRLRxCkY1d0xXUVZcVCb4Jr76Kuumsvu+7aVwSkimLFVV9Z7A0Seu+9B0hIzyQz87x/zAAJTBqZ5Ewy9+e6uMjMnDnnnpTfnHnOc+4jxhiUUkrVfjarC1BKKVU9NPCVUipEaOArpVSI0MBXSqkQoYGvlFIhQgNfKaVChAa+UkqFCA18pZQKERr4SikVIsKtLqCo+vXrm5YtW1pdhlJK1ShLly49bIxJKmu5oAr8li1bsmTJEqvLUEqpGkVEdpZnOR3SUUqpEKGBr5RSIUIDXymlQoQGvlJKhQgNfKWUChEa+EopFSI08JVSKkRo4CulqsTSnelM+nmr1WWoIjTwlVIB9+vmwwyfksz7ybvIcbqsLkf5aOArpQJq/toDjJm2mOaJ0cy9tR8xjqA6oT+k6U9CKRUwny7bw4Mfr6JLkwSmje5FnWi71SWpIjTwlVIBMWPhDp6Yt5ZzW9fj3RE9dc8+COlPRClVKcYY3vpxKy/P38hlHRvyxpBziIwIs7os5YcGvlLqtBljeOGbDbzz0zau7XYGLw/uSkSYHhoMVhr4SqnT4vYYHp+3hveTdzGsb3OeuqYzNptYXZYqhQa+UqrCCt0e7p+7ki9W7uMvF7fmoSvaI6JhH+w08JVSFZJf6Ob22cv4bkMqDw1oz20Xt7G6JFVOGvhKqXLLdroYN30xydvTePrazgzv28LqklQFVProiog0E5EfRGS9iKwVkbt99yeKyAIR2ez7v27ly1VKWSU9p4Ch7y5i8Y50/nVDNw37GigQh9NdwP3GmI5AX+B2EekETAS+M8a0Bb7z3VZK1UCpmfncNGkR6w9k8Z9hPbj2nCZWl6ROQ6UD3xiz3xizzPd1FrAeaAIMBKb7FpsOXFvZbSmlqt/utFyu/89CdqfnMm1ULy7v1NDqktRpCugYvoi0BM4BkoGGxpj94H1TEJEGgdyWUqrqbUnNYtjkFPIK3cwe14dzmuvIbE0WsDMkRCQW+AS4xxiTWYHnjReRJSKy5NChQ4EqRylVSav3ZDD4PwtxeQwfTuirYV8LBCTwRSQCb9jPNsZ86rv7oIg09j3eGEj191xjzCRjTE9jTM+kpKRAlKOUqqSU7WkMeXcR0fZwPr61Hx0axVtdkgqAQMzSEWAKsN4Y80qRh74ARvq+HgnMq+y2lFJV78eNqYyYmkxSvIOPbu1Hy/oxVpekAiQQY/jnAcOB1SKywnffo8ALwFwRGQvsAgYHYFtKqSr01ar93PPhcto1jGPGmN7Ui3VYXZIKoEoHvjHmV6Ckc6r7V3b9SqnqMXfxbiZ+uoruzesyZVQvEqIirC5JBZieaauUYvIv23jmq/Vc2C6Jd4b1IMqu7Y1rIw18pUKYMYZ/f7uZV7/bzJWdG/Hvm7rhCNewr6008JUKUR6P4Zmv1jP1t+1c36MpLwzqQrj2sq/VNPCVCkFuj2HiJ6v4aOkeRp/Xksev6qS97EOABr5SIcbpcnPvhyv4evUB7u7flnsua6u97EOEBr5SISSvwM2EWUv5edMh/npVR8ZdcKbVJalqpIGvVIjIzC9k7LTFLN2Zzot/7sKNvZpbXZKqZhr4SoWAI9lORkxNYdPBLF6/uTtXnd3Y6pKUBTTwlarl9mfkMWxyMnvS85g0oieXtNfGtaFKA1+pWmzH4RyGTk4mI6+QGWN60+fMelaXpCykga9ULbXhQCbDp6TgcnuYc0tfujRNsLokZTENfKVqoRW7jzJyagqRETbmTuhH24ZxVpekgoAGvlK1zO9bD3PL9CXUi3Uwe1wfmiVGW12SChIa+ErVIt+uO8ht7y+jZb1oZo7tQ8P4SKtLUkFEA1+pWmLeir3cN3clnc+IZ9ro3tSNsVtdkgoyGvhK1QKzFu3k8Xlr6N0ykSmjehHr0D9tdSr9rVCqhnv7x628+M0G+ndowJtDuxMZoe2NlX8a+ErVUMYYXpq/kbd/3MrVXc/glRu6EqHtjVUpNPCVqoE8HsOTX6xl5qKd3Ny7Oc9c25kwbW+syqCBr1QNU+j28NDHq/hs+V4mXHgmE6/soO2NVblo4CtVg+QXurlzznIWrDvIg1e057aLW2vYq3LTwFeqhshxurhlxhJ+33qEv19zFiPPbWl1SaqG0cBXqgY4mlvAqPcWs3pvBq/c0JVB3ZtaXZKqgTTwlQpyqVn5jJiSwrZDObw5pDsDOjeyuiRVQ2ngKxXE9qTnMmxyMgcznUwd1Yvz29a3uiRVg2ngKxWkth7KZtjkZHKcLmaN60OPFnWtLknVcBr4SgWhNXszGDk1BRH4YHw/Op0Rb3VJqhbQwFcqyCzZkcboaYuJc4Qza1wfzkyKtbokVUto4CsVRH7edIgJM5fSOCGSmeP60KROlNUlqVpEA1+pIPHNmv3cNWcFrRvEMmNMb5LiHFaXpGoZ7bSkVBD4eOkebpu9jM5N4vnglr4a9qpKBCTwRWSqiKSKyJoi9yWKyAIR2ez7X6cYKOXHe79t54GPVnJu6/rMHNuHhOgIq0tStVSg9vCnAQNOum8i8J0xpi3wne+2UsrHGMNr323m7/9dxxVnNWTKqJ7E6IVLVBUKSOAbY34G0k66eyAw3ff1dODaQGxLqdrAGMNzX6/nlQWbGNS9CW8O6Y4jXC9coqpWVe5ONDTG7AcwxuwXkQZVuC2lagy3x/DYZ6v5YPFuRvZrwZNXn4VNe9mramD550cRGQ+MB2jevLnF1ShVtQpcHu6bu4IvV+3njkvacP8f2ml7Y1VtqnKWzkERaQzg+z/V30LGmEnGmJ7GmJ5JSUlVWI5S1sorcDNh5hK+XLWfR67swANXtNewV9WqKgP/C2Ck7+uRwLwq3JZSQS0rv5CR76Xw46ZDPHddFyZc1NrqklQICsiQjojMAS4G6ovIHuBJ4AVgroiMBXYBgwOxLaVqmrScAkZOTWH9/kxevekcrul6htUlqRAVkMA3xtxcwkP9A7F+pWqqAxn5DJ+SzK60XCaN6MGlHRpaXZIKYZYftFWqttp1JJehUxaRll3AtNG96de6ntUlqRCnga9UFdh0MIthk5MpcHt4/5a+dG1Wx+qSlNLAVyrQVu4+ysj3UrCH2fhwfD/aN4qzuiSlAA18pQJq0bYjjJu+hLoxEcwa24cW9WKsLkmp4zTwlQqQ7zcc5C+zltEsMZpZY/vQKCHS6pKUKkYDX6kA+GLlPu77cAUdG8czfUxvEmPsVpek1Ck08JWqpPeTd/HY56vp1TKRKSN7Ehep7Y1VcNLAV6oSJv28lee+3sDF7ZN4e2gPouza8VIFLw18pU6DMYZXFmzi9e+3cNXZjfnXDd2wh+sF5FRw08BXqoI8HsNTX65j2u87uKlXM569rgth2t5Y1QAa+EpVgMvt4aFPVvHpsr2MO78Vj13VUTteqhpDA1+pcnK63Nw1Zznz1x7kvsvbceelbTTsVY2iga9UOeQWuJgwcym/bD7ME3/qxJjzW1ldklIVpoGvVBky8goZM20xy3el8/L1ZzO4ZzOrS1LqtGjgV7PMI1l8+trXLPrvYuo0qMPQv/6ZLud3tLosVYLD2U6GT0lhS2oWbw7pzpVdGltdklKnTQO/GqUdTGdk27vIz8733bOTpf9byQ0PXsMtLw63tDZ1qr1H8xg+OZl9GXlMHtmLi9rpJThVzaYTh6vRw394pkjYnzD35S9Yn7LZgopUSbYdymbw279zKMvJrLF9NOxVraCBX00O7TnCjtW7Snx84uVPU5BfUI0VqZKs25fJDe8sxOnyMGd8H3q2TLS6JKUCQgO/mmxZvh0p5eScvJx8fpq7sBorUv4s3ZnOTZN+JYI0PvjzU3QKvxhP5j8wxmV1aUpVmo7hV5OGLZIICw/DVeA/OIzHMOvpj5jy6Gyyj+YQ4YigSdvG3PjgQM4f1Efne1eDXzcfZvzMFJKiDzFr0Fs0iU8HA+TOwJh0JOHZcq/LGAOFK8BkQMQ5iC2h6gpXqpzEGGN1Dcf17NnTLFmyxOoyqswdfR5h4+ItFX5ex75tmTjrbn6a+xtpBzLo3r8Lvf94DmFh2qgrUOavPcCd7y+nVWIGM679B0kxmSctEQb1f0bMAXCnQkQXJKyB33UZ1w5M2mgw6YANTCHE3YstZkyVvw4VmkRkqTGmZ5nLaeBXn8y0LP7+53+w6qd1p/V8W5gNj9tDVGwkjVo14KaJ19L5vA40aK4HFCvj02V7ePDjVXRpksDUq1+ijn29/wUlBu8uvw1MAUQPQeIeKfbpyxiDOdwf3Ht9yx4ThSS+i9h7V+ErAePJweS8CXlf+DY7EIm5DbFV7spbJv8HTO4M8GRA5OVI9HDEFhuAilUgaOAHqYL8Aq6rN5qCvMofoI1whCMiXDH6Eu58Y5wO+5yGGQt38MS8tZzbuh7vjuhJVN5DkP9lOZ8dhSQ8jURdc/weU7gKkzYCTO6pizsGYKv7WkDqPr69/G8x2W+C5wCEnw3uneDeAxz7/bJDeFuk3ifemwW/g2sThLUCx4WInPiUaEweJvdzKPgJbI2Q6CFIRDs8Wa9B7hQwecdeCIQ1Rup9DiYTCEfC6mM82eDeBrbGSJjuhFSn8ga+juFXM3uknaGPDWLO85+Rn+Os1LoKnd7jAQtm/ETHvu24fPhFgSgxJBhjeOvHrbw8fyOXdWzIG0POITIiDBN2Gyb/K4rvnZckD5M7vVjg48mixLkQnvQAVF5kdTkfQNbzgC+IC37ws1QBuLdj8udDzhveTx6mEMQOtvqQ+AESVg/jycYcuR7c+33rs2HyPsXEPQ45kzjxBgLgBPcOTGo3vK/VhrHV9e79ix1MAcZxMVLnZUSijj/LeLIxBYvBtREkFnH0RcLbBPR7okqngW+Bmx8ZRJ0GCcx5/jPSD2bQultLtizfftp7/fk5TmY+9RER9nB6/KErcXX1o3ZpjDG88M0G3vlpG9d2O4OXB3clIswb0hLeBhPRGwqTy7cyT1bx2xFdvYF6ikiIvKJyhRdhjAuyX+Z42Je6cD7kvAOuHYCvNlMIbicm829I3dcxuTN9w1DHdkI8QD5kPXM8xP3zeP95DvnW61vO+RMm429InRe9S+XM9r05nViPyXJgov6ExD93/NOpMW4wWSBxxT99uA9icj8G13aIvACJvBoRm+85BgoWQeFysDWAyAHHh5uMJxOT9xm4NkN4ZyTq6tMe3jIFS7yfplzbIKIzxNyBkO07ptMVCW96WuutTjqkEyRW/7qOBy7+Gx5PJX4eAoJw82PXMfqpmwNXXC3i9hgen7eG95N3Maxvc566pjO2k6bLmsKNmCM3UHaY2iFmJLa4B4vd6w23l4B8vJ8UoiC8OVJvbrE93sow7r2YQ38sR42ARINxUXwv/ZhwpOFazJHrwOXv2NKxC7GfesJg2exIw8VQuME7zIW/T7RRSMJzEPlHTO40yH7TO3QkkRBzGxIzBgp+w6SPB4rMcJMEqD8fscVijowE12rf64sAHEi9WSDRmCODfW9CeUAU2GKRep8gYY0q9EpM/g+Yo3cX+T4c+51xAGHe2qKuQeKfPv5GVJ10DL8G2r56J2/d/R6rf98AHnC73Ke9rrCIMHoN6Mb1915Nsw5nkNiobgArrZkK3R7un7uSL1bu4y8Xt+ahK9qXeNzDFKzEZL0AhWvAVhccF/oOhBbg3auNBFsiUv8zxHbq99YULMfkzgbPEXD8AYm+DpHIU5Y7XcaTi0ntg/8QLUpA6vqOKfgL7TBv4KeP8u4ln8L7OvEcwPu6K8KOJP2IyXoe8v9bymL9wDEAsl6g+BtYFMTeC9n/wu8bW3h7sF8Cuf/xs9K6EN4aXEspPjwXBo4/YKv7arlfhTEGc+hS8OwtY8koJP5xJPr6cq87UDTwa4HD+9J48tqX2LluN7YwG4VOFx16t2H9ok24XRX746vfNJE3kl+gXuPQDP78Qje3z17GdxtSeWhAe267uOJjx8a1BZMz03tQ1HE+EjXY0pkqnozHIW8exYPcAbZG4NnnvRnREUn4Bybr3+D8H8X2krGB/XxsiZMx+f/DZDxY5MCs7/Gw1kjiJEz6BO9wCv6Gq0pga4Ak/YJJ9+6llyiih/d76jl46mOS4B3i8ftmI4Cdst/0ThaJrdGqci9tTB7mYHegHDtg4R2x1Z9XwXoqTwO/FtmzaR8Zh7M4s2sLomIieWH4q3w3+9cKryc8Iox5mTOwO+xVUGXwyna6GDd9Mcnb03hqYGeG921hdUkBYUwBJvNpyPsc7568A+IexBZ9A8ZzFJDjJ3wZ9yHvQVmT4dvbjwZbNFLvIySsiXcvNusfkDsdJMK7AVtdpO50JNzbDtpTsArSRgB+ZiAVI4ADqfMvJLI/npz3fXvv/j5hRELco5D1N0r+BCH4P4he0v1llReHreHSci9ujBuT2sP/zKuThbXAlrSg4jVVkgZ+Lfbzxwv5x5i3yPPTiK0sXS7syN8+fZD4xLgqqCz4pOcUMOq9FNbsy+Sfg7ty7TlNrC4p4IzJ886QsdVHpOR5GMY4If8bTOEGJLwtRF15yjEF4071HfysBxHdTxmPNoWbMUfvAffJzf7CIawdUAjhLZCYCYi9q2+7+d6xdNc2in9CiPCehZw4FXP4Gu+UzpOFNQf3QfzvxUdS+rEFm+9f0U81doi6EVvC46U871SezJcgd1YZ27NDzChscQ9UaN2BoIFfi7kKXdzZ91F2rd9b8YZrArF1Yuh2SWeOHsqke/8uDLx9APH1at8bQGpmPsOmJLPjSC5vDenOZZ0aWl1SreEpWAXZb3jbR9jqQcx4JOrako+JmDxM7ife4yAmB8LPRKIGgeMiRGwY54+Y9LsoHqiRkPBPkDA4eutJawyDuGchdxq4N5RQZQSENfMefzAeEIHw9kjd9yo8U8cYFybzGcj7BCTcNxPr2CcSFxAFYfWRep9a0kYjaAJfRAYAr+I9lD3ZGPNCSctq4JdfXk4+n732Nd/N+pm0A+nkZORhTmOGjz0ygpg6Mbyz/GXqNqxTBZVaY3daLkMnJ3M428nkET05t019q0tSZTDO3zBZr4B7O4S1QOLuQxwXAOAp3AFZT3nn8Ie1QOLuQey98bj2wuH+nDocZAPH5Uid16AgGdw7ILyd9xNFJU5QNJ5s8KRC2BngTsXkvu89/mA/1/uGZ4uu+Drde71TYsPbILbT68waFIEv3om0m4DLgT3AYuBmY4zf3gIa+KfPmedky4odPHrls+RmlmOqXhFiE+o2SKBJ28ZcObY/5w3qzf6tB6nTIKFGHuTdkprFsMkp5BW6mTa6F+c0r3mvQZWfx50GaWPAvQ7vuL4NHP2RhJdOK4Cri/HkYo7e5X1DOnauQ/QNSNxjFZ7aGSyB3w/4mzHmCt/tRwCMMc/7W14Dv/JchS5ev30y30z9AY+notPoINwejsftITLGQaHTRbdLOvPYnLuJSahcL5bqsnpPBiOmJhNmszFrXG86NIq3uiRVTYwpAPcu73TZ09xTrk6eow9A/jcUPz8iCuIewhYztELrKm/gV/UZAk2A3UVu7/Hdp6pIeEQ49066lfmuD/nk8FRadmmOLaz8P2ZXgQuP20NuZh6FzkJW/LCa54aWf86ylVK2pzHk3UVE28P5+NZ+GvYhRsSO+BkWMcaJcf6Ccf7kPcDthzEGU5CCJ/NFPFmvY1w7q7RWY/L9hD1AHuROrbLtVnVrBX+DZcU+UojIeGA8QPPmzau4nNASnxjHKz/+nUcGPMPOdXuwhdnIy873nopezg92hU4Xy79bQ/rBowEd41/7+0ZmP/Mxezbtp13P1lx8w7nkZObSqGUDulzYEZutYvsiP2xM5daZS2laN4pZ4/rQOCEwZ7Sqms04f8McvbPIPR5IeAWJvPTEMsZgMh4A57feNhSEYXLexcT/HVv0dVVUWB4l/hF6Tm7NHThVHfh7gGZFbjcF9hVdwBgzCZgE3iGdKq6nVktPzSDCHk5snRPDL3F1Y3kj+QU2L9vGge2pJDWrx5t3TWXb6l14XG5chWWfTBJuD+NoakaFA7/AWchnr37F/Pd+wOMxXD7iQv5879Ws/GENT9/wCk5f76D92w7y09zfiXBEgDFE14nmuS8foV3P8p0c9dWq/dzz4XLaNYxjxpje1It1VKhOVfsYk4cp3ATpf+HkqZTm6D2Q9O2J6xkU/ATO74qcdOby/st8AhPZH7FVwSdFqePt+3PK2bs2sPcN/PaObbaKx/DD8R607Q/sxXvQdogxZq2/5XUM//RsXLKVF0e8zv5tB8EYzjqvAxNn3kn9JvX8Lu92ubnn/L+ydeWO4x03SxMZ7WD27v8QX4GmbMYYHuz/dzYkbz4e7PaoCFp1aUH6gaOk7jpc6vPFJtz+6mgG3n5lqcvNXbybiZ+uonvzukwZ1YuEqIhy16hqB+PJ8HYiDWsChGGy/gm5M/HO3PE3bdmOxN2PxIwGwHP0Qcj3c3asxCAJzyORA6qmbuevmPTbONGuIwIk0ju1M7xiJwcGRXtkY4xLRO4A5uOdljm1pLAPVVnp2eTnOKnfJJGs9GxW/rAWe2QEZ3ZtSd2GCYRHlP4jSj94lPsveuJ4qAKs/HEt95z/ODO2voHH7WHhF0tY+/sGGrZsQP+hFzDvjW/YuGSr32mcIoLYBI/7xAFfj8fDkGa3MuTR67j5kUHlmta26ud1bFy8pVhdBXmF7Fy7m4K8sk/PNx7DpAdn0vuP3Wncyv/8+cm/bOOZr9ZzYbsk3hnWgyi7XgGsuhhPNuT/D0wa2HsjEWefeMy1BdxHIKITYvN/focxHihYCO7dEN4RIs4+6UIyhb6umQnFumYWryEXkzERnN9758YT5u3L4/yF0k+QKsR4sk6MN4sd/2ftClUZkeI4H+p9hMmZ7G1bYe+OxIypcGO3iqjy9sjGmK+Br6t6OzXN0UMZvDDsNVb9tA6xCRGOCPJzndhsNgqd3kB0RNm58eFrGfb49bgKXWxI3kxkTCRtzml1/I9j7j++KBaqx6TuOszCLxYz/cm5HNieSl52Po5oO5Mnzio1cMMdYfS7uhdL5q/wjvd7DAX53uXnPP8ZjVo15NKbzy/z9W1I3uL300N+jhNbuK1cfbg8HsOvnyQz+IFrit1vjOHf327m1e82c2XnRvz7pm44wjXsq5Jx78XkfuZtzWBrCtn/xhuQBUA4JrwTOC6G3A/AHMQ7H8RgYm7BFnfvSes6hEkb6m2pbNyAeNtKJ74L2L1X7MqZ4uvbH4mJvQtbzIhTa8p4GJw/ems43pb5f+V4NZGI48LjtyTqOkzel5zaoM0DjvPK8+05bRLRHqnzcpVuoyjth28BYwwTr3iGnWt3Hx9DPxaq7iINmpx5BXzw4ucs/r/lrE/Z7P37Eu+4/E0Tr+ObKd+xZ9P+ErfxwQvz2LNp//E3EGdu2WflFua72LR0K4VO1ymfAPJznHzwwmflCvz6TRKxR0aQl138GEFkjIOOfdux7veNft+oTnoRp7SL9ngMT3+1jvd+28HgHk15flAXwiswC0lVnCdvAWTcj7d5mL+dBRe4lnn/Hef7uee8jUdisMWOB3ytp4/e7Z0+WfRdv3A5JvttkCjIfpfj4WsKIOufeCSu2AFU40kD5w/4H7IphUSD4zLvG8yxu+w9MTGjIGcqx+fxi0HqvBmwdtbBQlsrVDFjDGt/38jyb1cTWzeGi248lyN707jvoicqfcWrssTWjSE7PafCz7NHRmAMx98oikpIiufjg1PKXEd+rpOhLf5CVloWRX/FYhKimb75daY8MptvZ/1CWHgY+Tn+P37bIyN4Z+U/adq2MeDtZT/xk1V8tHQPo89ryeNXdTqll32oMoWboHAZ2JJ8ly48vWMZxrXVe7ZrwRJvy4ToUZD9vLcdwmmLgAarIOuvkPclJXa3tDXwHjg1Wac+FtYMW9J3Rerc4ut1X866JBbs5yFR14LjUr/Dksa1Cwp+9b7pOC4rcTgqGAXFiVcVVV2B73a5Sf56GQe2pdK6W0vOvqhTQK8Ha4zh10+T+b8p37Fl+Xayj+YUG96ISYjGVeAqew+3EsQmNGhWn4M7D1X4uRGOcKJio8g8UvwPT2zCedf25smPy9ccaue63Txz47/Yt/UAiNCgeX0em3MPbbq1AiD7aA5H9qURXz+OV299l4X/XYIxBpvNRlh4GMOfvJ6bHvbu1Tldbu79cAVfrz7A3f3bcs9lbfUavvg6OR69G5w/e++QMJAoJHE2Et6q5Od50sB9AMJaHj8b1bh2YY4M9HWFPJYLxzqrVuZ3VSD2Ich5vfSOk1IH7zVy/Y33ObA1Wn2ifuPEpPb1E/g27/aKtTKOROq+gzj6nfYrCHYhE/jb1+zirbunsvrXDUTFRHLVhMsZ+fcbiLD738M5vPcId5//V7LSsnE5XYTbw2neqSkvf/sEUbGB+fj20qg3+OWTRVW+B1+SsIgwLh1yPs3bN2HW0x9X6I1FBJp3bMrwJwbz8pg3jw8D2cJsOKIdvJnyPM3aV+zcudTdhzEeQ4Pm9UsN6d0b9/LLJ8mA4fxBfWnewbudvAI3E2Yt5edNh/jrVR0Zd8GZFdp+bWUKN2LSbgFz4KRHBMJaY0s69dCZx5kCRx/wjbP7pq/G3orE/AWT+aiv1fLpX3jHv3AIPwdci0tfJvI67x62x88wZXgnbPU/L3aXJ2cGZP2TE2PvNpAYiH8B8j70XdawLRJ75/HOnbVVUMzSqWqpuw9zz3l/JTfL+wPPPprD5699zb6tB3hi7v1+n/OPMW9xeE/a8VkohQUutq3cyfS/zeXWf4ysdE1blm/n548Xlmu8XAQC+X4bHhFGi7Oacc1tAxgw5hJchW6Wf7+GdQs34sx1lrotW7gNR6SdCEc4j8+9jxadmlG3UR3mPP8p+7cd5KxzOzDksUE0adO4wnU1aFa+xmXN2jdhyKODit2XmV/I2GmLWboznRf/3IX3c9ndAAAVO0lEQVQbe4XGyXnGk+VtUyzxvhksJ45TGGMwzp/g6MndJY8vAe49GNcuJPzE98tz9P6Trjzle27OOxDWFAqWUbGwj6BcF0SxJXp79ZfIAbZ6SPz9GOf5kPEwJ3fNlLiHT11tzAhMWFNMzjveFsr2Pkjs7d7XHHV5BV5H6KjRgf/pq19R4CwerM68ApK/XMqBHak0atngpMecrPhhbbEph+Adq/5u5s8BCfzl360u96UJxWYjsWECR/aln/KYI9pBi05N6XxBBz579etTDqCKTYrd54h28PD0O7jgzydO2rA7bLww/6+sT97M/Knfs2DmT8VnzgiceXYLbvv3aDambKF+k0TOu643jijvH+fZF3bi7As7VeTlB9SRbCcjpqaw6WAWr9/cnavOrvibTU3gyf8F8j4A40GirsZ4DkPWy74LkRjvVZ8SpyLhZ3ov5p02wtuhsbSwFaHoMIzHuazkywyaPG9ohjX3dqo8RfiJwDZub01RV0NYB3BtgvwvfLX4+b2XGKTu2+DaiclcetIVtXzrjn8MiRqISBQSdSXGFu29Qpd7t7eDZNz9iL2X/5cZeWmxs2ZV6Wp04G9eug1Xwam/ZBGOCHZv2HtK4Jd2gXC3u+KNxvyJrRtDeES437pO1qRtY6au+zcrfljD7Gc/4cC2VNr3bsPwJwfTomPT48u1PrsF7zwwg8wj2TiiHQy8/Qpi68bw4UvzyM92El8/jrHPDykW9seICJ36tqNT33a06d6KKY+8j9vtwV3o5vxBfbh/8q04ohx0veisgLz+QNmfkcewycnsPZrHpBE9uaR9g7KfFOSMMeBa7x07j+gEtiTMkZvAtfLEMgU/4R3D9oDxDQmaXEzaGEj63nfxkV2UuScucRBWZOgrd3Lpy7uPIPFPYdKSOeWSiY5LkYTnvfPdTQbY+yHhJ9ZtPPdhcj/1XhPX7TtmFN7ce/A48o+ILcY7bTP/Kyj43Rf6dsCG1H0NcVxcvHTHRYjjotLrVaelRo/hv3XPe/z37fmntAewR0Ywec2/aHzmqSfs3HXuY2xI3kzR1x0eEcYfRl3Cve9MOP3ifXIycri52a1lXo3KEWXn0ffv4dyB/vdcysPj8eDMdRIZE1nuA5iFBYUc3HmYOknxxVowBJMdh3MYOjmZjLxCpo7qRe9Wwd/5sCzGfcR7bVf3DiDMO90wrKn/qzz5IzHei4EcvYuyD6A6kMQpiL338Xs8aeOh4MdSnvIHbHXfwJM339t3/lg/l6irkPi/BeQC7MYYKFyMcf4CUgeJ+hMSphelCYRg6ZZZpQbdc5W3/0oR9sgIul92tt+wB3ho2u3EJcYQGeP9iBoVG0mD5vUZ+9yQgNQUkxDDs189Sly9WKLjo4iOjyIyNpIuF3Qgtm4MCJzRphEPz7izUmEPYLPZiIqNqtBslQh7BE3bNg7asN9wIJPB7ywkt8DFnFv61oqwBzAZ93oPIpo8MNlAQfnDHgABz2FK/5O1ga0Z1F9QLOwBiC7t99uOxN3nXUPUFUjSL0jS90jDFGwJLwQk7MF3Fre9N7a4+7HFjtWwt0CN3sMH2LxsG6/fMZkNyVuwR9m5cswl3PLScOyRJV+oOzcrjx/m/Mqezftp16M1513XG7sjsD1Y3C436xZuwuP20OncdiXOGlInLN+Vzqj3FhMZYWPW2D60bVhz5kGXxnjSMKkXUrmpjQ5I+hWO/Ak8B/08HgUxo5HYvyAlHCD1HLkZCk+6eLfUhcS52CJqx4XdQ1XITMs8xuPxePcgdG52jfT71sOMm76E+rEOZo/rQ7PE4L1SUUUZ127M4asovb/LyRx4T1ASIBLi7sUWMwrjXIhJvxVvR8dCvNdSTfRdS7X0K3sZYzB5n0PudMAD0SOQqEEVvrqSCj4hMS2zqIr2T1fBY8G6g9z+/jJa1otm1tg+NIgPzBBC0AhrArY48Jwc+Md+Z0+aMBBzJ9jqe/vC2Oog0UMRu/dvWRz9oP5/MblzvLNY7H2RqOvKdVFuEUGir4Oq6vGugl6tCXxVM81bsZf75q6k8xnxTBvdm7oxJQ/F1VQiNkh4HpN+J95hHTcQCbZ4iJ4AeTO888jDm0Hc49gcvtlWMTf7X194cyT+1HnpSpVFA19ZZtainTw+bw19WiUyeWQvYh2199dRHBdC/U8xOTO90yrt/ZDoG70X14gdbnV5KkTU3r8wFdTe/nErL36zgf4dGvDm0O5ERtT+9sYS3hpJ+JvVZagQpoGvqpUxhpfmb+TtH7dyddczeOWGrkRoe2OlqoUGvqo2Ho/hiS/WMGvRLob0ac7TAzsTpu2Nlao2GviqWhS6PTz40Uo+X7GPCRedycQBHXQKrVLVTANfVbn8Qjd3vL+cb9cf5MEr2nPbxa017JWygAa+qlI5ThfjZy7hty1HeGrgWYzo19LqkpQKWRr4qsoczS1g1HuLWb03g38O7sqfezQt+0lKqSqjga+qRGpWPiOmpLDtUA5vDe3OFWc1srokpUKeBr4KuD3puQybnMzBTCdTR/Xi/Lblu+KVUqpqaeCrgNqSms3wKcnkOF3MGteHHi1Kb+illKo+GvgqYNbszWDk1BRE4IPx/eh0RrzVJSmlitDAVwGxZEcao6ctJs4RzqxxfTgzKdbqkpRSJ9HAV5X286ZDTJi5lMYJkcwc14cmdaKsLkkp5YcGvqqU/1u9n7s+WE6bBnHMGNObpDj/V1tSSllPA1+dto+X7uGhj1fSrVkd3hvVm4RovYyjUsFMA1+dlvd+287f/7uOC9rW553hPYi266+SUsFO/0pVhRhjeP37LbyyYBNXnNWQ124+B0d47e9lr1RtUKlG5CIyWETWiohHRHqe9NgjIrJFRDaKyBWVK1MFA2MMz329nlcWbGJQ9ya8OaS7hr1SNUhl9/DXAIOAd4reKSKdgJuAs4AzgG9FpJ0xxl3J7SmLuD2Gxz5bzQeLdzOyXwuevPosbNrLXqkapVKBb4xZD/hrdTsQ+MAY4wS2i8gWoDewsDLbU9YocHm4b+4Kvly1nzsvbcN9l7fT9sZK1UBVNYbfBFhU5PYe332qhskrcHPb7KX8sPEQj/6xA+MvbG11SUqp01Rm4IvIt4C/VoePGWPmlfQ0P/eZEtY/HhgP0Lx587LKUdUoK7+QsdOXsHhHGs9d14UhffTno1RNVmbgG2MuO4317gGaFbndFNhXwvonAZMAevbs6fdNQVW/tJwCRk5NYf3+TF696Ryu6XqG1SUppSqpUrN0SvEFcJOIOESkFdAWSKmibakAO5CRz43vLGTTwSwmjeihYa9ULVGpMXwRuQ54HUgCvhKRFcaYK4wxa0VkLrAOcAG36wydmmHXkVyGTllEWnYB00b3pl/relaXpJQKkMrO0vkM+KyEx54Fnq3M+lX12nQwi2GTkylwe3j/lr50bVbH6pKUUgGkZ9oqAFbuPsrI91Kwh9n4cHw/2jeKs7okpVSAaeArFm07wrjpS6gbE8GssX1oUS/G6pKUUlVAAz/Efb/hIH+ZtYxmidHMGtuHRgmRVpeklKoiGvgh7L8r93Hvhyvo2Die6WN6kxhjt7okpVQV0sAPUXNSdvHoZ6vp1SKRyaN6Eh+pveyVqu008EPQpJ+38tzXG7i4fRJvD+1BlF07XioVCjTwQ4gxhlcWbOL177dw1dmN+dcN3bCHV9W5d0qpYKOBHyI8HsNTX65j2u87uKlXM569rgth2t5YqZCigR8CXG4PD3+ymk+W7WHc+a147KqO2t5YqRCkgV/LOV1u7pqznPlrD3Lf5e2489I2GvZKhSgN/Fost8DFhJlL+WXzYZ74UyfGnN/K6pKUUhbSwK+lMvIKGTNtMct3pfPy9WczuGezsp+klKrVNPBrocPZTkZMSWFzahZvDunOlV0aW12SUioIaODXMvuO5jFscjL7MvKYPLIXF7VLsrokpVSQ0MCvRbYdymb4lBQy8wqZObYPvVomWl2SUiqIaODXEuv3ZzJ8SjIeA3PG96VzkwSrS1JKBRkN/Fpg2a50Rk1NIdoezqxxfWjTINbqkpRSQUgDv4b7dfNhxs9cQoM4BzPH9qFZYrTVJSmlgpQGfg32v7UHuOP95bSqH8PMcb1pEKe97JVSJdPAr6E+XbaHBz9eRecmCUwf3Ys60drLXilVOg38Gmjmwh08Pm8t57aux6QRPYl16I9RKVU2TYoa5s0ftvDy/I1c1rEhbww5h8gI7WWvlCofDfwawhjDi99s5D8/beXabmfw8uCuRIRpL3ulVPlp4NcAbo/h8XlreD95F0P7NOfpgZ2xaS97pVQFaeAHuUK3hwc+Wsm8Ffv4y8WteeiK9treWCl1WjTwg1h+oZvbZy/juw2pPDSgPbdd3MbqkpRSNZgGfpDKdrq4ZfoSFm0/wtPXdmZ43xZWl6SUquE08INQek4Bo95LYc2+TP51QzeuPaeJ1SUppWoBDfwgk5qZz/ApKWw/ksN/hvXg8k4NrS5JKVVLaOAHkd1puQydnMzhbCfTRvXi3Db1rS5JKVWLaOAHiS2pWQybnEJeoZvZ4/pwTvO6VpeklKplKnXmjoi8LCIbRGSViHwmInWKPPaIiGwRkY0ickXlS6291uzN4IZ3FuHyGD4Y31fDXilVJSp7quYCoLMx5mxgE/AIgIh0Am4CzgIGAG+JiPYA8CNlexo3T1pEVEQYH9/aj46N460uSSlVS1Uq8I0x/zPGuHw3FwFNfV8PBD4wxjiNMduBLUDvymyrNvpxYyojpiaTFO/go1v70bJ+jNUlKaVqsUA2YxkD/J/v6ybA7iKP7fHdp3y+WrWfW2YsoXVSLB9N6McZdaKsLkkpVcuVedBWRL4FGvl56DFjzDzfMo8BLmD2saf5Wd6UsP7xwHiA5s2bl6Pkmm/u4t1M/HQV3ZvXZcqoXiRERVhdklIqBJQZ+MaYy0p7XERGAn8C+htjjoX6HqBZkcWaAvtKWP8kYBJAz549/b4p1CaTf9nGM1+t58J2SbwzrAdRdj20oZSqHpWdpTMAeBi4xhiTW+ShL4CbRMQhIq2AtkBKZbZV0xlj+NeCTTzz1Xqu7NyId0do2Culqldl5+G/ATiABb4OjouMMbcaY9aKyFxgHd6hntuNMe5KbqvG8ngMz3y1nqm/bef6Hk15YVAXwrWXvVKqmlUq8I0xJbZvNMY8CzxbmfXXBm6PYeInq/ho6R5Gn9eSx6/qpL3slVKW0DNtq5DT5ebeD1fw9eoD3N2/Lfdc1lZ72SulLKOBX0XyCtxMmLWUnzcd4q9XdWTcBWdaXZJSKsRp4FeBzPxCxk5bzNKd6bz45y7c2Cs0ppsqpYKbBn6AHcl2MmJqCpsOZvH6zd256uzGVpeklFKABn5A7c/IY9jkZPak5zFpRE8uad/A6pKUUuo4DfwA2XE4h6GTk8nIK2TGmN70ObOe1SUppVQxGvgBsOFAJsOnpOBye5hzS1+6NE2wuiSllDqFBn4lLd+Vzqj3FhMZYWPuhH60bRhndUlKKeWXBn4l/L71MOOmL6F+rIPZ4/rQLDHa6pKUUqpEGvinacG6g9z+/jJa1otm1tg+NIiPtLokpZQqlQb+aZi3Yi/3zV1J5zPimTa6N3Vj7FaXpJRSZdLAr6BZi3by+Lw19G6ZyJRRvYh16LdQKVUzaFpVwNs/buXFbzbQv0MD3hzancgIbW+slKo5NPDLwRjDS/M38vaPW7m66xm8ckNXIrS9sVKqhtHAL4PHY3jiizXMWrSLIX2a8/TAzoRpe2OlVA2kgV+KQreHBz9ayecr9jHhojOZOKCDtjdWStVYGvglyC90c8f7y/l2/UEevKI9t13cWsNeKVWjaeD7keN0ccuMJfy+9QhPDTyLEf1aWl2SUkpVmgb+SbLyCxk+JYXVezN45YauDOre1OqSlFIqIDTwTxJtD+fM+jHcelFrBnRuZHU5SikVMBr4JwmzCa/c2M3qMpRSKuB0MrlSSoUIDXyllAoRGvhKKRUiNPCVUipEaOArpVSI0MBXSqkQoYGvlFIhQgNfKaVChBhjrK7hOBE5BOy0ug6f+sBhq4soB60zsLTOwNI6A6ukOlsYY5LKenJQBX4wEZElxpieVtdRFq0zsLTOwNI6A6uydeqQjlJKhQgNfKWUChEa+CWbZHUB5aR1BpbWGVhaZ2BVqk4dw1dKqRChe/hKKRUiNPBLISJPi8gqEVkhIv8TkTOsrskfEXlZRDb4av1MROpYXZM/IjJYRNaKiEdEgm5GhIgMEJGNIrJFRCZaXY8/IjJVRFJFZI3VtZRGRJqJyA8ist73M7/b6pr8EZFIEUkRkZW+Ov9udU2lEZEwEVkuIl+ezvM18Ev3sjHmbGNMN+BL4AmrCyrBAqCzMeZsYBPwiMX1lGQNMAj42epCTiYiYcCbwJVAJ+BmEelkbVV+TQMGWF1EObiA+40xHYG+wO1B+v10ApcaY7oC3YABItLX4ppKczew/nSfrIFfCmNMZpGbMUBQHvAwxvzPGOPy3VwEBOWFeI0x640xG62uowS9gS3GmG3GmALgA2CgxTWdwhjzM5BmdR1lMcbsN8Ys832dhTekmlhb1amMV7bvZoTvX1D+nYtIU+AqYPLprkMDvwwi8qyI7AaGErx7+EWNAf7P6iJqoCbA7iK39xCEAVUTiUhL4Bwg2dpK/PMNk6wAUoEFxpigrBP4N/AQ4DndFYR84IvItyKyxs+/gQDGmMeMMc2A2cAdwVqnb5nH8H6Unh3MdQYp8XNfUO7p1SQiEgt8Atxz0ifmoGGMcfuGbZsCvUWks9U1nUxE/gSkGmOWVmY9IX8Rc2PMZeVc9H3gK+DJKiynRGXVKSIjgT8B/Y2Fc20r8P0MNnuAZkVuNwX2WVRLrSAiEXjDfrYx5lOr6ymLMeaoiPyI9xhJsB0UPw+4RkT+CEQC8SIyyxgzrCIrCfk9/NKISNsiN68BNlhVS2lEZADwMHCNMSbX6npqqMVAWxFpJSJ24CbgC4trqrFERIApwHpjzCtW11MSEUk6NqtNRKKAywjCv3NjzCPGmKbGmJZ4fze/r2jYgwZ+WV7wDUesAv6A9wh5MHoDiAMW+KaQ/sfqgvwRketEZA/QD/hKROZbXdMxvoPedwDz8R5gnGuMWWttVacSkTnAQqC9iOwRkbFW11SC84DhwKW+38kVvr3TYNMY+MH3N74Y7xj+aU15rAn0TFullAoRuoevlFIhQgNfKaVChAa+UkqFCA18pZQKERr4SikVIjTwlVIqRGjgK6VUiNDAV0qpEPH/HnMEydZEhn4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x364a2b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w = model.get_weights()\n",
    "x1 = []\n",
    "x2 = []\n",
    "for i in x:\n",
    "    x1.append(i[1])\n",
    "    x2.append(i[2])\n",
    "plt.scatter(x1, x2, c=y)\n",
    "plt.plot(np.arange(-3, 3, 1), -(w[1] * np.arange(-3, 3, 1) + w[0]) / w[2])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What if data is not linearly seperable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4lFX2wPHvmT6TRkIQEFEsFBURFQtgASv2XeyLHXV/urprXXXtomtbcV3bLmvBir2sXSzYUUCxIKjYUekQkkymvuf3x8SQZCYkkEkmmZzP8+R5nHfeue8ZSc7cue+954qqYowxJn+4ch2AMcaY7LLEbowxecYSuzHG5BlL7MYYk2cssRtjTJ6xxG6MMXnGErsxxuQZS+zGGJNnLLEbY0ye8eTiouXl5dqvX79cXNoYYzqtWbNmLVXVHs2dl5PE3q9fP2bOnJmLSxtjTKclIj+05DwbijHGmDxjid0YY/KMJXZjjMkzltiNMSbPWGI3xpg8k5NZMaZlPpk2h4eueZKF3y5iixEDGXfxoWzQv3euwzLGdHCW2Duo1x9+h4kn3UE0HANg4fdLePfpGdz6wTVsOKhPjqMzxnRkNhTTATmOw+1/uacuqQM4SYdIdYTJl0zJYWTGmM7AEnsHtHzhSmqqImnH1VE+e3tuDiIyxnQmltg7oIKSEOpk3mS8tFdpO0djjOlsLLF3QMGCAKOPGonX1/AWiLiE/U7aI0dRGWM6C0vsHdT/3XgciDQ8qPDwtU8Rj8VzE5QxplOwxN5BvXjX6zjJZINjqkq4sob3nrECasaYplli74B+nv8rky+ZQjLhpD0XDcf49ZuFOYjKGNNZtDqxi0hARD4UkU9EZI6IXJGNwLqy+654jEQ8mfE5X8DLJlv3a9+AjDGdSjYWKEWB3VW1SkS8wDsi8qKqTs9C213S52/PbXJWTHmfMrbbe0g7R2SM6Uxa3WPXlKrah97an8xZybRI9/XLMh53uV1c+8rFuN3udo7IGNOZZGWMXUTcIjIbWAxMVdUPMpxziojMFJGZS5YsycZl89Yf/jYWf8jf4Jgv4GXXw4bTc8P1chSVMaazyEpiV9Wkqg4FNgB2EJHBGc6ZpKrDVHVYjx7NbtnXpe10wHaccv3RhIqDBAsD+AJeRhy8PefceWquQzPGdAJZLQKmqitFZBowBvg8m213NQedNoZ9T9qDhd8tptt6JRSVFuY6JGNMJ5GNWTE9RKRb7X8HgT2Bea1t14DX56XvwD6oKs/c9hJ3X/wQM176GMdJnwZpjDG/yUaPvTdwr4i4SX1QPKqqz2WhXQPM/eBrzt/7SpyEQ7QmxlOFL7Lp1htx/dRL8QV8uQ7PGNMBZWNWzKequo2qDlHVwap6ZTYCM6mVplcdMZGaygjRmlQJ30hVhPkff8czt72U4+iMMR2VrTztwH768hdWLatMOx4Nx5h635s5iMgY0xlYYu/AXC5Bm1gR4HLbP50xJjPLDh1Yn/696b5+ev11f8jPvla+1xjTBEvsHZiIcNnj51JYWkCwMIDH6yZQ4GfrUVtwwCl75To8Y0wHZZtZd3CbDNmIKT/9h3ee/IAVC1ey5ciBbL7TAKRxrXZjjKllib0TCIT87Hn0rrkOwxjTSdhQjDHG5BlL7MYYk2cssRtjTJ6xxG6MMXnGErsxxuQZS+ydRDKRZPFPS6mpjuQ6FGNMB2fTHTuBF+9+jUnn3U88GkcdZY9xu3D6rSfh83uzeh1VBw0/AOF7wakC/85I0dmIu09Wr2OMaVuW2Du4D1/8mNv+fA/RcLTu2OsPvYPjKOfedVpWr6WrLoeaZ4Ca1IHI82j0bSh/AXGXZ/Vaxpi2Y0MxHdyDVz3eIKkDRGtivD7lHcKVNVm7jiYXQ81T1CV1ABzQcKoXb4zpNCyxd3CLf1ya8bjb7aJiyarsXSjxJUimjTtiEJuRvesYY9qcJfYObvPhAxBXel0Yt8dNj77ds3chdx/QeKYnwLNx9q5jjGlzltg7uOOuOIJAyN+g6Jc/5OfEq4/C483eLRLxbALeIUDjXrsPKTg+a9cxxrQ9S+wd3Eabb8At0//OyN/tQGmvbgzYbhMufODPHHTamKxfS0rvAP9oUsndB+6+SOkdiGezrF/LGNN2RJvaoqcNDRs2TGfOnNnu1zUto04YtAZcZVYe2JgORERmqeqw5s6z6Y4mjbhCQCjXYRhj1pEl9k5MVZn+3CyevuVFqlZWsfPYndj/5D0oKiuynrYxXZgNxXRi91w8hSdvfp5IdcN57kVlhRxz6WH87ox9LcEbk0daOhTT6punItJXRN4QkbkiMkdE/tLaNk3zVixayWMTn01L6gCVy6u4628P8cytL+YgMmNMrmVjVkwCOEdVNwd2Av4kIltkoV2zBl+8/xVeX9MjadFwlPsnPE4uvpEZY3Kr1YldVX9V1Y9q/7sSmAtY1ag2VtKjGHXWnLQrl1cRjyXaKaKWUY2hmv4twxiTPVmdxy4i/YBtgA8yPHeKiMwUkZlLlizJ5mW7pC2GD6CkR3HGVam/Ke3ZbY29+vakyYU4y09AF22NLhqKs/wYNLEg12EZk5eylthFpBB4AjhTVdOKmKjqJFUdpqrDevToka3Ldlkul4vrX72UDQf1wetPT97+kI/x1/yhQ9w8VY2hyw6H2HQgmfqJzUCXH46q1Zc3JtuykthFxEsqqT+oqk9mo03TvN4b9+SG1y9jyxGD6nruIlBQEmL4Qduz1c6br1O7Gn0PZ+nvcRZujbNkXzTycusCjb4GWkkqqf8mVTmSyEuta9sYkyYbs2IEuAuYq6oTWx+SaSnHcThn1OV89vbcuvF2VaiuCPP2E9M5afBZPPefV9aqTY2+h674P0jMAWog+Q268jyccCs+rxM/QqaeuYbRxA/r3q4xJqNs9NhHAscAu4vI7Nqf/bLQrmnG7Nc/Z+mCZSQTybTnkvEksUicO86azLJfV7S4Ta28HmichCNQ9Y91n2HjHQgSSD8uIcQ7aN3aNMY0KRuzYt5RVVHVIao6tPbnhWwEZ9bs569/xUk6azxHXC6mP5taDBatiTa/OUfi28zHnRWp+jHrwrdLqixwg8qRXnD1BP/u69amMaZJVt2xE9t4yEZrnBUDqTH3muoolxx8Hb/rdhxjy0/g1GF/5ZtPvs/8AnfvJhoKZe51t4CIGymbAsHDQIpTP8GxSPdHSN2eMcZkk5UU6MRUlb/sfDHzP/qOeDTTJhng9Xvo2W89fv12Ecn46iGbUHGQyV/dQul6JQ3Od2qeh4oLaTgcE4TC03EVnpy6rrMCcCOu4iy/I2PMmrRbSQGTOyLCda9cwoGn7k1RWSEenweX24XH58EX9OELeDns3INY9vPyBkkdIBFL8NJdr6W16QruD8WXgas74AYpgsLTkYKT0PjXOEsPRhfvjC4ejrPsSDTxUzu9W2NMS3WM1StmnQULApw68XhOnXg8AL98s5D3/zcTj8/DzmN3ZObLszPe9IxF4vw47+eMbbpCh6DBsaDVqRuc4kKdSnT5UbXTFmvbi89OHevxOpJxv1RjTC5YYs8z62/ai0POOqDu8aZD+2VM7IECP5vvOKDJdkQEpHD1gcjzQJy6pA6k5qJXQ/QNCOzT+uCNMVlhQzF5brOhGzN4583xBVf3qN0eNwUlIfY8ZtcWt6OJHzPPitEYJDP3/DsrdVbhVN2Ns+IMnMpb0OTiXIdkzFqxHnseq1xRxRtT3mWTrTakoFuIz96aSzwaZ/iBwxh/zThCRcEWtyW+rdGaUGq1aIMnvODdKsuR544mf0WXjQWnGohA9A00fDeUPYh4265oqaoDsbfRyKsgRUhorO01a9aZJfY89dWsbzhvjytIJpJEwzGChQH6DurDjdOuIBDyr32D/t3B1RuSP5IakgHwg2cQeJu9Sd9p6KrrUnP2+W19QAw0hlZchJQ/1TbXVAddeVqqlo6GATcafgAtvgRX6LA2uabJbzYUk4dUlauP+ifhVTVEwzEAaqoifP/5Tzw+8dl1alPEi3R/FELjwLUeuHpBwXikbHKHKDSWNbG3WJ3U60nMQ9d1gVZzoq9CdHq9b0NJIAKrrkSdtHp6xjTLEnseWvTDEpb9vDzteCwS49X736p7vODrX/nkzTlUraxuUbviKsJV/Ddc672Da723cBWdiazjoqUOS5r6NiOAu00uqZEXgHD6E+KF2Pttck2T32woJg+53K4m67q43C5WLavk0oOvY/7H3+HxeYhH4xx+3sEce/nh+dX7XhfBI6D6LqD+ZiBe8O/RdlM6JUjqgyPDv1m+fXCadmE99jy0Xt9y1t+sF41ztD/oY8yJu3P1Uf/ky5nzidbEqK4IE4vEeXzis7z1+PTcBNyBSOFp4N8JCIAUpEopeAYgJRPa7prBQ1PXS38GfMPb7Lomf1liz1OXPHoOxd2LCRYG8PjcBAr8bDlyEKOPHMFnb88lEWu4EjVSHeWxG/+Xo2g7DhEfrtL/It2fQIonIKX3It2fRFwlzb94Xa/p2w4KTwb8qd67FIAUIKX/sYVfZp3YUEye2nBQHx768Q7ee2YGS39ezuY7DWCL4QNY8NUvuL3ujLVlVi2tzEGkuaPOKoi8klpk5R/ZYHqhePuDt3+7xeIqPB0NHgrRd8FVAP7dEGn5dFRj6rPEnsd8AR+jjhjZ4Nj6m/bC6/OkVVx3e91sv+827Rdcjmn0PXTlqaACJKDyH2joCKToopzdZxB3LwgdkpNrm/xiQzFdjNvj5i+3n4w/5KtLYF6/l6LSQv7wt7E5jq59qMbQlafXrqQNAzEgCjWPQezdNrmeU30/ztJDcZb9Aa15JrUgyZg2Yj32Lmi3w0fQa+P1eGzisyz6fgnb7rEVv//LfnTr0XbjyB1KrImbxFqD1jyB+HduUTOqcXCWgatbk9M+VZPo8mMhPhdIzYPXVXMg+i7S7fp1id6YZlli76IGbr8ZF085K9dh5IambyXYoufqcarvh6p/gqYKo2noSKTofEQa/UlFp0FiHr8l9dQ1aiDyEho/CfE2XYjNmHVlQzGm6/HtSGp1ZyMSQoIHNftyrXkBKv9RW8I4AkQh/AhaeWP6udH30uvrpJ6B+Iy1jbxhC04lGp+HOlWtasfkH0vspssRVwiKrwf8pPZhldQ0Q9+oFu3BqtW30aAHDkAEwg+hGmt42N2Dhnu9/haEp3Yzk+apU4FTfS9OxWVo+Ekcpxpn1ZXo4hHo8qNSm56sutrG7U0dG4oxXZIruA/qG4LWPAtaifh3A+92LZsRk1zY1BOgVSBldUck+Du06o4M53rAP7rZS2liPrrsyFR5ZCJo5BmovKp2CCgKWrtCNvwo6uqOFP5f8/GbvGc9dtNlibs3rsJTcBWdg/iGtXyao3fLzMddRSDdGl2jF1J6O0jp6pWsrj5I2f1Ik3VpQJ2VOFW3o8sOB11F3R60Gk59eDQoeQBQA9X3tCx+k/esx27MWpKi89Bl42i44XcACi9AJL2vJP6RsN57kPgC8IJn4Bo/RDS5BF12MDirSE3FbCG1SpAmJSs9dhG5W0QWi8jn2WjPmI5MvFsh3R8C3y6pcXLvEKT0X7hCBzf9GnGnXucd1Ow3A626HZyVrFVSh1RtfGPIXo99MnArcF+W2jOmQxPvYKTsrrZpPPoGkFjDCV5SNeMdUhUhBfAjxRe3TTym08lKj11V3wLSC4AbY9aeq2gNTwbBOwTKHgL/nuDeCPx7I90fSRUTWwNNLsSpuBBn8QicJXvjVD9oM2nyVLuNsYvIKcApABtuuGF7XdaYzid0PFRe2WjzcBe4+yEl14B3aGo4x3dbi5pTJ4yGJ0PVraz+JrAUKq9HE3ORkquyG79Jo5qA+KeAgnfr9IVsWdZus2JUdZKqDlPVYT169GivyxrT6UhwLAQPBXwghUAA3JtA8cXg3WqtipSpRtDlh0PVLaQP79RAzdNok9M3TTZobAa6eCS6Yjy64mR08XA02rZ7H9h0R2M6GBHBVXwJ0uMNKDoXXMXg/AIrz6hNCm8130gtDT8FiZ/IuNIWQHy1dWxMW1CnAl1xMuiKVHlorQKtQFf+EXXabvTaErsxHZWrBCpvBmdx7fz16lRSWHE6mvylZW1EXyd9lWw9mgR3n6yE25moxlBtvBagDUReJOOWh6pQ80KbXTZb0x2nAO8DA0VkgYiMz0a7xnRp0TfJPOUxiYafaFkb7nKa/jN3g3dglypEpslFOMvHo4u2RhcNxVk2Dk382HYXdCpqVw03FgWtaLPLZmUEX1WPykY7xph6nGWQcdZKHJwlLWpCQuPQmuchbWsVwLdLhysdnOpFexBxt0HbCXT5kbUlIWqHpuKzUqt7e7yGuAqyfk18O6WGu7TRtyYJpJ5rIzYUY0xH5duB1Fz1RiSUWs3aAuIdDMWX1e6lWnsj1tUHyp7GVTYJcXVrto1MVBWNvISz7Gicpb/DqfoP6lQ3fW7sE7TmOTT+deZzYp/gLD24tie9NU7FhaiTqSpmK0Sn1S78qn+/wQGN1A6ZtAHvkFRxufrbHEoQfCPBu23bXBMrKWAamT/7O36c+zMbbbEBm27dL9fhdGni2QQNHgg1z7N6nDwAngHg36PF7bhCh6DB/SD+eapejWfzVm//p5XXQM0jq3uiVd+gkf9B9ycb1MBRpwJdfgIkvgFxgSZR3w5I6e11G3Vr4id0xXH1yhvHoOZZNPkrUja5VXE2kPyhiWGRMJr4Fpzlqd68e6Os9d5FBLrdBJEX0JrHU8eCh0BgvzbdgtESuwGgpjrCRfv9na9mfYvLLThJZdD2mzLhuQsJFmTeHci0PSm+Gnwj0PCUVCXH4EFI6Ii1ngctEgTf9lmJSZO/QvghGo7/RyGxAGqehdChq8+tuBQSXwLx1fcQYx+gVbciRWenzgnfV1utsr4YxD5CE98ink2yEjeeQSDe9GtJKDUks/i+2mGTBFowHin8c1aSr4gLggcgwQNa3VZL2VBMFzfvw685f+8JHNrjRD5/Zx7RcJSaygjRcJQvpn/Nf//6QK5D7NJEBAkegKv7g7jKH8dVcOwaq0Jmk2qG2RwAsY9SCTJNDRp5rd7r4xCdCjRO2lEIP7r6YeKrDOeQqlmf+GEto14D33Bw96NhfXwPIBCfA8RqK2dGoPputKaFN6g7IEvsXdic977k3N0v56NXPyUWiaf9IcejcabeNy03wZmc0cR8nGVHo4s2x1k4BKfi0obj3a7uTW8hmKg/Jz5Jxql+QIOyw96hZNyMROPg6b92wa+BiAspewCCh4OUgBRB4MDaHnzjIZoaqP5v1q7d3iyxd2GTzruPaHjNFQTj0Qw9KZO3NLkUXXZE7bZ9DhCBmqfQFaesPsm3Q9MNOEvRxAKA1Abfns0znOQC3251jyQ0LjVLhPrDHgEI7IF4NmjFu0knrkJcJZfi6jkDV89ZSPH5TZ/sLMvqtduTJfYu7JtPmv+au/FWG7VDJKajqBvLb9DTjkL8UzQ+D6gdM3Y3Ue9J/OD8uvphyVWpG7Z1PfIASDek+K+rz3Gvh3R/HPyjgGDqG0HByUjJDVl8Z02QUnCVZnoCmimq1pHZzdMurHvvUn75Zs11Qn6cu4AlC5bRY4OW7c9pOrnEXDIuihIPJL4Fb23Nd/8ICH9H2ti4xhoMn4h3Cyh/GQ0/khpL9w5FQocirpJGzfdDSv+T3ffSAiICxZehK89m9Vx/F0gQKTqv3ePJFuuxd2HjLj4Ef2jNN+JUlVfundY+AZnc825JapPvRjQBns3qHkrBibVzs+unkCCEjkmfG+8sTJVFEE9qaEUK2yLydSaBPZGye1Pzzd2bQPBgpPvTSL3329lYj70L2+vY3Vi1rJL7r3yMWE2MRDz9hlg8mmDFwhU5iM7kggSPRKvvqZ3v/dtwjB982zQoPSDuXtD9KbTqJoi9n9rrteBEJHhog/ac6nug8iZS3wIcNPoGeLeD0kltsrp0XYlvG6RsUq7DyBrrsXdhIsKhZx/IE0vu5pqXLsYXTJ+ZECwMsO1eW+cgOpML4u6OdH8stTIST2p8PHR4xmES8fTF1W0irvXex9XjRVyhwxrM+9bkMqicSGqIo3YFrYYhPguir6W1Z7LHErvB4/UwdPRgRh85kkDB6q/h/pCPTYf2Y8f9227ps+l4xLMxrrK7cfX6AlfPj1MlhGUdFqnFpqfG5hvTMBp5ufWBmibZUIypc86dp7LdXlvz/KSpxKNx9hi3K3seuysrFlVQUl6E15dpUYoxTZBQ0885bVfZ0FhiN/WICKOPHMnoI0eiqjw+8VmO2uCPJONJxO3ikLP259jLDsflsi96pgX8I1M3XTOJfYRqos23iOuq7C/UZPTiXa9x72WPEl5VQ7QmRqQqwhM3Pscj1z2d69BMJyHiA9d6TTwbQ+NftWs8XYkldpPRg1c9QTTccIeZSDjKIzc803QNEWMacze1v3EMwve0ayhdiSV2k9GKRSszHg9X1JCIN/H12phGJHQ0TY74Rl5Ck4tbfQ1NLMCpvBVn1d/R6LvW8cASu2nCRlv2zXi850Y97CaqabnA/qkSAZmIFxJftKp5p+ZldOl+UH0HhCejK/+ErjwVbapIWRdhid1k9H//OA5/qOG8dn/Ixx9vPDZHEZnOSERqa8BkWIykSXD1rvdwKU7lzTjLjsWpuBJNfL/GtlVrYNVfSc2Try1toOHUNMvIS1l6B52TJXaT0dajtuTaly9hq103J1gYIFgUpLC0kFtOv5NDe47nqiNv4tdvF+U6TNMJSMFxQONveR7w9Ee8A4HaXZSW7psqlRufDjUPo0sPRmMzm244NpPMHxjh1G5OXZgldtOkLUcMpLisEFWlprKGZT8vZ/mvK6lYsoq3H3+fU4f9lSULOm9pU9M+xLMZUnpb7QyZAOAF305I2ep651p5A2glqwuQJYAatOLiNbS8pqmSGeq7dyGW2E2Tvnj/K2ZN/ZRIdTTtOcdRotVRHruxa/eMTMuIfxekx9tIjxeR9d7FVXY3Ur9cbuxdMm7cnfwRdSozN+obRsYeuwSR0GHZCLvTstUBpkmfTJtDtKbpjTgS8SSfvTW3yec7k1+/XcTLk9+gYmklO+y7DTvstw1ud8cpUpUPRATcfZp4srC2x572RGof0ozteaH0DnTFyakDWrtjU/Bw8O2SlZg7q6wkdhEZA9xM6uPzTlW9NhvtmtwqKS/C5XKRdJqeYeD2dP4vfe889QHXHv0vkokkiXiSVx94k0Hb9+ealy7C47W+T7sIHQ1Vt7C6JjqADwJ7rXGPV/ENgx7vpIqKaSX4RiKefm0dbYfX6r9KSdXevA3YF9gCOEpEtmhtuyb3dj1seLNzgr/55HuqK6rbKaLsi0ViXH/8rUTrlS2OVEWZ9+HXvPrA2zmOruuQghMhMAbw19ZrD6Q25Sie0PxrXQVI8CAkNM6Seq1sdLd2AOar6reqGgMeBg7OQrsmx4pKCxl95Mg1nuMklU+mtW4uci7Nnf51g1Kzv4lUR5l6/zQWfr+YmupIhleabBJx4+p2PdLjFaTbRKT8SVzdH0BcHWtTjs4iG4m9D/BTvccLao+ZPDD+mnEEC5su2eokHSqWrmrHiLLL6/c0+a1kzjvzOHmrszm0x3j+dfqdtuK2HYi7N+If1al3L+oIspHY07s7DXfCTZ0kcoqIzBSRmUuWLMnCZU176LFBd/7xxuX4M2zC8Zvi8qJ2jCi7Bu6wGYGCzB9cyYRDpDpKLBLjlclv8N+/3t/O0RmzbrKR2BcA9defbwD80vgkVZ2kqsNUdViPHk0VBjId0YDtNmXsmfvjcqf/unh8HjYZslEOosoOt9vNVc9eQGG3AoJFwdQesBm6KtFwjOf/+yrxWDz9SWM6mGwk9hlAfxHZWER8wJGATW7OMwefvi/BwkCD8Whf0MeIg7en98Y9cxhZ6w3YblMe+WUS5919GqfddDyhomDG85ykQ3hVTTtHZ8zaa3ViV9UEcDrwMjAXeFRV57S2XdOxdO9dyi3T/872+w7FF/RRUl7EYeccyIUP/DnXoTXrh7kLuPnUSZy/95VMueZJVi1Pny/tC/jY5ZCd2O/kPRm0Y/+M7RSUFFBUZjfzTMcnuShxOWzYMJ05cw01IIzJkhkvz+aKQ24gHk3gJB18QR+FJSHu+Oh6ynqVZnzN1x99y9m7Xkq0Jspvfx7+kJ+zJv2RPf7QtRe+mNwSkVmqOqy58zr/6hLTIU1/bhYnDT6L/YJHcfyAM5j2yLvtHoPjONw4/g6i4RhOMrVcPVYTo2JZJQ9MeLzJ1/XfdhP++e5V7HjAMLqvX8oWIwZw2RPnWlI3nYYldpN1Hzw/i6uOmMgPXywgHk3w8/yF/GP87bz6wJvtGseSn5ZRtaIq7XgynmT6s7PW+NpNt+7HhGfO57qpl9KrX0/uOGsyEw6/kfmzv2urcI3JGkvsJuv+e/4DaTVmouEYd17wYLvGESwM4DgZCksBoZLMN0jr+3LGfE7f4QKmPfwOP837mbef/IAzd76ET6bZLSTTsVliN1n38/yFGY8v/3Vlu04XLO5exJBdt8DtbVjMyx/yM/YvBzT7+jvOmkykOorjpAba1VGi4Si3nHFnm8Tb1ahTjcY+RhM/5jqUvGOJ3TQwf/Z3XLDPBH5XdhwnDPoLr9w7ba33kFxvw/KMx4vLi9q9qNaFD/6FTYZshD/kJ1QcxBvwsvfxo9h3/O7NvvbLmd9kPP7jFwtIJrr21mut5VRPRhcPR1eMR5fuj7PsCNRZnuuw8oaVrjN1vp/zE2ftckld/fXqlWFu+dOdLPtlOUddOLbF7Zww4Uj+Mf52ouHVwzH+kJ9jLj00Y12WtlRSXsztM67jm0++Z8lPy9hs240pX7+sRa8tKi1gxaKKtOOBgkDGxVqmZTT6NlTeBERWr1GPf4au+BPSfUouQ8sb9ttp6tx/xaNpY+ORcJSH/v4k0Zr0zTaaMuqIkZxx20l0712KiFBSXsTJ143joNPGZDvkFtt0637sdMB2LU7qAIecdUBqJWo9/qCPg07bp90/oPKJVt8NNF7olYD452hiQS5CyjvWYzd15n04H3XSh13EJSz+cSl9Bzas7Va1spoPnv+IZCLJ9vtuQ+l6JXXP7XPcaPY5bjTxWBw9jHMiAAAdVElEQVSP19MpE+Fh5x7E0p+X88J/X8Xj85CIJRh91M4cP+HIXIfWuTlN1IoSL+gKUlVJTGtYYjd11t+sF4t/XJp2PBlPUtarW4Njv21O4XK7UMBJJDnt5hPY/+S9Gpzn9TXexLjzcLlc/OnmEznuiiP45ZuF9Oq3HsXdO2/Bsw7Dvxskvmf1/qa/ccAzIAcB5R8bijF1jr7kUPyhhlUc/UEfexy9KwUlBXXHKpau4tqj/0W0JkZNVYRIVYRYJM7tZ05mwde/tnfYba6wWwEDttvUknqWSMF4cJXQcMPpIBResMbdkkzLWWI3dbbebUvOv/cMuvcuxePz4Av6GDN+d864dXyD8959egbiSh9aSSaSvPHwO+0VrumkxFWGlD8LBePBswX4RiOlk3AV2BBXtthQjGlgl0N2YsD2m/LZ23PpO3B9Bg5L3/AgHo3Xze2uz0k4xNaw+fWaxGNxpj87i8U/LmXQjv3ZYviATjkubxpSZxUkfwB3H8S1+sa1uMqQorOg6KwcRpe/LLGbOo7jMPHkf/PGlHfw+Dw4SYeNtujL31/8G8Vlq4chdthvGyadd1/a631BLyMO3mGtr/vz/F85a5dLiYSjJKJx3F43g3bsz9XP/w2fv/OO0Xdlqg5aeS2Ep9TeFI2hgTFIyd9JVfc2bcmGYkydZ257iWmPvEcsEie8qoZIdZRvZn/HDcff1uC83hv35A8XHYI/5ENcgogQKPCz17GjGLTDZnzwwkdce8y/uOGE21q0/P7qo/7JyiUV1FTWEI8liFRH+eL9r3h84rNt9VZNG9PwvRB+BIiCVgExiLyCVt6Q69C6BCvba+ocN+AMfslQDsDj8/D44rsoKA41OP71R9/y2oNvk4gnGHX4CLYYMZDrjruV957+kEh1FJHUwqQDT92HU64/JuM1Vyxaybh+pxKPpu8nuv5mvbj3q1uy8+ZMu3IW7wLOogzPBJCesxGxPuW6aGnZXhuKMXVqKjPvDiQiRMPRtMTef9tN6L/tJnWPP39nLu88MZ1YJFUPRhUi1VGeue0l9jtpDzYYsH5a28lEEpoYS7dl+52Yk75iNyVW+9P0Bumm9exj09TZYb9tcXvSfyW69y6ltGfDeezxWJxPps3hkzfnkIinetvP3PZSXVKvz0k6vP+/GVStrE57rrxPd3r1S98D1+v3svtRO6/rWzG55hua+bh7I0Qsqbc1G4oxdZb+vIxTtzuf8KowsUgct8eN1+9hwv8uYOjowXXnzZr6CRMOn1hXHMzldnHZ4+cy4YiJrFqavu1c6hzB5XKx0ZZ9+evk0xtsgD1/9necO/pyEvEE0XCMYGGA3pv05Ka3JzS5/2hXoclf0Or7ITEPvEOQ0DjEvV522tYkJL8HKcpam3Vtx+eiy48CjQJJUjuE+5HSfyP+EVm9VlfS0qEYS+ydWCKe4I2H3+WNh98lWOBn/1P2Yts9h7SqzVXLK3nuP1P59M0v2GBAb353xn5s0L933fMrl1RwzMZ/IhJuWDsmUOCvKx7WnIKSEPfNv7XBgp/KFVW8/tA7LPx+MVsMH8iIg4bh9rjX0Er+0/gX6PJxoDEgDvhA/Ej3RxHPpg3PVV2r6aEaeR2tuJDUzc0EeLdGut2MuDNX5lyn+BPfodX/hthn4NkUKTwV8W6Rtfa7IkvseS6ZTHLB3lcx78Ov6xJqoMDP7/+8Hyde/Yc2u+7Tt77If89/IG2+ur/Aj5NIZrwJ2pg/6OP4q47k0LMObKsw84Kz7DCIf9LoqIBvJK6yuwHQ+Dx01RUQ/xgkAMFDkaLz1riCU+NfossOAyL1jnrAsxnS/RlbP9CB2c3TPPPTlz/z73Pu5dNpXxAqDrL16MHM+3B+g15ypDrKw9c9TWFZIb87fd82mQNetaKaeDR9HD0RTbD5Tv2Z+8HXJONrvukZrYmx4Mv8Kz2QTapJiH+a6RmIfZj6r+SvtcMdtfcuNAzhR9Dkj0jppKbbDt9Hep2WRGohUWIu1OtVq7MKrXkS4p+BZwASOqzBQiPTMdnN005g6c/LOGOnvzHjxdlEwlGWL1zJm4++R6Q6knauOsrki6Zw5s4XE4us2yrQNdl2z63wB9MXmHh8bpYsWJpWHTJT5y9QGGDznfpnPbb84qJhLZV6JHXfQavvqx2mqS8K0ffRxA9NN538Gci0ZaAbnMV1jzT5C7pkb6icCJFnoeo2dMleaGL+2rwRkwOW2DuBJ29+gWhNrMFORk4y816eAPFYgh/n/szU+7K/efTmOw1gx/23JVCw+qt+oMDPRlv2ZeXiygxxCb7A6m8OHq+bbuXFjDrCbqCtiYhA8PdA4yEVP4SOSP1n4gtSY++NX+yFxLdNN+4bnqFdUh8Sni1XP1z1d9CVrB6yiYBWoRUXt/h9mNxoVWIXkcNEZI6IOCLS7LiPWTfzPviaRKz5sev6ouEobz3+ftZjERH+9tCZnHv3n9hh323Ycf9t+evk0ykuKyQaTr95Gij0M/zg7SnvU0ZJeTH7nLA7t354Df6gVfFrjhRfCL7tgQBIEeAH/y5I4Z9TJ3gHAxmG2zQOjW6uNmg3dBS4Shu9NgihcYi73tTT6Fuk9+wV4rNRbb+9a83aa+0Y++fAWOA/WYjFNKHf4L588f5XaQt23F43LpdkvGEpAkVtVGbW5XKx22HD2e2w4XXHPnzhI8Ql6Rt1KIz9835cPMWKPTVHnRUQ+yA11OIbjkgQKbsbTXybql/u2QzxbFh3voSOQcNTUom8jg/8Ixuc15i4iqH8abTqPxB9DaQYKTgeAo1uZosPNH24D9zYl/2OrVX/Oqo6V1W/zFYwJrOxZx6A19/wM9gX8LLtnkN4fPFdlJSnJ3Bf0M9Bp+7TXiFy4Gn7NBhygdTOS93WK2HznWzzhOY41feji3dFKy5EV56FLh6BxmYDIJ5NkMDudclaNYkmvgM8UDoFXPV3HPKAf+9mryeuMlzFF+Lq8Squ8ieR4EHps2GCY0kfsvFCYB9EuvZU1I7OPnY7gQ369+a6Vy5h4yEb4nK78Pq97DFuVy597BxCRSFufPNKuvcpI1gUIFQcxBfwcsxlhzFk1/abMzxgu0358+0nEyhMxRAo8LPBgPW57pVLbPpcMzQ+BypvIDWnvDpVNEtXoStOQhvdHNXIq+jikeiy36NLRsHKUxttNReGVZejkamtjkuKzgbvUCAIEkr9eAYgxZe3um3Ttpqdxy4irwK9Mjx1kao+U3vONOBcVW1ycrqInAKcArDhhhtu98MPa7hrb5oUrYni8aX2EP3561/xh/ys17ecZDLJnHe/pGpFNYN3HtRg8c+XM+bz3wseYP5H31Hep4xjLju8wTBKtuP7eta3FJSE6Dd4Q0vqLeBUXA41D5M2ni2FSMmNSGA0ULuac9kRNJx/3gTPAFzlz2UlPo3PgcRX4N44tZDJ/k1zJmvz2FV1z2wEpKqTgEmQWqCUjTa7In/Qz8xXPuH6426hpiqCk3TYeKsNufSxczL20L+a9Q3njL687sZmdUWYG064jYolFRx02pg2iW/wzptnvd28plVknn6oq+eoAxqeTPr88yYkF2QhsBTxbgneLZs/0XQYNhTTyfzyzUIuH3sDKxZVEKmOEovE+fqj7zh3jytwnPTkMOm8+9Nmq0TDUe6+eArJRJLqimqmPzeLj1//zKop5ogE9k4NczSmCahfVyXR1PzzDNxNz4ox+a9Vs2JE5PfALUAP4HkRma2q7XfHrgt67j+vkIw3nAXjJB1WLq7g83fmNei1JxNJPnv7i4ztJKIJnvzX80y+5BE83tSNMK/Pw9UvXMTAYWufFJb9uoI3H3mPcGUN248ZysDt07fUW9t6Jl2Gfw/wbgfxWanVo78tTir6S8NVnv4RtSUGmqvJE0CKzmm7eE2H16rErqpPAU9lKRbTAot+WEqiiSX7y35Z0eDxhy9+TFO3UBKJJJMvfphYJE6sXhn2C8dcxSO/TMLra3k5gveemcHf//BPVJVELMHD1z3NqMOHc85dpyEiTL3/Te6+6CGWLlhO+QZlnHDVUex97KgWt5/vRNxQOgmir6KRF1PVFoOHIr6tG54X+gMafhCcFUAT6xqkHOl2g1VQ7OJsKKaT2Wb3rRqs+vxNMp5k0I4Ne8k/fNH0OGtxWVHGD4hkPMmsVzLVKMmspjrCNUffTLQmRiyS2uQ6Go7y5mPvM+Ol2bz6wJvcfOokli5YDsDSBcv512l3MvX+7K+K7cxE3EhgH1zd/omrZEJaUgcQVzek+zMQOqp26Kb+tx8XSAlS/j/EP7Ld4jYdkyX2TmbPY3alrHdpg3nt/pCfPcbtSu+NezY4t+/A9TN+CLi9btbfrGfGsgSqSnVFuMXxzH79c1zu9F+jSHWUqfe/yT0XP0w03PCGXzQc5Z6LH27xNcxq4i7HVXwJst4sKDwfXOuDFIN/DNL9yTWW3VVnJRp+GK2+E41nHqIz+cGqO3YygZCf2z68lsf+8T/eevx9AoUBDv7Tvux93G5p5+50wHZ061FCrCZed2NUXEJpz24cfPq+fDP7+7Qa6ol4kqG7D05rqylrGjMXEZYsWJbxuaVNHDctI+JGCk+EwhNbdL5G30NXnlr7IA78Cw0egBRfbfc98pD12Duhwm4FnHDVUdwz71/cMfN6xpwwGpcr/Z/S7XFz83tXM/L3O+DxunF73Aw/cBi3vH81ux66E4N26F/Xo/9t4+lxF42le+/SFscydPfB6WUESBUG2/u4UfTcMHMPskff7rzx8LvcesadPH7Ts1QsXdXia5q1oxpDV54BWpP6IQFEIPI8RF/PdXimDdhGG13Eb//O9XtnyUSSNx99j2mPvkewMMh+J+/B1rut/XzlD174iAmH3wikevwej5u9jhvFn287iWmPvMeNJ93eYDjGH/RRWFpAeFUNNVUR/EEfbo+bG16/jAHb2TS9bNPou7WJvSr9Sf9euEpva/+gzDqxHZRMu1q5pIK3H59OuDLC9mOGNtjTdNoj73L3RVNY9MMSem5UzoZb9OWjqZ+kFS/rO2h97v7i5vYOPe+tObHvgav0jvYPyqwT20HJtKtuPUo4sImiY6OOGMmoI1bP1BjX79SMFSkXfr+Epb8sp3z9/NuhR1Uh8gxafSc4y8G3E1J45hqrMGaNbxiQqQMXQoK/a/vrm3ZnY+ym3bm9TVQGdLRusVS+0ap/oRWXpWquOEsh8kKqkFfylza/togf6XYTECBVrVGAIARGg3+vNr++aX+W2E2722/8Hmnb67lcwqbbbEy3HiU5iqrtqFMF1XcC9VaC4YBG0Ko72yUG8Y9CeryWWpFacDpSNhkpmYiIpYB8ZEMxHUBNVQ1TrnmK1x58G3EJex8/miPOOyhvdxk65OwDmD1tDnPenYeTdHB7PRQUB7loypk5i0k1Ne1TpA3+nye+SW1Xp41LAcQhPiP712uCuHtAwfHY5Mb8Z4k9x5LJJGfvdhk/fLGAeDS1E84j1z7FR1M/4aa3JuTlHGOvz8u1L13MlzPmM+/D+ay3YTnbjxmKx9v+v46a/AWtuBBiH6Ye+7ZHiv+OeDZo5pVrwd2z0S5HvxFw983edYypZYk9xz584WN+/vrXuqQOEIvE+eaTH/j0zS/YelT+lksduP1mGYuFtRfVGLrs8NSY929VE2MfosuPgB6vZ633Lu5eqH8ERN+lYdldP1JwSlauYUx9NsCWY/M++JqaqvSNE+KROPM+nJ+DiLqQyNTaeuf1Sys4qWORl7N6KSm5CQJ7Aj7AD65yKPkH4hua1esYA9Zjz7meG/UgUOBPW9rvOA7zZnzNquWVFJe1zabUXV7yx9qVmI1oGJLZ3eFLXCGk2z9TN1K1Elw97calaTP2m5Vjo44cWbvVXcPj6ijTn5vFH4eey6pllbkJLt95BoEE049LAXjaZhcocRUi7t6W1E2bst+uHIlFYrz79Ie898wMLnv8XDbaMv0mWiKaoGLJKp78Z3b2rjSN+HcF9/pA/drzXnD1BP+oHAVlTOvZUEwOzHnvSy7a/++oo6nNKeJJ9h2/O4u+X5I23h6PJpj+3EccP+GoHEWbv0TcUDYFrZyYKoiFQmB/pOhsROxPw3ReXe6313Ecvnj/KyLVUbYcMYBgYYav4m0oFo1z8QHXpNU8f+nu13GaqNtT2iv/Fu10FOIqRkouh5LLcx2KMVnTpRL7d5/9wIX7Xk24sgYRIZlIcvot4xlzwu7tFsPs1z/PuOl0PJqguLyIqhVVJBOrn/eH/Iw984B2i88Y0/l1mTH2ZCLJX/eawLJfVlBTGSG8qoZoOMatp9/FN598325xRMPRjPuQqipbjhjIpkM3xh/yUVASxB/0cfyVR7D9PjYlriNTVTQ+F43NQDV96qox7a3L9Ng/fv1zYjWxtOPxaJwX/vsqZ9x6UrvEsfXoLUnG0ysbBgr87HnMbuwydkd++vJnVi5exaZD+xEqat+hoq5CVSE+O1WUy90PfDus0ypfTXyPrjgZnMWAG3DQoitxhQ7KdsjGtFinSeyO4/Dxa5/xxXtfUda7G6OOGEFBSUGLX1+9sjpj4VLHUVYuab/de4rLijj1puP599n3Eo8lcJIOgcIAQ3bdghEHp8os9x3Yh74D+7RbTF2NOmF0xYmQmAuqIC5w94GyBxBXy3ePUnXQ5ceBs5AGZXFXXYx6+yPetpkyaUxzOkVij0XjXLD3BL7++DsiVRECIT+T/no/N7zW8h13ttp1cxJN9JRHHrx9tkNeowP+uDdbDB/IS/e8TnhVDSN/twM77r9txu3tTPZp1USIf07d8n4FEt+hFZcipbe0vKH4TNBVpNc6j6Hhh5GSK7ITsDFrqVWZRERuEJF5IvKpiDwlIt2yFVh9T9/yIl/N/IZI7VTASDhKeFUNEw6fSEt3gCrrVcofLvw9gdDq+h+BAj8bb7URux42vC3CXqNNhmzEaTedwLl3ncbwA4dZUm9PNU/TsGYLQAKir6GabHk7zoqmnqgdmjEmN1rbY58KXKiqCRG5DrgQOL/1YTW6yH3TiGYYH1+xqIKf5y9kg/69W9TO0ZccxuCdN+e5f79C1cpqRh0xkj2O3iUnVQVNLqV/c0txgCSpsfIW8G4Hmv57CUHE334zrYxprFUZTVVfqfdwOnBo68LJrOmbWpq2FL85Q0cPZujowa2OyXRivlEQfZlUEv+NgHdbRHxNvKghdarRyutoWEAMIACeDSFoN09N7mTz+/+JwItZbK/OmBN3xx9K/4Prvn4Z62/aqy0uafKYFF8ArjLgtxlHAZBipOSqFrehK8+AyIukfTiEjkW6P9o2G3YY00LN9thF5FUgU/a8SFWfqT3nIlLfbx9cQzunAKcAbLjh2m3ge+CpezP9+VnMff8r4tE4voAPt8fNpY+dk5cbUZi2Je5eUP4KWvMsJD4DT38k+HvE1bIVvpr4sXZjjgzDMLoCyVRYzJh2JC29+dhkAyLHAf8H7KGq4ebOBxg2bJjOnDlzra6jqnz+zjzmvDuP7uuXsfMhOxIsCKxDxMa0jkbfQ1eMp2FvvZa7P64ez7d7TKZrEJFZqjqsufNaNcYuImNI3SzdraVJvRXXYqtdNmerXWxusMktlRIyJnWg4YbVxuRGa8fYbwWKgKkiMltE/p2FmIzp0ESUpvtE3iaOG9N+WjsrJncbVhqTK+5NSCXwxtMmPeDfJQcBGdOQrYoxZi2JKwRFf2m0+5IHpBApODlncRnzG1uZY8w6cBWciLr7odX/BWcJ+EYihaci7p65Ds0YS+wmP6hqs1Nf1VkO0TcBAf8oxNW6ChgS2B0J2ApT0/FYYjedmkZeQyuvheQPqKscCk5DQuPSkrwTfhJWXcbqcgFJtOQaXEHbxMTkHxtjN52WRt9CV54FyR9SB5ylUHkDGr6n4XnJX2qTehQI1/5EoeJCNGnFukz+scRuOi2tnAg03rGoBqpua1ilMfIi6TVdAAQiL7ddgMbkiCV203klf8x8XCOgVfUex8ic2JNkLAtgTCdnid10Xu5+mY9LEKRw9WP/aDIvHHLVPmdMfrHEbjo8dSpxKi7FWbQdzqJtcFaehyaXIUVnA43rBQWh8HREVtdUF+8gCB2Veg6p/QlAwYmIZ5N2ex/GtBebFWM6tNS+ouMg8S11wyaR59H4LKT8JaTbzXWzYnCVQ8GfkNCRae24ii9EA/uikecAFxI8APEOadf3Ykx7scRuOrbYe5D8iYZj4QlwlkPklVSCDrRsOEV8QxHf0DYJ05iOxIZiTMeW+DLz9nMaRuNftH88xnQClthNx+beGDJuVxdCPJu2ezjGdAaW2E2Hpp5BoI2rKApIAAL75iQmYzo6S+ymw1JNwIqjgXijZ9xQOjlVZdEYk8YSu+m4om+Ds4L0xUVeJPFxLiIyplOwxG46ruRPoI176wA1aOK7dg/HmM7CErvpuLyDQDLMyJUQ4h3c/vEY00lYYjcdhqo2LN7l3R48/YH6s2I84CqDwJj2Ds+YTsMSu8k5dSpTZQIWDUYXbYmzbBya+BYRQUrvhdDRIGUgJRAci3R/HMk4BdIYAyCq2u4XHTZsmM6cObPdr2s6HlVFlx2SWohUN/tFQIqQHlMRV2kuwzOmQxGRWao6rLnzrMducis+G5Lf0nBKo4LG0PDjuYrKmE7NErvJrWRTs1sikJjXrqEYky9aldhFZIKIfCois0XkFRFZP1uBmS7C0x/INBwYBO9W7R2NMXmhtT32G1R1iKoOBZ4DLs1CTKYr8QwGz5Y0nPniAgkgwbG5isqYTq1ViV1VV9V7WEDmrpcxTUrNfLkTgkfU7nrkA/9opPsTiKs41+EZ0ym1uh67iFwNHAtUALbPmFlr4gohJZdAySW5DsWYvNBsj11EXhWRzzP8HAygqhepal/gQeD0NbRziojMFJGZS5Ysyd47MMYY00DW5rGLyEbA86ra7Fpvm8dujDFrr13msYtI/3oPDwJsfpoxxuRYa8fYrxWRgaTqqv4A/F/rQzLGGNMarUrsqnpItgIxxhiTHbby1Bhj8kxOioCJyBJSQzcdXTmwNNdBtAN7n/mlq7xP6Drv9bf3uZGq9mju5Jwk9s5CRGa25A50Z2fvM790lfcJXee9ru37tKEYY4zJM5bYjTEmz1hiX7NJuQ6gndj7zC9d5X1C13mva/U+bYzdGGPyjPXYjTEmz1hib4aI3CAi82o3FHlKRLrlOqa2ICKHicgcEXFEJO9mGYjIGBH5UkTmi8gFuY6nLYjI3SKyWEQ+z3UsbUlE+orIGyIyt/Z39i+5jqktiEhARD4UkU9q3+cVLX2tJfbmTQUGq+oQ4CvgwhzH01Y+B8YCb+U6kGwTETdwG7AvsAVwlIhskduo2sRkYEyug2gHCeAcVd0c2An4U57+e0aB3VV1a2AoMEZEdmrJCy2xN0NVX1HVRO3D6cAGuYynrajqXFX9MtdxtJEdgPmq+q2qxoCHgYNzHFPWqepbwPJcx9HWVPVXVf2o9r8rgblAn9xGlX2aUlX70Fv706KbopbY186JwIu5DsKstT7AT/UeLyAPE0FXJCL9gG2AD3IbSdsQEbeIzAYWA1NVtUXvs9U7KOUDEXkV6JXhqYtU9Znacy4i9RXwwfaMLZta8j7zlGQ4ZtPBOjkRKQSeAM5stE1n3lDVJDC09t7eUyIyWFWbvYdiiR1Q1T3X9LyIHAccAOyhnXh+aHPvM48tAPrWe7wB8EuOYjFZICJeUkn9QVV9MtfxtDVVXSki00jdQ2k2sdtQTDNEZAxwPnCQqoZzHY9ZJzOA/iKysYj4gCOB/+U4JrOORESAu4C5qjox1/G0FRHp8dssPBEJAnvSws2MLLE371agCJgqIrNF5N+5DqgtiMjvRWQBMBx4XkReznVM2VJ78/t04GVSN9oeVdU5uY0q+0RkCvA+MFBEFojI+FzH1EZGAscAu9f+Tc4Wkf1yHVQb6A28ISKfkuqcTFXV51ryQlt5aowxecZ67MYYk2cssRtjTJ6xxG6MMXnGErsxxuQZS+zGGJNnLLEbY0yescRujDF5xhK7Mcbkmf8HY0BWOdasMhYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x364a450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nb_features=2\n",
    "model = Perceptron(nb_features=nb_features)\n",
    "x, y = generate_data_perceptron(nb_features=nb_features, seperation=1)\n",
    "# plot and satisfy your self that data is not linearly seperable\n",
    "x1 = []\n",
    "x2 = []\n",
    "for i in x:\n",
    "    x1.append(i[1])\n",
    "    x2.append(i[2])\n",
    "plt.scatter(x1, x2, c=y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Algorithm didnot converge\n",
      "Accuracy on training data is 92.0\n",
      "92\n"
     ]
    }
   ],
   "source": [
    "converged = model.train(x, y)\n",
    "if (converged):\n",
    "    print ('Algorithm has converged')\n",
    "else:\n",
    "    print ('Algorithm didnot converge')\n",
    "    \n",
    "y_hat = model.predict(x) \n",
    "correct = 0 \n",
    "for i, y_real in enumerate(y):\n",
    "    if (y_hat[i]==y_real):\n",
    "        correct = correct + 1\n",
    "    \n",
    "print (\"Accuracy on training data is {}\".format(correct*100/len(y)))\n",
    "print (correct)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XeYVOX5//H3PX0bHQQpggSRoqIi9hZRsCSIvUUTTVCjoiYae1Riz0+JLSr2LuRr74rGFisKKqIiKChKh91l2+yU+/fHGZYtM7uzuzM7szP367r2gpk5c85zttznzOc8z3NEVTHGGJP7XJlugDHGmI5hBd8YY/KEFXxjjMkTVvCNMSZPWME3xpg8YQXfGGPyhBV8Y4zJE1bwjTEmT1jBN8aYPOHJdAPq69Wrlw4ePDjTzTDGmE7l008/XaOqvVtaLqsK/uDBg5kzZ06mm2GMMZ2KiCxNZrmURDoicp+IrBKR+fWeu0JEfhaRebGvg1KxLWOMMW2Tqgz/AWBinOenq+qY2NdLKdqWMcaYNkhJwVfVd4B1qViXMcaY9Eh3L50zReSLWOTTPd4CIjJFROaIyJzVq1enuTnGGJO/0lnw7wCGAmOA5cCN8RZS1RmqOlZVx/bu3eJFZmOMMW2UtoKvqitVNaKqUeBuYFy6tmWMMaZlaSv4ItKv3sPJwPxEyxpjjEm/lPTDF5HHgX2AXiKyDLgc2EdExgAKLAFOTcW2jDGmNT5YvJZvVpTzh92HZLopGZeSgq+qx8Z5+t5UrNsYY9oiElX+/d9FTJ+9kC17F3PsuEEEvO5MNyujsmqkrTHGpMLqDUHOnTmP9xat4dAxm3PV5G3yvtiDFXxjTI75YPFapj4xl/LqENcfvg1HjR2IiGS6WVnBCr4xJifUj3AG9yzioZPHMaJfl0w3K6tYwTfGdHrxIpxiv5W3xuw7Yozp1CzCSZ4VfGNMp2QRTutZwTfGdDoW4bSNfYeMMZ2KRThtZwXfGNMpWITTflbwjTFZzyKc1LDvmDEmq1mEkzpW8I0xWckinNSzgm+MyToW4aSHfQeNMVnFIpz0sYJvjMkKDSKcXhbhpIMVfGNMxjWOcK6evA1FFuGknH1HjTEZZRFOx7GCb4zJCItwOp4VfGNMh7MIJzPsO2yM6VAW4WSOFXxjTIewCCfzrOAbY9LOIpzskJLvuIjcBxwCrFLV0bHnegAzgcHAEuAoVV2fiu0ZYzoPi3CyhytF63kAmNjouQuBN1R1GPBG7LExJk9Eosqtb3zH8fd8SEnAwzNn7M7ROw2yYp9BKTnDV9V3RGRwo6cnAfvE/v8g8BZwQSq2Z4zJbhbhZKd0/gQ2U9XlAKq6XET6xFtIRKYAUwAGDRqUxuYYYzqCRTjZK+OHXFWdAcwAGDt2rGa4OcaYNrJeONkvnQV/pYj0i53d9wNWpXFbxpgMsginc0jnT+Q54CTguti/z6ZxW8aYDLEIp/NIVbfMx3Eu0PYSkWXA5TiFfpaInAL8CByZim0ZY7JD4wjn4VPGsXVfi3CyWap66Ryb4KX9UrF+Y0x2sQinc7KfkDGmVSzC6bys4BtjkmIRTudnBd8Y0yKLcHKD/cSMMc2yCCd3WME3xsRlEU7usYJvjGnCIpzcZD9BY0wDFuHkLiv4xhjAiXBu/+8i/mURTs6ygm+MsQgnT9hP1Jg8ZxFO/rCCb0yesggn/1jBNyYPWYSTn+wnbEyesQgnf1nBNyZPWIRjrOAbkwcswjFgBd+YnPf+4jWc/cQ8i3CMFXxjcpVFOKYxK/jG5CCLcEw89htgTI6xCMckYgXfmBxhEY5piRV8Y3KARTgmGfYbYUwnZxGOSVbaC76ILAE2ABEgrKpj071NY/KBRTimtTrqDH9fVV3TQdsyJudZhGPawn5DjOlkLMIxbdURBV+B10REgbtUdUYHbNOYnGMRjmmvjij4u6vqLyLSB3hdRL5R1Xc2vigiU4ApAIMGDeqA5hjT+ViEY1Ih7b8xqvpL7N9VIvI0MA54p97rM4AZAGPHjtV0t8eYzsYiHJMqaS34IlIEuFR1Q+z/BwDT0rlNY3KFRTgm1dJ9hr8Z8HTsbMQDPKaqr6R5m8Z0ehbhmHRI62+Qqn4PbJfObRiTayzCMelipwzGZIlIVLntzUXc/IYT4Tx08jhG9LMIx6SOFXzTRE1VkPUrS+m5eQ98fm+mm5MXVm8Ics7Mufxv0VqLcEza2G+UqROJRLjnwkd57vZXQACEYy+czPGXHm6RQhpZhGM6ihV8U+feix7jyekvoNFNvWMfvHwmNZU1/PG6EzLYstxkEY7paK5MN8Bkh4qySv7vxucaFPuNZv3zOSrLKjPQqty1ekOQE+/7iOmzF/Lb7Tbn+TP3sGJv0s4KvgHg+TteQxMMe1NV7r34sY5tUA57f/EaDrrlXeYsWc/1h2/D9KPHWF5vOoQVfAPAZ69/0ezrL93zBt9/sbSDWpObIlHl5tnfccI9H1ES8PDMGbtz9E6DLK83HcYKvgGg96CesQu18UUjUZ6+9aWUblPDS9GaV9HQV2iijxc5wiIckw3sc6QB4LCpB/POfz4gWFUb93WNKr8sWpGSbamG0dLzIPgGiBeIgPtX0OM+xNU1JdvIJtYLx2QLK/gGgF9tP4Tz7v0z06fcRdWG6rjLVFfUUFFayRuPvsOP3/xMSfdiuvQsYeDW/dlh/Da43e6ktqWV90HwTSAIGnSeDH+Dll2MdL89RXuUedYLx2QbyaaP0mPHjtU5c+Zkuhl5LRwKc3DhcUQjTX8v3F43BcUBamtC1FZv+iTgL/TTrU8Xbnn/anr07d7iNqKr9obo8jiveJHN5iBS0Ka2a3gRWvkYRFeAb2+kcBIigTatq71sIJXpSCLyaTK3j7XfQNOAx+tBxIVzC+KGIqEIFaWVzi1t6glWBVm5ZDWnjDqXe+ZPp6RHMe8/8zE/L1rB0O0Gs9OBYxqe/WtVgq2H0PASxDsC1QhE14KrKyL+FtsdrX4Vys4HQk7bg/9Dqx6Anv9BXMVJ7n1qWIRjspUVfNPEjgdsyyevzGvaJ19oUuzrq1hfyVnjLiQUChOsqiVYFcRf6KfPoF78672rKO5W5Czo3wdqngeiTVdSOpVo4Z+g4p+gNQBo4VFIyUWIxP91Va2F8ouBmnrPVkNkGVr1CFJ8WpJ73j7NRTga/hFq3wUpAv/4tB6ENLICIsvBMxRxWYRkNrFIxzSx/IeVnDnuQqdoV9fiL/DhDXipKq8mGolTpFvg8boZsu0WbLXjluy4/3bsdshAZP1EIBhnaR/OkaX+awEoPBJXl8virl9DX6LrTgSNMzjMMxJXr2da3ebWai7CiZb/P6h6EGe/XCAg3e5E/LuktA2q1WjpuRD8H4gPtBYKf4eUnG+fMHJcspGOFXwT14b1Fbx6/3/5bu4PDN1uMBNP3peLD7qGbz9e1K71BooDDB41kH8++SM+18dxlnAR98wfP7LZJ3EzeQ1/j66ZDMS52Owdh6vnI+1qc0vqRzhXTFjDUcMfQohAwW/AuwOUTm3aNilG+nxQF1epKkR+AKLgHpqwQGu0DGpeQiMrEd+O4Ns9FsFBtPRvUPMyDQ+WBVByCa6io1K+3yZ7WME3Kffz4hX8fquzmo11kuEv9HHylcM49PinaBjDNCeA9H4NcfeN+2p09cEQWUyDg4UUIF1vQAIT2tfgBBpHOLce/BQjur3Fpn3yg/hBy5u+WYqRbjch/n3Q0Ndo6ZkQWeN8CJBuSLebEd+YBm9xPsmcBBoBqkEKwbM10uNBQNGVY4E43WpdvaF4KtTOhchP4NsZCg6Hmqeh+nnADYVHIoUnIGKzo3ZGyRZ8G3hlktZ/aF/OvPVkPO3sbRKsquXNWdXg7gfUuyArBeDegrgjwMQPrl4J1ynd7wBXPycjl2JnvQXHgv+AdrU1kcYDqZ6bUsyI7m/T8AAWjB8zbaQhNFqFrvudU4ipBq2G6HJ0/R/QaOmmRVXR0rNBK6j7tKBVEFqAVj7czIVwILoayi+Dmqcg9AlU3g1rxkPFnc6nisgi2DAdXX963AFwqiE0+BbRDbcQLf0b0bLL0dpPcn6wXEfR8BI0+I5z7SXN7KKtaZVJfz6QIaO34KmbX2T1T2vx+r0s+/ZnytdVxJ14LRF/YQDp+SRa9SjUvOr0xik8EdyboWuPpWEEUgAl5yW8aAsgnkHQ+w0Ifer07vGOSfhpoL3i9cKh6kGnZ1ETEZw/s3DDpzUMvl0h+HrT18A5i69+AYpis5RGlkJkbZz110D101B0Crh6Juju2tjGyKd+e2ug9hMIfQG+TTep09DC2PWRDTg9oGLPVz8DBYcjXf+exPYa0mgZhJeAuz/ijn8QV1Wney0+xN0z/jLhHyDyC3iGJ1xPW2ntPLT6cYiWOZ8QAwcj4kvtNqKVzie72jmxay5BNHAQ0vWaZn/X28MKvmm1bfcaybZ7jax7HAlHOHuPS1ny5Y8EY/3zvX4PA4ZvzvoVZZSuKmvwfnEJq39ay2+7nobL5WLfYycw5Z8nUhiI9b/v+Ti64UYIzQd3X6T4DCTQ8pm6iAt8O6VuRxtptheOu68zalgbRyoB8AyByJJYryO389XlKsRVjEbXxHkPQA0aXVXvs46LhFmauJzMv+uV6PqpOAW9LWffEQjNrSv4qlF0/Z9A18VZthqq/w8tPBzxjqp7ViOrnesIWgP+vRHv8E2vaRQtvwaqH4u1L4p6toEeD+JyFW1arvYztOx8iKwCFPWORrpNR9z9nNejG9D1pzsHJ/E6hbLwKKTk0rrrGe0RrXwANtzExu+jBj+AykdR73ZQ8yIQhcCBSMlf2jUyXDdMcw6y1G4agFjzCurZMm09yyzDNykRrA7y/B2v8eZj7+INeDnk1APY7/g9WbpgGef9+gpqKoMNBms11r1vN55YdhcuV3amjC0NpFKtRVfvDdH1NLyOUAy93kTC36LBN53svmCS84mE2JnkupNoelG3EOl2C+LfK7Z+RddMcA4cDQSg5C+4in7vLBeaj1bcDbXvxc7KW0GKnLPLwIGxdX3pxE0J4yIXFJ2Bq+QsAKLVr0PZX3GKeRjwOtcGSi5FRIhW3g8bbqDJGA9XH6T3W4h40MgKdPWERt8PF7g3R3rNRsRFdP1ZEPwvDa9XFEDJBbiKjmvdPjei0VJ01Z407UEW62FV13YvuAcgvV5o03UP1RC6cnviX3PZDFefd1u1Prtoa7JGdWUNR/X9IzWV8bphblLSo5jt9hnFMRccyvCdftVBrWtZ/Qhn2qRRCQdSaXgJWnoOhBcB4hSpbjc1OANu8h5VtPRUCH7Ipvw/AN5RSI9HG5yxauhbdN0JQMj5VCBe8O6AdJ/RpOho9Qto2SXE7bkUl4B0R/q8UxddaO0n6PpTY9cN4vEiJeciRX9EoxXoqt3jbK8A6X4X4t+F6MrdQNfEX1XXm3EVHEi07Dqovi/OAh6k+z3g3Q5dNY768VId92BcvV9rdi9Vw851FSmJ+2lAa2ajZX9rZp/rkUKk67V1B8jW0GgVumpH4g1wRIpwbTa3VeuzkbYma/y8cDmSxJn7hnUVvPfUR7z/7MeM3mMEo3Ybzt5H7cbQ7Qanv5FxtHYuHPEMRno9g0ZWAdGkriGICHT7N1o1C6pnAVEoOAwpPK5JQRLvcOj9NgRfg8hq8G0P3h3jd+EMTICqJyA0j7hnkbhBusbO3hU8WzifKOrn1N5taT4acsPGYlf7PxBXnMVr0JrnnDEHWtb4xU2Cb0DBgc6/cYXRDf+CbreRsK9JNE5vqBjVKFpxG1TdBxoCKUZLzsdVeETDBaWYpOMwrUJDC9pU8MVViLqHOBfMG3A513bSJO0FX0QmAjfjhJf3qOp16d6myS6FXQqIhuNd0IwvGlG+eHsBX7y9gMevfZq9j9yFS2f+NY0tbKo9c+GIuw/Lf1jJ3RfcyGevf0FhSQG/PWMivzn9AH748kdKehSzxYgBm5YXD1J0HCQRR4irEAoObXk58UKPB9Dql2HDP2JdQzdGTR5w94eeLyDRlSAexL15nHX40S7XQNkFOAeNje93AV7ociXi7t9CS5S6AurZEsLfxl/M3cf5N7oq8arCX0L4O3D1SHBxWlCNINJ0Ej+tuA0q72VTD6f1UD4NdZU07Lbr28npLdZc76o6hYh7UBLLxSddr0LX/QHn00oY8IEEkJK/tXmdLW4znZGOON/5hcD+wDLgE+BYVV0Qb3mLdHLXaTuczw9fLCXaip489Z145ZH87rKmg4ci4QiRcARfIHU9KJKNcFYuXc0Xby+gpEcxOx6wLV6fE6usX1XGKSPPobK0sm5/3V43qkqg0E8kHGHzoX256oWL6DMwtb1L4lGtRitud3rzaCR2wfFsxNUtufeHv0er/uNM1+DqCd7RSGDfBu93Ip3daDKuQgpio4p3JVr7Jaw7PM4WfEiv5xHPEKIrd3aKcSIFh4N3Jyi/MN6LToQW2K/R/ofRVTslGIk9DFevFxsuH/oGXX+y00UWcT4R4AMq2XTQE2e8RO83kXoXnFtLw0vQygedA5lve2cshHuzVq8nKzJ8EdkVuEJVJ8QeXwSgqtfGW94Kfu5auXQ1fxt/JetXlhEORwjVxMlgm+Hxunk5+ETd4+rKGv599v288ei7RMIRthg5gHPuOpWRu2zV5jY2jnBuP26HuBGOqjLjbw/z7O2v4PG4EJcLj8/DP2dfzpbbbsGDV8xk1g3PUtvCPgaKA1z/6qWM3HV4s8t1Fs4EdufFHsUu2hYcjnT5e90BM1rzBpSei3NR1A14oOu1uAoOdl4vvx6q7k28Ee9uTtRUdT9xp+YoOAZX12kNntJoWexgFOfnIcW4NvusydOqEae7pFaAb6wT35Rd4DwH4N0W6Xod4hmcuK0dKFsK/hHARFX9Y+zx74CdVfXMeMtbwc9tqsqCDxaybkUpP337Mw9dMYtIKPmo59wZp/Hr4/YgUOjnogOv4vO3FzQ4cASK/Nw595/0/1W/VretNRHOhy98ytXHTm9yEbr3gJ48suTfXDjhKua+8WVS2/UGvEx75gLGHrBdywt3AhpZFeuWWQX+fRDviKbLaBRCnztdN33bN5guwzkb3yvxxV0KcC50hmiatXugaAquknOabE9X7Ra/e6l3B1w9n2j6fKL902rQaLvO6tMhW0baxpsQpMFPSUSmiMgcEZmzevXqNDfHZJKIMGq34YzZdxSPX/t0q4o9wB3n3s8pI8/h648WMu+/XzX5lBAKhnnq5hebvK+6soZH/vEfTh5xNlO2+yvP3v4ykXrXFFp7U/EX7notbo+jitJKFs5ZzBYjB+D2JnczmFBNiNunNnNG28mIuw9SdBJSfHrcYg/OeAnxbY/4d21U7NW5MY57CFC/oNYvI9U41xPinah6kILD4m6PkvOAxvMwBZCS85os3xyRgqwr9q2R7ou2y4CB9R4PAH6pv4CqzgBmgHOGn+b2mFaqqQqy6sc19Orfg8KSAtavLOWtme9TVV7NNnuNoP+wfnTfrGur+s+/Pet9glXNd9GM25bKIKFgmPN+fQXh2qajUyPhCG/P+oBDpuzPkG22AJwbupyzx6Us+/aXuojl7gseZe6b87nsP+e16Y5UibqXiksIVtcyeepBvHLfm0kf0H7+bgXhUBiPNz1/jhr5xcmJQwuc7p5FJza4SKvRKmfqZg2Df48WBxOphqDmdbT2XacPfcERiGdgnOXCzrgEV9ekRqlqxXSofJBNXTt9Tq8Z8UJ0Zfw3SSEbB6VJ1+vrxjc05io8AnWVoBW3xKaO3gopOQ/xtXhSnFPSXfA/AYaJyBDgZ+AYoH0jI0yHUFUeuHwmT970Ai63EAlF2H6/bZj35nxUta54ulxCt826cs6dp7Lrb5L74/n6w+8STsPQtVdJs9M0bLxIm0jZ6nKm7noJ175yCaP3GMH/nvmE5YtXNsjTg1VBPnznG464+W3mrqxsdS+cfY/ZnW8+XtTkoKWqjNh5GL6Aj2tfuZTpU+7k5+9WIAIev5eaivgTxQWK/bg9yX0iSIaGf3Tm5vFsBdE16LrjYiM5wxD6DK2eiXZ/DPEOQYMfQdnZoIITlUTRLtNwFTY9U3b2MYiuO94Za6BVgAetvB+63YwE9q1bLlr5KFRMd7YrLrTgRKfPfoKRsBpZC5X30bALaW1sG4UJ9rQgNmK5dywaav6gIoEJaZtIr7NIa6SjqmHgTOBV4Gtglqp+lc5tmtR47t+v8ORNLxCsClK9oYbamhAfvfgZweraBsUzGlXWLS/l6mOn8+2cxVRX1fCv0+7imAFTOGnYmTx184uEap3l161Yz8cvz6V8beL+0vscvTuvhmbSs3/Lt0pMpKYqyK1nOTHJ529/RXWjQls1qC+LjjmQ+asqk4pwGtv/pH0YOmYwgSJn4je3x42/wMdf7/lzXW+h0btvzb1f/Ysn19zHs+UP88RPdzLu4B0QV8OU01/oY/LUg1MyX71GK4mu+wO65mC0dCq6el903Smx3ikbPxGFnMfrDkNXjoHSP8WKaiVODxvnZjLR0ML426h6AkIL642+DQM1aNn5zpk/EK163hlRq+U4E8hVQ9VDztl1IqHPnflkmqhxukkS5zVXIRI4EPHvnPJ5bnJV2vvhq+pLwEvp3o5JrZnXP9uq2KW2OsRDV87i8zfn182nA3DnXx/k/ec+Yei2W/D8na/j9XuprYk/xYLL7WLELsMIh8KsW14ad5lkff/5UirLK/H6PHj9XkLBECrCul23Zd1u2+Evq+DKbfpz9E6t70ft83u56a0ree/pj/nopU/p3qcrB56yHwO2atqXvahLYd17rnruQu6/9HGe+teLuDwuIqEIE0/+NSdecWS79nUjLb+86dwsCS9+Nhc3RaH0r9D7+U3rjlY488hU3EX8Ka0jTmTk2w4qb6PpiNtqZ4K54rPi9pPH1ZP490FwOb1kQnMgsi62XjfgRbpcE39dJiGbWiHPRaNRKsuqKOxS0OC+swcXHtdit8LG/IU+glVNi7nL7cLjdbe4vqJuhTyxbAb+Ah+Tup7Y5My8tbwBLx63m+rKGsKFAVYcshfVgzenZMFitpq7gJnf35a23Lw51ZU1dddFNh4QWkMja9GqxyA835kPv/A4cHVHV+5A/FG1beFC+nzmjAgN/4iuPSrWLz3BVA1SgPSYiXi3JrpyTIL5dzxIn4/j3t5x01xBP9Kw8AeQnjPBPRCtftK5m5d7AFJ0POIZmoL9zA02tYJp0QszXuf+Sx+nqrwar8/DnofvwtEXTGLQ1gMYtsOWfPV+glGRcbg9rgZn9vVFI1FqE9wa0V/o3NKwZ79uXP7k+QQKnZhk0hkTefrWl+IeQJIVqgkRIkTVoL6sOGQvogEfm7/xIaNqq7j8zcvTd5E0tBDdcAOEPgNXNyg8JTZVghPbFBQFGoy0bdW6w0vQtUfEzuCDsZu1PwzdZhD/DLmt3LHJ1wrR8stAS5tZvzj3KvDExhN4hjuzbjbm6u7cryDeGkSckcHrT4fwDyAeZ71dptX19pGik6DopPbuWF6zM/wMqA2G+PilzyhbXc62e49k4PCWhqenRiQc4YUZr/PiXa9Turqc8rUbmvQkcXtcjNhlK3739yP5+6E3UFtd2+KNLkSgoEsB1eU1rbopRqDQz2X/+Qv9ttyMAVtt3iDHjoQj3Db1Pl574L+ISMKDSXPqRzje9RsY9Op7PPL25fQb0vqRjElvM7wUXXtow1GdUgAFx+PqkvyQedVo3Auc0XV/dHrUNO6W6BnjjFCNLG30DldsnvxynPO7ZKYMAFy9kd7vAlF05WgSFnspAvxIz0cQjzPhndZ+hq77PQ2jnwB0uQpX4W9b3LSGlzgHG89wy+aTlBUDr1qrsxT85T+s5NnbXmbJV8sYuetW/Ob0CXTvk9y82D98uZTzfn0l4dowkUgEVdjvuD05d8apab/R9LQjb+Tjl+e2mM27vW52mrg9v592NA9fOYuPX55LKNiwG6TL7aK4eyEiLsbsO5qTrz6W60+6lQXvN73YJy7B5RIi4YZFo1ufrsz8ZUazXToryypZ8tVPnD9+WtPRuQJbjBzIyiWrmnSVbBDhfLWYPq99gCca5dmyh+o+RaSa1s5x7koVjTeexI/0eR9xlSR+v0bRijtjE3yVg3sLpMuliH/vumWiK0YTP7YR6P4AlJ7mzKRJBPA6UUvPp5xFIj+gtV9A5T00P4tmALreiKtgf1Qj6MptiHuTFvxIt1ucrpyNZ+usnYdW3Aihb50IpvisBr14TGpZwU+TBR8u5IL9/0G4NkQ4FMHr9xIo8nP7x9fRb8vmzxxVlRO2PINVSxsWhECRn7/cfTr7HrN72tr9/RdLmbrbxUlHJF6/h1nL76G4WxFrflnHBftPY/VPaxGXEAqGmfD7fZj67z81OEgt+245Z467gMry6roTUJfbxcWPnc0d5z7AhvWV1FbX4nK78Po9XPrEX9jlkB2Tas89Fz3Ks7e+TE3sYOX2uCjqWsStH17DOXteRtnqcqKx2Kh6i34sP3hPon4fvWd/RJcvv0OAzYduxoPf3Zb8N60VNPguuv4MEt6jV4qRHg8h3tFxX45WvxSbpKzxwTiA9LgXid3YJbpyLHHvkYsP2exLiCx2ukmGvwffDkjhiU3mZolWPeNcWI38Qt2BgbDT5923I1J0GuLbftPy6890BkQ1KPo+KDgcV9crE35PTMexDD9Npk+5k5rKTX/UoWCISCjM3Rc+wt9nNT+j4+LPl1C+pukfa01lkBdnvJ7Wgv/1h/G72SUiLhdVG6op7lZEr817cM/86Xz90Xes/WU9w8duSZ9BvZu8Z8Cwfjy65A5mP/IuCz74li1GDWLy1AMpKAqww/htefGu1/ls9hf03XIzJk89iCGjk+8hc8o1xzF41ED+c+NzlK/ZwNiJYzjx8qPoPaAnt398HXed9xAfvvQZa3fehhU7jMRbWs6A/5uNb9U6RMBX4OPMW09p1fcAYkPp8TR7kwuNbkDLrqDZG7JryLnnbhzOHDTnxH0NatCKW5EeDzkPC46EqkdoeGDwQcFvnYOv51dI16sTtwNwFR4Khc6Mm6rB2KRoveJeTAWQLlei6xY6n1w05OTr7sFIyfnNbscJ01JrAAAODklEQVRkHyv4rVBdUc1P3/zS5PloVPn0tc9bfH8oGE4YX7Qlo26Nnpv3wOVOvgtbSbcievXvUfdYRJKamKyoaxGTzpjIpDMmNlxf92KOuXAyx1w4OflG1yMijD9hL8afsFeT13oP6Mmpd/+Zr2fOZX5sLpwpW4/mqbIVfPfp9wwY3p8TLjuCETsPS7h+jVY40wTHhvpr6Au07FJnFkNcaOBApMsVDYqiRiucCbWCb9N87xg/BMYnvDcr5S2cJYcX1/1XSs5Bw4ug9iOn8GrYmcir5JLm15GAiB9amABM3D2h18vOnPfhJeAZBr5d0h5BmtSzgt8KHp/HyaLjvFZQ3Hiejqa22nFL3J6mBd9f6GP8CXumoIWJjZ2wHQXFAWoqaxqMYhWXOI8FUGJxi5dz7z4ta2832Fi8m4qLCCOf+EuL79XQ12jp+RBZDAjq2xNKpsK6E+t1LYw49xqNLEd6PrrpvaVnber33pyCyUiXS5tpRLybk9fj2XSgFfEjPe5Gw4ud0a7uwQ3uG5suIm7w7+V8mU6rc/xFZwmvz8ueR+yCt9GoTH+Bj9+c3vKQbbfHzUWPno2/0IfX56yjoDjA0O0Gc+Afx6elzRt5vB5uevtKhmwzCF/Ai7/QR++BPbnprSuZteIejr/4MEbvsTX7nbAnt7x/NTsftENa25MKkahy8+zvOOGejygJeHjmjN05eqdBSZ95RoOfomsnQ2QhTpYdhtp3YO0faHpj8VoIfYnGRqBq5OfYVLnNFfsAFF+Iq+u0FnqbNHcR2YcUn93kWfEMdaYK6IBib3KHXbRtpcryKi77zXUs/HQxbq+HcDDEbpN24sKHpyY9H8qqH1fz6oNvsW55KTvuvy27/mZsSudSSWb7tcEw/X/Vt9N+LG/NdMYamo+WXwOhL8DVFQpPhoKjYXWcG3YAznlQnG6IUox0vQEJjEdrP0XX/4n49z4VkIDT/774rBa/x9Gya6H6/jiveJDu9yP+nZt9vzHWSyfNlnz1E8u/X8ng0QPT2q/bNJXsHakANLwIXXt4bJToRgXg2xFqPyD+FAOu2Ffjroh+pNeLiGcQGi2P3bS7ca8aLwQmIV0vd/LxJKhG0fWnQO37bOre1At6PIXL0/J9cY2xXjppNnjUQAaPajolrEmf1t5UHEAr7tg0r0yd6maKPdSdoWsVm870A84NPWLT74qrC1r0h0bT+bpBCp3bByZZ7CE2P3yP+9HwTxD6Etx9wbt9p/30ZbKXFXzTKbT5puKhr4g/StQb+4oT6XiGI91ucaZHqP2fM+d64bFI0akNFpPic8EzFK2815n33bcbUjK1TfckBZw55ePMK29MqljBN1kvUS+cpHiGQeQHmt4hScE9FCKLaHDh1dUHejyBuAJI9+YHaYkIFExCCia1ZneMyRjrpWOyVnt74QBI8Wk07QUTgIJDkF6PQ9GfwNXf+So6A+n9Oi5Xy11sjemM7AzfZKU2RziNiHcUdL8LLZ/mnOlLAAqOi919yYuUnA0lTbs9GpOLrOCbrNOuCCcO8e+K9H4Z1VrAaxdDTd6ygm+yRlt64bSGTbVr8p0VfJMVUhXhGGMSs78ok3GpjnCMMfFZwTcZk+4IxxjTkBV8kxEW4RjT8dL2FyYiVwB/Ajbe3uliVX0pXdsznYdFOMZkRrpPqaar6v9L8zZMJ2ERjjGZZZ+hTYewCMeYzEv3X9yZInIiMAf4q6quT/P2TBaqH+Fcd9g2HL2TRTjGZEK7Cr6IzAbiTdh9CXAH8A+cWav+AdwInBxnHVOAKQCDBiV/U2uT/RpHOA+fMo6t+1qEY0ymdMgNUERkMPCCqo5ubrnOdAMU0zyLcIzpOBm/AYqI9FPV5bGHk4H56dqWyS7WC8eY7JTOU64bRGQMTqSzBDi1+cVNZ2cRjjHZLW0FX1V/l651m+xjEY4x2c/+Ik27WYRjTOdgBd+0mUU4xnQuVvBNm9SPcCZv35+rDh1tEY4xWc7+Qk2rWYRjTOdkBd8kzSIcYzo3K/gmKRbhGNP52V+saZFFOMbkBiv4JiGLcIzJLVbwTVwW4RiTe+wv2DRhEY4xuckKvqlTP8IZYhGOMTnHCr4BLMIxJh/YX7SxCMeYPGEFP49ZhGNMfrGCn6cswjEm/9hfeB6yCMeY/GQFP49YhGNMfrOCnycswjHG2F98Hqgf4dxw+LYcOXaARTjG5CEr+DnMIhxjTH1W8HOURTjGmMasAuQgi3CMMfG42vNmETlSRL4SkaiIjG302kUiskhEvhWRCe1rpklGJKrcPPs7TrjnI7oEPDx75u4ctZN1uTTGONp7hj8fOAy4q/6TIjISOAYYBWwOzBaRrVQ10s7tmQQswjHGtKRdFUFVvwbinUFOAp5Q1SDwg4gsAsYBH7RneyY+i3CMMclI1ylgf+DDeo+XxZ4zKWS9cIwxrdFiwReR2UDfOC9doqrPJnpbnOc0wfqnAFMABg0a1FJzTIxFOMaY1mqxQqjq+DasdxkwsN7jAcAvCdY/A5gBMHbs2LgHBdOQRTjGmLZoVy+dZjwHHCMifhEZAgwDPk7TtvKG9cIxxrRHuzIAEZkM3Ar0Bl4UkXmqOkFVvxKRWcACIAycYT102sciHGNMe7W3l87TwNMJXrsauLo96zcOi3CMMalgp4hZzHrhGGNSyQp+llq9Ici5M+fx3qI1FuEYY1LCKkgWsgjHGJMOVvCziEU4xph0soKfJawXjjEm3ayiZAGLcIwxHcEKfgZZhGOM6UhW8DPEIhxjTEezCpMBFuEYYzLBCn4HsgjHGJNJVvA7iEU4xphMs4rTASzCMcZkAyv4aWQRjjEmm1jBTxOLcIwx2cYqUBpYhGOMyUZW8FPIIhxjTDazgp8iFuEYY7KdVaQUsAjHGNMZWMFvB4twjDGdiRX8NrIIxxjT2ViFagOLcIwxnZEV/FawCMcY05m52vNmETlSRL4SkaiIjK33/GARqRaRebGvO9vf1MxavSHIifd9xPTZC5k0pj/PnbmHFXtjTKfS3jP8+cBhwF1xXlusqmPauf6sYBGOMSYXtKvgq+rXQM4WP4twjDG5JJ0Z/hARmQuUA5eq6rtp3FbKWS8cY0yuabGCichsoG+cly5R1WcTvG05MEhV14rIjsAzIjJKVcvjrH8KMAVg0KBBybc8jSzCMcbkohYLvqqOb+1KVTUIBGP//1REFgNbAXPiLDsDmAEwduxYbe22UskiHGNMLktLRiEivYF1qhoRkS2BYcD36dhWqliEY4zJde2qaCIyGbgV6A28KCLzVHUCsBcwTUTCQAQ4TVXXtbu1aWIRjjEmH7S3l87TwNNxnn8SeLI96+4IFuEYY/JJ3mYWFuEYY/JNXlY4i3CMMfkorwq+RTjGmHyWNwXfIhxjTL7Li4pnEY4xxuR4wbcIxxhjNsnZgm8RjjHGNJSTFdAiHGOMaSqnCr5FOMYYk1jOFHyLcIwxpnk5URG/XFbGyQ9+YhGOMcY0IycK/oDuBWzdt4RLDh5hEY4xxiSQEwW/e5GPh0/ZOdPNMMaYrObKdAOMMcZ0DCv4xhiTJ6zgG2NMnrCCb4wxecIKvjHG5Akr+MYYkyes4BtjTJ6wgm+MMXlCVDXTbagjIquBpe1YRS9gTYqakw1ybX/A9qkzyLX9gdzfpy1UtXdLb8iqgt9eIjJHVcdmuh2pkmv7A7ZPnUGu7Q/YPm1kkY4xxuQJK/jGGJMncq3gz8h0A1Is1/YHbJ86g1zbH7B9AnIswzfGGJNYrp3hG2OMSSCnCr6I/ENEvhCReSLymohsnuk2tZeI/FNEvont19Mi0i3TbWovETlSRL4SkaiIdNqeEyIyUUS+FZFFInJhptvTXiJyn4isEpH5mW5LqojIQBH5r4h8HfudOzvTbWoPEQmIyMci8nlsf65s1ftzKdIRkS6qWh77/1RgpKqeluFmtYuIHAC8qaphEbkeQFUvyHCz2kVERgBR4C7gPFWdk+EmtZqIuIGFwP7AMuAT4FhVXZDRhrWDiOwFVAAPqeroTLcnFUSkH9BPVT8TkRLgU+DQzvpzEuferUWqWiEiXuA94GxV/TCZ9+fUGf7GYh9TBHT6o5mqvqaq4djDD4EBmWxPKqjq16r6babb0U7jgEWq+r2q1gJPAJMy3KZ2UdV3gHWZbkcqqepyVf0s9v8NwNdA/8y2qu3UURF76I19JV3ncqrgA4jI1SLyE3A88PdMtyfFTgZeznQjDOAUjZ/qPV5GJy4k+UBEBgPbAx9ltiXtIyJuEZkHrAJeV9Wk96fTFXwRmS0i8+N8TQJQ1UtUdSDwKHBmZlubnJb2KbbMJUAYZ7+yXjL71MlJnOc6/SfKXCUixcCTwDmNkoBOR1UjqjoG59P+OBFJOn7rdDcxV9XxSS76GPAicHkam5MSLe2TiJwEHALsp53koksrfk6d1TJgYL3HA4BfMtQW04xY1v0k8KiqPpXp9qSKqpaKyFvARCCpC+2d7gy/OSIyrN7D3wLfZKotqSIiE4ELgN+qalWm22PqfAIME5EhIuIDjgGey3CbTCOxi5z3Al+r6k2Zbk97iUjvjT31RKQAGE8r6lyu9dJ5EhiO0wNkKXCaqv6c2Va1j4gsAvzA2thTH+ZAz6PJwK1Ab6AUmKeqEzLbqtYTkYOAfwFu4D5VvTrDTWoXEXkc2AdnFsaVwOWqem9GG9VOIrIH8C7wJU5dALhYVV/KXKvaTkS2BR7E+Z1zAbNUdVrS78+lgm+MMSaxnIp0jDHGJGYF3xhj8oQVfGOMyRNW8I0xJk9YwTfGmDxhBd8YY/KEFXxjjMkTVvCNMSZP/H9hEJnMI1J7FQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x32d5e70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w = model.get_weights()\n",
    "x1 = []\n",
    "x2 = []\n",
    "for i in x:\n",
    "    x1.append(i[1])\n",
    "    x2.append(i[2])\n",
    "plt.scatter(x1, x2, c=y)\n",
    "plt.plot(np.arange(-3, 3, 1), -(w[1] * np.arange(-3, 3, 1) + w[0]) / w[2])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finding a seperating plane when data is d dimensional. Note that we visualize only first two features, so data might not look seperable but might really be seperable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XWcVNX7wPHPudM7m8DSEoIiipSEiqiIgdjYhY36xe7GbsX8qdiFiqJiS4iChQICgighKb3B1vR9fn/MsrDMLBtTG+f9evF6yZ255zy7ss/eOfEcJSJomqZpjYeR6gA0TdO0+NKJXdM0rZHRiV3TNK2R0Yld0zStkdGJXdM0rZHRiV3TNK2R0Yld0zStkdGJXdM0rZHRiV3TNK2Rsaai0xYtWkinTp1S0bWmaVqDNWfOnC0iklvd+1KS2Dt16sTs2bNT0bWmaVqDpZRaVZP36aEYTdO0RkYndk3TtEZGJ3ZN07RGRid2TdO0RkYndk3TtEYmJatitIarpLCUDx//jBkTf8WV7uSE0cM4YuQhGIZ+RtC0+kIndq3GvGU+Rg+4hc1r8gj4AgA8d+Wr/PXzP1w77rIUR6dp2jb6MUursalvzyB/fUFFUgfwlvqY+s4M1q/YmMLINE3bkU7sWo39MW0B3lJfxHWLzcLfs5alICJN06LRiV2rsVadcrHaLFFfa9YmO8nRaLESEUqLygiFQqkORYszndi1Gjv20iOx2CpPyxgWg+zcLPYd3D1FUWl18e2b0zmtzSWcnHshI5pdwDv3fYhpmqkOS4sTndi1GmvbpTVjJt5ATqssnG4HdqeNrn068fh3Y/SqmAZk5sezeHb0KxRu2kooEKKs2MP7j0xi/AMTUx2aFidKRJLeab9+/UQXAWu4TNNk7ZL1ON0OWu7WItXhaLV0yb7XsXLRmojraRkuPs5/HYsl+nBbQ+H3+vn46a+Y/MZ0AI48fwgjrh6O3WlPcWSxU0rNEZF+1b1PL3fUas0wDDrs1S7VYWh1tHH1lqjX/b4A3hIv7ix3kiOKHxHh5iPvY+mcf/F5/AC8c++HzPpyDk/+cC9KqRRHmBwxf35WSjmVUr8ppeYrpRYppe6JR2CapiVGp33aR73uznThynAlOZr4+uO7hSybt7IiqQP4PH6Wz1vJH98tTGFkyRWPgVEfcJiI9AJ6A8OUUvvHoV1N0xLg4ofPweGqPCzhSLNz0cNnN/i5ksW/LsEXZUmut8zH4l+WpCCi1Ij5/6KElZT/1Vb+J/kD95qm1UjPg/fmga9uY6+Be+BKd9KheztufP0Kjr5waKpDi1nzts1wpEWOpTtcdpq3a5aCiFIjLpOnSikLMAfoCjwvIjdHec8oYBRAhw4d9lu1qkYHgWiaptWYp8TDWR0vp6SgtNL19Gw341e/gCu9YQ811XTyNC6fu0QkJCK9gfbAAKVUjyjvGSci/USkX25utUf2aZqm1Zor3cUT0+9ht73aYXfZsbvs7LZXO574/p4Gn9RrI66rYkSkUCn1PTAMaDozFZqm1Ru79+zIa389xcZVmwFo1bHpPUjGnNiVUrlAoDypu4DDgUdijkzTtCZLRPhz5mLmTJlPRk46Q848iOZtcmrVRlNM6NvE44m9DfBm+Ti7AUwQkS/i0K6maU1QKBTivtOeZM7k+XhLfdgcNt64833u+ugGBhzdJ9XhNQgxJ3YRWQDo77amaXEx86NfK5I6UFEm+oEzx/LhxlexO2ypDK9BaNiLVjVNa3SmvPVD1PLQAIt++jvJ0TRMOrFrmlavGNaq01JD30CVLPq7pGlavXLU+UNwuh0R1y1WC/sM6paCiBoendg1TatXBp04gENPOxCHy47VbsXpduBMd3L3xzditem6hTWhy/ZqmlYvLZ+/krlT/yQjx83gkwc26KqT8aLL9mqa1qB16dWJLr06pTqMBkkPxWiapjUyOrFrmqY1Mjqxa5qmNTI6sWuapjUyOrFrmqY1Mjqxa5oGQFFeMXnrC0jFEmgtvvRyR01r4javzePBs57i79+WoZSiVccW3PL2VXTr3zXVodWIiCCeD6H0FTALwT4QlXEDytox1aGljH5i17QmLBQKce3Bd/LXL0sI+oMEfAHWLlnPjUPvoWBjYarDqxEpfhyKHoDQSpBC8E1B8kYgoXWpDi1ldGLXtCbsj2kLKcorxgyZla6HgiG+fX16iqKqOTG3QtlbgGeHqyaIByl5JVVhpZxO7JrWhG1avQUJRY6p+70B/lu2IQUR1VLwX1D2aC9AYE7Sw6kvdGLXtCasW/8uCJGJ3el2sO/g7imIqJYsbUD8UV5QYNFj7JqmNUFdenWiz2H74kjb/tRrs1tp1jqHQ047IIWR1YyytAbHIGDnMr8OVPqoVIRUL+jErmlN3JiJNzByzGm07dKKFu2bcdz/juK53x7C4YqsiV4fqawnwTkMsIf/GK1R2U+jbD1SHVrK6LK9mqY1CiIeMEvBaI5SKtXhJIQu26tpWpOilAssrlSHUS/oxK5p9UgwEOSb16Yz+c3vsVgNjr5oKEPPGYzFYql1O8pQtb5Paxx0Yte0ekJEuOPYh1j40z/4ynwALPtjBb9+MYe7Pry+Rm2s+HMVY0e9xN+/L8NitXDIqQdw5XMX6dOHmpiYJ0+VUrsppaYrpRYrpRYppa6OR2Ca1tT88d1CFv28PakDeEt9/Pb1H/zz+7Jq78/fUMA1g+9k8ayliCkE/UFmfPQLtw57IJFha/VQPFbFBIHrRaQ7sD8wWim1dxza1bQmZf70hXhLfRHXQ4EgC374q9r7vxw3laA/WOlawBdkxcLVLJmzPG5xavVfzIldRNaLyNzy/y4GFgPtYm23IRIJhGfmNa0OclplY3dF7qK0OWxk5WZWe/+/C1bh9wYirivDYO2S9XGJUWsY4rqOXSnVCegDzIry2iil1Gyl1OzNmzfHs9uUE7MEs/BGZGMfZGNfzC3HIf75qQ5La2CGnDkIw4j8kVSG4qARA6u9f68BXaP+YjBDITrv2yEuMWoNQ9wSu1IqHZgIXCMiRTu/LiLjRKSfiPTLzc2NV7f1ghRcDt6vAT8QguA/SMF5SHBNqkPTGpCsFpk88OWtZLfMxJXhxJXupHnbZjw65S7SMqpfxnf0xUNxpjkwjO1ruO1OGz0O6k7nHolL7D6Pj1AolLD2tdqLywYlpZQN+AL4VkSerO79jWmDkgSXIVtGAN6dXrFB2tkYmbelIiytGpvX5vH9+z/hKfUycHjfelV7PBQKseyPlVgsBrv36hj1Kb4q61ds5MXr3mTOlPnYnXaGXXgY5997OnZntEJZsfnjuz95ZvQrrFu2AZvdyrCLDmPUYyOxO2xx7ysW4p+LFD8KgcVgyQX3aIy0k1IdVp3UdINSzIldhbd4vQnki8g1NbmnUSV273fI1htASiJftA/CaPZ68oPSdumHD3/h0fOfQ0yTYCCE3WlnyBmDuO7lyxrtjsV4Wz5/JVcPuh1f2fYCXHaXnUEnDuC2d+vPwjgJLEDyzqHyg5cLMq7GcF+YqrDqrKaJPR5DMYOAc4HDlFLzyv8Mj0O7DYN1D5DICSuwg6130sPRdq20qIzHLngOv8dPwBdETMFX5uP7D35izpQFqQ6vwXjv4U8iJmr9Hj8/fTKrXh3QIcVjifw07YGSZ5GoVSEbh3isivlRRJSI9BSR3uV/vopHcA2Bsu4GjqGAc4erBignKu3sVIXVIIgIJYWlBPzRfjEmxtypf2KxRu7G9Jb6mPbujKTF0dCt/mstYkZ+2rc5bGxYWY8WRwT/jn5dQmDmJTeWJNLVHeNAZT8G6ZeCkQvKDY6hqOYfoywtUh1avfX7N38wsssVnNrqIk7MPo+nLn8JvzfxT1A7TizuSCkwLPrHoaa69esS9fvl9wZot0frFERUBctuVb9mNEteHEmm/yXHgVI2jPTRGC1/wmj1B0bO8yirXl5WlX9mL+eeUx5nw8pNBAMh/N4AU96cwaPnPxdTuyI+JLQJkapXaPQ9omfEMXAAjjQHR5x7SEz9NyVn3HoSjp2WVjrSHBwz6nAym2WkKKpIKv1qKn+aBnBB2tko1TDKEteFTuxa0r3/8Cf4PTuNz3r9/PLZ7DqNz4oEMYseQjb2RzYPRTbtj1n6ftT3utxO7nj/WhxpdhxpDqx2Kw6XneGXHE6vQ/ep09fTFLXr2oanfryfvof3xJFmp0W7Zlxw3+lcPvb8VIdWiXIMgqxHwWgNWMKfqN0XoDJuSHVoCaWLgGlJt/afdURbjWW1W9m0Jo+cVtm1ak+KH4ey96iYJBMfFD+IWHJQzqMi3j/wmP14d+ULzJw4C0+JlwHD+9Cxe/u6fClN2u49O/LI5DtTHUa1DNcwxHkUSCkoF0o1/oqXOrFrSbfX/nuw+u//IoZEgv4g7brWbnxWxA9l44lc+eBFSp6LmtghvBno2EuPqFVfWsOllAKVnuowkkYPxWhJd+YtJ+FIs7PjknFnmoMTrjia9Oxalpc1i4HIMXMAQhvqHKOWeCIm4p2KWXg95tY7dRmOONJP7FrSte3Smmd+fpCXb36bhT/+TWbzDE694XiOu+zI2jdmZIfHTaOtSbY17jHzksJSpr//ExtXbmLvA7ox8Ji+UZdyJoqYJYjnEwj+BdZuKNcIlFF9sTIoT+qF/wPfr0AZYCCeSUj6FRhN+BDqeNFnnmoNnlk2EYruBXasrOlENX8XZds3VWEl1PL5K7n+0DEEAyF8ZT5c6U7adGnFUzPvw5We+OPhJPQfsuVkEA/h77szvHej+Ycoa8fq7/dOR7ZeC1K20yt2VO53KEvLRITd4CVz56mmpZSRdjIq+0mw7gMqB+yDUM3HN9qkDvDgWU9TurWs4lAOT4mXtf+s4/2HP01K/1J0H0gh23+ZekGKkKIxNbvfNzVKUgeUFfw/xy3Opkondq1RUM6hGC0+wWg1C6PZ6yhbj1SHFBeb1+Yx//tF5K0vqLi2ZV0+61dsjHiv3xtg2viZyQnM9yORcxsm+H+NuuIpgnITPf0oUGmxx9fE6TF2TauH/L4Aj4x8ll8/n43NYcPvC3DIqQdww6v/C++erSJ3Jm2MXVmjz2tQs/6VawRS9j6Rq5kUOA6ONbomTz+xa1o99Oqt7/LrF3PwewOUbi0j4A0w86Nfeef+j2jWOoeO+7SPqETpcNk56oIhyQnQeTywcylgGziH16hCprLtBRk3A47w07tKB5WByhmHUjvvFNVqS0+ealo9IyIcn3lu1PNPM5unM3Hz66xdup5rB9+Jz+Mj4A1gtVvZs18XHvrmjqTUQxezBMm/AEJLQSRcbMfSEdXs7RqvjAm3Uwj+XwAnOAahVPzrxjcmNZ081UMxmlbPiEilOuc7KisKD12036MN7656gV8m/c7mtXl069+VHgftlbR68spIR7IeBM+E8F4C53CUY3Ct+1dGNjiPTlCUTZdO7JpWzxiGQZc+nVg2d0XEa3sfuGfFf9sdNg457cBkhgaEf/FI0b3gmUh4AtUKvq8h+yVw7J/0eLRIeoxd0+qhq56/BKfbgWEJPwFbrILLbeHysWelODLAPwM8HxOe+PQDZSAepHB0oz68oiHRiV3TqiDmViS0JSV9dx+4B89/346jziigW59Shp+dx4vTlrJ7+1uRqCd2JY+UTaTyZrCKV8D/e7LD0aLQQzGathMJbUAKr4fAPEAhlvao7MeSuuFJgqto3+4rrnlspwnU4L/gnQyuY5IWS6TgLl6ruha+ljz6iV3TdiASQvLPgsAcIAD4IfQvkj8yuU/v/tkQtbxsGeJP/CYkEUGCK5DgqogNR8p1AhCtbIEJ9gEJj02rnk7smrYj/69gFhCxq1KCiGdi8uIwmgHRVpjYwGiV0K4lsCB8YMmWE5EtxyFbjkICO5wd6jgivIlIbUvudsCJynoMzCLMkpcwt96NeL9O+bBRU6WHYjRtR6H/QKKVAfZBaGXy4nAcBMpZXk9lxydmC8p1SsK6FXMrkn9e+FCKbUIrkfxzIHcGykhDKQOyn4HAbMQ3I7yxyHU8hNYgW44ECQJ+xPspWF6CZuNRhi4TkEw6sWvajmz7EnW/vkpD2fpXe7uICb5vkbJPQSmUawQ4jqj9+m5lg2ZvIwWXgbmZ8Idra3is37qLA5pj5f0Sop4ZGwTfZHCdWB6fAnt/lD38PRExkbxTKxf2kjIILkfK3kSlX564mLUIOrFr2g6UrTvi2L+8Tvi2OiY2MJqDa/gu7xURpPA68E1n26oR8f0CzmGo7EdqH4u1K7SYAsGl4Vise6NUYn9kJbSJyPothOvChDZXfWNoBUhxlBd84PkcdGJPqriMsSulXlNKbVJKLYxHe5qWSir7OUgfDZb2YLSEtDNQzSdWX8MkMK9SUg/zgPdrJLCobrEohbLtibL1THhSB1D2/aJXV1Q2sPfZxZ22KoawAF0mIOniNXn6BjAsTm1pWkopZcdIvxQj9zuMlj9iZN4Z3vpeHf9PQGR9l/Awxk/xDjMx7IPA2h3Y8ZeYE2x9wbZflbcpa4fwL8KICV8XuM5IQKDarsQlsYvIDCA/Hm1pWoOlMoFoBbhsUIvCWKkg/vmY+echmweBBMB5Alj2BGs3yLihvOrirucJVM5z4dU8yk34F4MTHIeg0k5NytegbZe0MXal1ChgFECHDh2S1a3WiElwDYTWgLVrwo9SE7MEKRsPvmlgNEeljUTtXBfFeQwUPx55swKc9fcDrfhnI/kXUjG2buZBcAlkPYbhOqrG7Sjr7pD7A/hmhCd8bX3C5Xm1pItb2V6lVCfgCxGp9ugaXba3fhEJQOCv8PI6655JqxBYVyIepOBK8M8Kj9+KD1zHozLvQ0Xd1BNjf2YJkncShDawfajFBRnXYrjPr/xe30yk8OptfwMsqOxnUY4D4h5XvJh5p0JgfuQLRhtU7vf1/t9DU6LL9mo1Ej5U+EbCW8FNMHIh58Xwiox6SoruDSd1fOGkDuD5ErF0RqVfEv/+yj7YKakDeKD4ScR1CspIr7iqHIOh5a/gnwMosPet/zXGA/9Ev25uWyGT+MOxtfjSO0+bMAmuDj9dSlF4Q4p4wptM8kfW2x2DIoHw8rmISUoPlL2VmE5906L0R/h4uMCfkZeVHeU4AOXYP+FJXcSPSLQJ21qw5Ea/rpyAI7a2tZSI13LH94BfgG5KqbVKqYvi0a6WWOKZQGRBJwkn+Pp6Urz4qbLQlJQkpk+jOdG394egJqtlEkBCWzALLkU29kY29sbMOx0J/lu3xtyX71AeYBsXuC8M7zLVGpy4DMWIyJnxaEdLstBmolfqk/AEWpKImITrejuqX3lhuBFLJwgt3/kVsA9MSHzKPTK8db7S+nQDjLZgTf7koIgZLlQWWkvF/7/APCTvdMidVquj6QCU62REtkLJ8+XlABSknYNyj4577Fpy6F/HTZhyDAaibEaRENiqnZ+JmUgIs/hpZNN+4SfPzUMwPVOqvU9l3Ut43HfbRKkNVDoq46aExKns/SHjJsAZPnQZF1h2RzV7NSUTi+L7qXzMf8dfygLiRzyf1ayNwCLMgqswtxyHFI1BOY5EtZyFyp2MavUbRuaN+mm9AdOTp02Z80gofQ2Cy9i+jdwFrpPCG04STIofh7LxVDwJm+tg6/WI8dIuV5Eoe39o8TFS+hoEl4OtF8p9AcrSOmGxGu6zEdeJEFwEKmuXq4cktCW83M/aGdO0seCHvyjcVESPg/Yit33zmOIQsxCKbifqtn885eUHqmnDNwMpuILwvIGE67l4v0A1n1CvJ821mtOJvQlTyg7NxyNl74cnJFUaKu3MpBwuLOKFsneJTFBepKT65YHK2gWV9UDC4ovap+HeZb1xMUuRrdeD70dQdkwzyPin2vHRiy1QKAL+AMdffgijHh+FYdTtaVi23lFeFCyaNJSt567vF0G2jqHy9z0IUooUP47KebFOcWn1i/6s1cQp5cRwn4/RYiJG87dRruF1Hl6o1Z4IMw+q6ie0qk79p0Lp1lJevOFNztztUn774EhCpTMAP0gJhvJyyqUr6D1oI2XFHgK+IF+Om8KPbx2L+OfUui8RX3ktmiomj42saguVIUXlyxgjXtDH2jUiOrFrMZPAUsy8s5GN3TE39AofsiDRzsTcgZFLlf/8rN3iHmMiBANBrh50B589/w2+0k30OnALFmvlyWhnmsnpo7cnUm+ZwWevmkjBRUhoXe06lAARB4BUsJUXKqtmzblyEX2FDylb4aPFn07sWkwktAnJPx0CswknHQ94JiIFuy7TqpQ9vMwuYvOLE5VxTYKija+fPv2dTau3EPAFycgJEQxET5jNWlXeE1BWYgEJIGXvAbBh5SYW/riYksLSaLdXUEY6WPeI8ooFnEejLC2qjVkpO7iOJ2J9unJBml6l3FjoMXYtJlI2vnxt+Y7DMD7wz0WCy3Y5GafclyBGDpS+AKEtYNsLlXFztePE9cXiWUvwlITHqjeusRMKRib2UBDm/bh9Z6rdaXLI8QVAgNL85dw34h4W/fQPNoeNgC/AKTccz/n3nF7lcJjKehDJP7f86d0POMFIR2XcUOO4VeZdiFkMvu/LSzL4wXVmeH5FaxR0YtdiE/iLcILZibKGV6zsKrErFa7810Cr/7XdvTWONDu+Mj+hoOKlMW254qG1ONPCv+RM08BTqnjv6fAZpc60EG06+jnuvHzAxWNXGCz88W8CviB+b/ip/uOxX9Bxr3YcdtbgqH0qWw9o8U14wju4HGy9UWmn1GrtulJOVM6z4UM1QuvB2gllZMX2zdDqFT0Uo8XG1oPwYcY7kSBYuyQ9nGQ67KyDsNq2PxtN+bAZL9/XlrwNdgQ3hvMAinmbgcOaMfjYIkY/8B/PfLkUZ5pBcVEWs6fmEfBVHpP3lvqY8Piu16IrSyuMjKsxcp7BSL+w1huStrfTEmXvpZN6I6QTuxYTlXYmKAeVJ+Qc4fMwG/ma6PRsN0/+cC+d9+2AzWHllP9t4dIxG2jWKoCiFAJ/0Cb3BS5//iXuGH86R56VgT2tBbhOodQYh2GJXoly65ZoR8xpWs3poRgtJsqSC80nIEX3gf+3cJJ3nYLKuD7VoSXF7j07Mm7+ExRuXEGGHIfacSmilIF/Dso/A5V+IaRfWPFSq/QQTrcDX1nlAl6GxWC/IxrGHINWf+kndi1mytoFo9kbGK3/wmj1B0bm7dWfD1rPiQQRz1eYBVdjbr0d8UepV76DrMxFKBXt9KQyxPttxFWLxcLV/3cJjrTt9XGsdivurDRGjmmYcw5a/aGf2LVGz1PqpazIQ06rrBrt+BQJIgUXhw+nljLAQDyfIxnXYLgvjH6TSiP6+nADVEbUWwafvD8tO7RgwmOTWL9iE72H7MPJ1x5H8zY5Nf7aNC0andi1Rstb5uOpS19ixke/ohRkNEvnqucv4cAT+u/6Rt/UHZI6hNfne6F4LOI6EWU0i7zHcRDRE7t9l2d+duvflTsnNI1hKy159FCM1mg9eNZTzJz4KwFfAL83QN66Ah486ykWz9p1oSzxfrtDUt+BsoLv16j3KGVH5bwaPtBapZcf6GyHjBtRtr3j8NVoDZVIALPkVczNR2BuOhSz6GHELEpon/qJXWuUtqzLZ87k+RXrw7fxe/188Mgn3P3xLkr8qkzCzzxRtu8b7qpvs/eGlj+HDykRD9j3Rxl6WKWpk8KrwPcTFYXXyt5BfN9Bi89RKjEnVOkndq1R2rI2D6s98rlFBP5btmGX94aHTqIdaWcFezVVJ5Ud5TgU5TxaJ/VaEjER34+YRQ9glvxf7Wvp1EMSWFw5qQPgDxdi836VsH51Ytcapd32akfQH3k6lMVqYd/B3Xd5r7L1gIwbAUd4SEW5QWWVH6xRzw+mbqDCE9ajkMIroOxNKHke2TwM8U5NdWixCSwg6tyLlCEJrKbZYIZixMxHSt8Nlxa17o5yj0RZd091WFo95c5M45Trj+PjsV/iLV8rrgyF0+3g9JtOrPZ+w30u4joO/LPCK17s+1exnFGLC+9X5WWDt1UFDQABZOsN4JiVsCGLhLO0BWVULqUEgAMsHRPWbYNI7BJah2w5qXxCyweB3xHPJ5DzEsqxf6rD0+qp8+89g7ZdWjPhsUls3VxEryE9uOD+M2nVMbdG9ysjG5xHRX1t7ZJ1vHb7eBbMWEx2bian3XgCR4w8JCVH5SVS6dZSJj3/Db98PoecVlmMuPoYeg/pEfd+xDOJymfKbmOAfw44Dox7n0lhPzB84pZ4qDRnoywo18kJ61bV6nCEOOnXr5/Mnj27xu83C28E7+dETGZZdkO1mNrofpi0+m39io1c1udGPCVexAz//DjTHJxyw/Gcd/dpKY4ufkq3lnJZ35vIX19QMQntSHNw8UNnceKV1RzoUUtmweXgmxb5gnKjcsaFj0NsoCS0Dim8FgILAQMsrVBZT6DsvWrdllJqjohUeyBxwxhj980g6gqF0EaQgqSHozVt7z/8Kb4yX0VSh/Ca+Q8fm0RZcTUHjDQgk/7vW/I3FFRaWeQr8zHuxre5vN9NHJt+Dhd2v5ofJvwcc1/KdWr5ISA7s4OtT8ztp5KytMVo/gEqdwYqdwqqxZQ6JfXaaBBDMRjpEIqWwAVo2FvXtYZj9d//8e3r3/HDhz8TCkY+aFhsFtYuWcee+6WuquWWdflMfuN7Nq/ZQs9D9uGgEQOw2es2NzDrizn4PYGI6wF/kGVzVwCw5p91PHbh85QVezj6oqF1D9wxBJwjwPNR+O/KCihUzoso1TDSVHWUJbaDzGsjLt8xpdQw4GnAArwiIg/Ho90KaSOh+Ekqj8HZwDEEZaTFtSutbkSEgD+IzW5tlENj08bPZOwlLxIMhAgFo585GvAHadEuyq7UJPlz5mJuG/4AZsjE7w0w9d2ZvPfQxzz90/240qs5Mi+KnNY1OyrPV+bn1dvGc9QFQ+p8SLdSCpU1BnGfC76fw+e3Oobqn+86inkoRillAZ4Hjgb2Bs5USsV1q51KOxdcxwD28K4+nGDricp6MJ7daHVgmibvPjCRk5qdz7Huszln9/8x8+NZqQ4rrjylXsaOegmfx19lUrc7bQwc3pdmrVOzdl1EePDsp/CW+iqGTrwlXv5bup6PnvyiTm2edNVwHGmKttFvAAAgAElEQVQ1W41SWliKJw7DUMq6O8p9Dsp1nE7qMYjHGPsAYJmI/CsifuB94IQ4tFtBKQMj60FU7lRU9pOoFh9jNH+vzgcMaPHz5l0f8N5Dn1C6tQwxhU2rtvDIyGeYM2XX1RAbkoUzF2OxRv9RUYbC5rBx8GkHcvNbVyY5su3+W7qe4vzIM1P93gDT3/uxTm32OmQfLnnkbBxpDtIyXTjSHNgc0Yd17C47znQ9LFpfxGMoph2wZoe/rwUGxqHdCMrSGiytE9G0Vgd+X4CPn/4yoqa4r8zPm2MmsN8RiZ0gSha7yx5lHXLYwOF9ue29a3C5U5vUbA4bVa1wqyoZ18QJo4/myPOHsGzuCmxOG89f9Sp/z1pW6T3ONAen3XgClioODtGSLx5P7NEGVCP+hSmlRimlZiulZm/evDkO3WqpVrSlqNLKkB2tW7Y+ydEkTo9Be2FzRiZHp9vBSVcNT3lSB2jVMZf2e7SJmN9wpDk49tIjYmrb5Xay7+DuvHHHeyyftyri9cNHHsxZt42IqQ8tvuKR2NcCu+3w9/ZARJEHERknIv1EpF9ubs02iGj1W3bLLCy26E9pnfbtkORoEsditfDAF7eSnu0mLcOF0+3E7rRx4pVH0/fw+nPa0V0fXU9O62zSMlw40uw4XHYGDO/D8FGHx9z2xlWb+XPmYgK+yFUyWzcXN8oJ84YsHkMxvwN7KKU6A/8BZwBnxaFdrZ6z2qycc+cpvDVmQsW2fQCHy84F952Zwsjir1v/rnywbhy/fzOPksJS+hzWg5Yd6tcDSruubXh35f8x+9v55K3LZ+8D9qTzvvHZtr55bR42hy2iWibA+n83xqUPLX5iTuwiElRKXQF8S3i542sisijmyLQG4ZTrjiM92827908kf0MhnffdjVGPjWSfA7ulOrS4szvtDDpxQKrD2CWrzcr+x+4HQCgU4ocJPzP1nRlY7VaGXTCEAcP71unputM+u0V9WrfaLfQ6VNebr28aREkBTdNqR0QYc9Kj/DHtT7yl4U9TTreDo84fwhXPXlSnNl+/630+HvtFRXuGxcCdlca4BU/Qom3q1u83JY2rpICmNSAiwpcvT+HsTpcz3HUml/e7ifnfJ/dD7LzpCysldQBvqY+vX/uO1X//V6c2z7/ndK5+YRSde3akedschp41mBfmPKqTej3UOPbqalo98uETn/PW3RMqloEum7uC2495kEem3BWXISq/L8AXL05m8pvfY1gMjr5oKMMvHorFun0i+/dv5lVK6hVEmDt1AR32alfrfpVSHH7OwRx+zsGxhK8lgU7smhZHwUCQd+//KHJtv8fP63e8x+Pf3R1T+6ZpcstR97Nk9jJ8ZX4AVi/+j1lfzeW+STdXjJ9n5Lix2a0EdjpsxLBacGfpHZ2NnR6K0eodERMJ/IUEFiISfQt/fVW4aSuhQPSYVy5cHXP7c6f+ybK5/1YkdQhXXJw/fWGlQ7qHnj0YZYn88VaKej8BrMVOJ3atXpHAn8jmQ5D8s5D8c5BNgxBf4mvPiAifv/gtI7tewYnNzuOuEx6p01h0ZotMlBF91Um7PdvEGiZ/zvgLT4k34nrAH2ThzMUVf2/ZIZdb37kKV7qTtEwXaZku0nPcPPDFbaRl1L4gmNaw6KEYrd4QswzJPw+kZIerZUjhpdBiWkLLnr5889t89n+TK4ZQfv1iDvO/X8SL8x6jTedWNW7H7rAx4tpjmfjkF5WGYxwuO+fdc0bMcTZrk4PDZcfn8Ve6bnPYIqoxHnTSQPoP682fM//GarPQ46C9sNr0j3xToJ/YtfrDN4WoB6pICLx1q1BYEyWFpUx67ptKiVhE8Hn8THh0Uq3bO+/u0zjrtpNwZ6ehlKJ1p5bc+u7V9B26b8yxDjlzEEaUIRar1cJBIyJLNDlcDvod2YveQ3ropN6E6P/TWp2EgiFmfzuPvHUFdD9gTzr3iEMJATMPJHITDPgQc3PUokTxsOafdVjt1ohdlaFgiL9+WVLr9gzD4KzbTubMW0cQDATrfNBFNJnNMnj42zu477QnKSksA4Ts3CzGfHxDvahZo9UPOrFrtbZu+QauO+QuPMVezJCJiDDw2P24bfzVsVX4sw8g/E9yp+Su0lD2xB1a3rJDCwK+YMR1pRTtYxgXV0rFNalvs/cB3Ri/+kVWLlqDYTHosFc7XatFq0QPxWi1ds8pj5O/oZCyYg/eMh8+j59ZX87lq3FTY2pX2XqA41Bgh8k95QJbr/Bp7wnSvE0OA4f3xb5TBUe7y8bpN5+YsH5joZSic48OdOzeXid1LYJO7FqtbFq9mbX/rIso1+sr8/HFS1Nibl9lP4nKuhts/cDWG9JvQeW8glKJ/ad6yztXcthZg7E5bVhtVlp2bMFdH96Q0vNLNa2u9FCMVit+XxBVxbmW0Sr/1ZZSFnCdhHKdFHNbteFwObj+lcu58vmLWT5vBYtnLaVgYyGlW0txZ7mTGkuiFOUX89fPS8holk73/feo8/mkNSUSgOBSUG6UNT5VJrWa0Yldq5V2XVuT2SydzTvtrLQ7bRx6RuKGS5JBRHj1lnf4YtxUEMGwWHjuyle5d9LN9Dks9hUtqfTBY5N4a8wH2OxWTBEyctJ5ZPKdtN+zbUL6Mz1ToOhWIAQSQqwdUdkvoKztE9KfVpmu7qjV2p8zF3Pb8AcIBU0CvgDOdCetO+Xy9E8PNMjNL/8uWMWX46bw75+r+ee3pRETqe7MNCZsfAV7DEfMpdK86Qu547iHKy3nVErRunMuby59Lu5j9BJchmwZAey4kcoAS1tUi6kJH1ZrzGpa3VE/sWu1tu/g7rz+zzNMfuN7NqzcRO8hPRh88sCErABJtClv/8DTl40j4A9ihqKsoQcEYf70hfQf1ifJ0cXHpOe/iahdIyIUbipiyZx/6dYvvvMIUvoeESubMMEsgMBcsFebl7QY6cSu1UmLts0a/DmXnlIvT1/+csQuzmhCwehJvyEozi+Jet00hdKtZfHv0FwHRKuXo8DcEv/+tAj6M5HWZP09a2mlUrdVCQVNeg3Zh5LCUsqKPUmILL4OGjEQmyPyGc5X5qOsqHJi37hqM79/8wfrlm+oe4f2Q6i0ZHUb8YdXOmkJp5/YtSbL6XYiZtVP4ja7FWUxOP++M7ju4LtYsXA1Cuhx0F7c9OaV5LZPXO2aeJHAQo48eTqv3BK5AQvg5Zvf4aCTBhIMBHl45LP8Mul3bA4bAV+AXkN6MOaj63G4HLXqU6WdgJS9DqF1wLYhIBeknYGytI7tC9JqRCd2rcnq1r8L6TnuiGqJdqeNA47vR5dendj/uH5cd/BdlG4tZds6gwUzFnPN4Dt4a+lzNXriTxWz7FMougun8iPmPkT7gL5hxSZ8Hh/vPfQJv342G783ULFsdf70hbx4/Vtc/X+XVLpHRHY54aqUC5pPRMreAu/XoDIQ51n88FkW0959CLvTyrALhzLg6D56c1WC6KEYrckyDIMHv7qdnFZZpGW4SMtwYXPYOO2mE7nj/es489YRLPhhEQF/kB0Xj5khk5L8UmZ/Oy91wVdDxAvFdxNemWKS1Sz6E7vdYWBz2Pj8hckRcw1+b4DJb0zHNMNlI8zS8ZibBiEbu2FuPjy8pLEKykjHSP8fRovPIecdxpzxJ2NHvcRvX83lx49/4/7Tn+SFa9+I29erVaaf2LUGbeFPfzNx7BdsXpNH/6N7c9KVw8lsnlHj+zvtsxvvrXmJ+d8voji/hH0P7k6z1jkVr6/5Z33EihIIn5S0fsWmuHwNVZHgKsQzAUIbUY6DwTkMpew1uznwFzs+t536v028+kAbfJ7tnzAcLpNjL/BiGEbUGu8AAV94tZDhfweKxwLlcwyh1bD1esR4BuU4tOL9fl+Ad+79kK9emYavzEe/o3pz4PH9mf/DXxHnr345bgrHjx5G+z12XY+ncPNWvnhxMotnLaXTPh04YfRRtOyQW7PvQxOlE7vWYH3z+nc8d+Vr+D0+RGDFglV8/ep3vDj3UbJzs2rcjsVqoe/hPaO+1n3gHkx+Y3pE4jMsBnv03T2m+HdFvN8hhdcQXl0SQLxTofQ1aP5eeKijOiojXO643PEX5JG/ycYn43IxrEIwoDhsRD4X3BJO1D0P2Zu5U+az87aWPfbrjMVqIPnPU5HUK3iR4icrJfZ7Tn6cedMX4i9/+v950u/89tXcqLuSlVL8MXXBLhP7+n83MnrALfjKfPi9AeZO/ZPPX/iWx6ffrcs97IIeitEaJL8vwAvXvIGvzFeRjPy+AEWbi/joyc/j1s/gU/Ynp1U2Vvv2ZyC708YefXdn7wP2jFs/OxIJIFtvIjyMsi0hlkHw3/I14jVg7QqWdmz7EVcKLrhlAx/8uYinPlvG+/MWcc1j67Cm9QXg8rHn48pwVXydFpsFp9vBVc9fAlIGUhq9n9D24/5W/bWG+TskdQgPW4WCZtRTpQyrgTt71+UaXrz+TUoLSyt+MQT9QTwlXsZe+lLNvg9NVEyJXSl1qlJqkVLKVErpXQda0qxatCbq9YA/yKwv5satH7vDxrOzHmT4xUPJzs2kedscTrnuOB765vbETfwFFxN9HbgXvF/WqAmlFCpnHFjag0oDwitbXG6TTnt5Sc8iXA454zoAOnZvzysLxzLi6mPoecjeHHfZkYyb/wTd+ncN36+qSMCW7TVgVvy5OupkcigYilpLXynFAcfvOm3MnboA04zcHf/v/FX4PJFDZFpYrEMxC4ERgP71qSVVZvMMgoHoE4LZrWo+DFOjvpplcOVzF3PlcxfHtd2quUCqWIap0mrcirK2hxZTILAAzHyEAJS9C8E1YO+LSh+NsnaueH9u++Zc8sg5ke0oA0m/Eoofp3KZACcq4/qKv7Xt2hozyvJRm8PG/sftx+9f/1Fx+pPFauG+z26p9nAQZ5qj0tj8NharoU+E2oWYvjMishjQS5a0pGvVMZc99tudv39bRiiw/enWkebglGuPTWFksZHQJgQrGLlgrgF2fFp1odxn1ao9pRTYe4X/G8B5ZO1jMguh7BMqJ3UF6aPDk7rl9ui7O516dGD5HysI+Lf/0rXZrfzvqQvIePMK/pz5Nza7lX0GdatRYj72siOZ8PhnlYZ3bA4rB596YL1eappqeoxda7DGTLyRPfp0xpFmx52VhsNl59wxpzLwmP1SHVqtSWgjZt6ZyObDIO9EkGJQmeEhkG1DKa6TwTEsuXGJIHmnQWjRzq9A6QuIuX3nqlKKh7+5nYNO3h+r3YphMejWvwtPfH8PzVtbsDuh35G96HXoPjV+2j7r9hEMGNYHu8tOWqYLR5qdbv27cuVzF8Xxq2x8qq3uqJSaCkTbLna7iEwqf8/3wA0iUmXJRqXUKGAUQIcOHfZbtWpVXWPWmpCCjYW8ett4fp70O3anneGXDOXMW0+qVHBszT//UbBxK116d8KdWfOhivpCRJAtwyG0kspj607IvBOlbGDbD2XdLS59rf93Y3l1x5bVftoW/+9I/nlAtGEvOyr7MZTz6IhXQsEQoWAIm5qDFN0BoQ2ACi/ZzLwHZdSuxv3apetZuXA17bq2pvO+Tbe2e9yqO4rI4fEISETGAeMgXLY3Hm1qjZunxMP/+t9MwYathILhhDfh0Un8PWspD351e8X7duvWjt26tUtVmLELzAdzPZETpkEI/oPKvCMu3Sz7YwX3nfYEeesLQKBF++bcOeE6uvTqVPVNwVVUHg7aUQjM6EXELFYLhvyL5F1GpWWS3m8QswDV7NVaxd5+jzbVrnfXttNDMVrSiAhL5iznp09/Y9Oa6qv8TX17BsX5pRVJHcDn8bNgxl8sn78ygZEmmbmR6D+KQQitjUsXpUVl3HDY3axbvhFfmR+fx89/S9dzw5C78ZRsT7ybVm/mrhMf4WjnmRybfg5jr1qC17OL5z/HoCpfkrJX2V4rZhs/+H9DgtFXNWnxEdPkqVLqJOBZIBf4Uik1T0SOiktkWqNSsGkrtx51P/8tW49hMQj6gxx+zsFc/eKoKo9oW/jzP1F3fSrDYPm8lXTp1YltQ4kNegLfti9ItGMFXXE7xHvGh79U+gW5TTAQZMZHv3LU+UMoK/YwesCtFOUVY4ZMgv4gU975k+Vzu/HMlwtRaqcVL+5Ldl3UK7gciLK6R9kh9B/EYWgpWTat2cIPE37B7/Uz8Ji+dO3dufqbUiimJ3YR+URE2ouIQ0Ra6aSuVeWhs59m5V9r8Jb6KCvy4PcGmDb+R756eVqV93Ts3g67M/LwDgVkt8zk8Quf55i0sxlmP4ObjriXtUvWJfArSBxlaQuuk6DSjlIbGDkoV3xq3uetK8BXFll33u/xk7euAIBp78zAW+qtdOBI0B9k9VInC+efBKQBFjDaQ/bzGOVr4Ktk2y/8dexMfGDdo+5fTJJNGz+TC/a6mtfvGM9bd0/gmkF38H/XvE4qTp+rKT0UoyVcUV4xC2currQsEcL1wD999qsq7zv6oqERqycsVgu5HVrw5pgJfPfejwR8AcyQybzvFnLlAbdRlF+ckK8h0VTmPaiMu8DaHSy7QdpIVItPUEZ6XNrf+4A9cbgjy+/a0xwVO2iXzVsZdc24iLBm1WEYredhtF6M0fI7DOcR1fap3OeDclI5zbjAdTLKUv9LHgMUF5Tw5MUv4vf4K+rm+Dx+vn5lGgt//DvV4VVJJ3Yt4byl3oqNKTsrK6r64IqcVtk88f097N6rIxabBavNwn5H9mTUo+eyevHaSmeTiggBb4BvXpse9/hrQiSIeKchJS8h3qmIRN88VRWlFCrtZIwWkzByp2Fk3owycqq/sYZ6H9YjvDTUtb2ImMNlp1u/LvQ6dB8Adu/ZAWdaZPI3DIPd9qr9odfK0hrV/GNwHB5eumm0h4zrUZlj6v6FJNnvX/+BxRb5b9fn8fHdez+mIKKa0Vu3tITL3a0FGc3S8f2XX+m6xWapdkt51z6deemPxykpLMVis+ByO5n85vdR3+vz+Pk3BZOqYuYjeWeAuRnEG35KNZpBswn15snUMAwennwnnz77NZPfmI5SiiPPP5QTrji6Yn7i8HMO5u17PsTv9Vds47farbTp0oqeB+9dp36VtSMq57m4fR1JV+XcjYpa/6a+0IldSzilFDe+Ppq7TnyUoD9IKBjC4bKTnuPmnDtPqVEb6TsUi+q4d/uo73Gk2enaJ36TWutXbAyvzCkoYcDRfel7+L5RJ3ql6IHy1SvlT+lSCiEfUnQPKueZuMUTK7vDxmk3HM9pNxwf9XV3lptnf32IZ0e/wpypC7BYDQ457UBGP31hw56cjsGAo/tgRjnv1u6yM/SswSmIqGaq3aCUCP369ZPZs6vcy6Q1UmuXrGPS89+wbtkGeg3Zh+EXH14pYdeUiHDt4DtZMudfAr7wahJlKDJy0nljyTNk5MQ+Lj1z4q88MvJZQqEQQX8Ip9tJr0P35p5Pb8JiqbyV3dywL5HL+gCsqFaLGmRSbBSrjeLkhw9/4dHzw586zGAIw2rhxCuGcckj5yY9lppuUNKJXWuQPCUeXr7pHaa+M4OAL8h+R/Zk9DMX0qZzq5jb9pb5OLXVRRETiU63g+tevpwhZ1Reu21u6AFErjhpyIldqyx/QwEzJ87C7/Ez4Ji+dOwe/VNjosVt56mm1UeudBdX/d8lXLXTeZzxsPDHv6NO9npLfUx7d0ZEYscxFHxTiNh2b7QBKQIV32qTWvI1a53DCaOTW6cnFnpVjKbtxGqrumqg3Rl5NJ3KvAOMlkT8OJkbkLxTENF1w7Xk0old03bS46C9sERJ7k63g2EXHhZxXVlyIfspYOd7AuGVMt6q1+prWiLoxN4IlW4tZdxNb3NWx8sY2fUKxj/0MQF/tC3rqSXiR6TqdeypYrVZuW/SLaRlunBlOLG77NidNoZfPJT+w3pHvUcF/yYysQNShvjjd6KTptWEHmNvZAL+AFcdeDvr/91YsYFn/P0TmffdQh6ZfGe9mMgTswjZelf5uLSJWLuhsh5A2fZJdWgV9jmwG+//N45fPptNaWEpfQ7vuevqgpa2oCxRCiE6wjtJNS2JdGJvZH765Dc2r8mrtCvT5/Gz+Ncl/P3bMroPTG2NDhFB8i+A4N9UHNQc/AvJPwdafIOyxL6qJV5cbieHnXlQzd5sHwQqO7xBacfyu8qKcp2ckPg0rSp6KKaRWfTzP3hKvBHXzZDJP78vS0FEOwkuhOAyKpL6NhJAyt5PSUjxoJQF1Xw82PoQLnxlB8vuqGZv1Zvdp1rToZ/YG5k2XVrhcNnxeSqvq7bYrLTcrUWKotpBcDXRnyf8EFyS7GjiSlnaoJqPR8wCkADK0jLVISVd/oYCJo79knnTF9K2SytOuf54uvXrkuqwmhz9xN7IHH72wRErOgxDkZbhZMDwPimKage2bkSeFATgBFv0icmGRhk5TTKpb1qzhUv2vY5Pnv6SJbOX88OEX7j+0Lv48ZNZqQ6tydGJvZHJbJ7BE9PvoUP39tgcNmx2K936d2XszPtqfIBwoqxYuJop766lqLgHsGMVQQOUC5V2WqpC0+Lg7bsnUFJYRsAfnt8REXxlfp6+/GVMM8qBG1rC6KGYRqhrn868umgseesLsFgNsnNTu/MxGAhy76lPMHfKApShsNkVI29qx7HnFWIoH9gPQWXehDL0Ds2GbPaUBZUO6djGU+Jl0+ottO7U9D7FpIpO7I1Y8zbxq+cdi4+e+Jy5UxZUjPt7S+HFO7P4eeqBPDrlrhRHp8XCU+pl0U//4Eyzk9HMzZa1eRHvMUMh3FlpKYiu6dKJXUu4L8dNiZjMDQVN/py5mNKtpbizal/hUUu9qe/8wFOXvYzVasEUE8MwsDtt+L3bVzzZ7Fb6HdU7LhU3tZrTiV1LOJ83+q5XpcDvC6LTesOzctEanrp0HD6Pv1LBYofLjs1hxe60E/AH2Xv/Pbn5zSvi1q+YpRD8K3werLVr3NptbHRi1xLuoJMG8PUr0wjudOZp686tyGmpx9WTRUJbwD8DsIHjUJSRUee2vn51WsUk6Y4sVgs3vPw/mrXOIbd9c9rsHr8NZ2bpm1D8BCgrSBCxdkbljKtXm9rqC53YtYQbefdpzPpqLkVbivGW+rA5bFhtFm56Y3SqQ2twCjZt5YcPfqa4oIS+h/dk7wP2rFGZCLP0XSh+OFz2AAViQvZYlDOyqFlNbN1cFHWiVEQIBcw6H6VXFfH9DMVPAt7tZRuCS5CCS1EtPo1rX42BTuxawmXnZvHKwrFMe2cmC39cTPtubTn6oqH1ZnK3oZg9eT73jHgM0zQJ+IJMeGwS/Yf14Y4Pro16ZN82ElwOxY8Avkq1bKTwGmg5s06rkQ44vj8/ffpbxGEkoWCIXofGN6kDSOmbwM4F40IQ/BcJLkdZ9SaoHcW0jl0p9ZhS6m+l1AKl1CdKqex4BaY1Li63k2MvPYJb3r6Kc+44RSf1Wgr4A9x/+pN4y3z4vQFEBG+pj9+/+YOZH/26y3vF8zkRh4AAYIBvWp3iOeikAXTt0xmne/t+BKfbwSk3HE+LdgkooWBGrrYBwsMyZmH8+2vgYt2gNAXoISI9gSXArbGHpGnazv76eQnRjrH0lvr49s3vd33zzoXJKpgg0Y70q57FauHRqXdxxbMXsd+RvRh88v7c88lNXHDvGXVqr1rOQ6m8qa2chMDaPebmRfyIuTXq97ghimkoRkQm7/DXX4GaHTmvaVrt7GIYvboxduU8Ail7j8ihDAHHIXUOyWa3cdT5Qzjq/CF1bqOmVNpIxDMRQlvYfnC4CzJuQhl1XyMv4kWK7gXPZ4AJltaQeS/KUcOqnvVUPEsKXAh8Hcf2NE0rt8+B3aKew+p0Oxh2QTWJ1dYXXMcCLsK/IQzACelXoiy7qDFfQyIBJLgsvOomQZSRiWo+CdIvB1svcByOavYyhvvsmNqVwuvA8znhw8iDEFqLFPwPCfwVl7hTRVX30UMpNRVoHeWl20VkUvl7bgf6ASOkigaVUqOAUQAdOnTYb9WqVbHErWlNzrzpC7nz+IcBCPiCWO1WDhoxkJvfvKLap3YRgcBsxPMVKAfKdTzKFvskp1k2CYrvA4IgQbAPQGU/iTLq/3SbhDYgm48Adj6T1gDn0RjZY1MR1i4ppeaISL9q3xfrmJJS6jzgMmCoiJTV5J5+/frJ7NmzY+pX05qi4oISZn70K0X5Jex3RE/26Lt7ymIR/5zwoSnsWP/fBrbeGM3fTVVYNSb+uUjBJSDFkS9au2O0mJT8oKpR08Qe0xi7UmoYcDNwSE2TuqZpdZeRk87wSw5PdRgASOmrRD7tBiCwAAmuQVnr+ZGA1t1Bdo4fwFp+YErDFesY+3NABjBFKTVPKfViHGLSNK0hCK0nyiGvoGxgbk56OLWljGxIO5Pw3EPFVVBOlPviVIUVF7GuitHFGjStqXIcWH7q1c7HHAbBumdKQqotlXEbYukIZa+BuRXs/VEZN6Ks7VMdWkz0zlNN0+pEpV2AlE0EKWL7BigXpF+GMhpGNUelFMp9DrjPSXUocaUTu6ZpdaIsLaDFZ0jJi+HiYkZzlPsilPPIVIfW5OnErjUJIlKjYll1a9sPvhnhbe/2/ihr6laqJJuytERl6cNS6hud2LVGTXy/IMX3Q3AZorLAfSHKPQql4rM3TwJLkfxzCRfYMgETcR2LynwwYb9INK06+jBrrdES/3yk4FIILgUEpBBKXkCKH49P+yJI4eUg+SClhLfs+8D7FXi/iEsfmlYXOrFrjZaUPEvlzTMAHih7BzHjsO0iuDT6sj7xlNdm0bTU0Ilda7yCS6NfVwaYG+PQgZ8qf4Rk518ompY8OrFrjZd1j+jXpbyKX8zt7wXYorzgBNfxsbevaXWkE7vWaKmMqwDnTletoKzIxgGYeacg/rrXLFLKisp+gvDOxfIEr9LAticq7cw6t6tpsdKJXWu0lK0nKufl8idrg4qt41IC+MI1TfIvRPzz6/qu4TgAAAYeSURBVN6HYzAq92twXwau01FZD6OavY9SUQ6F0LQk0csdtUZNOQaiHJ+FT8jZNDC83b0SL1LyFKrZ63Xvw9IWlXFlbIFqWhzpJ3ataTC3lK8zjyL4T3Jj0bQE04ldSyoxS5GopVITzNjFAcuWjsmLQ9OSQCd2LSkksBhzy0nIpn7Ixj6YBZchZn7S+lfKAWnnULlEK4ATlX5V0uLQtGTQiV1LOAnlIflnQ3AREAKC4JuB5J+b1FPhVcb1kH4xqHTAAKMdZD2BchyQtBg0LRn05KmWcOL5CGSnmt0EIfQfBGaDvX9S4lDKQKVfibivAAIoZU9Kv5qWbPqJXUu84DIij1ArF1qT1FCgvAa3TupaI6YTu5Z4tt5Ejm0TXqVi3Svp4WhaY6cTu5ZwynUCGBmAZYerDrDvh7LtnaqwGg2JWJuvNXU6sWsJp4x0VPOPwXkcqEwwWoTroue8lOrQGjTT8y3mpiHIxr0xNw7ELH0tqZPRWv2lJ0+1pFCWlqjsR1MdRqMhvu9h641UlCWWAih+GpEgKn1UKkPT6gH9xK5pDZAUP0XUWvOlLyESSkVIWj2iE7umNUSh1dGvixekOLmxaPVOTIldKXWfUmqBUmqeUmqyUqptvALTNG0XrF2iX1dpoDKSG4tW78T6xP6YiPQUkd7AF4A+rlzTkkCl/397dxMaVxmFcfz/VEuQVlGoIjahdlGLpQrCUC1SxE+iSIuKoIItKIhgRVFR6yCuhIKgGwsiKG7iF6i4yKK2IHSj4lj8SEmrRRRTFSOCCi6k9LiYKUzjpDN37mTee+88PwjkTW4mz2GSw/2aOY/x//eaPwtWPox0RqcfsRGSq7FHxF9tyxWAL8mbDYHGNqPz9sCZlwDLYdlFcM6zLFuxPXU0K4Dcd8VIeh7YDvwJXJs7kZn1RGNb0NiW1DGsgLrusUvaL2mmw8c2gIioR8QEMAXsPM3jPCCpIakxP99hsruZmQ2EBvWCBklrgOmI2Nht21qtFo1G/7MmzcxGkaQvIqLWbbu8d8W0j4HfChzO83hmZpZf3nPsuyWtB04APwIP5o9kZmZ55GrsEXHHoIKYmdlg+JWnZmYVM7CLp5l+qTRP89RNr1YBvy9RnGFxDcXgGorBNfRnTUSc322jJI09K0mNXq4EF5lrKAbXUAyuYWn5VIyZWcW4sZuZVUxZGvurqQMMgGsoBtdQDK5hCZXiHLuZmfWuLHvsZmbWo9I1dklPSApJq1JnyaoKg0kkvSDpcKuODySdmzpTVpLulHRI0glJhbyrYTGSJiUdkXRU0tOp82Ql6XVJv0maSZ2lX5ImJH0sabb1d/RI6kwLlaqxS5oAbgQWmQtWeFUYTLIP2BgRlwPfArsS5+nHDHA7cCB1kCzUnKCxB7gZ2ADcLWlD2lSZvQFMpg6R03Hg8Yi4FLgKeKhoz0OpGjvwEvAkJR3oUYXBJBHxUUQcby0/BcZT5ulHRMxGxJHUOfqwCTgaEd9HxL/A28C2xJkyiYgDwB+pc+QREb9ExMHW538Ds8DqtKlOlXvQxrBI2goci4ivJKWO07eKDSa5D3gndYgRshr4qW09B1yZKIsBki4GrgA+S5vkVIVq7JL2Axd2+FYdeAa4abiJsjtdDRHxYUTUgbqkXTQHkzw31IA96FZDa5s6zUPSqWFm61UvNZRQpz2a0h31VYWklcB7wKMLjsaTK1Rjj4gbOn1d0mXAWuDk3vo4cFDSpoj4dYgRu1qshg7eBKYpYGPvVoOkHcCtwPVR0PtlMzwPZTIHTLStx4GfE2UZaZKW02zqUxHxfuo8CxWqsS8mIr4BLji5lvQDUIuIUr2JkKR1EfFda1nKwSSSJoGngGsi4p/UeUbM58A6SWuBY8BdwD1pI40eNfcuXwNmI+LF1Hk6KdvF07Lb3ZoX+zXN00qFu02qBy8DZwP7WrdtvpI6UFaSbpM0B2wGpiXtTZ2pF62L1juBvTQv2L0bEYfSpspG0lvAJ8B6SXOS7k+dqQ9XA/cC17X+B76UdEvqUO38ylMzs4rxHruZWcW4sZuZVYwbu5lZxbixm5lVjBu7mVnFuLGbmVWMG7uZWcW4sZuZVcx/ZvGKu3PVk1UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x364aeb0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nb_features=10\n",
    "model = Perceptron(nb_features=nb_features)\n",
    "\n",
    "# use seperation=1 for non-seperable \n",
    "# use seperation=2 for seperable \n",
    "\n",
    "x, y = generate_data_perceptron(nb_features=nb_features, seperation=1)\n",
    "# plot first two dimensions\n",
    "x1 = []\n",
    "x2 = []\n",
    "for i in x:\n",
    "    x1.append(i[1])\n",
    "    x2.append(i[2])\n",
    "plt.scatter(x1, x2, c=y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Algorithm didnot converge\n",
      "Accuracy on training data is 77.0\n",
      "77\n"
     ]
    }
   ],
   "source": [
    "converged = model.train(x, y)\n",
    "if (converged):\n",
    "    print ('Algorithm has converged')\n",
    "else:\n",
    "    print ('Algorithm didnot converge')\n",
    "    \n",
    "y_hat = model.predict(x) \n",
    "correct = 0 \n",
    "for i, y_real in enumerate(y):\n",
    "    if (y_hat[i]==y_real):\n",
    "        correct = correct + 1\n",
    "    \n",
    "print (\"Accuracy on training data is {}\".format(correct*100/len(y)))\n",
    "print (correct)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}