ML_Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine Learning\n",
    "## Classification Problem\n",
    "\n",
    "**Unexpected challenge emerged when it came to splitting the data to train+test and evaluation sets**\n",
    "\n",
    "![Hysteresis](https://github.com/asyaparfenova/air_condidioner_effectiveness_cost_carbon_footprint/blob/main/images/hysteresis.png?raw=true)\n",
    "\n",
    "The initial idea was to train the model on the temperature data for the time periods, when the air conditioner was “off” and then to re-construct the “uncooled” temperature, when it was on. But here comes the hysteresis problem: it takes time to cool down the room and, more importantly, the room stays cooler for some time after switching off the cooling and that\n",
    "time can not be used for training.\n",
    "\n",
    "![Cutting Tail](https://github.com/asyaparfenova/air_condidioner_effectiveness_cost_carbon_footprint/blob/main/images/classification.png?raw=truee)\n",
    "\n",
    "To estimate hysteresis lag regression machine learning was used, as the time after changing the state the air conditioner, when our model can not properly predict the right state or, in\u000b",
    "other worlds, to recognize that the state was changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.preprocessing import MinMaxScaler, OneHotEncoder, FunctionTransformer, PolynomialFeatures, KBinsDiscretizer\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.utils import resample\n",
    "\n",
    "import statsmodels.discrete.discrete_model as sm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be working with july subset for speed and simplicity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('data/july.csv', index_col='Unnamed: 0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let's start with the splitting**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['sensor.ac_power', 'sensor.atmospheric_pressure',\n",
       "       'sensor.bathroom_humidity', 'sensor.bathroom_temperature',\n",
       "       'sensor.equipment_temperature', 'sensor.kitchen_humidity',\n",
       "       'sensor.kitchen_temperature', 'sensor.office_co2',\n",
       "       'sensor.room_humidity', 'sensor.room_temperature',\n",
       "       'sensor.weather_humidity', 'sensor.weather_temperature',\n",
       "       'binary_sensor.balcony_door', 'binary_sensor.kitchen_window',\n",
       "       'binary_sensor.room_window', 'climate.air_conditioner',\n",
       "       'climate.air_conditioner_old', 'cover.balcony', 'cover.windows',\n",
       "       'person.one', 'person.two'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sensor.ac_power                     0\n",
       "sensor.atmospheric_pressure         0\n",
       "sensor.bathroom_humidity            0\n",
       "sensor.bathroom_temperature         0\n",
       "sensor.equipment_temperature        0\n",
       "sensor.kitchen_humidity             0\n",
       "sensor.kitchen_temperature          0\n",
       "sensor.office_co2                1192\n",
       "sensor.room_humidity                0\n",
       "sensor.room_temperature             0\n",
       "sensor.weather_humidity             0\n",
       "sensor.weather_temperature          0\n",
       "binary_sensor.balcony_door          0\n",
       "binary_sensor.kitchen_window        0\n",
       "binary_sensor.room_window           0\n",
       "climate.air_conditioner             0\n",
       "climate.air_conditioner_old     44640\n",
       "cover.balcony                       0\n",
       "cover.windows                       0\n",
       "person.one                          0\n",
       "person.two                          0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- We can drop *'climate.air_conditioner_old'* and *'sensor.office_co2'* columns, because CO2 level is not important for our modelling and old air conditioner dataset is empty for July.\n",
    "- Also 'sensor.ac_power' is not important for our nearest goals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.drop(['sensor.office_co2'], axis=1, inplace=True)\n",
    "df.drop(['climate.air_conditioner_old'], axis=1, inplace=True)\n",
    "df.drop(['sensor.ac_power'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(44640, 18)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also redefine our air conditioner state into binary values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['climate.air_conditioner'].unique()\n",
    "df['climate.air_conditioner'].replace({'cool':1, 'dry':0, 'fan_only':0, 'off':0}, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And create a feature for time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['time_hour'] = pd.to_datetime(df.index).hour"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let's define our X and y**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop('climate.air_conditioner', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df['climate.air_conditioner']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((44640, 18), (44640,))"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape, y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Train/Test split**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((35712, 18), (35712,))"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape, y_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**X_train:** Feature Engineering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sensor.atmospheric_pressure</th>\n",
       "      <th>sensor.bathroom_humidity</th>\n",
       "      <th>sensor.bathroom_temperature</th>\n",
       "      <th>sensor.equipment_temperature</th>\n",
       "      <th>sensor.kitchen_humidity</th>\n",
       "      <th>sensor.kitchen_temperature</th>\n",
       "      <th>sensor.room_humidity</th>\n",
       "      <th>sensor.room_temperature</th>\n",
       "      <th>sensor.weather_humidity</th>\n",
       "      <th>sensor.weather_temperature</th>\n",
       "      <th>binary_sensor.balcony_door</th>\n",
       "      <th>binary_sensor.kitchen_window</th>\n",
       "      <th>binary_sensor.room_window</th>\n",
       "      <th>cover.balcony</th>\n",
       "      <th>cover.windows</th>\n",
       "      <th>person.one</th>\n",
       "      <th>person.two</th>\n",
       "      <th>time_hour</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-07-31 19:31:00</th>\n",
       "      <td>1014.0</td>\n",
       "      <td>43.0</td>\n",
       "      <td>23.7</td>\n",
       "      <td>28.1</td>\n",
       "      <td>50.3</td>\n",
       "      <td>24.2</td>\n",
       "      <td>38.6</td>\n",
       "      <td>22.1</td>\n",
       "      <td>36.0</td>\n",
       "      <td>22.9</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-29 18:56:00</th>\n",
       "      <td>1014.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>23.8</td>\n",
       "      <td>28.5</td>\n",
       "      <td>44.7</td>\n",
       "      <td>23.2</td>\n",
       "      <td>43.7</td>\n",
       "      <td>22.9</td>\n",
       "      <td>38.0</td>\n",
       "      <td>20.2</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-30 08:10:00</th>\n",
       "      <td>1018.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>23.8</td>\n",
       "      <td>28.9</td>\n",
       "      <td>44.7</td>\n",
       "      <td>22.9</td>\n",
       "      <td>39.9</td>\n",
       "      <td>23.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     sensor.atmospheric_pressure  sensor.bathroom_humidity  \\\n",
       "2020-07-31 19:31:00                       1014.0                      43.0   \n",
       "2020-07-29 18:56:00                       1014.0                      44.0   \n",
       "2020-07-30 08:10:00                       1018.0                      46.0   \n",
       "\n",
       "                     sensor.bathroom_temperature  \\\n",
       "2020-07-31 19:31:00                         23.7   \n",
       "2020-07-29 18:56:00                         23.8   \n",
       "2020-07-30 08:10:00                         23.8   \n",
       "\n",
       "                     sensor.equipment_temperature  sensor.kitchen_humidity  \\\n",
       "2020-07-31 19:31:00                          28.1                     50.3   \n",
       "2020-07-29 18:56:00                          28.5                     44.7   \n",
       "2020-07-30 08:10:00                          28.9                     44.7   \n",
       "\n",
       "                     sensor.kitchen_temperature  sensor.room_humidity  \\\n",
       "2020-07-31 19:31:00                        24.2                  38.6   \n",
       "2020-07-29 18:56:00                        23.2                  43.7   \n",
       "2020-07-30 08:10:00                        22.9                  39.9   \n",
       "\n",
       "                     sensor.room_temperature  sensor.weather_humidity  \\\n",
       "2020-07-31 19:31:00                     22.1                     36.0   \n",
       "2020-07-29 18:56:00                     22.9                     38.0   \n",
       "2020-07-30 08:10:00                     23.0                     67.0   \n",
       "\n",
       "                     sensor.weather_temperature binary_sensor.balcony_door  \\\n",
       "2020-07-31 19:31:00                        22.9                         on   \n",
       "2020-07-29 18:56:00                        20.2                         on   \n",
       "2020-07-30 08:10:00                        18.7                         on   \n",
       "\n",
       "                    binary_sensor.kitchen_window binary_sensor.room_window  \\\n",
       "2020-07-31 19:31:00                          off                       off   \n",
       "2020-07-29 18:56:00                          off                       off   \n",
       "2020-07-30 08:10:00                          off                       off   \n",
       "\n",
       "                    cover.balcony cover.windows person.one person.two  \\\n",
       "2020-07-31 19:31:00          open          open   not_home   not_home   \n",
       "2020-07-29 18:56:00          open          open       home   not_home   \n",
       "2020-07-30 08:10:00          open          open   not_home   not_home   \n",
       "\n",
       "                     time_hour  \n",
       "2020-07-31 19:31:00         19  \n",
       "2020-07-29 18:56:00         18  \n",
       "2020-07-30 08:10:00          8  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define a function for creating a \"outside and inside temperature difference\" feature and normalize it to the range 0.0 - 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def temp_diff(data):\n",
    "    new_data = data['sensor.room_temperature'] - data['sensor.weather_temperature']\n",
    "    return pd.DataFrame(new_data)\n",
    "\n",
    "temp_diff_scaled = make_pipeline(\n",
    "    FunctionTransformer(temp_diff),\n",
    "    MinMaxScaler(),\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define a function for creating a \"outside and inside humidity difference\" feature and normalize it to the range 0.0 - 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def hum_diff(data):\n",
    "    new_data = data['sensor.room_humidity'] - data['sensor.weather_humidity']\n",
    "    return pd.DataFrame(new_data)\n",
    "\n",
    "hum_diff_scaled = make_pipeline(\n",
    "    FunctionTransformer(hum_diff),\n",
    "    MinMaxScaler(),\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's make pipelines for creating \"binned time\" and \"polynomial time\" features and normalize them to the range 0.0 - 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "bin_then_encode = make_pipeline(\n",
    "    KBinsDiscretizer(n_bins=5, encode='onehot-dense', strategy='uniform'),\n",
    "    OneHotEncoder(sparse=False, handle_unknown='ignore')\n",
    ")\n",
    "\n",
    "poly_then_scale = make_pipeline(\n",
    "    PolynomialFeatures(degree=3, include_bias=False),\n",
    "    MinMaxScaler()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "fe = ColumnTransformer([('label_encoder', OneHotEncoder(drop='if_binary'), ['binary_sensor.balcony_door',\n",
    "                                                                            'binary_sensor.kitchen_window',\n",
    "                                                                            #'binary_sensor.room_window',\n",
    "                                                                            #'cover.balcony',\n",
    "                                                                            #'cover.windows',\n",
    "                                                                            'person.one']),\n",
    "                                                                            #'person.two']),\n",
    "                        ('scaler', MinMaxScaler(), [#'sensor.atmospheric_pressure',\n",
    "                                                    #'sensor.room_humidity',\n",
    "                                                    'sensor.room_temperature',\n",
    "                                                    'sensor.weather_humidity',\n",
    "                                                    'sensor.weather_temperature']),\n",
    "                        ('temp_diff_scaled', temp_diff_scaled, ['sensor.room_temperature',\n",
    "                                                                'sensor.weather_temperature']),\n",
    "                        ('hum_diff_scaled', hum_diff_scaled, ['sensor.room_humidity',\n",
    "                                                              'sensor.weather_humidity']),\n",
    "                        ('polynomial_time', poly_then_scale, ['time_hour']),\n",
    "                        #('binned_time', bin_then_encode, ['time_hour']),\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We intentionaly made some collinear and excessive features - will pick up the right working combination later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "fe.fit(X_train)\n",
    "X_train_trans = fe.transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.        , 0.        , 1.        , 0.42647059, 0.03030303,\n",
       "        0.625     , 0.28804348, 0.6200495 , 0.82608696, 0.68241966,\n",
       "        0.56373798],\n",
       "       [1.        , 0.        , 0.        , 0.54411765, 0.06060606,\n",
       "        0.484375  , 0.47826087, 0.65841584, 0.7826087 , 0.61247637,\n",
       "        0.47932933],\n",
       "       [1.        , 0.        , 1.        , 0.55882353, 0.5       ,\n",
       "        0.40625   , 0.56521739, 0.25247525, 0.34782609, 0.12098299,\n",
       "        0.04208104],\n",
       "       [1.        , 0.        , 1.        , 0.44117647, 0.1969697 ,\n",
       "        0.72395833, 0.19021739, 0.59282178, 0.56521739, 0.3194707 ,\n",
       "        0.1805704 ],\n",
       "       [1.        , 1.        , 0.        , 0.66176471, 0.01515152,\n",
       "        0.42708333, 0.58152174, 1.        , 0.2173913 , 0.04725898,\n",
       "        0.01027369]])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_trans[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modelling!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Baseline Model**\n",
    "\n",
    "As the baseline model we would choose the \"always off\" prediction (since AC was much more often off).\n",
    "\n",
    "Let's check what would be it's accuracy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7225582437275986"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "baseline_accuracy = 1 - y_train.sum() / y_train.count()\n",
    "baseline_accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Selection\n",
    "\n",
    "In our feature transformer we actualy added more features than even planned to do after data explorations.\n",
    "\n",
    "It is because we decided to double check feature importance. We'll do it by running Logistic Regression model with *statsmodels* library and removing features with high p-values.\n",
    "\n",
    "**LogReg | Statsmodels**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.188453\n",
      "         Iterations 9\n",
      "x1      3.759355\n",
      "x2     -3.278611\n",
      "x3      0.966681\n",
      "x4     -9.936939\n",
      "x5     -3.925014\n",
      "x6     12.661729\n",
      "x7     -1.785244\n",
      "x8     -5.302263\n",
      "x9    -14.063536\n",
      "x10    38.276029\n",
      "x11   -25.824555\n",
      "dtype: float64\n",
      "                              Logit Regression Results                             \n",
      "===================================================================================\n",
      "Dep. Variable:     climate.air_conditioner   No. Observations:                35712\n",
      "Model:                               Logit   Df Residuals:                    35701\n",
      "Method:                                MLE   Df Model:                           10\n",
      "Date:                     Thu, 21 Jan 2021   Pseudo R-squ.:                  0.6809\n",
      "Time:                             20:45:33   Log-Likelihood:                -6730.0\n",
      "converged:                            True   LL-Null:                       -21089.\n",
      "Covariance Type:                 nonrobust   LLR p-value:                     0.000\n",
      "==============================================================================\n",
      "                 coef    std err          z      P>|z|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "x1             3.7594      0.091     41.334      0.000       3.581       3.938\n",
      "x2            -3.2786      0.122    -26.913      0.000      -3.517      -3.040\n",
      "x3             0.9667      0.050     19.269      0.000       0.868       1.065\n",
      "x4            -9.9369      0.257    -38.734      0.000     -10.440      -9.434\n",
      "x5            -3.9250      0.168    -23.343      0.000      -4.255      -3.595\n",
      "x6            12.6617      0.400     31.621      0.000      11.877      13.447\n",
      "x7            -1.7852      0.312     -5.714      0.000      -2.398      -1.173\n",
      "x8            -5.3023      0.208    -25.463      0.000      -5.710      -4.894\n",
      "x9           -14.0635      0.848    -16.588      0.000     -15.725     -12.402\n",
      "x10           38.2760      1.895     20.199      0.000      34.562      41.990\n",
      "x11          -25.8246      1.285    -20.103      0.000     -28.342     -23.307\n",
      "==============================================================================\n"
     ]
    }
   ],
   "source": [
    "logit = sm.Logit(y_train, X_train_trans)\n",
    "f = logit.fit()\n",
    "print(f.params)\n",
    "print(f.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These features did have p-value higher than 0.1, therefore were excluded (commented out) from *Column Transformer* and next modellings:\n",
    "- 'binary_sensor.room_window',\n",
    "- 'cover.balcony',\n",
    "- 'cover.windows',\n",
    "- 'sensor.atmospheric_pressure'\n",
    "- 'person.two'\n",
    "- 'sensor.room_humidity'\n",
    "\n",
    "On the same reason we've chosen 'polynomial_time' over 'binned_time'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SciKit-Learn Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**LogReg**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(max_iter=1000)"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_lr = LogisticRegression(max_iter=1000)\n",
    "model_lr.fit(X_train_trans, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9097782258064516"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_lr.score(X_train_trans, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*LogReg/Cross-Validation*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation scores: [0.91334173 0.90550189 0.91038925 0.90422851 0.91094931] \n",
      "Avg. cross-validations score: 0.91\n"
     ]
    }
   ],
   "source": [
    "accuracy = cross_val_score(model_lr, X_train_trans, y_train, cv=5, scoring='accuracy')\n",
    "print(\"Cross-validation scores:\", accuracy, \"\\nAvg. cross-validations score:\", round(np.mean(accuracy), 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CONTINUE FROM HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Decision Tree**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.967"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_ds = DecisionTreeClassifier(max_depth=8)\n",
    "model_ds.fit(X_train_experimental, y_train)\n",
    "round(model_ds.score(X_train_experimental, y_train), 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation scores: [0.96598068 0.96472071 0.96513582 0.95701484 0.96261551]\n"
     ]
    }
   ],
   "source": [
    "accuracy = cross_val_score(model_ds, X_train_experimental, y_train, cv=5, scoring='accuracy')\n",
    "print(\"Cross-validation scores:\", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.995"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_rf = RandomForestClassifier(max_depth=15, n_estimators=50, random_state=42)\n",
    "model_rf.fit(X_train_experimental, y_train)\n",
    "round(model_rf.score(X_train_experimental, y_train), 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation scores: [0.99272015 0.99160017 0.99313918 0.99187903 0.99355923]\n"
     ]
    }
   ],
   "source": [
    "accuracy = cross_val_score(model_rf, X_train_experimental, y_train, cv=5, scoring='accuracy')\n",
    "print(\"Cross-validation scores:\", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### SVC - support vector classificator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.951"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_svc = SVC()\n",
    "model_svc.fit(X_train_experimental, y_train)\n",
    "round(model_svc.score(X_train_experimental, y_train), 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation scores: [0.95086098 0.94876102 0.95309437 0.94679362 0.95057407]\n"
     ]
    }
   ],
   "source": [
    "accuracy = cross_val_score(model_svc, X_train_experimental, y_train, cv=5, scoring='accuracy')\n",
    "print(\"Cross-validation scores:\", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Gradient Boosting Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.994"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_gb = GradientBoostingClassifier(max_depth=4, learning_rate=0.6)\n",
    "model_gb.fit(X_train_experimental, y_train)\n",
    "round(model_gb.score(X_train_experimental, y_train), 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation scores: [0.99174017 0.99048019 0.99075889 0.99103892 0.99271913]\n"
     ]
    }
   ],
   "source": [
    "accuracy = cross_val_score(model_gb, X_train_experimental, y_train, cv=5, scoring='accuracy')\n",
    "print(\"Cross-validation scores:\", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, RandomForest shown the best results (GradientBoosting is also promising, but need more hyperparameters optimisations).\n",
    "\n",
    "Let's say we chosen RandomForest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "#transforming y_test\n",
    "y_test_trans = y_test.replace({'cool':1, 'dry':0, 'fan_only':0, 'off':0}, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "#transforming X_test\n",
    "X_test_trans = fe_experimental.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.993"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(model_rf.score(X_test_trans, y_test), 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# WHAT'S NEXT?\n",
    "\n",
    "- build a column with the data about how long ago AC was turned on/off\n",
    "- look on dependancy of prediction probability (LogReg) on this delay\n",
    "- use this information on Regression problem\n",
    "- split the data for Regression model\n",
    "- build and train Regression model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color='orange'>Let's catch the tail</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sensor.ac_power</th>\n",
       "      <th>sensor.atmospheric_pressure</th>\n",
       "      <th>sensor.bathroom_humidity</th>\n",
       "      <th>sensor.bathroom_temperature</th>\n",
       "      <th>sensor.equipment_temperature</th>\n",
       "      <th>sensor.kitchen_humidity</th>\n",
       "      <th>sensor.kitchen_temperature</th>\n",
       "      <th>sensor.office_co2</th>\n",
       "      <th>sensor.room_humidity</th>\n",
       "      <th>sensor.room_temperature</th>\n",
       "      <th>sensor.weather_humidity</th>\n",
       "      <th>sensor.weather_temperature</th>\n",
       "      <th>binary_sensor.balcony_door</th>\n",
       "      <th>binary_sensor.kitchen_window</th>\n",
       "      <th>binary_sensor.room_window</th>\n",
       "      <th>climate.air_conditioner</th>\n",
       "      <th>cover.balcony</th>\n",
       "      <th>cover.windows</th>\n",
       "      <th>person.one</th>\n",
       "      <th>person.two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>52.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:01:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>52.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:02:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>52.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:03:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>52.0</td>\n",
       "      <td>18.9</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:04:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>52.0</td>\n",
       "      <td>18.9</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     sensor.ac_power  sensor.atmospheric_pressure  \\\n",
       "2020-07-01 00:00:00              1.0                       1004.0   \n",
       "2020-07-01 00:01:00              1.0                       1004.0   \n",
       "2020-07-01 00:02:00              1.0                       1004.0   \n",
       "2020-07-01 00:03:00              1.0                       1004.0   \n",
       "2020-07-01 00:04:00              1.0                       1004.0   \n",
       "\n",
       "                     sensor.bathroom_humidity  sensor.bathroom_temperature  \\\n",
       "2020-07-01 00:00:00                      42.0                         24.5   \n",
       "2020-07-01 00:01:00                      42.0                         24.5   \n",
       "2020-07-01 00:02:00                      42.0                         24.5   \n",
       "2020-07-01 00:03:00                      42.0                         24.5   \n",
       "2020-07-01 00:04:00                      42.0                         24.5   \n",
       "\n",
       "                     sensor.equipment_temperature  sensor.kitchen_humidity  \\\n",
       "2020-07-01 00:00:00                          29.7                     39.5   \n",
       "2020-07-01 00:01:00                          29.7                     39.5   \n",
       "2020-07-01 00:02:00                          29.7                     39.5   \n",
       "2020-07-01 00:03:00                          29.7                     39.5   \n",
       "2020-07-01 00:04:00                          29.7                     39.5   \n",
       "\n",
       "                     sensor.kitchen_temperature  sensor.office_co2  \\\n",
       "2020-07-01 00:00:00                        23.8                NaN   \n",
       "2020-07-01 00:01:00                        23.8                NaN   \n",
       "2020-07-01 00:02:00                        23.8                NaN   \n",
       "2020-07-01 00:03:00                        23.8                NaN   \n",
       "2020-07-01 00:04:00                        23.8                NaN   \n",
       "\n",
       "                     sensor.room_humidity  sensor.room_temperature  \\\n",
       "2020-07-01 00:00:00                  38.3                     22.6   \n",
       "2020-07-01 00:01:00                  38.3                     22.6   \n",
       "2020-07-01 00:02:00                  38.3                     22.6   \n",
       "2020-07-01 00:03:00                  38.3                     22.6   \n",
       "2020-07-01 00:04:00                  38.3                     22.6   \n",
       "\n",
       "                     sensor.weather_humidity  sensor.weather_temperature  \\\n",
       "2020-07-01 00:00:00                     52.0                        18.7   \n",
       "2020-07-01 00:01:00                     52.0                        18.7   \n",
       "2020-07-01 00:02:00                     52.0                        18.7   \n",
       "2020-07-01 00:03:00                     52.0                        18.9   \n",
       "2020-07-01 00:04:00                     52.0                        18.9   \n",
       "\n",
       "                    binary_sensor.balcony_door binary_sensor.kitchen_window  \\\n",
       "2020-07-01 00:00:00                         on                          off   \n",
       "2020-07-01 00:01:00                         on                          off   \n",
       "2020-07-01 00:02:00                         on                          off   \n",
       "2020-07-01 00:03:00                         on                          off   \n",
       "2020-07-01 00:04:00                         on                          off   \n",
       "\n",
       "                    binary_sensor.room_window climate.air_conditioner  \\\n",
       "2020-07-01 00:00:00                       off                     off   \n",
       "2020-07-01 00:01:00                       off                     off   \n",
       "2020-07-01 00:02:00                       off                     off   \n",
       "2020-07-01 00:03:00                       off                     off   \n",
       "2020-07-01 00:04:00                       off                     off   \n",
       "\n",
       "                    cover.balcony cover.windows person.one person.two  \n",
       "2020-07-01 00:00:00          open          open   not_home   not_home  \n",
       "2020-07-01 00:01:00          open          open   not_home   not_home  \n",
       "2020-07-01 00:02:00          open          open   not_home   not_home  \n",
       "2020-07-01 00:03:00          open          open   not_home   not_home  \n",
       "2020-07-01 00:04:00          open          open   not_home   not_home  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['off', 'cool', 'fan_only', 'dry'], dtype=object)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['climate.air_conditioner'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['ac_on'] = df['climate.air_conditioner'].replace({'cool':1, 'dry':0, 'fan_only':0, 'off':0})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sensor.ac_power</th>\n",
       "      <th>sensor.atmospheric_pressure</th>\n",
       "      <th>sensor.bathroom_humidity</th>\n",
       "      <th>sensor.bathroom_temperature</th>\n",
       "      <th>sensor.equipment_temperature</th>\n",
       "      <th>sensor.kitchen_humidity</th>\n",
       "      <th>sensor.kitchen_temperature</th>\n",
       "      <th>sensor.office_co2</th>\n",
       "      <th>sensor.room_humidity</th>\n",
       "      <th>sensor.room_temperature</th>\n",
       "      <th>...</th>\n",
       "      <th>sensor.weather_temperature</th>\n",
       "      <th>binary_sensor.balcony_door</th>\n",
       "      <th>binary_sensor.kitchen_window</th>\n",
       "      <th>binary_sensor.room_window</th>\n",
       "      <th>climate.air_conditioner</th>\n",
       "      <th>cover.balcony</th>\n",
       "      <th>cover.windows</th>\n",
       "      <th>person.one</th>\n",
       "      <th>person.two</th>\n",
       "      <th>ac_on</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:01:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:02:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>18.7</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:03:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>18.9</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:04:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>18.9</td>\n",
       "      <td>on</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>off</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     sensor.ac_power  sensor.atmospheric_pressure  \\\n",
       "2020-07-01 00:00:00              1.0                       1004.0   \n",
       "2020-07-01 00:01:00              1.0                       1004.0   \n",
       "2020-07-01 00:02:00              1.0                       1004.0   \n",
       "2020-07-01 00:03:00              1.0                       1004.0   \n",
       "2020-07-01 00:04:00              1.0                       1004.0   \n",
       "\n",
       "                     sensor.bathroom_humidity  sensor.bathroom_temperature  \\\n",
       "2020-07-01 00:00:00                      42.0                         24.5   \n",
       "2020-07-01 00:01:00                      42.0                         24.5   \n",
       "2020-07-01 00:02:00                      42.0                         24.5   \n",
       "2020-07-01 00:03:00                      42.0                         24.5   \n",
       "2020-07-01 00:04:00                      42.0                         24.5   \n",
       "\n",
       "                     sensor.equipment_temperature  sensor.kitchen_humidity  \\\n",
       "2020-07-01 00:00:00                          29.7                     39.5   \n",
       "2020-07-01 00:01:00                          29.7                     39.5   \n",
       "2020-07-01 00:02:00                          29.7                     39.5   \n",
       "2020-07-01 00:03:00                          29.7                     39.5   \n",
       "2020-07-01 00:04:00                          29.7                     39.5   \n",
       "\n",
       "                     sensor.kitchen_temperature  sensor.office_co2  \\\n",
       "2020-07-01 00:00:00                        23.8                NaN   \n",
       "2020-07-01 00:01:00                        23.8                NaN   \n",
       "2020-07-01 00:02:00                        23.8                NaN   \n",
       "2020-07-01 00:03:00                        23.8                NaN   \n",
       "2020-07-01 00:04:00                        23.8                NaN   \n",
       "\n",
       "                     sensor.room_humidity  sensor.room_temperature  ...  \\\n",
       "2020-07-01 00:00:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:01:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:02:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:03:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:04:00                  38.3                     22.6  ...   \n",
       "\n",
       "                     sensor.weather_temperature  binary_sensor.balcony_door  \\\n",
       "2020-07-01 00:00:00                        18.7                          on   \n",
       "2020-07-01 00:01:00                        18.7                          on   \n",
       "2020-07-01 00:02:00                        18.7                          on   \n",
       "2020-07-01 00:03:00                        18.9                          on   \n",
       "2020-07-01 00:04:00                        18.9                          on   \n",
       "\n",
       "                    binary_sensor.kitchen_window binary_sensor.room_window  \\\n",
       "2020-07-01 00:00:00                          off                       off   \n",
       "2020-07-01 00:01:00                          off                       off   \n",
       "2020-07-01 00:02:00                          off                       off   \n",
       "2020-07-01 00:03:00                          off                       off   \n",
       "2020-07-01 00:04:00                          off                       off   \n",
       "\n",
       "                    climate.air_conditioner cover.balcony cover.windows  \\\n",
       "2020-07-01 00:00:00                     off          open          open   \n",
       "2020-07-01 00:01:00                     off          open          open   \n",
       "2020-07-01 00:02:00                     off          open          open   \n",
       "2020-07-01 00:03:00                     off          open          open   \n",
       "2020-07-01 00:04:00                     off          open          open   \n",
       "\n",
       "                    person.one person.two ac_on  \n",
       "2020-07-01 00:00:00   not_home   not_home     0  \n",
       "2020-07-01 00:01:00   not_home   not_home     0  \n",
       "2020-07-01 00:02:00   not_home   not_home     0  \n",
       "2020-07-01 00:03:00   not_home   not_home     0  \n",
       "2020-07-01 00:04:00   not_home   not_home     0  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "44640"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df['ac_on'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['ac_on_time'] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-43-3956d6b6bca5>:7: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['ac_on_time'][row] = time_on\n"
     ]
    }
   ],
   "source": [
    "time_on = 0\n",
    "for row in range(len(df['ac_on'])):\n",
    "    if df['ac_on'][row] == 0:\n",
    "        time_on = 0\n",
    "    else:\n",
    "        time_on += 1\n",
    "        df['ac_on_time'][row] = time_on\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ac_on</th>\n",
       "      <th>ac_on_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:20:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:21:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:22:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:23:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:24:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:25:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:26:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:27:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:28:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:29:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:30:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:31:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:32:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:33:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:34:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:35:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:36:00</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:37:00</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:38:00</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:39:00</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     ac_on  ac_on_time\n",
       "2020-07-01 05:20:00      0           0\n",
       "2020-07-01 05:21:00      0           0\n",
       "2020-07-01 05:22:00      0           0\n",
       "2020-07-01 05:23:00      0           0\n",
       "2020-07-01 05:24:00      0           0\n",
       "2020-07-01 05:25:00      0           0\n",
       "2020-07-01 05:26:00      0           0\n",
       "2020-07-01 05:27:00      0           0\n",
       "2020-07-01 05:28:00      0           0\n",
       "2020-07-01 05:29:00      0           0\n",
       "2020-07-01 05:30:00      0           0\n",
       "2020-07-01 05:31:00      0           0\n",
       "2020-07-01 05:32:00      0           0\n",
       "2020-07-01 05:33:00      0           0\n",
       "2020-07-01 05:34:00      0           0\n",
       "2020-07-01 05:35:00      0           0\n",
       "2020-07-01 05:36:00      1           1\n",
       "2020-07-01 05:37:00      1           2\n",
       "2020-07-01 05:38:00      1           3\n",
       "2020-07-01 05:39:00      1           4"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['ac_on', 'ac_on_time']][320:340]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Repeat for 'ac_off'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-45-c1596258e8ed>:9: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['ac_off_time'][row] = time_off\n"
     ]
    }
   ],
   "source": [
    "df['ac_off'] = df['climate.air_conditioner'].replace({'cool':0, 'dry':1, 'fan_only':1, 'off':1})\n",
    "df['ac_off_time'] = 0\n",
    "time_off = 0\n",
    "for row in range(len(df['ac_off'])):\n",
    "    if df['ac_off'][row] == 0:\n",
    "        time_off = 0\n",
    "    else:\n",
    "        time_off += 1\n",
    "        df['ac_off_time'][row] = time_off"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ac_on</th>\n",
       "      <th>ac_on_time</th>\n",
       "      <th>ac_off</th>\n",
       "      <th>ac_off_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:20:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:21:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>322</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:22:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:23:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:24:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:25:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>326</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:26:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>327</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:27:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>328</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:28:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:29:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:30:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:31:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:32:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:33:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:34:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:35:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>336</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:36:00</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:37:00</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:38:00</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 05:39:00</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     ac_on  ac_on_time  ac_off  ac_off_time\n",
       "2020-07-01 05:20:00      0           0       1          321\n",
       "2020-07-01 05:21:00      0           0       1          322\n",
       "2020-07-01 05:22:00      0           0       1          323\n",
       "2020-07-01 05:23:00      0           0       1          324\n",
       "2020-07-01 05:24:00      0           0       1          325\n",
       "2020-07-01 05:25:00      0           0       1          326\n",
       "2020-07-01 05:26:00      0           0       1          327\n",
       "2020-07-01 05:27:00      0           0       1          328\n",
       "2020-07-01 05:28:00      0           0       1          329\n",
       "2020-07-01 05:29:00      0           0       1          330\n",
       "2020-07-01 05:30:00      0           0       1          331\n",
       "2020-07-01 05:31:00      0           0       1          332\n",
       "2020-07-01 05:32:00      0           0       1          333\n",
       "2020-07-01 05:33:00      0           0       1          334\n",
       "2020-07-01 05:34:00      0           0       1          335\n",
       "2020-07-01 05:35:00      0           0       1          336\n",
       "2020-07-01 05:36:00      1           1       0            0\n",
       "2020-07-01 05:37:00      1           2       0            0\n",
       "2020-07-01 05:38:00      1           3       0            0\n",
       "2020-07-01 05:39:00      1           4       0            0"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['ac_on', 'ac_on_time', 'ac_off', 'ac_off_time']][320:340]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "fe_experimental.fit(df)\n",
    "X_total = fe_experimental.transform(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "probs = model_lr.predict_proba(X_total)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((44640, 26), (44640, 2))"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape, probs.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "probs_on = []\n",
    "probs_off = []\n",
    "for row in range(len(probs)):\n",
    "    probs_on.append(probs[row][1])\n",
    "    probs_off.append(probs[row][0])\n",
    "    #df['probs_on'][row].replace(probs[row][0], inplace=True)\n",
    "    #df['probs_off'][row] = probs[row][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['probs_on'] = probs_on\n",
    "df['probs_off'] = probs_off"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sensor.ac_power</th>\n",
       "      <th>sensor.atmospheric_pressure</th>\n",
       "      <th>sensor.bathroom_humidity</th>\n",
       "      <th>sensor.bathroom_temperature</th>\n",
       "      <th>sensor.equipment_temperature</th>\n",
       "      <th>sensor.kitchen_humidity</th>\n",
       "      <th>sensor.kitchen_temperature</th>\n",
       "      <th>sensor.office_co2</th>\n",
       "      <th>sensor.room_humidity</th>\n",
       "      <th>sensor.room_temperature</th>\n",
       "      <th>...</th>\n",
       "      <th>cover.balcony</th>\n",
       "      <th>cover.windows</th>\n",
       "      <th>person.one</th>\n",
       "      <th>person.two</th>\n",
       "      <th>ac_on</th>\n",
       "      <th>ac_on_time</th>\n",
       "      <th>ac_off</th>\n",
       "      <th>ac_off_time</th>\n",
       "      <th>probs_on</th>\n",
       "      <th>probs_off</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.491696</td>\n",
       "      <td>0.508304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:01:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.491696</td>\n",
       "      <td>0.508304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:02:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.491696</td>\n",
       "      <td>0.508304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:03:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0.528091</td>\n",
       "      <td>0.471909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01 00:04:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1004.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>24.5</td>\n",
       "      <td>29.7</td>\n",
       "      <td>39.5</td>\n",
       "      <td>23.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.3</td>\n",
       "      <td>22.6</td>\n",
       "      <td>...</td>\n",
       "      <td>open</td>\n",
       "      <td>open</td>\n",
       "      <td>not_home</td>\n",
       "      <td>not_home</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0.528091</td>\n",
       "      <td>0.471909</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     sensor.ac_power  sensor.atmospheric_pressure  \\\n",
       "2020-07-01 00:00:00              1.0                       1004.0   \n",
       "2020-07-01 00:01:00              1.0                       1004.0   \n",
       "2020-07-01 00:02:00              1.0                       1004.0   \n",
       "2020-07-01 00:03:00              1.0                       1004.0   \n",
       "2020-07-01 00:04:00              1.0                       1004.0   \n",
       "\n",
       "                     sensor.bathroom_humidity  sensor.bathroom_temperature  \\\n",
       "2020-07-01 00:00:00                      42.0                         24.5   \n",
       "2020-07-01 00:01:00                      42.0                         24.5   \n",
       "2020-07-01 00:02:00                      42.0                         24.5   \n",
       "2020-07-01 00:03:00                      42.0                         24.5   \n",
       "2020-07-01 00:04:00                      42.0                         24.5   \n",
       "\n",
       "                     sensor.equipment_temperature  sensor.kitchen_humidity  \\\n",
       "2020-07-01 00:00:00                          29.7                     39.5   \n",
       "2020-07-01 00:01:00                          29.7                     39.5   \n",
       "2020-07-01 00:02:00                          29.7                     39.5   \n",
       "2020-07-01 00:03:00                          29.7                     39.5   \n",
       "2020-07-01 00:04:00                          29.7                     39.5   \n",
       "\n",
       "                     sensor.kitchen_temperature  sensor.office_co2  \\\n",
       "2020-07-01 00:00:00                        23.8                NaN   \n",
       "2020-07-01 00:01:00                        23.8                NaN   \n",
       "2020-07-01 00:02:00                        23.8                NaN   \n",
       "2020-07-01 00:03:00                        23.8                NaN   \n",
       "2020-07-01 00:04:00                        23.8                NaN   \n",
       "\n",
       "                     sensor.room_humidity  sensor.room_temperature  ...  \\\n",
       "2020-07-01 00:00:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:01:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:02:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:03:00                  38.3                     22.6  ...   \n",
       "2020-07-01 00:04:00                  38.3                     22.6  ...   \n",
       "\n",
       "                     cover.balcony  cover.windows person.one person.two ac_on  \\\n",
       "2020-07-01 00:00:00           open           open   not_home   not_home     0   \n",
       "2020-07-01 00:01:00           open           open   not_home   not_home     0   \n",
       "2020-07-01 00:02:00           open           open   not_home   not_home     0   \n",
       "2020-07-01 00:03:00           open           open   not_home   not_home     0   \n",
       "2020-07-01 00:04:00           open           open   not_home   not_home     0   \n",
       "\n",
       "                    ac_on_time ac_off ac_off_time  probs_on probs_off  \n",
       "2020-07-01 00:00:00          0      1           1  0.491696  0.508304  \n",
       "2020-07-01 00:01:00          0      1           2  0.491696  0.508304  \n",
       "2020-07-01 00:02:00          0      1           3  0.491696  0.508304  \n",
       "2020-07-01 00:03:00          0      1           4  0.528091  0.471909  \n",
       "2020-07-01 00:04:00          0      1           5  0.528091  0.471909  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "probs_on = df.groupby('ac_on_time')['probs_on'].mean()[:240]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ac_on_time\n",
       "0    0.082312\n",
       "1    0.409684\n",
       "2    0.426038\n",
       "3    0.420536\n",
       "4    0.422022\n",
       "Name: probs_on, dtype: float64"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probs_on.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [],
   "source": [
    "probs_off = df.groupby('ac_off_time')['probs_off'].mean()[:240]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(240, 240)"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(probs_on), len(probs_off)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>On</th>\n",
       "      <th>Off</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.082312</td>\n",
       "      <td>0.211339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.409684</td>\n",
       "      <td>0.403675</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.426038</td>\n",
       "      <td>0.432142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.420536</td>\n",
       "      <td>0.461828</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.422022</td>\n",
       "      <td>0.460740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>235</td>\n",
       "      <td>0.844975</td>\n",
       "      <td>0.965330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>236</th>\n",
       "      <td>236</td>\n",
       "      <td>0.845878</td>\n",
       "      <td>0.964896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>237</th>\n",
       "      <td>237</td>\n",
       "      <td>0.846419</td>\n",
       "      <td>0.960558</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>238</td>\n",
       "      <td>0.846419</td>\n",
       "      <td>0.960325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239</th>\n",
       "      <td>239</td>\n",
       "      <td>0.840088</td>\n",
       "      <td>0.960324</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>240 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Time        On       Off\n",
       "0       0  0.082312  0.211339\n",
       "1       1  0.409684  0.403675\n",
       "2       2  0.426038  0.432142\n",
       "3       3  0.420536  0.461828\n",
       "4       4  0.422022  0.460740\n",
       "..    ...       ...       ...\n",
       "235   235  0.844975  0.965330\n",
       "236   236  0.845878  0.964896\n",
       "237   237  0.846419  0.960558\n",
       "238   238  0.846419  0.960325\n",
       "239   239  0.840088  0.960324\n",
       "\n",
       "[240 rows x 3 columns]"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probs = pd.DataFrame({'Time': probs_on.index, \"On\":probs_on, \"Off\":probs_off})\n",
    "probs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>State</th>\n",
       "      <th>Probability</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>On</td>\n",
       "      <td>0.082312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>On</td>\n",
       "      <td>0.409684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>On</td>\n",
       "      <td>0.426038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>On</td>\n",
       "      <td>0.420536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>On</td>\n",
       "      <td>0.422022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>475</th>\n",
       "      <td>235</td>\n",
       "      <td>Off</td>\n",
       "      <td>0.965330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>476</th>\n",
       "      <td>236</td>\n",
       "      <td>Off</td>\n",
       "      <td>0.964896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477</th>\n",
       "      <td>237</td>\n",
       "      <td>Off</td>\n",
       "      <td>0.960558</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>478</th>\n",
       "      <td>238</td>\n",
       "      <td>Off</td>\n",
       "      <td>0.960325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>479</th>\n",
       "      <td>239</td>\n",
       "      <td>Off</td>\n",
       "      <td>0.960324</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>480 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Time State  Probability\n",
       "0       0    On     0.082312\n",
       "1       1    On     0.409684\n",
       "2       2    On     0.426038\n",
       "3       3    On     0.420536\n",
       "4       4    On     0.422022\n",
       "..    ...   ...          ...\n",
       "475   235   Off     0.965330\n",
       "476   236   Off     0.964896\n",
       "477   237   Off     0.960558\n",
       "478   238   Off     0.960325\n",
       "479   239   Off     0.960324\n",
       "\n",
       "[480 rows x 3 columns]"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probs = probs.melt('Time', var_name='State', value_name='Probability')\n",
    "probs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAn4AAAFgCAYAAAAl5HQWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxV9Z34/9c55+5L9oQQIKwm7Isgi/uGFasOttpaFZe6Tp2tjnXp+O04nU6ntf11U2fGLlZR6zbaaouMSwWsaKQCAiI7hISQfb37veecz++Pm1yNCRAgkATez8cDyT3ncz/nc+6J4Z3P8v5oSimFEEIIIYQ44ekD3QAhhBBCCHF8SOAnhBBCCHGSkMBPCCGEEOIkIYGfEEIIIcRJQgI/IYQQQoiThAR+QgghhBAnCQn8hDgB/dM//RPl5eW0trb2OHfvvfdSXl7ON77xjR7nIpEIkydP5q677gJgyZIlnH/++d3KhMNhWlpaMq/vu+8+ysvL+/kO+iaZTFJfX3/U9Tz88MOUl5ezb9++fmhV//jggw8oLy/n5ZdfHpT1HYl9+/ZRXl7Oww8/PGBtEOJkJ4GfECeg0047DYCNGzf2OPfBBx/gdDpZs2YNlmV1O7dx40Ysy2LevHkA3HHHHXz729/OnP/4449ZtGgRO3bsOIat75uamhouu+wyVq9ePdBNEX2Ul5fHQw89xMKFCwe6KUKctCTwE+IEdKDAr7KyktraWi699FJCoRCbN2/udn7dunUAzJ07F4AzzjiDCy+8MHN++/btNDQ0HMum99m+ffuorKwc6GaIw+Dz+fibv/kbJk6cONBNEeKkJYGfECegU045hdzcXDZs2NDteEVFBbquc8cdd6BpGu+//3638+vXr6eoqIixY8cez+YKIYQ4TiTwE+IEpGkas2fPZtOmTd2OV1RUMHHiRMaMGUN5eTkVFRWZc0opNm7cmOntg+5z/B5++GHuv/9+AK6//voec/82bdrEkiVLmD59OmeccQbf//73SSQS3crU1NTwrW99i/nz5zNt2jQuv/xyXnjhhW5lDjRn8LPHX375Za6//noA7r///kPOMQyHw3z/+9/n3HPPZcaMGVx22WW8+OKLB33P5s2b+fu//3tOP/10pkyZwoIFC/jnf/5n6urqupV79tlnueyyy5gxYwbz5s3jzjvv7DEU/vrrr/PlL3+ZWbNmMXv2bG666SbWrl170Ot/tu33338/p556KnPnzuWee+6hsbGxW5nGxka++93vcsEFFzB16lRmz57N9ddff8hr9OV9XXMDV69ezb/927+xYMECZsyYwQ033MDWrVu71aeUYunSpVx66aVMnz6d888/nx//+MfEYjGg5xy/rtd/+MMf+OlPf8rZZ5/NtGnTuOqqq7p9b3bV/cQTT3DRRRcxffp0vvSlL1FRUcHChQu57777+vRZCiHAMdANEEIcG6eddhpvvfUWlZWVjBkzBqUUa9as4fLLLwdg/vz5PPfccySTSVwuF7t27aK9vZ358+f3Wt/ChQtpbGzk+eef54477mDatGndzt9www1cfvnlfPGLX2TlypU8+eSTKKX4l3/5FwCqq6v5yle+QiKR4LrrrqOwsJA33niD//f//h+VlZXcc889h3Vvd9xxB//zP//DV7/6VWbPnn3AsslkkmuvvZYdO3bwla98hYkTJ7Jq1SoeeOABYrFYJoD8rG3btnHNNdcwevRobrvtNrxeL+vWreOVV16hoaGBp556CoBXX32VBx98kMWLF7NkyRJaWlp48sknWbJkCW+++SbBYJA1a9bwzW9+k7PPPpurrrqKWCzG008/zU033cSyZcsYNWrUQe/1pz/9KSNHjuQf/uEfqK+v56mnnmLTpk38/ve/x+PxEI/HufbaawmFQlx77bUMGzaMyspKnn32WW677TZWrVpFIBDoUe/hvu+BBx6gqKiIb3zjG7S3t/PrX/+aW2+9lRUrVuBwpP8p+bd/+zeeffZZzjvvPL72ta+xZ88eHn/8cSorK3nkkUcOeI8///nP8Xq9fP3rXyeVSvH4449z++23s3LlSnJzcwH40Y9+xG9+8xsuuOACbrjhBtatW8ett96aubYQoo+UEOKE9PHHH6uysjL1hz/8QSml1NatW1VZWZlasWKFUkqpP//5z6qsrEy9//77SimlXnjhBVVWVqb27t2bqeO6665T5513Xub1Sy+9pMrKylRFRUXm2L333qvKysrUb3/728wxy7LUwoUL1TnnnJM59k//9E9q4sSJ6uOPP+5W7vbbb1fl5eVq+/bt3er7vM8fr6ioUGVlZeqll1466OfwzDPPqLKyMvXqq69mjtm2ra655hp1xhlnKNM01S9+8QtVVlamqqurlVJKfec731EzZsxQra2t3er65je/qcrKyjLHb7nlFvXFL36xW5mVK1eqSy65RH344YdKKaX+9V//Vc2aNUvZtp0ps3XrVnXRRRep5cuXH7DdXfd3/vnnq0gkkjn+8ssvq7KyMrV06VKllFLLli1TZWVl6p133un2/meffVaVlZWp119/vdfP63Df9+Uvf1mZppkp99hjj6mysjL17rvvKqWU2rFjhyovL1cPPPBAt/p+8pOfqLKyMrVjxw5VXV2tysrK1C9+8QullMq8Puecc7rdY1fbnn/+eaWUUlVVVWry5Mnqn//5n7vV/R//8R+qrKxM3XvvvQf8HIUQ3clQrxAnqEmTJhEMBjMLPCoqKjAMgzlz5gDpXjPDMFizZg2QXthRXFxMaWnpEV3vi1/8YuZrXdeZPHkyTU1NAFiWxcqVKznzzDOZMmVKt3J33HEHSinefvvtI7ruoaxcuZK8vDwuvfTSzDFN03jooYd45pln0PWePwYffPBB3n77bXJycjLHwuEwbrcbgGg0CkBxcTG7d+/mkUceyaSCOeecc1i2bFmmF7K4uJhIJML3vvc9du3aBUB5eTmvv/46F1988SHbf8011+Dz+TKvL7/8crKzs1m5ciUAl1xyCe+//z5nnnlmpkwymcx83dXWzzvc91100UUYhpF5PWnSJIDMsPPKlStRSrFkyZJu77v55pt59dVXD/p9dc4553S7x67FH111r1ixAtM0uemmm7q977bbbjtgnUKI3kkfuRAnKF3XmT17dmaBR0VFBVOnTs0M3wWDQSZNmpSZz/XRRx9l0rgcifz8/G6vPR4PqVQKgNbWVqLRaK+LRsaPHw+k5/8dCzU1NZSWlqJpWrfjI0aMOOB7NE2jtbWVxx57jG3btlFVVcX+/ftRSgFg2zYAd955Jx999BEPP/wwDz/8MBMmTOD888/nqquuygQ61113He+++y5PP/00Tz/9NCNHjuS8887jyiuv7NPq1nHjxnV7bRgGI0aM6PZ5aZrGL3/5S9avX09VVRVVVVWZz76rrQe6z76+Ly8vr9trl8vVrVxXe8aMGdOtXFZWFllZWQe9x0PVvXfvXgBGjx7drVxBQcEh6xZCdCc9fkKcwObMmcPWrVtJJBJ8+OGHPebvzZs3j40bN9LS0sKePXu6Lew4XL31nHXpCph60/WPe9c/9gfy+ZyDfWVZVo+g71BWrlzJZZddxuuvv05xcTHXXXcdS5cu5fbbb+9Wrri4mFdeeYUnnniCJUuWYJomv/zlL7nkkksyPamBQICnn36a559/nltvvRW/389TTz3FFVdcwR//+MdDtqW3ttu2nel968pn+Nhjj+Fyubjkkkv4+c9/zqOPPnrQeg/3fQd7vnDkz6cvdXcFo719j3T1wgoh+kZ6/IQ4gc2dO5dUKsXy5cvp6Ojo0aM3f/58fvOb37Bs2TKUUkfV43cweXl5+Hw+du/e3ePcnj17gHQQBZ8GAV2LTrp0DRsfrpKSErZt29bj+KpVq3jttdf41re+1ePcv//7vzN69GheeumlbkOQnw/UuupdsGABCxYsAGDt2rXccMMNPPXUU8ydO5c9e/YQCoWYOXMmM2fO5O6772bnzp1ce+21/Pa3v+Wyyy47aPs/3xOaSqWoqanJDNk/8sgjNDc3s3z58m69bYcKKo/0fQdSUlICpBfxdPXiAtTX1/Of//mfXHfddZlnfLi6FsBUVlZSVlaWOR4Oh2lubj6iOoU4WUmPnxAnsClTpuDz+XjuuedwOp2ceuqp3c7Pnj0bh8PB73//e0pKSg65wrQrKDvY8GFvDMPgrLPOYvXq1d2SRiul+NWvfoWmaZx77rkAFBYWArBly5ZMubq6OtavX9+jzr605eyzz6apqYk333yz2/Enn3yy26rRz2pra6OkpKRb0FdbW8sbb7wBfNq79Y//+I/cc8893Xq7Jk+ejNPpzHxW3/ve9/jGN75BJBLJlBk3bhxZWVmH7OmCdOqaz9b/4osvEgqFMom129ra8Hq9mcAL0kHzc889162tvd3jkbzvQM455xwgnd7m8+1fvnx5ryuL++rCCy9E0zSeeeaZbsd/97vfHfb3ohAnO+nxE+IE5nA4mDVrFqtXr2bOnDl4vd5u5/1+P9OmTWP9+vVcccUVh6yvay7Ws88+S1NT0yF7qz7r7rvv5oMPPmDJkiUsWbKEwsJC3nzzTSoqKrjpppuYMGECAIsWLeKxxx7jm9/8JjfeeCOJRIJnnnkmk26kS1fA9uqrr6KU4oorrug1tcfVV1/NSy+9xDe/+U2uvfZaxo4dy8qVK1m9ejXf//73uy1Y6HL22Wfz2muv8Z3vfIdp06axb98+XnjhhUw+uq4g7uabb+aBBx7gxhtv5OKLL0YpxSuvvEIikeCaa64B4KabbuLWW2/l2muvZfHixbjdbt566y2qqqr44Q9/eMjPbd++fdxwww1ceuml7Ny5k9/97nfMmjWLxYsXZ9r69ttvc/vtt3PxxRcTCoX4wx/+QFVVVbe29naPR/K+A5k0aRJXXXUVTz31FA0NDSxYsICdO3fy3HPPsXjxYiZOnHjEeyGPHTuWa6+9lqeffprm5mZOP/10Nm3axGuvvQb0PhwuhOidBH5CnOBOO+00Vq9efcD8fPPmzWP9+vV9mt+3YMECFi1axIoVK6ioqOCiiy7qcztKS0t54YUX+NnPfsZzzz1HPB5n/Pjx/Md//AdXXnllptzEiRP52c9+xqOPPspDDz3E8OHDufXWW4nH4zz00EOZcuPHj2fJkiW8/PLLbNq0iXnz5vW6ctTj8fDUU0/xs5/9jGXLlhEKhRg/fjw/+9nPWLRoUa9tffDBB/H5fLz99tu88sorFBcXs3jxYhYuXMjXvvY1KioqmDx5MldddRVOp5OlS5fyk5/8BNu2mTp1Kr/61a8yw+Znnnkm//3f/81jjz3Gf/3Xf5FIJDjllFP4yU9+0m0l9IH867/+K2+88QY/+MEP8Hg8XH311dx1112ZIPfqq6+mo6ODF198ke9973sUFBQwc+ZMHnnkEa6++moqKiq48cYbe9R7pO87mO9+97uMGTOGF198kbfffpuSkhLuvPNObrnllsOqpzff/va3yc3N5aWXXmLlypVMnDiRX/3qVyxZsgSn03nU9QtxstDUwWZdCyGEEAOsq/fR7/d3O97a2sr8+fP5xje+wT/+4z8ORNOEGHJkjp8QQohBbdOmTZx66qksW7as2/Guod7p06cPRLOEGJKkx08IIcSglkwmueSSSzLbyw0fPpxt27bx/PPPM2PGDJYuXdqnhTJCiEEU+G3ZsoUrr7ySP//5zwdd8h+JRPjxj3/MG2+8QTQaZc6cOfzLv/xLj6ShQgghThx1dXU8/PDDrF69mubmZoqKili0aBF33nlnj0VLQogDGxSB3+7du7nxxhupr69n1apVBw38brvtNjZt2sQ999yD3+/nkUceoa2tjWXLlhEMBo9jq4UQQgghhpYB7Rs3TZNnnnmGK6+8kkQiccjyH374IatWreKHP/whV1xxBRdddBFPPPEEoVCoR+4oIYQQQgjR3YAGfmvXruXHP/4xX//617n77rsPWX716tX4/X7OOOOMzLG8vDxOO+003nnnnWPZVCGEEEKIIW9A8/iNHz+et956i/z8fF5++eVDlt+9ezejR4/ukXC1tLSU5cuXH9a1W1sj2PaxHeXOzw/Q3Bw+ptcQR0+e09Agz2lokOc0NMhzGhry8wO0tkbIzfUfunAfDWjgV1BQcFjlw+Fwr9v++P1+wuHD+wbuzw/xYPLzj3ybInH8yHMaGuQ5DQ3ynIYGeU5DQ3/HK0Nq546DrUM53KX8zc3hY97jV1gYpLExdEyvIY6ePKehQZ7T0CDPaWiQ5zQ0FBYGaW4O92uQPqQSHwUCgV73j4xEIke1AbgQQgghxMlgSAV+Y8eOpbq6ukfP3969exk7duwAtUoIIYQQYmgYUoHfmWeeSUdHB++9917mWEtLCx9++CGnn376ALZMCCGEEGLwG9SBX0tLCx999FFm4cZpp53G3Llzueuuu3jxxRd58803ufHGGwkGg3zta18b4NYKIYQQQgxugzrwW7lyJV/96lfZvHlz5tgjjzzC+eefz0MPPcR9991HcXExTzzxBNnZ2QPYUiGEEEKIwW9QbNk2EGRVr+giz2lokOc0NMhzGhrkOQ0Nx2JV75BK5yKEEEII0a+UjZaKAxoKDTTo/A9oWucfHXTjIJUMHRL4CSGEEGLwsC1ce9aixzrAMkE3UA4XOFzpv9HANtEss/vfZgotHkazTWLTL8YO5kMyTuD9Z8FMoFx+lMMJgGYm0RMRtGg7RrgZzUodslmp4eVEZy/Gyh1+jD+AY0sCPyGEEEIMGq5dfyVQ8dzRVaIUkTOvw72zAlfVhkMXd7pRaGhKAQoUdP4HlA3Kxlm7jaxlD2HljgBNR+k6aDpm4Rhip15+dO09jiTwE0IIIcSg4a5cC0CydAZW9jCwrXSPnJlEMxPpeMwwULoDDMenfxtOcLjwrX0FV9UGookv4d5ZAUB0xiLsQF66dxBQhgPl9mN7s7CCBeD0HLRNWiKCd8Ny3Nvfw9Gyr9s5o3U/sZlfHDJDwRL4CSGEEGJQ0GIhHPU7UbpBZP5XUW7fYdfh3L8VZ+02fH99GUd7HbY7QHzKBWAcecij3H6ic68kNnVhegjattCUDUqlA8chEvSBBH5CCCGEGCRcVRvQlCJZMvGIgj6AxIT5OGu34d7zYfr1+NOOKuj7LOXLxvIN7fRxgzqPnxBCCCFOHq7K9QAkx8w64jqSo6Zhu/2Z14nx8466XScS6fETQgghThSpBK6azSjdwMovxfblpNORDABnzSfokVZQCi0exgg3Y3uziE88C+XNwlG3A2fDbrRkDGwLK1iIo2E3SneQHDn1yC9sOEiMn4v3kxWYBaOxc4r776ZOABL4CSGEEIOc0VqLa/caHK370aNtKMOJ6kpv4nB3pjlROKs/Rk/FM+9LDS8ndN6t/TbU2VeOht0E3/5lr+c8W1Zie7MwIq29nk+OmASugy+2OJT4lAvQExHiZWceVT0nIgn8hBBCiMFM2QRW/QYj1NSn4mbBaJTTg6OxMr3I4cM/EJ135TFuZHeu3en5danCsVg5w1EuL3YgD0fdDlx7N2BEWrH8uSTHzML2ZoECR8s+9EgLsakLj/r6yhMgcvo1R13PiUgCPyGEEGIQc9Zuwwg1YflyiJ72JexgQWfC4iRaKoFmJtOpTqwUZuEYrLyRABjNVWT938/xbH8Xs2gcybGnHp8G21Ymd1503lVYuSWZU4myM4h1NKLHQ5gFY0D/dKlB4vi07qQngZ8QQoiTkpaM4qjdgabs9GIA3QDbSu8U4fahPpfbTTndKNeRrTQ9Gu5tq4F00JQqnd7n91n5pUTnXIF/zf/ir3ieVNE4lD/nWDUzw1m7DT0RwcwuxsrpucuFnVWInVV4zNsheieBnxBCiMOnFFoyimYmO/c3Te9nqkIpjOY69EQYrPQ2WtgpUJAaMRnlDfZ/W8wkejyUXiRgmWA4MDoacFZtwtFclW6nZWL7srF9uSjDQDOTOBor07nY+nrLmkZo4d9hDhvf//dwAHqkFWfnYo3EhPmH/f5E2Rk4a7fhqt6Eb90rRM664Ri0srtuK3MHaGGJODAJ/IQQQhyQloji+fitzh0TbPRoB3qkBSPckj72OTZwoCxnynARn3R2eogP0BJh9Gh7es/UZBQtGUsHb4CVVYTtzwPDgWYmcDTsxmivJzrrUpLj56LFw/g/eAFH4170WHuf7sUINXWbJ6c0ndSwCSi3Hy0RSW/NpRlgm+iJCJjJTz8HM4mejOLe9m7/BX5KYTTtRU9GQanMYdufi5VdDCjcW99BU4pE6fQjC5o1jeicK3Du34q7cj2JsjMwh03on/Z/nm2hR1pxVW8EIDnmOA0ti8MigZ8QQogD8m78Pzxb3+n1nO30oJye9P6mnfuZ6g4HpsuPcgdQDmd6Gy3DiR7rwFm7De/Hb/Xpus6G3b0e97//HMrhxvvxm5mts5RuYHuC6YS/ugNsC+XykRo5mVTJRGxvFkp3oEdb0SPtaMpGaRpW/ijUZ/K9HYwebiHn99/Fte9jIsn4Ua86xTLxv/8s7j1rez1tOz1oto1mpYPPRNnpR3wpO5BHbOqF+DYsx7/6d5jDxmF7XPjjqc4eOQ3V2TOnKYUWa8cIt5AcNY3YrEtB2Xg/ei2dasXtRzlcaKl4ujcX0oFypBU91t651y2YeaNkOHeQksBPCCFEr7RkDPfODwCIzvxiemWmN4gdyMf256Fc3h5DeYWFQVoaQ73WZzRW4tn2bqZXT7m92N7szqDNi3J608GbZWGEGtAjbZ29cDpmfinOxt14tqwi+M5vAbCCBYTOuwU7WNRtkcCB2NnF2NlHltPNDuSRKhqPs2EXruoNJA8nKbBtpXv0OlOqaIko/neX4tq/FeVwkSoaB13D5crGaK/PpDqxggUkS6cfdS9dfMr5uHetwQg3Y+xuQQHuQ7zHu/nPYDjQI624d6055DUUGrYvGyuQT2z6xUfVXnHsSOAnhBCiV+6dFWhmglTxKcSnHX2KDatwDJHCMX0qa1Le41iqdBpatB333o+wvdmELvxb7ED+Uberr5Lj5uBs2IV799pDB36Wie+vL+Gq3oQWj4CukyqZhB3Ix72rAi2VwHYHCF1wG1Z+aY+3a9H29JzJ/poTaTgJXfR3OOp3gbIJBt2EOuKAAgUaKjPcbHuz0BIR/BXP4d34OgDKcBJZcDVKd6BZyXSQ7nCl69aNzvmTOcc9X6A4fPKEhBDiZKNsHHU70eNhrEBeerhT2WhmAj3aAbaJVTAa95ZVAMQnnTuw7e2i6UTOuJbUiEmYxWXY/tzjevnk6Bn41vxveseJqo2AwtFUhaOpKjPf0cwtITnmVLyb3sBZvxNI94Rh27j2fZypK1VcRmTeVQccDlXHYD9Y259LctwcAPTCIMkD9Mx20ZSNv+J5lG4QPvfrpEom9XubxPEngZ8QQpyozCTu3R+iJaOg6emhRNvCtftDHO11farCyiokNWIQ/YNvOA9vmLUfpecNTsVVtYHgqsd7LeNorsKzswJI95yFzr0ZK28kWjyMq2oDRkcjibGzsfrY8zmQEqcswAoWoFzeTG5AMfRJ4CeEEMeLbaVTnNgmWCk0y0y/tlLpY4rOIbPsdE65o7xWYNVvce3f0utpy5eDlV+KHmlBS8XTw4qGM72LAgpn/W40K5neRUE79Py5k0Vs+hfSn5eyUYYTK2c4ZuFYbE8AzbZw1mzGvftDbLef8Hm3YgfygHQPXmLi2QPc+sNnFp8y0E0Q/UwCPyGEOEa0eBjn/q0492/BuX9bOrddHyhNSy96COQTm7no8Cb2m0k0y8S39hVc+7dgu/0kxs8DVDpnnVKY+aUkR888+Hwsy0SPtR/XOXRDgZVbQujCvz3geXPYeGKzLku/kBx2YhCSwE8IIY6GUmAmMToacFVvwlG/E822wEphtNamJ813FUUDw9GZ4sSB0h2drx1gOEEp9Gg7WqwDI9qGEW3DseJXtC+6Czt72EHaYOPctxnPJyu6pUFRhpPQebce2bCi4ZCg70hJwCcGMQn8hBDic7REFOz0DhBaPJLOUdb5x4i0oEfa0GId6MkoWiKSDvR6oXSD1LAJJEsmkRoxCTurqG9BgWWiR9vxrXs1PZ9s5a8JnXtLeisxpwfl9qElojia9uKs2YKreiN6rCNzTeVwodx+oqd9aUjMJRNCHD8S+AkhxGc46nYQfOu/Molo+0IZTmxPkFTJRFIjp6T3eUVL71PqPFS2tF4YDuxgPuHTryGrowFHWy05r/7np9fT9B5bjVmBfOITzyYxYR44jzK5sBDihCWBnxBCdFEK37o/oimF7fKiKRvb5cP252L78zr/zkl/7cvGdvtQLh905TPrb0434XNvwf/e79AjLYCGloqjJ6Mow4mZX4pZNI5k6fT0qksZYhRCHIIEfkII0cm5bzOO5ipsT5C2xQ8cWW9dP7OD+YS+8PfdD1pmOsg72pW/QoiTjgR+QoghQZkp9PY69HgENB3bE+iZ/NYycVVtAE3DChZi5ZZkgiMt2o6jbT+205vuoVM2mplCi4fQzARWVhHeDa8BEJt64aAI+g5IdkcQQhwh+ekhhDj2UgmMtlr0aFt6bpploiWi6LF2HM37MNpq04spPiezIlaBbSbJofu8u8TYOUTnLEZ5AugdjQT+8iSOln2Z82ZuCeFzb8Zoryew6onMhvcHY/lySJSdfnT3K4QQg5QEfkKI/mfbBFb8EkdjJRoKLZU4+jo1HSuQj+0Joikbo3U/7j0f4qrehO32ocfDaFYKy5+HlTcivZVW636ylv1/aKk4mrIx80amkxGbSdB10B3Y3iDKcGK016NHWonN/pt0ahUhhDgBSeAnhOh3jvqduPZvzbxWuoGVPQw7UIDqDLiU24ftCWDllGDljUB1G1r9dJGC6vy6oDiXlpZY5rgeasL/wYs4a7dhdO6Tmhg9k+j8r6BcPrRklMA7T+Ks3QZAbMoFxGZdKgsghBAnNQn8hBD9zr37r0A62IpPvRDlcB31QgTtc/Pa7GABoQvuQIu1o1kWyuFEebMy55XLR+j823BvexflCZIce+pRXV8IIU4EEvgJIQ6PUujhZvRoG2bRuJ77uJpJXFUbAUhMmI9yeY9dWzQN5Zgj3pUAACAASURBVPv8zL/P0A0Sk845dtcXQoghRgI/IU5yeqgJ96416YUXqUQ6VYiy0rtR2FY6b1ykDS0VRzlcaEqhdQ6txqZemB4+/QxX9SY0M4FZMLrnqlshhBADSgI/IU4SWqQNV/UmjNYajI7GdEoQ207vLXvgPrPudaTiANjeLLRYCM/mt0mWzsDKH5Up4+oc5k2Mm9P/NyGEOKm0JqPsjbVQl+jA0HQKXQECDjcKhVt3MMyVhUs3CJlxOsw4XsOFz3CSzgmgUEqhAJ/hQpf5vYAEfkKc0LRoO47mKlyV63Ht/ajHNl+QXniRGDsbs3AsyulGGS7QdZRupPeGdbixfTkolzedDkUplNuP76+/x7N1Ff73nyNRfibO6k0Yrfsxom0oTSc5etYA3LEQ4kRRF+/gvyrfwT7EL6Yu3SB5gP2yu/gNFxP8hRS6Ajg0A4eu49AMAoaLEk8OHsNBTbydqlgL1bFWmpJh8px+it1ZmMoiZCYImQnCVhyz8+doqTePy4dNwzPEsgBI4CfEYJdKpBdG6EbfVqQqhXPfZrwbX8fRUv3pYU0nWTqd1LAJWNnD0kO5VgqzcBzKG+xTU9RnFlhEZ16Cs3oTjtYaHBXPf1pGN4hPPAflCfT9HoUQJySlFO+17iZqJRnnK6DUm4ezjwu91rVXY6MocWczMTgMSykaEyFidgodjYiVpDERImlbeHUnOU4vcdskaiXRAA0NTQNbKSJWkg0dNQe8lgY9wsumZITtkYYDvqctVUNtvJ3rRs4l3+Xv0z0NBhL4CTGIede+gveTFQAoTQPDiXK4MfNHYQ6bgO32gaaRGl6O8uVk8ud1pVKxnR6s/FGYhWNJnLIA25/bf41zuomccQ2BvyzFyi4mOWYWqWETsAN5spWYEAKA/Yl2ljd8AsCq5p04NJ3R3jxGeHJw6AaGpuHQdLy6iylZw3Hr6bDEVopNoXSgdlnxNEZ5e//ZZSqbhJXCZ7jQDvCLsVKKxmSYXZFGQlYC07YxlUVK2bSnYtTE20jaJsPdWZR68xjlzaXIHaQ5GaEhEcKtOwg43AQdHgION07NIG4n+d/9H9GQDPF41XvcNf4CjM8vdBukJPATYrCyLdw73gfSvXWassFMoplJXDWf4Kr5JFPUyiqi/dJ7cFWuxbV/K7bLR2z6F0icsiC9PdkxYg6bQNuV3z1m9YsTg6lsqmOt5Di95Dp9A92cE5qlbP5Yv4kiV5D5uWMHfF7b5o5aAErc2SgUtYkOdkWb2BVt6lH2zaYtLCyYxMzskeyONhEyE+Q5fYz05Bywfoem43AcfHtFTdMocgcpcvc+smErha1sHJ/7hbXEk32QWn3cNvoM/lT/MYnOHsihQgI/IQYpR/0u9FQcM7uYjsvvA9sCy0SPh3E07MLRVIlmmTjqdmB0NODdsBzXnrUARE/7EklZXCGOguocHjOVhY5G0OHptUfFUja7I03UJ0LE7CT+iJsc20uJJ4dsh4fmVIQX9q9jf7wdgAKXn6nBEmZnl+LQdWri7bh1B6Xe3CHTYzKY7Yo08WFbVebrK0tm4jX695c/WynaUlGaUxHilolCcYq/sMd1lFJsDqUDv4uLJjPOX0DYTLAn2kRTMoKl7MyfvbEWauLtvFz3ETsiDZmAdUbWyAP25PUXXdPQtcMfpfAYTq4sGXpzmSXwE2KQcu37GIDUqKnpA53z/Gynm2Qwn+T4uUB6l4ysNx7Bu/nPAJi5IyRZsTgiUSvJrkgTOyON7Iw00G7GM+dGeLI5J/8Ush1e2swYjYkw9YkOdkYaidmpTytp/vRLl2ZgozCVTcBwk1IWTckIK5t3sLJ5R7dru3UHE/yFlPmLGOHJIaUsTGXh1V14DSdew4lTM455EDDU7Yo2Zr7eFqnnR7veosxfxMRAMeWBokxwppSiKRkmYiYZ6c3pFnTbnQFbfaKDDjNOSzJKUzJMwjZx6w5SyiLxub21HZrO5OBwStzZ5Di9lAWKaE5GaE5F8BsuxvjyAQg43EzLGtGj3bZSbOyo4Y/1m9gU2p85PiO7Z1lxdCTwE2IwUgpn9SYAkiOnHrSoOWwC8Qnz8eysACB66uU9kyqLk1LUSrItXE97Kk7ESqTnUhkudDQsbBoSIfZEmwmZcQxNx1Ld10+6dQdu3UHCNqmJt/O7mg97vU6RK8A4fyF+w4XTY7CjpYHaRAdRKwnAtGAJf1M8HaduUBltYW17FZtDtRiaxghPDmEzQWMyzOZQbaaHqDcOTcejp4NAWylSyiJlpwNEv+FmhDeHYncWuU4feU4fuS4fAcN9UgWLuyLpIdQrimewvr2aylgLH4dq+ThUi056yFOhiOxMEu7MxznWl8/VJbPxO9zsi7Xxav3GTA/t56Ws9OrZgOGm0B3AZ7iIWSl2R5vY2FHDRtLz8ordWZR2zsubHBx+yCFnXdOYmT2SQneAJ6s/IGolGeHJpsAli8T6mwR+QhwPto3Rsg9n3XactdsxmquITzqX+IyLey1utNViRFqxPUGsgtJDVh879XIcrTVYOcMxS8r7u/XiGGpNRWlNRgk6PDh1nZpYG/XJMD7DSZbDA4ClFJayMZVNxErQkYrj1I30BHlNpyrWQmsqiq7pGGgYmp4O+iINWL2k8OmNqWyMzon3E/yFTPAXUuzOQtc0krbJh21VrG2vQkMj2+El3+VnmDuYmQjfpbAwSGMgBKQDz6Rtku3wZoKv8f4CxvsLMJWNjpYJCFo6V1BuDzfQmori1h0Ymk7cThG1UsSsJKayCVsJwlaiR/vbzBhtoViPwNGrOzk1exSzc0rxdfZ2nag53cJmgrpEB05NZ3rWCGbnlNKajLIlXMeWcB17o+l8eF38hguFYk+0mUcqV6Gj026m98POcng4NXsUWQ4PuU4fhe4AXt1FUpnoaPg/N6+uJRlhS7iOtlSMreF66hIdmWtNDQ7v8z2M8ORwa+npvNW0jbk5Y47+QxE9aEqpvmVuPcE0N4ex7WN764WFQRobQ8f0GuLo9etzskyc+7dihBqxXV40M4WzbgeO+h3oyViP4uEzl5AcO7vHcc/GN/BteI34hPlEF1zdP20b4obi/08J2+T1hk8wNJ2Szt4Ln+GiJRlhR6SRHZEGGpPhY3Z9DRjnK6DEk0PA4cJUNjErhVIKXdPIdnoZ68unwBXAVgqtc4Xl0TiWzyllW8SsZGc6Dx2nbuDSDRyaTmsqxr7O/GstqWgmoO42DN3JqRnku/wUuALku/w4NJ2UshjuzmZysHjA5hq2paKdPaE6pd5chrmzDqstG9r38WLteib4C7lx1Pwe56NWkpZkBIdmMKIwB7PDImTG+V3Nh+yLtwHg0R3MyRnNeQVlmRW2h6sjFec3Ve/RnIrgM5zcO+Eimb95hAoLgzQ3h8nP77+ezwHv8fvTn/7Ef//3f1NdXc2IESO4/fbbWbx48QHLt7S08KMf/Yi//OUvJJNJZs2axf3338+YMWOOX6PFSU+LtuFbvwxsC7NwTHoHjOYqnPu39BrgAViBfFLFZZjDy9DDzfjW/wn/+89h+7Ixh01Ai4fxfLIiHTi2pXstUocY5hWD22v1m1nbXnXQMm7dQZE7SNhMkLBTDHdnM9yTTcI2CZlxNMDQdAxNR9d0fIaTbIeXmJWiJt6GqWxGeXMocmeBUuneQdK9fKf4i8hx9m2vZGMIdIA5dQOn7iWLnvc0zB1kWC+rNmtibbzfuoedkUYUClspYnaqW4/UZ+U6fYzx5tGUiuDTnVxcNIVCd/8PN6Zsi7+27eXDtr1AOuCuTbR3G2rX0chz+ch1+tI7UuhOPIaTgMNNsTuL4Z7sbsHZzs6VshN8vW+V6DNc+LzpXs88j5/GUIgsp5dbSk+nMtZC0OGm0BU86t7QLKeHm0sX8Gr9JiYGhknQN8gMaOC3fPly7r77bq6//nrOOuss3nrrLe699148Hg8XX9xzCEwpxZ133klVVRXf+ta3yMnJ4Re/+AXXX389f/zjH8nOPtjSayH6h9FcTXDFr9Fj6Tkw7sp13c6buSWYRePT+9kqhVk0jlRxGXYw/9NCSqGHmvDsrCDrjUcws4sxIi1oZnpOlNI0UsNOISXDtkPWJ6Fa1rZX4dB0zswbT0MiRLuZnmvnM1yc4i9kgr9IVrMeYyO8OVzp7b7yMmalaE6GaUyGaU5GUKR7Rzd17Kc5FaE1Fc2U3V3ZxFn5E9DQSCmLcb4Cxvry+/TMUrZFZayZvdEWGhIholaShG1iaDrtZpzQZxbPQHoO48RAMQ5NpzrWSnMqQlMy/ac3OhoTA8M4LXc0Y7z57IqkF3aM9xcc1mfk0A0m+Pt3X+0sp5frRs7t1zpF/xjQwO8nP/kJixYt4tvf/jYAZ511Fu3t7fz85z/vNfCrrKxk3bp1/PCHP8z0Co4fP54LL7yQt99+myuuuOK4tl+cPBy12wmu/DWYqcy+tqmicSTHzsbRtBelGVgFpaSKxmJnFx+6Qk0jOvdKcLpx7forjvY6AJIjphCffC5mfik4D56bSgw+pm1RFWulJt7GX1p2AXBR4SROzxs3wC0Tn+U1nIz05jLyc0mBzy8oZ2u4jrCZIN/lZ117NRs6ani7aXumzDvNO3HpBgHDjVt3UOgOMtydRXlgWLe5jvtibTxX8yFtZu8jAJDupbywYCK5Th9xO8Uwd7BbSpSkbdKcjNBhxolaSeJWer5juxmjNt5OfSLEJ+E6PgnXZXae8Bsuhrmz+u2zEieeAQv8qqurqaqq4q677up2/Atf+ALLly+nurqaUaNGdTuXSKQn9Pr9n26N0tXL19bWdoxbLE5mni0ru/XGJcbPSwduhoNE2RlHVqnhIDrnCqKzLsNRvxPlCWLlSeqCoaghEeKNxi3sijSRUp/uGTreV8D83LED2DJxOHRNY/JnFiKM9xcyOTic7eEG/A4XSim2hetpSIZpsdO9grWJDjZSw+uNWxjtzWO0N4+kSg/jWsqm0BWgLFDESE8O/s5g0UahozHck33QYVWX7mC4J5vh9D6aFTLjrG2rYmNHDU3JCArFlD6soBUntwEL/Hbv3g3A2LHdfyiOHj0agD179vQI/CZOnMi8efN49NFHGTduHLm5ufzgBz/A5/Nx4YUXHp+Gi5OOFgvh3L8Vpem0fflBlDsAej8OzRkOzJKJ/VffCWqwrkNLWCZP7VuTGR7sSmMxwpPD9KwR8o/wEDclOJwpnwkGv1A0mYiZIGaniFkp6hMhqmItfBzaz95YC3tjLZmy83LGsKhoco8dIfpL0OHh3IIyzi0ow7Qt2s14n+d0ipPXgAV+oVB61Vcg0H3SbFdvXjjc+0q3Bx98kFtuuYVLLrkEAJfLxaOPPtojSDyU/lwhczCFhb1vESMGl4M9J3tdBUrZMG4GBaXSI3espGyL5dWbyXZ5mZU/iiyXB0vZ/LVhL8urN9O6PcqoQC6F3iAJM4Xb4eSy0mnke/pnc/RQMs7q+l3URTtoTkRwaDo+h6vHn2JfFuOzCjMB3dLtH9CaijLKn8udU84h1y1bkp3oP/cK6Xl/cTPFuuZq2hIxQDEmmM/k3L6nMekPh3u1E/05nSj6O14ZsMCv67f3zyfW7Dqu99KjsmvXLq6++mpKS0v59re/jcfj4YUXXuAf/uEf+PWvf82cOX3fokrSuYguh3pOWRvfxQGERs4iJc/zmHmvZTevNWwG4Nmdf8WjOzMbqXfZ2dHIzo5PdybY0FTNV0pmM95XcMRJepVSfBKu49W6jUQ6Ew4fSpbDwwR/IbZSfNSxD4ems7hoBmaHRSMn9/fIyfxz7xS9kMyCY5NB/TmczM9pKDmh0rkEg+nfND7fsxeJRLqd/6wnnngCgMcffzwzt++MM87gmmuu4fvf/z4vv/zyMWyxOBkZbbU4WvZhOz2kRk4Z6OacsJRSrGmrBNIJXGvj7Zn8a3lOH+fmlzF/9Fg21dQQMuO4dSfr26vZHmngieoKNNJpUcb7C5mRNYIidxCP7iRup+hIxalLdLAv3kbYTGBoGpZSRKwEETNJ1ErStV/FWF8+07NGkOP0opQiZpvEO5MHx22TiJVkT6SJNjPGuvbqTPsXFk7sNZWIEEIMNgMW+HXN7auqqqK8/NOUFXv37u12/rP279/P+PHju6Vt0TSN2bNns3Tp0mPcYjHU6KEmPFtWYbTWoEfbic654tN9b/vIvWUVAMkxs8BwHotmCmB3tJmmZIQsh4fbRp+BpWxStoWh6bh1B5qmke3yUh4YlnnPlOBwVjRt573W3SRsk7htHnLLrwPxGS7OLyhjbs6YQ87JU0pRHWulLtGBjcJruJgWLDnsawohxEAYsMBv9OjRjBw5kv/7v/9j4cKFmeNvvPEGY8aMoaSk5w/SsWPH8vvf/5729vZuwd+GDRsYMULmXolPGY2VBFf8Cj3xaf4r76bXDyvwc9ZswbOzAqUbJMrPPhbNFJ26evvm5JRmkhW7DrFrgK5pXFBYzgWF5VjKJmTG+bijlk/CtYQ6kyG7dAdZDg8FrgCjvLnkOn3YnbtW+A1XOqGt4cJ5GJPvNU2j1JdHqS/vaG5ZCCEGxIDm8bvzzju5//77yc7O5txzz+Xtt99m+fLl/PSnPwXSu3RUVVUxYcIEAoEAN954I6+++io333wzt912Gx6Ph1deeYU1a9Zk3iNObnqoCdeedXg/fhPNSpEsmUhi4tn4/7IUR3M1RlstVs6hp0BriQj+958FIDZjEdZxnqR9MmlLRdkSqkNHY072ofcl7o2h6eQ4fZyZP54z88f3cwuFEOLEMaCB35e+9CWSySSPP/44L774IqNGjeKHP/xhZsXuypUruf/++1m6dCnz5s1j5MiRPPvss/zoRz/ivvvuQ9d1ysrK+O1vf8vpp58+kLciBpCjYQ+uPWtx1u/AaK/PHI9PmE903lWgGyRHz8Kz831cu/5KbPblmTJaMoqqbcRZsx8tlc6i72jbj6tyPXqsg1ThWOKTzz/u93SiU0rRkoqyqWM/77bsxEYxOVBMlqSiEEKIY0pTgzU51jEmq3pPAErh3roK39pX0Dq/jZXDTXLUVJJjZ5MqmQSd87UcDbvJev0X2N4s2i+/H/fWd3Dt24zRUp157+eZ2cWEz7u1+1Zros8+u3LfUjaV0Rb2RJvYF2+jJtaWWbwBMMFfyBXFM8g+SOAn/z8NDfKchgZ5TkPDCbWqV4gjocXDOGu3YbTXYzRX49q/BYD4xLNJjp6FmT8KjJ7f1mbhWKxgAUaoiezf/zt6Mp1sV+kGFJSQdGejXF7QNGy3n+TomVj5pZnAURycrRQNiRBxO4WpbLaHG9gUqiFupch2eomYyW6BHkDAcDPSm8OC3HGHvbeoEEKIIyOBnxgStGg7gXeX4qjfndkrF0AZTiKnf43kmFMPUYFGYtxp+DYsR09GMXNHEJt5CalhEygsKSAsv/ketv3xNqpjbVTHWtkRaThgDryuDeYLXQHKA8MY1bmrRbbDc8S594QQQhwZCfzE4GdbBN55AmfjHpRukBo2AbNgNFYgD3PYBOxg33qLEuVnYrTXYeWNIj7pHDhG2yidDFY0befPTdu6Hct2eMlxetHQKPZkMTNrBAWuAO1mHIemk+/qnx02hBBCHDkJ/MSg51v7Ks7GPdi+bNovuRvlPbJEucrtJ3LWDf3cupNPfSLEiqbtAMzMGskITw7j/AUUuQK99uB5JP+hEEIMGhL4iUFLi4fxfvQanh3voXSD0Nk3HXHQJw5MKUVzKkKOw3vIzeRtpXilbgM2irk5o7m8ePpxaqUQQoj+IIGfGJSc+zbjf/cp9FQcpelE516JVThmoJs1pEWtJLsiTbh1B17DSVMyTE28na2hOtrMGMPcQa4bOZdcp++Adaxu2UVVrJWA4WZh4aTj2HohhBD9QQI/MegYjZUE3nkik4A5Onsxdk7xQDdrQFXHWqmKtTA7u/SIhk4bEiGWVn9Amxnr9byORn0ixGOV73JuwSm4dQf5Tj8jvbnomoatFP/X8Anvte4G4NJhU/HKEK4QQgw5EviJgacUjrodOBp3g+7A88lKNCuVTsA8/6sndUqVmlgbyxs2UxlrAWB9+z5uGDUPv+GmLRUl6PDg1A2iVpJPQrWYyibX6cO0LeoSISJWAl3T+ai9mrhtUuQKEnS4iVkp8lw+hrmzmOAvJN/l57mateyONvGn+o8z1/cbLgpcAZqSYSJWEgONy4unMzVL9qYVQoihSAI/MaActdvxrf8TjuaqbseTJRPTu26cREFf0jZ5at8akrbFV0tOJWlbPF79PgnbxKM7cOtO6hIdPLrnHRQqE4gVuYM0JMNYyj5o/ZMDxVxZMuuAe+DeMGoe77fsoTEZJqUsqmOttKaiRDqDzqDDw1dKZjHWJzn3hBBiqJLAT/Qf20aPtmH7c0DTD1ncvWUVvrV/QFMK2x0gMW4O6DrK7SdeduZJlW5FKcUrdRvZE20G4LG972JoOgnbZHKgmC8Nn4mlbJ7at4Z98TYAfIaLmJWkNtGBRnr3ixynl9ZkFEPTGeYOku30YitFwOFmarAE/SCBtKHp3fa5VUpRnwgRthIUuAJkOTwHfb8QQojBTwI/0W+8Hy3Du/nP2G4/ZsEYtGQUPR7CLBpPvPxMrPxRn5Zd+wreT1YAEJu6kNi0heBwDVTTB9yatr1s6KjBpRmUeLIzQ7ujvXlcVXIqzs4g+OulC9gebmCYO0iBK0DCNqlNdJDn9B10u7MjoWnpfHxCCCFOHBL4if6hbNy7/wqAnojgqtmcOWWEmnDv+oDYpHOJzVmMo24H3k9WoHSDyIKvkRw3Z6Ba3S8sZVOfCFETb8NSNsPcQYa7s/u8CKM1FeW1hvTntXj4DKYEh/NG4xZakhGuGD4zE/QBuHRHt/l1HsPJWJ/sJSyEEKJvJPAT/cJoqkKPdWD5cwld+Lc4WvZhe4Iopxv37rW4t7+Ld8tKzJJyvOuXARCb/oUhF/QlbJOK1j00JEK0pWK0paJ0mPHPbCKX5tYdLC6ewbQ+LIJY3bIbS9lMC5YwPWsEAIuKphyD1gshhDjZSeAn+oWrehMAqVHTsLOKSGYVZc5F80uxPQF8Hy0jsPK3aFYS25tNfNK5A9TaI9OWivHMvjXUJjq6HdeAPKePkd5cHJpObbyd2kQHz+9fyyehWuJ2isZkGFspXLrBhQUTM712ETPB2ra9AJyTf8rxviUhhBAnGQn8RL/oCvySo6b1ej4+5XxcVRtxtFQDEJ25aMjM6UvZFhs7anizcSthK0G+08/Z+RPIcfrIcXrJdnpxfGYxi1KKitZKljdsZlNof4/6Xti/Dl3TmBwcznute0gpm3L/MJlPJ4QQ4piTwE8cNb29HqOjAdvlwywad4BCBpHTryFr+U+xcoaRHDf3+DbyMK1u2cVfmnfh1A3iVoqYnQJgnK+Aq0fMxmccOGjVNI0FeWMp9eayI9JIocvPME8WDk1nTete3mnZyfM1axnnL2BvNL2I4+z8CcflvoQQQpzcJPATR0yLtuOs245z/1YAUiOnHDQFi5U7nLYvfQdlOEE/dLqXgdKYCPNGwxYsFFjpY8PdWZyeN47pWSMw+pCqBmCEN4cR3pxuxxYWTsRUFu+17mFHpBFIB5OjfXn9eg9CCCFEbyTwE0dEDzWRtfyn6IlI5tiBhnk/S3kCx7JZR00pxZ/qN2GhmJU1knMLylAo8p1+tH7IYadpGouKpjAtawRRK4lTM3oEh0IIIcSxIoGfOHzJOIEVv0ZPRDDzRmLmjcIO5JEaOXWgW9artW1VrGrewQUF5czIHnnQshs7atgVbcKrO7m4aDJ+h7vf26NpGqO8uf1erxBCCHEoEviJwxZY/TSO9jrM7GI6Fv4duDwD3aReJW2TPzduY3XrbgD+ULeRkd5c8l1+AGylWNdeTSJiYiQ1dkYa2RquB9JDssci6BNCCCEGkgR+4rDo7fW49n2McroJn3fLoAr6GhIhNnbU0JqK0pgIU5fowEZhoDHMk8X+eDsv137EzaWnE7ESvLh/PbujTd3qcGo6p+eNY07O6AG6CyGEEOLYkcBPHBbX3o8ASI6ajh0sGODWfKo23s6vq94jYZuZYzoaIzw5XFw0mWHuIA/vWcneWAs/2Pk6ccvERuE3XJxdcgpNoTABw8283DEEpKdPCCHECUoCP3FYXFUbAEiOnjnALflUazLKk9UfkLBNTvEXMjVYQr7LT4knG5f+6bf4FcUzeGrfGqJWOjVLmb+IK4bPYNzwQhobQwPVfCGEEOK4kcBP9Jne0YCjdT+200NqePmAtsVUNiEzzvr2fVS07iZq/f/t3Xd8VFX+//HXzKQX0kMgQEJHqYHQRRBBugoWFBXXRQXFiuwifP1Z1t1VsCCoKzYU1oagoqJBFhFFsVFEEEKHhBAgJKSXyczc3x8h0RggBDIzSeb9fDx4PJh778z9TA73kTfn3HNuKS0DIrghtidep1lSpl1QY/7eZiiGYeBv8an0DFwRERFPoOAnZ618mLe0eWewuP6fjs1hZ2NOKt9l7SWrtLDSvnj/M4e+csFedeeeRBEREVdT8JOz5nPw5DBvi64uP3eWtZCFKevJthUBZc/HDbT4EusXSv/wVrQMiKiVdfZEREQaMgU/OSuWzBS8TqSVDfM27eDScxuGwSdHfyXbVkS0TzCXRrbjguAmmBX0REREakTBT85KwKYVAJS07evyYd5f8w6zpyADf7M3f23RV7NuRUREzlHdfWCq1Bleh3fifWQXDh9/ijsNcem5i+xWPj+6DYBh0Rcq9ImIiJwHBT85M8NBwOZPASjueCmGb6BLT7/m+C4K7Fbi/MPpHtLcpecWERFpaGoc/PLytN6ZxzAMAn7+EK+sQzgCQijucLFLT59Rks+PJw5gAsY07qx7+kRERM5TjYNfv379uOuuu1i5ciUlJSXOqEnqgpOhz2/ntxhmL/L7TgAvH5eW8EXGdhwY9AhpQYxfI5eeteJWyQAAIABJREFUW0REpCGq8V36EydOZOXKlaxevZrAwECGDBnCqFGj6N+/PxaLFsRtEAyDgA0f4bdzHYbZQv6gv2Jr6roFmw3DYFNOKsn5R/ExW7g0yr2LRYuIiDQUJsMwjHN54+bNm/nss8/44osvyMjIICwsjGHDhjF69GgSExNru85al5mZj8NxTl/9rEVFBde/R4EZBgEbl+O34+uy0Dfwr5Q26+j009oNB7m2Yg4X5/DjiQPsKzwOwLCoCxgQ0cap566X7eSB1E71g9qpflA71Q9RUcFkZuYTERFUa595zsGvnGEY/PTTT6xZs4avv/6agwcPEhMTw5gxY7jyyitp1apVbdVaqxT8/sQw8D68A7+t/8M7Y39Z6Lv4Fkqbd6rV0+TbSrA6bPhbfNhdcIz1Wfs4XJyDg8pt4W/25rLoC0gMaeH0hZnrVTt5MLVT/aB2qh/UTvWDM4LfeS/IZjKZ8Pf3r/hjGAZ5eXm8++67vPrqqwwePJjHH3+c8PDw2qhXnMR/8wr8f/sSAIdPAAX9JtRq6DtuzWft8d38mptWJeQBmDHhb/GmiV8Izf3C6BMWT6CWbhEREalV5xz8tm7dSlJSEl988QWHDx/G29ubgQMHMmXKFC655BIAPvvsMx599FGmTZvGm2++WVs1Sy2zZB3Cb/saDJOJooQxFLfrD961E7pK7DbWHN/J9yf248DAjIkQL38K7VZCvf3pG9aSbiHN8DFrLXERERFnq/Fv26effpqVK1eSlpaGyWSiZ8+e3HHHHQwbNozg4OBKx44dO5Yvv/yS7777rtYKllpmOAj4aRkmw6C4w8UUdxxcKx/rMAy25B7ifxnJ5NqKMQHdQ5ozKKIt4T6uXQtQREREytQ4+L322mtceOGF3HjjjYwcOZLo6OgzHt+jRw+6det2zgWKc/ns/QnvjAM4/BtR1HXEeX+e3XCwPS+drzP3cKQkF4BYvxAuj+lCrF/oeX++iIiInLsaB7+kpCRatmx52v0Oh4O0tDSaNy97ysItt9xy7tWJ0/nt/BaAwu5jMHz8z+uzkvOO8MnRreTaigEI8fJnSFR7ujZqpsWXRURE6oAaL+A8cuRIVqxYcdr9H374IVdeeeV5FSWuYbIWYslKwzBbsLboel6ftS33MO+kbSDXVkykTyCXN+7Mfa0uISGkuUKfiIhIHVFtj9/Ro0f5/vvvK14bhsHPP/+MzWarcqzD4eDTTz91+vIbcu78fv0Ck91GUcIovI7tx4RBaUSL83oqx6+5aSw7vBkHBheHt2FIVAeFPRERkTqo2uAXHh7OggULOHDgAFC2fMuSJUtYsmTJad9z00031VqBUnssJ9IJ2JIEgDWuG15H9wBga1yzBZJ/yTlEod1Kl0axbM09zOfHtmEAAyPaMiSyvYK/iIhIHVVt8PP29mbhwoUcOnQIwzC4+eabmTx5Mv37969yrNlsJjw8vEaLNq9YsYKXXnqJ1NRUYmNjmTx58hmHih0OBy+//DLLli0jIyODuLg4pkyZwqhRo876nJ7Kd/f63/++7ye8ju0DoLRx67P+jB9O7GfF0W0ArDy2vWJNvqFRHbg4vI1Cn4iISB12VpM7mjZtStOmTQF44oknSExMrJi8cT6SkpKYPn06EydOZMCAAaxevZoZM2bg5+fH8OHDT/mef//73yxZsoRp06bRoUMHPvvsMx544AGCgoIYOHDgedfUYNms+OzfUPHSZ98GTNZCDJMZW9TpJ+v80W+5h/nsZOiL8w8npSgLMyaubNKV7iHn/+9BREREnKvGs3rHjh1bayd/9tlnGTFiBLNmzQJgwIAB5OTkMG/evFMGv5SUFN5++23+8Y9/cM011wDQt29fDhw4wLp16xT8zsDn4C+YrUXYIppjslmx5BwFwBbZ4oyLNTsMg535R/kx+wB7CjIAGBLZnkGR7cgtLabUsBOhdflERETqhWqD3wUXXMCcOXMYM2YMAB06dKh2OM9kMrF9+/YzHpOamkpKSgrTpk2rtH3YsGEkJSWRmppapVdx9erV+Pn5VRkKfuutt6r7Gh7Pd3fZBJ2Stv0wlRQQsLlsZvaZhnl35x/jfxnJHC7JAcDLZOai8NYMjGgLQCNvPydXLSIiIrWp2uB35ZVX0qJFi0qva+M+rn37yu4v+/OagHFxcQDs37+/SvDbuXMnLVu2ZP369TzzzDPs2bOHZs2acd999zFy5Mjzrqmh8jqyG++M/RjevpTEd8dkLcJ/82eYME45scNhGKw89hvrT+wHINjLl/5hreke2pwAy7nP/hURERH3qjb4PfHEE5VeP/nkk7Vy4ry8PACCgoIqbQ8MLBs2zM/Pr/KerKws0tPTmTVrFvfeey/NmjVj6dKl3H///YSHh9OnT5+zPn9ERFD1B9WCqKjg6g9yIsNuw/HZhwCYe44gqmkkAI5ugzGOHSSkYzdMfxjqtdptvL5zPb+cOITFZObyuM4MbtoeH0vDfpauu9tJzo7aqX5QO9UPaqf6obbzitt+mxtG2WzQP/celm83m6uuLV1aWkpWVhYLFizgkksuAcru8du3bx8vvPBCjYJfZmY+DodxruWflaioYDIy8px6jur4bVtNQFY69uAocuL7Q3k9ncuG7sm2AtaK4z8/uo1fThzCz+zNDc0SaekXSU5WkesLd6G60E5SPbVT/aB2qh/UTvVDVFQwmZn5tRr+qg1+l156aY0/1GQysXr16jMeExxc9j+NP/fsFRQUVNr/R4GBgVgslkpLyZhMJvr168eyZctqXGdDZ87Pwv/XVQAU9LoKLN5nPD7TWsCPJw5gAm5p0UfP1hUREWlgqg1+5cu41Lbye/tSUlJo3759xfaDBw9W2v9HcXFxOBwObDYbPj6/32tWWlqq9eP+zDAI/GEJJruVkrhu2Jp2qPYt/8vYgR2D7iHNFfpEREQaoGqD33//+1+nnDguLo5mzZqxcuVKhg4dWrF91apVxMfHnzJwDhgwgNdff52kpCSuuuoqAGw2G+vWraNHjx5OqbO+8tn7E97pO3H4BFDY86pTHmMYBsuP/MqW3EM08QshtegE3iYzl0a2P+XxIiIiUr+59Y79qVOnMnPmTEJCQhg0aBBr1qwhKSmJuXPnAmWTOVJSUmjTpg1BQUH07duXgQMH8s9//pPCwkLi4+N55513SEtL45lnnnHnV6lTTIU5BGxcDkBhz3EY/qe+gfebrD1szEkBILXoBAD9wlsR4u3vmkJFRETEpc7qHr9Zs2ZV3Ot3Nvf8nc09fgDjxo3DarWycOFCli5dSvPmzZk9e3bF0ixr165l5syZLF68mN69ewMwf/585s2bxyuvvEJOTg4XXnghCxcupFOnTtWez1P4b12F2VqENfZCrC1P3RO6I+8IqzOSMQFXN0nA1+JNbmmRnsAhIiLSgJ3VPX4BAQGVXtem6667juuuu+6U+8aNG8e4ceMqbfPz82PGjBnMmDGjVutoKMz5Wfju+QEDE4XdL4dT3PuYWnSC9w9vwqDsGbtdQ5q5vlARERFxuRrf4+ese/6kdvhtXYXJYaekZQ8coTFV9h8ryWNx6o+UGnYSQppzcXjVBZxFRESkYTqve/wyMzNJS0vDbDbTvHlzQkJCaqsuOQfmvOP47v0Jw2SiqMuwKvvthoO3Dv1EkaOUDkGNuTKmi2ZDi4iIeJBzCn7ff/89Tz/9dJXn8SYmJjJr1iwuuOCCWilOasZ3z4+YDAclrRJxNIqusn9fwXGySgsJ8w7g2qbdsZiqLpItIiIiDVeNg9+3337L5MmTCQoK4sYbb6RFixY4HA4OHDjAp59+yoQJE3jrrbfo2LGjM+qVM/A6ugcAa1zCKff/knsIgISQ5viYG/Yj2ERERKSqGv/2nz9/Pi1atOC9996rMrQ7depUxo8fz5w5c1i0aFGtFSlnwWbFKzMFw2TCFl118esSh43teUcA6NYo1tXViYiISB1Q47G+5ORkxo8ff8r7+SIjI5kwYQJbtmypleLk7HllHMDksGMPa4bh8/ss7DxbMVaHje156ZQadlr4hxHuE+jGSkVERMRdatzjFx0dzYkTJ0673263Exqqx325mvfJYd7Sxq0rtqUVZfNKynd4m8z4msue09utkZZuERER8VQ17vGbMmUKixcvZt26dVX27dixg0WLFjFp0qRaKU7OXvn9fbbGZcuzGIbByozt2A0HxQ4bObYiLCYznRs559nLIiIiUvdV2+M3ceLEU26//fbbadOmDa1atcJkMpGWlsZvv/1GSEgI27Ztq/VC5QxsVryOH8TAhK1xKwD2FGSwvzATP7M318cmsj0vnVi/UPwtPm4uVkRERNyl2uB36NChKtvCwsIAKCgoYOvWrRXbY2LKFgzesGFDbdUnZ6H8/r780Bge3v8VzfxCKbCXADAwog2tAyNpHRjp5ipFRETE3aoNfmvWrHFFHXKuDAPf/WVBe3tgMHbDwcGiLAAaefnRJ6zqDF8RERHxTE5ZzC0rK4vw8HBnfLT8kWHgv/Hjk0/rMPNVo1B8zBaGR13I7oIMeofF4222uLtKERERqSPOKfgtX76cVatWUVhYiMPhqNhut9spKChgz549us/PBfw3r8B/x1oMs4VvugzmoLeDxOBYeoXF0yss3t3liYiISB1T4+D36quv8uyzz+Lt7U1QUBAnTpwgJiaG7OxsioqK8PPz46abbnJGrfIHPgc24//blxgmMzkDbubTknSwW0kMjXN3aSIiIlJH1Xg5lw8//JAOHTqwfv16lixZgmEYLF68mA0bNvDwww9TUlJC165dnVGrnGQ5kU7g9+8CUNjjCraERlJotxLj24hYv6oLa4uIiIjAOQS/tLQ0rrjiCoKCgmjevDkhISFs2LABi8XChAkTGDlypB7X5iSWE4cJXP8ujT5/BpPNSknLRArbX8RXx3cBkBjaApPJ5OYqRUREpK6q8VCvl5cXgYG/P/IrLi6OnTt3Vrzu3bs3c+fOrZ3qBACv9J34b12N99HdFdusLbpS0Odafsw+yDFrHmHeAfQIaeHGKkVERKSuq3Hwa926NZs3b+aaa64BoGXLlpUmcuTm5mK1WmuvQg9nKs4n+MuXMRkODC8fSlr3prjDxTgaRZFnK+bL42Whe1TjTprBKyIiImdU46HecePG8eGHHzJ9+nQKCwsZPHgwGzZs4IUXXuDzzz/nzTffpEOHDs6o1SN5HT+IyXBgC29G9lWPUtjrKhyNonAYBp8c2UqJw0b7wGg6BDV2d6kiIiJSx9W4x+/666/nyJEjvP3223h5eXHZZZcxatQoXnjhBQCCgoKYPn16rRfqqbyOHwSgNKYdhk8AUPYc3s+P/caO/CP4mr0Y2biTO0sUERGReuKc1vG7//77ufvuu/HyKnv7M888w/XXX092djYJCQlERETUapGezJKZAoAt8vf7977L2scPJ/ZjMZm5IbYnET6Bp3u7iIiISIVzfnKHl5cXmZmZpKWlYTabadu2LSEhWkqkVhkGXsfLgp89smx9vp35R/kiYzsAVzfpRis9g1dERETO0jkFv++//56nn36a7du3V9qemJjIrFmzuOCCC2qlOE9nzjuO2VqIwy8YR0Aox0ryeP/wJgzg0sj2dG4U6+4SRUREpB6pcfD79ttvmTx5MkFBQdx44420aNECh8PBgQMH+PTTT5kwYQJvvfUWHTt2dEa9DZbXsX0Efv8uhYljKY29sGzbH4Z5DWDJ4Y2UOGx0DG7CoIi2bqxWRERE6qMaB7/58+fTokUL3nvvvSpDu1OnTmX8+PHMmTNHizjXhOEg4KdlWHIz8E1e93vwOzmxwxYRx3FrAUdL8giw+DCuSTct1CwiIiI1VuPlXJKTkxk/fvwp7+eLjIxkwoQJbNmypVaK8xQ+B7fgdeIwAN5H94C9FKg8sWNvYQYArQMi8TWf862ZIiIi4sFqHPyio6M5ceLEaffb7XZCQ0PPqyiP4rDjvyUJAMNkxmQvxevYPnDY8co8BIA9ojl7CsqCX5vAKLeVKiIiIvVbjYPflClTWLx4MevWrauyb8eOHSxatIhJkybVSnGewHfvT1hyj2EPiqC4w8UAeB9OxuvoHkwOG/bgSGw+/uwvzASgtYKfiIiInKNqxwwnTpx4yu233347bdq0oVWrVphMJtLS0vjtt98ICQmp9Ag3OT1z3nECNiwHoKjbSBz+jfDfsRbvtB14H9sHQEmrXhwqyqbEYSPSJ5BQb393liwiIiL1WLXB79ChQ1W2hYWFAVBQUMDWrVsrtsfExACwYcOG2qqv4SktxpKbgSMghKB1izHZSiiJ64Y1vjs47BhePnjlHAHA4RdE8QUD2ZN9AIA2AertExERkXNXbfBbs2aNK+rwGMFrF+J9ZFfFa3tgGIV9rgWTCSxelDZui0/abwAUdR4G3r6/T+zQMK+IiIich3OeHmq329m2bRtpaWn4+PjQpEkTrd1XDZO1EK+juzFMJvDyAcOg4KKJFc/gBSiN7YBP2m/YA8MpaduXoyW5HCrKxoyJlgF6FJ6IiIicu3MKfl999RWPPfYYR48exTAMAEwmE9HR0TzyyCMMHjy4VotsKLyO7MFkGJRGtybvsrvAcIDZUumYkta9seRmUNKyB4U4eOvQzzgwSAhpjp/F202Vi4iISENQ4+C3YcMG7r77biIiIrj//vtp3bo1hmGwb98+3nnnHe655x4WL15M9+7dnVFvveZ9ZDcApTFty4Z2TZaqB3n5UNhzHHbDwbupP3CitJCmfiFc3rizi6sVERGRhqbGwe/5558nNjaWZcuWERwcXGnfhAkTuOqqq3jppZd49dVXa63IhsLrZPCzNWlX7bFrj+9mf2EmQRZfbojtibf5FCFRREREpAZqvI7fr7/+yjXXXFMl9AEEBQVx9dVX68kdp2AqzMEr5wiGlw+2iBZnPDalMIu1mbswAdc27U6IlnARERGRWlDj4Fcdk8lEaWlpbX9svVcxzBvdGiyn72jNt5WwNH0zBnBReGtaBUa6qEIRERFp6Goc/Lp27cqyZcsoLCyssi8/P5+lS5fSubPuR/uz8iVc/jzMuzv/GAsOrGNVxg625Bzihf1fc6K0kCa+jbg0qoM7ShUREZEGqsb3+N11111MnDiR0aNHc+ONNxIfHw9QMbnj6NGjPPbYY7VdZ/1mGBX395XGtK3YbHXY+OjIFnJtxRwqzq7YHu8fwbVNE/Ay1XqHrIiIiHiwGge/xMREnn/+ef7xj38wZ84cTCYTAIZhEBUVxdy5c+nTp0+tF1qfmfMzsRScwO4bgD2sacX2b7P2kmsrJsa3EbF+oewtyCAxtAUXR7TFfPLnKiIiIlJbahz8Tpw4waWXXsqgQYP47bffKh7pFhsbS8eOHfHyOuc1oRss8+FkAH4NCOLwiYP0CYsnz1bMusy9AIxu3Il4Lc4sIiIiTlbjlDZ27FiuueYapk6dSpcuXejSpYsz6mpQCg/9SiMguVEY645t4/sT+8i1FWMzHHQMbqLQJyIiIi5R4+CXlZVFVJSeGXu2HA47wccOAODdtCN+jkKySssmxjT1DWFE9IVurE5EREQ8SY1nD4wZM4YlS5ZUDPGerxUrVjBq1Ci6dOnCiBEjWL58+Vm/Nz09nR49evCf//ynVmpxhtTD2wi0Wcn28WNgq/5Maz2Y21r05//aDuPOlhcT6h1Q/YeIiIiI1IIa9/iZzWb27dvHsGHDaNGiBREREZjNlfOjyWRi0aJF1X5WUlIS06dPZ+LEiQwYMIDVq1czY8YM/Pz8GD58+BnfaxgGs2bNIj8/v6ZfwaWOH9wIQF50SwLNFgKwEBcQ7uaqRERExBPVOPh99913hIWFAVBSUsLhw4fP+eTPPvssI0aMYNasWQAMGDCAnJwc5s2bV23we+edd9i3b985n9sVimxWYjLTAAhu3g2Hm+sRERERz3bWwa+0tJQ9e/Ywb9482rRpg7//+T1GLDU1lZSUFKZNm1Zp+7Bhw0hKSiI1NZXmzZuf9r1PP/008+bN47bbbjuvOpzJarPSNq9sfT6jaXs3VyMiIiKe7qyC35tvvsmLL75YMazq4+PDhAkTeOCBB855+Zby3rqWLVtW2h4XFwfA/v37Txn8HA4HDz74ICNGjODiiy8+p3O7iqMoHz+HnQIvb4yAUHeXIyIiIh6u2tS2fPlynnzySWJjY7niiiswm838+OOPvPnmm9jt9oph2prKy8sDICgoqNL2wMBAgNPeu7do0SJSU1NZsGDBOZ23XEREUPUHnafMo5kAWC1eREUFO/18cu7UPvWD2ql+UDvVD2qn+qG280q1we+dd96hW7duLFq0CF9fX6BsYsX999/PkiVLmD59Oj4+PjU+sWEYABVP/vjz9j9PGIGyXsLnnnuO+fPnExx8fv9gMzPzcTiM8/qM6hilJQCUmi1kZOQ59Vxy7qKigtU+9YDaqX5QO9UPaqf6ISoqmMzM/FoNf9Uu57J3717GjBlTEfqgLKz95S9/wWq1nvMEi/Lg9ueevYKCgkr7y9ntdh588EGGDx9O//79sdls2Gw2oGz4t/zvdYnjZPCzWfQ0ExEREXG/aoNfUVHRKXvXmjVrhmEY5ObmntOJy+/tS0lJqbT94MGDlfaXS09PZ8uWLSxfvpyOHTtW/AF4/vnnK/5el5T3+NnMCn4iIiLiftUmEofDUWU4FsBisQBlPXHnIi4ujmbNmrFy5UqGDh1asX3VqlXEx8fTtGnTSsdHR0ezbNmyKp9z9dVXc/3113PVVVedUx3O9HuPn8XNlYiIiIicwzp+tWnq1KnMnDmTkJAQBg0axJo1a0hKSmLu3LlA2ePhUlJSaNOmDUFBQXTu3PmUnxMdHX3afe5kaKhXRERE6pCzSiTZ2dlVFmrOyckBysLZqRZx/nOP3amMGzcOq9XKwoULWbp0Kc2bN2f27NmMHDkSgLVr1zJz5kwWL15M7969z6bUOqU8+Nk11CsiIiJ1gMkon0Z7Gh06dDjlUC+UzcA91T6TycT27dtrp0InccWs3qLdq4n9YQVbm7QhdshdTj2XnDvNbqsf1E71g9qpflA71Q/OmNVbbVfU2LFja+1kHsdmBcCuoV4RERGpA6pNJE888YQr6miYShX8REREpO6odjkXOXemkz1+Dou3mysRERERUfBzLlspAHYFPxEREakDFPycqLzHz/BS8BMRERH3U/BzItPJHj+H7vETERGROkDBz4l+D37q8RMRERH3U/BzInPFUK+PmysRERERUfBzKvPJHj9DPX4iIiJSByj4OZHFbgPU4yciIiJ1g2YdOJHZfrLHT8FPRETqqKKiAvLzc7Cf/J0lrmOxeBMUFIK/f6DLzqng50SWk0O9KPiJiEgdVFpqJS/vBKGhkXh7+2IymdxdkscwDIPS0hKys4/j5eWNt7drsoKGep2ofKhXwU9EROqivLxsgoJC8PHxU+hzMZPJhI+PH4GBIeTnZ7vsvAp+TlQR/CwKfiIiUvfYbFZ8ff3dXYZH8/Pzp7TU6rLzKfg5i2HgVdHjp1m9IiJS9zgcdsxmi7vL8GhmswWHw+6687nsTJ7GYcMElJrMWMy6lVJEROomDfG6l6t//gp+TlL+nF6r2YzFpB+ziIiIuJ8SiZP8HvwsCn4iIiJ1mGEYLn2fOymROIt6/ERERNxq7949PPLITC6/fBiDBvXhiiuG8fDDM9m9e1fFMdu2/crf/35fjT/7u+/W8c9/PlKb5bqEEomTmE4uhFnW46f7J0RERFxpz57dTJlyC3l5edx//9+YO/dFpk69j/T0NCZPvoVt27YCsGLFx+zfv6/Gn//+++9y9OiR2i7b6TTrwEnKh3pLzRa81OMnIiLiUu+//w5hYeE89dQ8LJbfZy5fdNFAbrjhahYteo2nnprnxgrdQ8HPWU4+taPEonv8REREXO3EiSwMo+p9eAEBAdxzzzSKi4v5178eJSlpBQAXXZTIrFmPMHLkGNLSDrFw4cts2PAT2dnZNGoUQp8+/bj77mk0atSIu+66nV9+2VTxvvnzF9C9eyI5OdksWPAC69Z9TWFhIe3bd+COO+6mS5duLv/+p2N59NFHH3V3Ee5QVGTFmfdkWrLT8T2wmcP+gXi36Uugl6/zTibnJTDQl8JC1y2eKedG7VQ/qJ3qh/J2ys/PISgo1N3lOEVeXi7/+99KfvhhPTabjYCAQEJDy75ry5ataNu2Ha1bt+HIkcMUFOQzd+5/uPDCTphMJm699SZKS0uZMuVuLr98LJGRkSxbtoT8/Hz69buICy7oSHLyb4SHh/Ovfz1NmzZtMQyDqVNvZ/funUyefCejRl3B/v37WLjwFXr16kNUVPRpaz1dOwQG+lJUZCUgoPYeBKEePyf546zeAPX4iYiIuNS4cdeSmZnJe++9zbPPzgYgNDSM3r37cPXV13HBBR2JjW1GaGgY3t4+dOrUGYCdO5OJiWnC//t//6BJk6YAdO+eyPbt2yp6+Vq2bEVAQBB2u63ifZ988hF79+7m1VcX0aHDhQD06dOP2267mZdffpHnnvuPq38Ep6Tg5ywnh3pLzWZN7hAREXExk8nE7bffyfXX38SPP65nw4af2LRpA198kcSqVSu5776/cdVV11Z5X/v2HfjPf17D4XCQmprCoUOp7N+/j4MHD5zxfBs3/kRUVDRt2rTDZrNVbO/X7yL++983KC0txdvb/U/yUvBzEpNd6/iJiIi4W3BwMEOGDGPIkGEA7NqVzOOPP8yLL85j6NDhp3zPe++9xX//+wY5OTmEh0fQocMF+Pn5U1RUeNrz5OTkcOzYUQYN6nOa/dlERkad/xc6Twp+TqIFnEVERNzj2LGj3Hbbzdx22xRGj76y0r527Tpw2213MmvWdNLTD1d576pVK3nhhee48857GTlyTMV9gf/v/z3Irl3Jpz1nUFAQ8fEteeihx065PySkbtxLqUTiJJUe2aYfs4iIiMtEREQhl3FoAAAYHUlEQVRisVj48MOllJSUVNmfknIAX19fYmObYTZX/h3966+/EBoayoQJN1WEvsLCQn799Rccjt9nhVosld/XrVt3jhxJJzIyig4dLqz4s27d1yxd+h5eXnWjr02JxFm0gLOIiIhbWCwWpk37O3v37uHWW29i+fJlbN68ke+//47585/h1VdfYtKkKQQFBREUFExWVhbff/8dx48f58ILO5Kdnc1//jOPzZs3smpVElOn3kpWVibFxUUV5wgKCiY1NYWNG38mNzeXkSMvJzIymvvuu5MvvvicTZs28Pzzc1m06HWaNo3FVEeygJZzcRKf1K14ZaayJTSS1vG96kyDS1VafqJ+UDvVD2qn+sETlnNp0SKevn0vIi0tlf/97ws+/vgD1q1bi8lkYurU+xgz5goAmjaN5ccfv+ejj5YSGhrK2LHXAPDFF5/zyScfsXPnDhITezN69BV8+eUqhgwZRkhIKNHRMXz99Ro+/vhD2rRpS/v2HRg8eAgHDx7ggw/e57PPPiU3N4dbbrmVG264+Yy1unI5F5NRH58wXAsyM/MrddnWtoDv3sZv38+8FX8hIwfc7rTzyPmLigomIyPP3WVINdRO9YPaqX4ob6cjRw4SExPn7nI83unaISoqmMzMfCIigmrtXBrqdRLDVnZPgc1SN8b0RURERBT8nKR8coeCn4iIiNQVCn7OcnIBZwU/ERERqSsU/Jzl5ALOdgU/ERERqSMU/JykfKjXbnH/41lEREREQMHPacqDn6EePxEREakjFPycxHRyAWf1+ImIiEhdoeDnJOaTkzscdeQRLSIiIiIKfk5iPtnj57DU3mrbIiIiIudDwc8ZHHZMhgM7JjBZ3F2NiIiICAAah3SC8okdVrMZi1nZWkRExB02bdrA+++/w7ZtWyksLCAqKpq+ffszfvwNNGnS1N3luYVSiTNUBD8LXib9iEVERFxt4cJXuOeeKTgcBvff/3eefno+1113Iz///CO33DKBn3/+0d0luoXbU8mKFSsYNWoUXbp0YcSIESxfvvyMx2dkZPDQQw9xySWXkJCQwLhx40hKSnJRtWenfEav1WLBouAnIiLiUt98s5aFC1/httvuYM6cuVx66VC6d09k7Niree21/9KmTTseeWQWx48fd3epLufWVJKUlMT06dPp378/L774Ir169WLGjBmsXLnylMdbrVZuvfVW1q9fzz333MMLL7xAp06duO+++1ixYoWLqz8908kZvVazWcFPRETExd588zXi41ty882Tquzz9/dnxoyHyM3N4YMPlpCefpiLLkrk66/XMGvW3xg6dAAjRgxm9ux/UVxc7Ibqncut9/g9++yzjBgxglmzZgEwYMAAcnJymDdvHsOHD69y/DfffENycjJLly6lS5cuAPTv35/Dhw/z6quvMnr0aJfWfzr2kGgyYlrzta+Xgp+IiNQ7m3NS2Zid6u4y6BHanISQ5jV6T3Z2Nrt2JTNhwk2nPaZ58xa0bduO9evXcfnlYwF48sl/MmrU5TzxxDPs2PEbr7zyH8LDw7nttjvO6zvUNW5LJampqaSkpHDZZZdV2j5s2DD27dtHamrVf3CBgYGMHz+ezp07V9reqlUrUlJSnFpvjZgtbO55Od9GN1PwExERcaEjRw4DEBNz5skbsbHNOHIkveJ1//4DuOuu+0hM7MVNN91CQkIP1q9f59Ra3cFtPX779u0DoGXLlpW2x8XFAbB//36aN6+c8vv27Uvfvn0rbSstLeXrr7+mbdu2Tqy25uyGAYDFZHJzJSIiIjWTEFLznra6wjj5+9ermgcoWCwWTh4KQOfOXSvtj4qK5tixY7Ven7u5Lfjl5eUBEBQUVGl7YGAgAPn5+Wf1OU8//TQHDhzgxRdfrNH5IyKCqj/oPPhZyx7VFhTgR1RUsFPPJedPbVQ/qJ3qB7VT/RAVFcyxY2a8vBrWyFRsbCwAR4+mn/G7pacfJiYmBoul7JiAAP9Kx1ssFsDhkp+P2Ww+7XVT23nFbcGvPJGb/tQjVr7dXM36d4Zh8NRTT/Hmm28yadIkhgwZUqPzZ2bm43AY1R94jnLzigCwFtvIyMhz2nnk/EVFBauN6gG1U/2gdqofytvJ4XBgszncXU6tatQolI4dO7N27RomTZpyyjyRnn6YnTuTuf76m7Dby76/w2FU+lkYhoFh4JKfj8PhOOV1ExUVTGZmfq2GP7fF/ODgsmT75569goKCSvtPxWq18sADD/D6668zadIk/v73vzuv0HNkM8r+oegePxEREde65ZbbOHjwAK+9tqDKvpKSEp588nH8/QMYO/ZqN1TnXm7r8Su/ty8lJYX27dtXbD948GCl/X+Wn5/P5MmT2bRpE7NmzeLmm292frHnwK7gJyIi4hZ9+vTjjjvuZsGCF9izZxcjRowmPDyC1NSDvP/+uxw9eoTHHnuCxo1jSE8/7O5yXcptwS8uLo5mzZqxcuVKhg4dWrF91apVxMfH07Rp1dk4drudO+64gy1btlQsBVNXaXKHiIiI+9xww8107tyV999/h3nzniE3N6fikW3XXjuBpk1j3V2iW7h1Hb+pU6cyc+ZMQkJCGDRoEGvWrCEpKYm5c+cCkJWVRUpKCm3atCEoKIj33nuPn376ifHjx9OkSRN++eWXis8ymUx07dr1dKdyufIePz2yTURExD26dOlGly7dznhMkyZN+fbbDVW2/9//PeqkqtzLrcFv3LhxWK1WFi5cyNKlS2nevDmzZ89m5MiRAKxdu5aZM2eyePFievfuzRdffAHAkiVLWLJkSaXPslgsbN++3eXf4XQ01CsiIiJ1jVuDH8B1113Hddddd8p948aNY9y4cRWvFy9e7Kqyzlt58DOjoV4RERGpG9Qd5STq8RMREZG6RqnESeyUT+7Qj1hERETqBqUSJ9HkDhEREalrlEqcREO9IiIiUtcolTjJ78FPkztERESkblDwc5LfF3DWj1hERETqBqUSJ9FQr4iIiNQ1SiVOouAnIiIidY3bF3BuqBT8RERE3GvTpg28//47bNu2lcLCgopn9Y4ffwNNmjStOG7z5o089dS/SU8/TM+evXn00X/x+OMP89NPP+Dt7cPChW81mGf7Kvg5iYKfiIiI+yxc+AoLF75Cv34DuP/+vxMWFsbBgwdYtuw9Vq78jMcfn03Pnr0BeOml57Hb7cyZ8xwRERGsWpXEunVfM23aDFq2bEVMTBM3f5vao+DnJL9P7tCsXhEREVf65pu1LFz4Crfddgc33zypYnv37okMHz6Kv/3tXh55ZBaLFy8hMjKS3NwcOnfuWhEE1637GoCxY6/G1MB+jyv4OYlNPX4iIiJu8eabrxEf37JS6Cvn7+/PjBkPcf3143jxxef43/9WAnDoUCorV35Gt27d+eWXTQAMGNCTESNG83//96gry3cqpRIn0ZM7REREXC87O5tdu5Lp1++i0x7TvHkL2rZtx65dySxY8AbR0Y3p27c/Cxa8wX33/Y0rrhgHwIIFb/CXv9zqqtJdQj1+TlIe/Mw0rC5iERFp+Hz2/oTv3h/dXQYlrXtjbd2rRu85cuQwADExTc94XGxsM37++Uc6deqMt7c3oaFhdOrUGYCoqGiAitcNibqjnMSOhnpFRERczTh5j72X15n7tiwWCycP9Sjq8XMSPblDRETqK2vrXjXuaasrymfgpqcfPuNxhw+n0bhxY1eUVKcolTiBYRhazkVERMQNwsLC6dixM9988xUOh+OUx6SnH2bXrp3073+xi6tzP6USJ3BQ1ttnxoS5gU0DFxERqetuueU2Dh48wGuvLaiyr6SkhCeffBx//wDGjr3aDdW5l4Z6naCit8+sXC0iIuJqffr044477mbBghfYs2cXI0aMJjw8gtTUg7z//rscPXqExx57gsaNY9xdqssp+DlB+f19WspFRETEPW644WY6d+7K+++/w7x5z5Cbm1PxyLZrr53QYB7BVlMKfk5QsYafevxERETcpkuXbnTp0q3a45YsWV7p9V/+cmuDW7+vnJKJE2hih4iIiNRFSiZOoB4/ERERqYuUTJzg9x4/i5srEREREfmdgp8T/L54s5ZyERERkbpDkzucIMInkFi/EHpExbm7FBEREZEKCn5O4G22cEf8xURFBZORkefuckRERE7LMAxMGqFyG8PFDwzWUK+IiIiHsli8KC21ursMj1ZaasVicV0/nIKfiIiIhwoKCiU7OwOrtcTlPU+ezjAMrNYSsrMzCAoKddl5NdQrIiLiofz9AwHIyTmO3W5zczWex2LxIjg4rKIdXEHBT0RExIP5+we6NHiIe2moV0RERMRDKPiJiIiIeAgFPxEREREPoeAnIiIi4iEU/EREREQ8hMfO6jWbXbNKuavOI+dH7VQ/qJ3qB7VT/aB2qh9qu51MhlZsFBEREfEIGuoVERER8RAKfiIiIiIeQsFPRERExEMo+ImIiIh4CAU/EREREQ+h4CciIiLiIRT8RERERDyEgp+IiIiIh1DwExEREfEQCn5OsGLFCkaNGkWXLl0YMWIEy5cvd3dJHs9ms9GlSxfat29f6U9CQkLFMd9++y1XXXUVXbt2ZfDgwSxcuNCNFXuWHTt20LFjR44cOVJp+9m0ydatW7nppptISEjgoosu4tlnn6W0tNRVpXuU07XT0KFDq1xb7du3Jysrq+IYtZNzORwO3n33XcaMGUNCQgJDhgzhiSeeID8/v+IYXU/udzbt5OzryWOf1essSUlJTJ8+nYkTJzJgwABWr17NjBkz8PPzY/jw4e4uz2Pt37+fkpISZs+eTXx8fMV2s7ns/z6bNm1iypQpjBgxgnvvvZeNGzcyZ84cDMNg0qRJbqraM+zbt4/Jkydjs9kqbT+bNjl48CB/+ctfSEhI4LnnnmPv3r3MnTuX/Px8Hn74YXd8nQbrdO1UUFBAamoqDzzwAL169aq0r1GjRoDayRVee+01nnvuOSZNmkTfvn3Zv38/8+fPZ8+ePbz++uu6nuqI6trJJdeTIbVqyJAhxn333Vdp27333msMHz7cTRWJYRjGJ598YnTo0MEoLCw85f6bb77ZuOaaayptmzNnjpGYmGiUlJS4okSPU1paarz11ltGQkKC0atXL6Ndu3ZGenp6xf6zaZNZs2YZAwcOrNRGb7/9tnHBBRcYR44ccc0XaeCqa6eNGzca7dq1M/bs2XPaz1A7OZfD4TB69uxpPProo5W2f/bZZ0a7du2M7du363qqA86mnVxxPWmotxalpqaSkpLCZZddVmn7sGHD2LdvH6mpqW6qTHbs2EGLFi3w9/evsq+kpIQNGzacst1yc3PZtGmTq8r0KBs3buTpp5/mr3/9K9OnT6+072zb5LvvvuOSSy7Bx8en4pjhw4djt9v59ttvnf8lPMCZ2gnKri1fX99KPel/pnZyroKCAi6//HJGjx5daXurVq0A2L17t66nOqC6dkpJSXHJ9aTgV4v27dsHQMuWLSttj4uLA8qGG8U9du7ciY+PD5MmTSIhIYGePXvy8MMPk5+fT2pqKqWlpWo3F2vdujWrV6/mrrvuwmKxVNp3Nm1SVFREenp6lWPCw8MJCgpSu9WSM7UTlF1boaGhTJs2jcTERBISErj//vvJyMgAUDu5QFBQEA899BA9evSotH316tUAXHjhhbqe6oDq2qlNmzYuuZ4U/GpRXl4eUNa4fxQYGAhQ6eZNca3k5GRSUlIYOHAgr7zyCnfeeScrVqzgjjvuULu5SWRkJBEREafcdzZtcrpjyo9Tu9WOM7UTlF1bx48fp23btixYsICZM2fy888/M3HiRIqLi9VObrJlyxZeeeUVhgwZouupDvtjO7Vu3dol15Mmd9QiwzAAMJlMp9xePpFAXG/u3LmEhITQvn17AHr27ElERAR/+9vf+O6774Cq7VZO7eZ6p7uWypnN5jMeYxiG2s1FHnroIQzDoGvXrgAkJibSunVrJkyYwCeffMLAgQMBtZMrbdy4kSlTptCsWTP++c9/VvQC6XqqW/7cTuCa60ktWYuCg4OBqj1EBQUFlfaL6/Xq1asi9JUbNGhQpdd/brfy12o31zvdtfTHNin/H++p/odbWFiodnORLl26VPySKtejRw+Cg4NJTk5WO7nY559/zi233EKTJk148803CQsL0/VUB52qncA115OCXy0qH3NPSUmptP3gwYOV9otrZWZmsnTp0iqTa4qLiwGIiIjAYrFUabfy12o312vRokW1bRIYGEjjxo0rrq9ymZmZ5Ofnq91coLCwkA8++IDk5ORK2w3DoLS0lLCwMLWTC73xxhtMmzaNbt268fbbbxMdHQ3oeqprTtdOrrqeFPxqUVxcHM2aNWPlypWVtq9atYr4+HiaNm3qpso8m8lk4uGHH+att96qtP3zzz/HYrHQr18/EhMTWbVqVcVwB8AXX3xBcHAwnTp1cnXJHs/X1/es2qR///589dVXWK3WSsdYLJYqa2BJ7fP19WX27Nm88MILlbZ/+eWXFBcXV7SB2sn5li5dypNPPsmIESN47bXXKvX86HqqO6prJ1dcT5ZHH3300dr5OgJlXeYvvfQSJ06cwGQy8cYbb/DRRx/xyCOP0LZtW3eX55H8/f3Jzs7m7bffxuFw4HA4+Pjjj5k/fz4TJkxgzJgxxMTEsGDBAvbu3Yu/vz/Lly/n1Vdf5e6776Z3797u/goN3o4dO/jyyy+55ZZbKoYyzqZNWrZsycKFC9mwYQMhISGsXbuWp556imuuuYYxY8a48ys1SH9uJ7PZjJeXF4sXLyYnJwcvLy++/PJL/vWvfzFgwAAmT54MqJ2cLTMzk1tvvZXGjRvzwAMPkJmZyZEjRyr++Pj40LJlS11PblZdO/n5+REcHOz86+nslx6Us/Xuu+8aQ4cONTp16mSMGDHC+Oijj9xdksezWq3GK6+8YgwbNszo1KmTcemllxovv/yyYbfbK45ZtWqVMXr0aKNjx47G4MGDjddff92NFXuWDz74oMrCwIZxdm3y888/G9dcc43RqVMnY8CAAcYzzzxjWK1WV5XuUU7XTu+//74xevRoo0uXLsaAAQOMOXPmGEVFRZWOUTs5z0cffWS0a9futH+WL19uGIauJ3c723Zy9vVkMow/9PuKiIiISIOle/xEREREPISCn4iIiIiHUPATERER8RAKfiIiIiIeQsFPRERExEMo+ImIiIh4CAU/EREREQ/h5e4CRETc6cEHH+Sjjz6q9rjY2FjS0tJYvHixnuYiIvWWFnAWEY+2efPmSg+v37hxI0uWLGH8+PH06NGjYnuzZs04dOgQ/fv3JzIy0h2lioicN/X4iYhHS0hIICEhoeK13W5nyZIldOvWjSuuuKLSsX8MgiIi9ZHu8RMRERHxEAp+IiJn4cMPP6R9+/b8+OOPlV4nJydzzz33kJCQQJ8+fZg9ezZ2u52PPvqIYcOG0a1bN6677jqSk5MrfV5OTg6PP/44AwYMoFOnTowYMYJFixahu29ExJk01Csich5uv/12evTowYMPPsiqVatYuHAhu3btYufOndx8880YhsFLL73EPffcw+eff46XlxeFhYXceOONpKenM2HCBGJiYvjhhx/497//zYEDB3jkkUfc/bVEpIFS8BMROQ/dunVj7ty5AIwcOZK+ffuyfv16PvnkE9q2bQtAQUEBCxYs4NChQ8THx/P666+zf/9+PvjgA9q3bw/AhAkTePbZZ3n55ZcZP348HTp0cNt3EpGGS0O9IiLnYciQIRV/Dw4OJjw8nPj4+IrQB2UzggEyMjIAWLVqFe3atSMqKoqsrKyKP+Wf9dVXX7nwG4iIJ1GPn4jIefjz0i5eXl5ERERU2maxWABwOBwApKSkUFxcTN++fU/5menp6U6oVEREwU9E5LyUh7o/MplMZ3yP3W6nR48e3HXXXafcHx0dXSu1iYj8mYKfiIiLxcbGUlBQQL9+/Sptz8nJ4fvvvycuLs5NlYlIQ6d7/EREXGzw4MEkJyezdu3aSttfeukl7r33Xnbv3u2ewkSkwVOPn4iIi02ePJlVq1Zx1113cd1119G2bVs2btzIxx9/zMUXX8zFF1/s7hJFpIFS8BMRcbHQ0FCWLFnC/PnzWblyJUuWLKFp06bceeed3H777ZjNGowREecwGVomXkRERMQj6L+VIiIiIh5CwU9ERETEQyj4iYiIiHgIBT8RERERD6HgJyIiIuIhFPxEREREPISCn4iIiIiHUPATERER8RAKfiIiIiIe4v8DjfttPM194a4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.rcParams['figure.figsize'] = (10,5)\n",
    "sns.lineplot(data=probs, x='Time', y='Probability', hue='State', palette=\"Set2\", linewidth = 2)\n",
    "plt.title(\"Without class balancing\")\n",
    "sns.set(font_scale = 1.5)\n",
    "plt.savefig(os.path.join('./images', 'class_not_balanced_probs.png'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}