examples/sklearn/demo_new_features.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zyACqF5CQKUU"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Trusted-AI/AIF360/blob/master/examples/sklearn/demo_new_features.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3P12Gr-RQW2e",
    "outputId": "4a6126f9-198f-4b88-c9cf-f44a932ecaf0"
   },
   "outputs": [],
   "source": [
    "#Install AIF360\n",
    "!pip install 'aif360'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KFgI8U9ZQX7g",
    "outputId": "b47381c9-8f71-4e59-ee4d-e63730211573"
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow.compat.v1 as tf\n",
    "tf.disable_eager_execution()\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)\n",
    "\n",
    "from sklearn.compose import make_column_transformer\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import GridSearchCV, train_test_split\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "from aif360.sklearn.preprocessing import Reweighing, ReweighingMeta\n",
    "from aif360.sklearn.inprocessing import AdversarialDebiasing\n",
    "from aif360.sklearn.postprocessing import CalibratedEqualizedOdds, PostProcessingMeta\n",
    "from aif360.sklearn.datasets import fetch_adult\n",
    "from aif360.sklearn.metrics import disparate_impact_ratio, average_odds_error, generalized_fpr\n",
    "from aif360.sklearn.metrics import generalized_fnr, difference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zHi_Esd7QKUW"
   },
   "source": [
    "## Loading data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JBSgJWDJQKUX"
   },
   "source": [
    "Datasets are formatted as separate `X` (# samples x # features) and `y` (# samples x # labels) DataFrames. The index of each DataFrame contains protected attribute values per sample. Datasets may also load a `sample_weight` object to be used with certain algorithms/metrics. All of this makes it so that aif360 is compatible with scikit-learn objects.\n",
    "\n",
    "For example, we can easily load the Adult dataset from UCI with the following line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 238
    },
    "id": "CVuqa6BpQKUX",
    "outputId": "cc44625b-359a-46cd-fc15-cc15e90412b3"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>education-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Non-white</th>\n",
       "      <th>Male</th>\n",
       "      <td>25.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">White</th>\n",
       "      <th>Male</th>\n",
       "      <td>38.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Farming-fishing</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Male</th>\n",
       "      <td>28.0</td>\n",
       "      <td>Local-gov</td>\n",
       "      <td>Assoc-acdm</td>\n",
       "      <td>12.0</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Protective-serv</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Non-white</th>\n",
       "      <th>Male</th>\n",
       "      <td>44.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>10.0</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>7688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>White</th>\n",
       "      <th>Male</th>\n",
       "      <td>34.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>10th</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Other-service</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 age  workclass     education  education-num  \\\n",
       "race      sex                                                  \n",
       "Non-white Male  25.0    Private          11th            7.0   \n",
       "White     Male  38.0    Private       HS-grad            9.0   \n",
       "          Male  28.0  Local-gov    Assoc-acdm           12.0   \n",
       "Non-white Male  44.0    Private  Some-college           10.0   \n",
       "White     Male  34.0    Private          10th            6.0   \n",
       "\n",
       "                    marital-status         occupation   relationship   race  \\\n",
       "race      sex                                                                 \n",
       "Non-white Male       Never-married  Machine-op-inspct      Own-child  Black   \n",
       "White     Male  Married-civ-spouse    Farming-fishing        Husband  White   \n",
       "          Male  Married-civ-spouse    Protective-serv        Husband  White   \n",
       "Non-white Male  Married-civ-spouse  Machine-op-inspct        Husband  Black   \n",
       "White     Male       Never-married      Other-service  Not-in-family  White   \n",
       "\n",
       "                 sex  capital-gain  capital-loss  hours-per-week  \\\n",
       "race      sex                                                      \n",
       "Non-white Male  Male           0.0           0.0            40.0   \n",
       "White     Male  Male           0.0           0.0            50.0   \n",
       "          Male  Male           0.0           0.0            40.0   \n",
       "Non-white Male  Male        7688.0           0.0            40.0   \n",
       "White     Male  Male           0.0           0.0            30.0   \n",
       "\n",
       "               native-country  \n",
       "race      sex                  \n",
       "Non-white Male  United-States  \n",
       "White     Male  United-States  \n",
       "          Male  United-States  \n",
       "Non-white Male  United-States  \n",
       "White     Male  United-States  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X, y, sample_weight = fetch_adult()\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YoUxNPTHQKUY"
   },
   "source": [
    "We can then map the protected attributes to integers,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "KoIzyUpIQKUY"
   },
   "outputs": [],
   "source": [
    "X.index = pd.MultiIndex.from_arrays(X.index.codes, names=X.index.names)\n",
    "y.index = pd.MultiIndex.from_arrays(y.index.codes, names=y.index.names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FpMD-pRaQKUY"
   },
   "source": [
    "and the target classes to 0/1,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "4FuupoA3QKUY"
   },
   "outputs": [],
   "source": [
    "y = pd.Series(y.factorize(sort=True)[0], index=y.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3T-NvfWyQKUZ"
   },
   "source": [
    "split the dataset,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "Vc4qGCNGQKUZ"
   },
   "outputs": [],
   "source": [
    "(X_train, X_test,\n",
    " y_train, y_test) = train_test_split(X, y, train_size=0.7, random_state=1234567)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1YBoWRhSQKUZ"
   },
   "source": [
    "and finally, one-hot encode the categorical features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 366
    },
    "id": "D1RI5-mUQKUZ",
    "outputId": "633dc1a0-c526-4d4f-9af0-21550a873740"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>workclass_Federal-gov</th>\n",
       "      <th>workclass_Local-gov</th>\n",
       "      <th>workclass_Private</th>\n",
       "      <th>workclass_Self-emp-inc</th>\n",
       "      <th>workclass_Self-emp-not-inc</th>\n",
       "      <th>workclass_State-gov</th>\n",
       "      <th>workclass_Without-pay</th>\n",
       "      <th>education_10th</th>\n",
       "      <th>education_11th</th>\n",
       "      <th>education_12th</th>\n",
       "      <th>...</th>\n",
       "      <th>native-country_Thailand</th>\n",
       "      <th>native-country_Trinadad&amp;Tobago</th>\n",
       "      <th>native-country_United-States</th>\n",
       "      <th>native-country_Vietnam</th>\n",
       "      <th>native-country_Yugoslavia</th>\n",
       "      <th>age</th>\n",
       "      <th>education-num</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">1</th>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>42.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1887.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 103 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          workclass_Federal-gov  workclass_Local-gov  workclass_Private  \\\n",
       "race sex                                                                  \n",
       "1    1                      0.0                  0.0                0.0   \n",
       "     0                      0.0                  0.0                0.0   \n",
       "     1                      0.0                  0.0                1.0   \n",
       "     1                      0.0                  0.0                1.0   \n",
       "     1                      0.0                  0.0                1.0   \n",
       "\n",
       "          workclass_Self-emp-inc  workclass_Self-emp-not-inc  \\\n",
       "race sex                                                       \n",
       "1    1                       0.0                         1.0   \n",
       "     0                       0.0                         1.0   \n",
       "     1                       0.0                         0.0   \n",
       "     1                       0.0                         0.0   \n",
       "     1                       0.0                         0.0   \n",
       "\n",
       "          workclass_State-gov  workclass_Without-pay  education_10th  \\\n",
       "race sex                                                               \n",
       "1    1                    0.0                    0.0             0.0   \n",
       "     0                    0.0                    0.0             0.0   \n",
       "     1                    0.0                    0.0             0.0   \n",
       "     1                    0.0                    0.0             0.0   \n",
       "     1                    0.0                    0.0             1.0   \n",
       "\n",
       "          education_11th  education_12th  ...  native-country_Thailand  \\\n",
       "race sex                                  ...                            \n",
       "1    1               0.0             0.0  ...                      0.0   \n",
       "     0               0.0             0.0  ...                      0.0   \n",
       "     1               0.0             0.0  ...                      0.0   \n",
       "     1               0.0             0.0  ...                      0.0   \n",
       "     1               0.0             0.0  ...                      0.0   \n",
       "\n",
       "          native-country_Trinadad&Tobago  native-country_United-States  \\\n",
       "race sex                                                                 \n",
       "1    1                               0.0                           1.0   \n",
       "     0                               0.0                           0.0   \n",
       "     1                               0.0                           1.0   \n",
       "     1                               0.0                           0.0   \n",
       "     1                               0.0                           1.0   \n",
       "\n",
       "          native-country_Vietnam  native-country_Yugoslavia   age  \\\n",
       "race sex                                                            \n",
       "1    1                       0.0                        0.0  58.0   \n",
       "     0                       0.0                        0.0  51.0   \n",
       "     1                       0.0                        0.0  26.0   \n",
       "     1                       0.0                        0.0  44.0   \n",
       "     1                       0.0                        0.0  33.0   \n",
       "\n",
       "          education-num  capital-gain  capital-loss  hours-per-week  \n",
       "race sex                                                             \n",
       "1    1             11.0           0.0           0.0            42.0  \n",
       "     0             12.0           0.0           0.0            30.0  \n",
       "     1             14.0           0.0        1887.0            40.0  \n",
       "     1              3.0           0.0           0.0            40.0  \n",
       "     1              6.0           0.0           0.0            40.0  \n",
       "\n",
       "[5 rows x 103 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ohe = make_column_transformer(\n",
    "        (OneHotEncoder(sparse=False), X_train.dtypes == 'category'),\n",
    "        remainder='passthrough', verbose_feature_names_out=False)\n",
    "X_train  = pd.DataFrame(ohe.fit_transform(X_train), columns=ohe.get_feature_names_out(), index=X_train.index)\n",
    "X_test = pd.DataFrame(ohe.transform(X_test), columns=ohe.get_feature_names_out(), index=X_test.index)\n",
    "\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TsmP5ozGQKUa"
   },
   "source": [
    "Note: the column names are lost in this transformation. The same encoding can be done with Pandas, but this cannot be combined with other preprocessing in a Pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 366
    },
    "id": "_pBLAXtIQKUa",
    "outputId": "b18a38cf-fa47-4728-c66b-0c258b625e21"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>education-num</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>workclass_Private</th>\n",
       "      <th>workclass_Self-emp-not-inc</th>\n",
       "      <th>workclass_Self-emp-inc</th>\n",
       "      <th>workclass_Federal-gov</th>\n",
       "      <th>workclass_Local-gov</th>\n",
       "      <th>...</th>\n",
       "      <th>native-country_Guatemala</th>\n",
       "      <th>native-country_Nicaragua</th>\n",
       "      <th>native-country_Scotland</th>\n",
       "      <th>native-country_Thailand</th>\n",
       "      <th>native-country_Yugoslavia</th>\n",
       "      <th>native-country_El-Salvador</th>\n",
       "      <th>native-country_Trinadad&amp;Tobago</th>\n",
       "      <th>native-country_Peru</th>\n",
       "      <th>native-country_Hong</th>\n",
       "      <th>native-country_Holand-Netherlands</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <td>25.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>1</th>\n",
       "      <td>38.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <td>44.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>7688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <th>1</th>\n",
       "      <td>34.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 103 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           age  education-num  capital-gain  capital-loss  hours-per-week  \\\n",
       "race sex                                                                    \n",
       "0    1    25.0            7.0           0.0           0.0            40.0   \n",
       "1    1    38.0            9.0           0.0           0.0            50.0   \n",
       "     1    28.0           12.0           0.0           0.0            40.0   \n",
       "0    1    44.0           10.0        7688.0           0.0            40.0   \n",
       "1    1    34.0            6.0           0.0           0.0            30.0   \n",
       "\n",
       "          workclass_Private  workclass_Self-emp-not-inc  \\\n",
       "race sex                                                  \n",
       "0    1                    1                           0   \n",
       "1    1                    1                           0   \n",
       "     1                    0                           0   \n",
       "0    1                    1                           0   \n",
       "1    1                    1                           0   \n",
       "\n",
       "          workclass_Self-emp-inc  workclass_Federal-gov  workclass_Local-gov  \\\n",
       "race sex                                                                       \n",
       "0    1                         0                      0                    0   \n",
       "1    1                         0                      0                    0   \n",
       "     1                         0                      0                    1   \n",
       "0    1                         0                      0                    0   \n",
       "1    1                         0                      0                    0   \n",
       "\n",
       "          ...  native-country_Guatemala  native-country_Nicaragua  \\\n",
       "race sex  ...                                                       \n",
       "0    1    ...                         0                         0   \n",
       "1    1    ...                         0                         0   \n",
       "     1    ...                         0                         0   \n",
       "0    1    ...                         0                         0   \n",
       "1    1    ...                         0                         0   \n",
       "\n",
       "          native-country_Scotland  native-country_Thailand  \\\n",
       "race sex                                                     \n",
       "0    1                          0                        0   \n",
       "1    1                          0                        0   \n",
       "     1                          0                        0   \n",
       "0    1                          0                        0   \n",
       "1    1                          0                        0   \n",
       "\n",
       "          native-country_Yugoslavia  native-country_El-Salvador  \\\n",
       "race sex                                                          \n",
       "0    1                            0                           0   \n",
       "1    1                            0                           0   \n",
       "     1                            0                           0   \n",
       "0    1                            0                           0   \n",
       "1    1                            0                           0   \n",
       "\n",
       "          native-country_Trinadad&Tobago  native-country_Peru  \\\n",
       "race sex                                                        \n",
       "0    1                                 0                    0   \n",
       "1    1                                 0                    0   \n",
       "     1                                 0                    0   \n",
       "0    1                                 0                    0   \n",
       "1    1                                 0                    0   \n",
       "\n",
       "          native-country_Hong  native-country_Holand-Netherlands  \n",
       "race sex                                                          \n",
       "0    1                      0                                  0  \n",
       "1    1                      0                                  0  \n",
       "     1                      0                                  0  \n",
       "0    1                      0                                  0  \n",
       "1    1                      0                                  0  \n",
       "\n",
       "[5 rows x 103 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(X).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8bMlpomVQKUa"
   },
   "source": [
    "The protected attribute information is also replicated in the labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HSpthk1lQKUa",
    "outputId": "b98fb738-2dd1-44b7-fe64-2dbbcb59a6a2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "race  sex\n",
       "1     1      0\n",
       "      0      1\n",
       "      1      1\n",
       "      1      0\n",
       "      1      0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "13UfQJx_QKUb"
   },
   "source": [
    "## Running metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iMTOzFVLQKUb"
   },
   "source": [
    "With the data in this format, we can easily train a scikit-learn model and get predictions for the test data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tHeMqub4QKUb",
    "outputId": "7b06fd6c-bbc9-45d4-b6f2-daf3455a44fa"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8455074813886637"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = LogisticRegression(solver='liblinear').fit(X_train, y_train).predict(X_test)\n",
    "accuracy_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aVu8VzgpQKUb"
   },
   "source": [
    "Now, we can analyze our predictions and quickly calucate the disparate impact for females vs. males:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8mVD0ZWSQKUb",
    "outputId": "204a3f64-8306-469f-c39a-2fbf90f17039"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.26889803976599136"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "disparate_impact_ratio(y_test, y_pred, prot_attr='sex')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cNonH6t1QKUc"
   },
   "source": [
    "And similarly, we can assess how close the predictions are to equality of odds.\n",
    "\n",
    "`average_odds_error()` computes the (unweighted) average of the absolute values of the true positive rate (TPR) difference and false positive rate (FPR) difference, i.e.:\n",
    "\n",
    "$$ \\tfrac{1}{2}\\left(|FPR_{D = \\text{unprivileged}} - FPR_{D = \\text{privileged}}| + |TPR_{D = \\text{unprivileged}} - TPR_{D = \\text{privileged}}|\\right) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "C5_ZlM4dQKUc",
    "outputId": "2fd17013-5f9f-4131-8826-d3a50b7ef713"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.09875694175767563"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "average_odds_error(y_test, y_pred, priv_group=(1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H0VZVtSrQKUc"
   },
   "source": [
    "In that case, we chose to look at the intersection of all protected attributes (race and sex) and designate a single combination (white males) as privileged.\n",
    "\n",
    "If we wish to do something more complex, we can pass a custom array of protected attributes, like so (note: this choice of protected groups is just for demonstration):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LmcK2STxQKUc",
    "outputId": "eccc550b-b7cb-49c7-dca7-5ef1f2d621c2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3844295196608744"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "race = y_test.index.get_level_values('race').to_numpy()\n",
    "sex = y_test.index.get_level_values('sex').to_numpy()\n",
    "prot_attr = np.where(race ^ sex, 0, 1)\n",
    "disparate_impact_ratio(y_test, y_pred, prot_attr=prot_attr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vZRn0EJiQKUd"
   },
   "source": [
    "## Debiasing algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FJhOugvqQKUd"
   },
   "source": [
    "`ReweighingMeta` is a workaround until changing sample weights can be handled properly in `Pipeline`/`GridSearchCV`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KyGPWqeVQKUd",
    "outputId": "83e7a7b8-a6a5-4c27-fd73-fb300bb00b58"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8402004864745338\n",
      "{'estimator__C': 1}\n"
     ]
    }
   ],
   "source": [
    "rew = ReweighingMeta(estimator=LogisticRegression(solver='liblinear'),\n",
    "                     reweigher=Reweighing('sex'))\n",
    "\n",
    "params = {'estimator__C': [1, 10]}\n",
    "\n",
    "clf = GridSearchCV(rew, params, scoring='accuracy', cv=5)\n",
    "clf.fit(X_train, y_train)\n",
    "print(clf.score(X_test, y_test))\n",
    "print(clf.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WGz3Hn8UQKUd",
    "outputId": "7f2e11c0-dd77-4a0b-b813-02961a3f756a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5874230323221501"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "disparate_impact_ratio(y_test, clf.predict(X_test), prot_attr='sex')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "svwzVk2DQKUe"
   },
   "source": [
    "Rather than trying to weight accuracy and fairness, we can try a fair in-processing algorithm:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "waeF--xEQKUe",
    "outputId": "e68f4dbc-b526-49f2-e859-10d3e1f7d59a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8345249502469226"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adv_deb = AdversarialDebiasing(prot_attr='sex', random_state=1234567)\n",
    "adv_deb.fit(X_train, y_train)\n",
    "adv_deb.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PEXZO3CsQKUe",
    "outputId": "31b00ed9-8ccd-4cb6-b55a-a45bbca77368"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1200164835077702"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "average_odds_error(y_test, adv_deb.predict(X_test), prot_attr='sex')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ihLM59GVQKUe"
   },
   "source": [
    "Note that `AdversarialDebiasing` creates a TensorFlow session which we should close when we're finished to free up resources:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "pm75UZQEQKUe"
   },
   "outputs": [],
   "source": [
    "adv_deb.sess_.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MwUp9d4GQKUf"
   },
   "source": [
    "Finally, let's try a post-processor, `CalibratedEqualizedOdds`.\n",
    "\n",
    "Since the post-processor needs to be trained on data unseen by the original estimator, we will use the `PostProcessingMeta` class which splits the data and trains the estimator and post-processor with their own split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dCcMevWgQKUf",
    "outputId": "213d4a62-3898-4fb8-a61e-c56395d53660"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8200044224957618"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cal_eq_odds = CalibratedEqualizedOdds('sex', cost_constraint='fnr', random_state=1234567)\n",
    "log_reg = LogisticRegression(solver='liblinear')\n",
    "postproc = PostProcessingMeta(estimator=log_reg, postprocessor=cal_eq_odds, random_state=1234567)\n",
    "\n",
    "postproc.fit(X_train, y_train)\n",
    "accuracy_score(y_test, postproc.predict(X_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 283
    },
    "id": "oSElX6mEQKUf",
    "outputId": "7c6189a7-b615-4ce1-bf6a-ffdac35b0d49"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAEKCAYAAAAGkryaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABaH0lEQVR4nO3dd1hUZ/YH8O8ZOoIIig0QlI5gJahRY4sJ7s8YFY1tExPjqklcjUaT7Ka4MWuams2apqYZExN7EkvUxF1LqhE7HXRQVFAUpUgd5vz+uDPuSB3KMDNwPs/DA3Pn3jvnjsiZe+/7nkPMDCGEEEJYL5W5AxBCCCFEw0gyF0IIIaycJHMhhBDCykkyF0IIIaycJHMhhBDCykkyF0IIIaycSZM5EUUTUTIRpRHR81U8/y8iOqn7SiGimwbPTSeiVN3XdFPGKYQQQlgzMtU8cyKyAZACYCSAiwCOApjCzAnVrP9XAL2ZeQYReQCIBRAJgAEcA9CXmW+YJFghhBDCipnyzDwKQBozn2PmUgAbATxYw/pTAHyt+/l+AD8yc44ugf8IINqEsQohhBBWy9aE+/YCkGHw+CKAflWtSES+ALoC+G8N23pVsd0sALMAoFWrVn1DQkIaHrWZZGYCly8DvXsDKhnJICzUsWPHrjGzp7njEELcyZTJvC4mA9jKzOV12YiZ1wJYCwCRkZEcGxtritiaxOOPA7t3A8ePmzsSIapHROfNHYMQojJTngNeAuBj8Nhbt6wqk/G/S+x13bZZUKuBrl3NHYUQQghrZMpkfhRAIBF1JSJ7KAl7R8WViCgEgDuA3wwW7wNwHxG5E5E7gPt0y5otSeZCCCHqy2TJnJk1AOZCScKJADYzczwRLSWiMQarTgawkQ2G1TNzDoBXoXwgOApgqW5Zs6TRABkZksyFEELUj0nvmTPz9wC+r7Ds5QqP/1HNtp8C+NRkwVmQjAygvFySuWi+jh071t7W1vZjAOGQYlVC1IcWQJxGo5nZt2/fqxWftJQBcC2aWq18l2TezMXGAh9/DLz6KuDZsgaE29raftyxY8dQT0/PGyqVyjTFLYRoxrRaLWVnZ4dlZWV9DGBMxeflE7IFSE9Xvksyb+aWLwc2bgQcHc0diTmEe3p65kkiF6J+VCoVe3p65kK5ulX5+SaOR1RBrVbmlvv41L6usFJqNbB1KzB7NuDqau5ozEEliVyIhtH9H6oyb0sytwBqNeDtDdjZmTsSYTLvvAPY2ADz5pk7EiFEMyTJ3ALItLRmLidHuVc+dSrgVamQoRBCNJgkcwsgybyZW70aKCwEnnnG3JG0eF988UUbIup74sSJ2wMXkpOT7QMDA7sDwK5du1yHDRsW0NDXiYmJ8fvss8/cAWDSpEm+x44dcwQAZ2fn3g3Z765du1x//PHHVnXdzsvLKyIzM9OoAc+rVq1q+8gjj3Spe3RVGzJkSMC1a9dsAOCf//xn+27dunUfM2ZM1w0bNrj9/e9/79hYr6On1WrRv3//oJycHBUA2NjY9A0JCQnTfyUnJ9s39mvq1eV9NjRr1izvHTt2NOj+m4xmN7OiIqUuuyTzZqqkBFi1CoiOBiIizB1Ni7dx40aPPn36FKxfv96jd+/el5viNTdt2lSnErhlZWWwq+ae23//+19XFxeX8pEjR95qlOCawKFDh9L0P3/yySee+/fvT/H39y/TLco1dj81vS+GNm/e7Na9e/ciDw8PLQA4ODhok5KSquzWaSkWLVp09bHHHvMdM2ZMfn33IcnczM7r/ptLMm+mvvwSuHIFWLTI3JFYjBkz4BMXB+fG3Gd4OAo//fSO5kyV5Obmqo4ePeqyf//+5DFjxgT+61//MjqZazQaPPnkk94HDhxwIyKePn36tRdeeOHqokWLOu3du7dNSUmJKjIysmDDhg3nVRU6JUVFRQWvWLEi45577ikEgMcff9zn0KFDrT09Pcu2bdt2rnPnzpqoqKjg8PDwwj/++MMlJiYmJzg4uPiNN97oVFZWpnJ3d9ds2rTpXGFhoWr9+vWeKpWKN2/e3Padd9650KNHj+LHHnvM99KlS/YA8Pbbb1+47777bmVlZdnExMR0u3Llin3fvn0Lqmt1vXXr1tYvv/yyV3l5OXl4eGh+++23FMPnv/rqK7eKcfj4+Gh2797t8swzz3QBACLCr7/+mpSXl2cTExPTraCgwKa8vJzefffd89HR0QVeXl4RsbGxic8880znixcvOowaNSpw2rRp19zd3ctjY2NbrV+//sLly5dtqzqOhQsXdj537pzDhQsXHLy8vEqWLFmS+dhjj3UtKysjrVaLbdu2nY2IiCgxjHnDhg0es2fPvlbTv+dPP/3kvHDhQp/CwkKVu7u7ZsOGDem+vr5lUVFRwREREYVHjhxxKSwsVH322WfqZcuWdUpOTnZ68MEHc1atWnUZAO69917/zMxM+5KSEtWcOXOuLFq0qNLrffDBBx4ffvhhh7KyMurTp8+t9evXnweASZMm+Z0+fboVEfG0adOuLVmy5GpQUFDpzZs3bS9cuGDbpUsXTU2xV0cus5uZTEtrxrRaYOVKoFcvYPhwc0fT4n311Vdthg4dmtujR48Sd3d3zU8//WT0B4qVK1d6XrhwwT4hISE+JSUlYebMmdcBYPHixVfj4uISU1NT44uKilQbN250q2k/RUVFqsjIyFtpaWnxAwcOzH/++ec7658rLS2luLi4xFdeeeXKyJEjC06ePJmUmJiYMGHChJylS5d2DA4OLn3kkUey58yZcyUpKSkhOjq6YPbs2T4LFy68EhcXl/jNN9+cnTNnjh8APP/8850HDBhQkJaWFj9u3LibmZmZlS4tX7582Xbu3Ll+27dvP5ucnJzw7bffnq24TlVx6N6PjqtWrTqflJSU8Pvvvye5uLhoP/30U48RI0bkJiUlJSQmJsb369evsML7f6F9+/Zlhw4dSlmyZMkdRU+qOw4ASE1NdTx8+HDyzp071e+++67nk08+eSUpKSnh9OnTiV27di2tGPOxY8dcBg4cePvKRUlJiUp/iX3kyJH+JSUlNG/evC7ffffd2fj4+MTp06dfW7Ro0e3BLPb29tq4uLjExx57LHvixIkBH3300YWkpKT4TZs2tcvKyrIBgA0bNqTHx8cnnjx5MmHNmjUd9Mv1jh8/7rh161aP2NjYpKSkpASVSsWrV69u+9tvvzlnZmbapaamxqekpCQ89dRT1/XbREREFP73v/91qer3xhhyZm5m+oIxfn5mDUOYwp49QGIisGEDQGTuaCxGbWfQprJ582aPefPmXQWAmJiYnC+++MJj8ODBhbVtBwD//e9/W8+ZMydbf5m3Q4cO5QCwZ88e17fffrtjcXGx6ubNm7ZhYWFFqOHSsUqlwsyZM3MAYMaMGdfHjx9/+/78lClTbpesVqvV9mPHjvXOzs62Ky0tVfn4+JRUtb9ffvmldWpqqpP+cUFBgU1ubq7q999/d92+fXsaAEyePDl39uzZlTpSHjx4sFVUVFR+SEhIqeExGaoujv79+xcsWrTI56GHHsqZMmXKDX9/f23//v1vzZ4926+srEw1YcKEG3fffXdRze9q7ccBANHR0TddXFwYAAYMGHBrxYoVnS5evGg/efLkGxXPygEgNzfX1t3dXat/XPEy+9GjRx1TU1Odhg8fHgQo99g9PT31l/0xbty4mwDQs2fPooCAgCJfX98yAPDx8Sk5d+6cfceOHYvefPPNDrt3724DAFlZWXbx8fGOHTt2vP0BYu/eva5xcXHOPXv2DAWA4uJiVfv27TWTJk26mZGR4TB9+nSfBx54IHfcuHF5+m08PT01+isT9SHJ3MzUasDeHujcufZ1hZVZvlwpHjBxorkjafGuXLli8/vvv7smJyc7zZ07F+Xl5URErNVqL9Z3n4WFhfTMM8/4HjlyJCEgIKBs4cKFnYuLi+t0tZMMPuS5urreTkBz587tMn/+/Kxp06bl7tq1y3Xp0qVV/oVgZhw/fjzR2dnZJHP4q4vjtddeyxo7dmzud9995zZ48OCQ3bt3p44aNarg8OHDydu2bXObMWNG17lz516ZO3fu9dpeo7bjaNWq1e33Zc6cOTmDBw++9c0337iNHj068N133z1f8T6zjY0Nl5eXw8bGpuKu9K9FAQEBRSdPnkyq6nlHR0cGlA9eDg4Ot+NRqVTQaDS0a9cu10OHDrnGxsYmubq6aqOiooKLioru+HdnZpo4ceL1999/v1K3z7i4uIRvvvmm9erVqz03bdrksWXLlnQAKC4uJicnJ23F9Y0ll9nNTK0GfH2VojGiGTl6FDh0CFiwQAoIWIAvvvjCfdy4cTmXL18+c+nSpTNZWVmnvb29S/ft22fUZc0RI0bkrVmzpl1ZmXICd+XKFZvCwkIVAHTs2FGTm5ur2rlzp3tt+9FqtdCPcl+3bl3bqKioKgc85efn23Tp0qVMv55+uaura3l+fv7tLDVo0KC8119/vb3+8a+//uoEAP3798/Xb7d58+bWeXl5lTLb0KFDb/3xxx+uSUlJ9vpjMjaO+Ph4h6ioqKJly5Zl9ejR41ZcXJxjSkqKvbe3d9kzzzxz7ZFHHsk+fvy40bcxqjuOihISEuxDQ0NLXnzxxav333//zZMnT1Zar2vXrsWJiYkO1b1Wjx49inNycmz379/fCgBKSkooNjbW6LKMN2/etHFzcyt3dXXVnjhxwvHUqVOVZhdER0fn7dq1y/3SpUu2gPLepqSk2GdmZtqWl5fj0Ucfvfn6669fOnPmzO336OzZs449e/Y0+mpGRZJCzEympTVTK1YAbm7AzJnmjkQA2LJli8f48eNvGC578MEHb3z55Zcexmy/YMGCbG9v79KQkJDuwcHBYZ988olHu3btyqdNm5YdGhrafdiwYUE9e/asdYS5k5OT9o8//mgVGBjY/fDhw66vv/56ZlXrvfDCC5enTJni371799C2bdveHhAVExNzc/fu3W1CQkLC9u7d67J27dqM48ePtwoKCgrz9/fv/t5773kCwBtvvHH5l19+cQkICOi+fft2906dOlW6t9y5c2fNqlWr0seNGxcQHBwcNm7cuG7GxvHWW2+1DwwM7B4UFBRmZ2fHEyZMyN23b59raGho99DQ0LBt27Z5PPvss1eMeW8BoLrjqOjLL7/0CAoK6h4SEhKWmJjoNHv27Epn/vfdd1/uDz/8UO00L0dHR964cePZ559/3js4ODise/fuYYcOHTL6XnVMTEyuRqOhbt26dV+8eLFXVf/uffv2LX7xxRcvjRgxIigoKChs+PDhQRkZGXbp6el2gwYNCg4JCQl7+OGHuy1duvQioHygSE9Pd7jnnnvqPUuBqhvlaG0iIyM5NjbW3GHUWdu2ylXY1avNHYloNGo1EBCgjGB/801zR9OoiOgYM0fWdbtTp06l9+zZs8YRxkI0hvPnz9tNmTLF79dff001dyzGWr9+fZtjx445//vf/651hsWpU6fa9ezZ06/icjkzN6O8PKU4mJyZNzNSulUIs/H19S2bMWPGNX3RGGug0WjopZdeMvpKRlVkAJwZybS0ZkhKtwphdjNnzrxR+1qWY8aMGQ2O12o+uTRHMi2tGZLSrUIIM5Bkbkb6ZC5n5s2ElG4VQpiJSZM5EUUTUTIRpRHR89Ws8xARJRBRPBF9ZbC8nIhO6r52mDJOc1GrgVatgHbtzB2JaBRSulUIYSYmu2dORDYA3gcwEsBFAEeJaAczJxisEwjgbwAGMvMNImpvsIsiZu5lqvgsgX5amhQHawakdKsQwoxMeWYeBSCNmc8xcymAjQAerLDOXwC8z8w3AICZr6IFkTnmzYi+dOvixfLpzIJJC9TaNbcWqETU98EHH7z9l7asrAzu7u49a/t3ru/vQnFxMUVGRgbrCww1FVOOZvcC7qjBfBFAvwrrBAEAEf0CwAbAP5h5r+45RyKKBaAB8AYzf2vCWJscs5LM5SSumZDSrVZBWqA2PXO3QHVyctImJyc7FRQUkIuLC3/zzTetO3ToYLJM6+joyEOGDMn7+OOPPZ544omc2rdoHOYeAGcLIBDAUABTAHxERG10z/nqilNMBfAOEflX3JiIZhFRLBHFZmdnN1HIjeP6deDWLTkzbxakdGvdzJjhg6io4Eb9mjHDp7aX1bdA/eyzz9K/+eYboyq/6Wk0GsyaNctbX/Vs2bJl7QFg0aJFncLDw0MDAwO7T5kyxVerrVxaOyoqKvjw4cO3y3Y+/vjjPgEBAd0HDBgQdPnyZVv9OjNmzPAJDw8P/ec//9nhq6++cuvRo0dIaGho2N133x2UkZFhm5ycbL9+/XrP1atXd9BXgLt8+bLt/fff7x8eHh4aHh4e+sMPP7QCgKysLJuBAwcGBgQEdJ80aZJvTS1Qw8LCQoODg8MGDBgQVPH5quIAgN27d7voO5GFhoaG3bhxQ3X+/Hm7yMjI4JCQkLDAwMDue/fudQH+d1Vg6tSpXfQtUF955ZX2hlcAqjuOhQsXdh47dmzXPn36hIwfP75rbGysY0RERGhISEhYUFBQ2JkzZyqVbd2wYYOHvlmK3r333pu7ZcuWNgDw9ddfe8TExNxOsgcOHHDu1atXSGhoaFjv3r1DTp06VWmfeXl5qokTJ/pFRESEhoaGhn355ZdtAKC6eCZMmHBz48aNdfodayhTJvNLAAz/g3nrlhm6CGAHM5cxsxpACpTkDma+pPt+DsBBAJUuTzHzWmaOZOZIT88qq/9ZLBnJ3oxI6VarIC1Q79RSWqACwMMPP5yzadMm98LCQkpMTHQeMGDA7ed79uxZfPTo0aTExMSEJUuWXHr22We9K+7z73//e6dhw4blnTlzJvGnn35KfvHFF73z8vJU1cVz1113FZ0+fbrOt0MawpSX2Y8CCCSirlCS+GQoZ9mGvoVyRv4ZEbWDctn9HBG5Ayhk5hLd8oEA3jJhrE1O5pg3E2o1sHWrMoLdtdpy0MLQp59KC1RIC1RjjgNoeAtUAOjXr1/RxYsXHT766COPe++9945/n5ycHJtJkyZ1TU9PdyQiLisrqzTo5eDBg6337dvXZtWqVR0BpZZ6WlqafXXx2Nraws7Ojm/cuKGqGIupmOzMnJk1AOYC2AcgEcBmZo4noqVENEa32j4A14koAcABAIuZ+TqAUACxRHRKt/wNw1HwzYGcmTcTUrrVKuhboD711FO+Xl5eEe+9917HnTt3uld1WdxY+hao27dvP5uSkpLw5z//+VpjtkB98sknr6akpCS8995750tKSqrcr751aFJSUkJSUlLC1atXT7u5uTVa8qgujtdeey3r448/Pl9UVKQaPHhwyIkTJxz1LVC9vLxKZ8yY0fW9995rW9v+jTmOii1Qv/vuuzQnJyft6NGjA3fs2FHpE7S+BWpF0dHRN5csWeLzyCOP3HEf+7nnnvMaMmRIfmpqavzOnTvTSktLK73XzIytW7em6ePLzMw806dPn+Ka4ikrKyNTtaatiknvmTPz98wcxMz+zLxMt+xlZt6h+5mZeSEzhzFzBDNv1C3/Vfe4p+77J6aM0xzUasDDA2jd2tyRiHqT0q1WQ1qgSgvUJ5544tqiRYsuR0VF3XHFIC8vz8bb27sUANasWVNl1Y9hw4blrVy5soP+w98vv/ziVFM8WVlZNm3atNEY9kM3NXMPgGuxZFpaMyClW62GtECVFqj+/v5lL774YqXpz88991zWP/7xD+/Q0NAwjUZT8WkAyvup0WgoJCQkLCAgoPuLL77oVVM8e/bsaV3xcr6pSQtUMwkKAnr2BLZsMXckol5KSgBfX6B3b2WOeQshLVCFpbOEFqj33Xef/4oVKy726NGjyrEODSEtUC2IVgucPy9n5lZNSrcKYZHM3QK1uLiYxowZc9MUibwm0gLVDDIzgdJSSeZWS0q3CmHRzNkC1dHRkefOnVvp8r+pSTI3A5mWZuX0pVs3bJDSrUIIiyCX2c1ApqVZOSndKoSwMJLMzUDOzK2YlG4VQlggSeZmoFYDnToBjo61ryssjJRubRJaLfCf/6DV+vVo85//oFUDarvcdvbsWbsRI0b4+/r6hvv4+IQ/9thjPsXFxVXeJ0lPT7eLjo6uNFWrIsOOYHW1cOHCzi+//HIHY9dvaMc1Q2+99ZanvqjLiRMnHPU11uPj4x169+4d0tD9R0dHd0tISLAHlNrsQUFBYfpa7vXp+masinXwjfXaa695vvPOO0YXubFEcs/cDGSOuZWS0q1NYtMmuM2fD9/8fNgQgbVaUOvWKP/3v3F+0iTju2wZ0mq1GDt2bMDMmTOvzp8//6xGo8HUqVN958+f77VmzZqLhuuWlZXBz8+vbO/evedq269hRzBr8uyzz97uTLVly5Y2Y8aMufHWW29lAsCJEyeSjN2PVqsFM8PG5n+fZ2JjYx3Ly8spLCzs9tz2Q4cOpXTq1KnqSdwW4K9//ev1qKiokKeffrrJB641FjkzN4P0dEnmVklKt5rcpk1wmz4d3a5cgV1hIVS3bsGmqAiqK1dgN306um3ahBobmVRn586drg4ODtr58+dfB5Ta2atXr87YtGlTu/z8fNWqVavaDh8+PKB///5Bd999d7Bhj/P8/HzVn/70p27+/v7dR44c6d+jR48Q/dmfviNYcnKyfbdu3bpPnjzZNyAgoPvAgQMDCwoKCABWrlzZLjw8PDQ4ODjs/vvv98/Pz6/x725GRobtyJEj/YODg8OCg4Mrncnm5uaqBgwYEBQWFhYaFBR0u4NXXl6eaujQoQHBwcFhgYGB3T/66CN3AHjyySe9/P39uwcFBYXNmjXLG/jfVYFNmza5rV27tsO6des8+/XrFwTceQXgpZde6hAeHh4aFBQUtmDBgs6A0v/dz88vfNy4cX5BQUHdz549e0cTl3Xr1rV94IEHbtZ0jPHx8Q6DBw8O7N69e2jfvn2D9f3lY2Ji/KZNm9alZ8+eId7e3hG7du1ynThxol+3bt26x8TE+Om3nzZtWpfw8PDQgICA7vq4Ktq+fXvrXr16hYSFhYWOGjWqm77We1Xvh6urq9bb27vkwIEDdT6rtxSSzJtYWRmQkSHJ3OpI6VaT02qBefPgW1JS9d+lkhKo5s+Hb30uuZ85c8apZ8+edzRV8fDw0Hbq1Kk0ISHBAQDi4+Odv/vuu7NHjx5NNlxv+fLlnm3atCk/e/Zs/GuvvXYpISGhysvEFy5ccJw3b97VtLS0eDc3t/L169e7A8C0adNuxMXFJSYnJycEBwcXrVq1qsqSoXpz5szpMnjw4Pzk5OSE+Pj4hD59+hQbPu/s7KzdvXt3WkJCQuKhQ4dS/v73v3trtVps3769dceOHcuSk5MTUlNT48ePH5+XlZVl8/3337unpqbGp6SkJLz22mt3VJybNGlSrr4T25EjR1IMn9u+fXvrtLQ0x9OnTycmJiYmnDx50nnPnj0uumN1mDt3bnZaWlp8UFDQHdXljhw54tK/f/873ushQ4YEhYSEhPXo0SMEAGbOnOn7wQcfXIiPj09cvnz5xSeeeKKLft3c3FzbEydOJL3xxhsZkydPDli8ePGV1NTU+KSkJCd9mde33377UlxcXGJSUlL8L7/84nrkyJE7yrpmZmbavvbaa50OHz6ckpCQkNinT5/CV199tUNN70efPn1uHTx40Govucll9iaWkaH80ZLBb1ZGSrea3IEDaFVQgBrvP+fnw+bgQbQaPhy1lk6tq8GDB+dV1Tns119/dZk/f/5VALjrrruKg4KCquy05uXlVaLvFNa7d+/C9PR0BwA4duyY08svv+yVn59vc+vWLZshQ4bUeKvg119/dd26dasaUK4gtG3b9o6YtFotPf30096///67i0qlwtWrV+0vXrxo26dPn6IXXnjB54knnvB68MEHc6OjowvKysrg4OCgnTRpkt/o0aNvTpo0yejbFHv37m19+PDh1mFhYWEAUFhYqEpKSnLs1q1baadOnUpHjBhR5b9Bdna2XceOHcsMlxleZs/NzVWdOHHCZeLEif7650tLS2+PXfi///u/myqVCn369Cls27Ztmb6WelBQUNHZs2cd7r777qLPP//cY926de00Gg1lZ2fbnTp1yrFfv363a64fPHiw1dmzZx2joqJCAKXpSd++fQvatm1bXt370b59e01SUpLVjmSSZN7EZFqaFSopAVatAqKjgYgIc0fTbF26BDsi1Fhfmgh88SLqPI0gPDy86Ntvv72jEUpOTo4qMzPTPiwsrOTIkSPOzs7ODRpmZ29vfzt2GxsbLioqUgHArFmzum7dujVtwIABRatWrWp76NChBp39rVmzxuP69eu2Z86cSXRwcGAvL6+IoqIiVY8ePUqOHz+esG3bNreXXnrJa//+/XkrVqzIPHnyZOKOHTtab9261f3DDz9s//vvv6fU/ipKp7Cnn346c/HixXeU4U1OTrav6b1ycHDQ6o+9KuXl5XB1ddUkJSVV2QnT0dGRAcDGxuaO91SlUkGj0VBSUpL9e++91+HYsWOJnp6e5TExMX4Vu9UxMwYNGpS3c+dOdcX9V/d+FBcXq5ycnJqkXakpyGX2JibJ3ApJ6dYm4eWFMq0WNVbhYQZ5e6OspnWqMmbMmPzi4mKVfgS3RqPBk08+6TNx4sRrhq1HqzJgwICCjRs3ugPAsWPHHFNSUqrs6FWdwsJCVZcuXcpKSkpo48aNtTZ2GThwYP7y5cs99XFev379jqsVubm5Nu3atStzcHDgnTt3ul6+fNkeUEbgu7q6ap988smchQsXZp08edI5NzdXpevXnbt69eqMpKQko+8Jjxo1Ku+LL75op7/XrFar7S5dulTrCWBgYGCVXcv0PDw8tN7e3qWffvqpO6AMovvtt9+Mfk9v3Lhh4+TkpPXw8CjPyMiwPXjwYKVxFEOHDr0VGxvrEhcX5wAo4wlOnz7tUNP7kZKS4hAeHm50D3ZLI2fmTUytVsZQ+fiYOxJhFCnd2mSGDcMtV1eUFxVVf5Lh6oryoUPrfoldpVLh22+/TZs1a5bv8uXLO2m1WgwfPjx31apVl2rbdvHixdkPPfSQn7+/f3d/f//igICAYnd398oNs6vx/PPPX46Kigr18PDQ9OnTp6CgoKDGWwkffvjhhUcffdQ3KCionUqlwnvvvXf+3nvvvX3MM2fOzBk1alRAUFBQWI8ePQq7du1aDCiX8//2t795q1Qq2Nra8gcffHD+5s2bNqNHjw4oKSkhAHj11VczjI17/PjxefHx8Y533XVXCKDcq9+wYYPa1ta2xqsno0aNuvnf//7XdezYsVW2dwWAr7/++txf/vIX3zfffLOTRqOhcePG5QwYMMCoRDpgwICi8PDwQn9///BOnTqV9u3bt6DiOp07d9asWbMmffLkyd30l/CXLFlyyc3NTVvd+3H06FGXN99887IxMVgi6ZrWxKZOBX777X9n6MLC7d4NjB6tlG6dOtXc0Zidqbum6UezVzUIzsEB2s8/x7n6Tk+rL41Gg9LSUnJ2dub4+HiH++67L+js2bNx+svB4k4FBQU0cODA4GPHjiXZ2lrH+eIvv/zitHz58o7ffvutxf9lrq5rmnW8082ITEuzMlK6tUnpEvU5w3nmzCBX14bNM2+I/Px81eDBg4PLysqImfGvf/3rvCTy6rm4uPDLL798Wa1W2wcGBlbqo26Jrl69avfmm2/WepXGkkkyb2JqNfCnP5k7CmEUfenWt9+W0q1NaNIk5E6ciNMHD6LVxYuw8/ZG2dChuKUy0wgfd3d3bVxcXKJ5Xt06xcTE5Jk7hroYN26cVcVbFUnmTaioCMjKkmlpVmPlSindaiYqFWCK6WdCNFcm/axLRNFElExEaUT0fDXrPERECUQUT0RfGSyfTkSpuq/ppoyzqaSnK9/lMrsVUKuBLVuA2bOldKsQwuKZ7MyciGwAvA9gJICLAI4S0Q5mTjBYJxDA3wAMZOYbRNRet9wDwBIAkQAYwDHdtmZrON8YZFqaFZHSrUIIK2LKM/MoAGnMfI6ZSwFsBPBghXX+AuB9fZJm5qu65fcD+JGZc3TP/Qgg2oSxNglJ5lZCSrcKIayMKZO5FwDDOY0XdcsMBQEIIqJfiOh3Ioquw7YgollEFEtEsdnZ2RWftjhqNeDgAHTsaO5IRI2kdKv5KT1QW2H9+jb4z39aoRF6oEoL1P9p6haoffv2DTZ8PiQkJEzfyKY6hs1u6uruu+8Oys7Orte/i7UydwU4WwCBAIYCmALgIyJqY+zGzLyWmSOZOdLT09M0ETai9HRl8Ju5RuUKI0jpVvPbtMkNnTv3wJgxQXjyST888EAQOnfugU2b6tUxDfhfC9QxY8bcPH/+fJxarY67deuWav78+ZVOEuraArVdu3ZGF5CxFM8++2z23LlzrwP/a4GamJiY0L1795K6tkAtL7/z8KtqgXrr1i2btLQ0OwA4fvy4yeufT5ky5fqKFSssPyk0IlOmlUsADOuceeuWGboIYAczlzGzGkAKlORuzLZWR/qYWwEp3Wpemza5Yfr0brhyxQ6FhSrcumWDoiIVrlyxw/Tp3eqb0KUFqnlboI4dOzZn/fr1HgCwfv16j5iYmBz9c8nJyfZ9+/YNDgsLCw0LCwuteLyAUrhn9uzZ3vpYli9f3g4Azp8/bxcZGRmsP9Pfu3evCwBMnjz55vbt29vW9D43N6ZM5kcBBBJRVyKyBzAZwI4K63wL5awcRNQOymX3cwD2AbiPiNyJyB3AfbplVk2SuYWT0q3mpfRA9UVJSdV/l0pKVJg/37c+l9ylBap5W6BOmTLlxs6dO90BYN++fW3Gjx9/U/9c586dNT/99FNKQkJC4qZNm84tWLCgCyp455132rm5uZXHxcUlnjp1KvHzzz/3TEpKsv/00089RowYkZuUlJSQmJgY369fv0IA8PT0LC8tLaWsrKwWc6ndZKPZmVlDRHOhJGEbAJ8yczwRLQUQy8w78L+knQCgHMBiZr4OAET0KpQPBACwlJlzKr+K9cjNBW7ckDnmFm3PHiAxUSndSjX2+xCmcOBAK9RStxz5+TY4eLAVhg+XFqhW1AK1ffv25W5ubpq1a9e6BwQEFLm4uNz+RFZaWkqPP/64b0JCgpNKpcL58+crNWnZv39/66SkJOcdO3a4A0B+fr5NQkKCY//+/W/Nnj3br6ysTDVhwoQb+vcfANq2bau5cOGCfceOHa22eUpdmPTuLTN/z8xBzOzPzMt0y17WJXKwYiEzhzFzBDNvNNj2U2YO0H19Zso4m4KMZLcCUrrVvC5dsgNRzWVSiRgXL9arBeqpU6fu6Bhm2AIVUM5467pfQxVboGo0GgKUFqjvvffehZSUlITnnnvuckl1Vx6MZNgCNSkpKaFt27Zlhi1QIyIiil566SWvRYsWdbKzs8PJkycTJ0yYcGPXrl1thg4dGmjs6+hboCYlJSUkJSUlXLhwIW7BggXXgJrfq+paoE6YMOHGs88+6ztlypQ7TsyWLVvWoX379mWJiYkJZ86cSSgrK6u0LTPTypUrL+hjuXTp0pnx48fnjRo1quDw4cPJXl5epTNmzOiqH9QHACUlJdTQf1NrIkOxmogkcwunL926YIGUbjUXL68yaLU1XxJhJnh7SwtUK2yBOm3atBtPPfVU1vjx4+8onZqbm2vTqVOnMhsbG3zwwQdtKw6oA4CRI0fmfvjhh576bmenT592yMvLU6WkpNh7e3uXPfPMM9ceeeSR7OPHjzsDysC87Oxsu+Dg4BJjj9faSTnXJiLJ3MJJ6VbzGzbsFlxdy1HFWd1trq7lGDpUWqBaYQtUd3d37bJly7Iqrv/0009fjYmJ8d+4cWPb4cOH5zo5OVX6cLVgwYJr6enpDhEREaHMTB4eHmXff//92X379rmuWrWqo62tLTs7O5dv2LBBDQA///yzc+/evW/ZtaAP5tICtYnMmwesW6fcO5fbsRZGrQYCApQR7G++ae5oLJqpW6DeHs1e1aVoBwctPv/8HOpw37cxSAvUurGEFqiPPfaYz9ixY28++OCD1fZUt1bSAtXM9CPZJZFbICndajmURH0O8+f7Ij/fBkQMZoKrazn+/e/zTZ3IAWmBWleW0AI1PDy8qDkm8ppIMm8i+pM/YWGkdKvlmTQpFxMnnsbBg61w8aIdvL3LMHToLXNVW5IWqHVn7haozzzzTO1XgZoZSeZNgFlJ5vfea+5IRCVSutUyKT1QpQWqEEaS0exNIDtbyRcy+M3CSOlWi6fRmDsCIayDJPMmICPZLZSUbrVoJ07A0cMDvU6dQqVpTkKIO0kybwKSzC2QlG61aFot8Nhj8CsogM2jj8KvEZqmCdGsSTJvAunpyndJ5hZEX7p18WKZYmCBPv8c7ikpcGIGkpPhvH492jR0nzY2Nn31DTlGjRrVrbaGJxUlJyfbr169utaiL9YqKioqWN9Apja7du1yHTZsWKMN6Z00aZLvsWPHHAHg008/de/WrVv3fv36BR0+fNj50Ucf9alt+5oUFBTQXXfdFazRaJCcnGzv6OjYJyQkJEz/VV0b3MZQ37a1o0eP7nbmzJk6XZGSZN4E1GqgXTvAxcXckYjbpHSrxcrNhWrBAnQpKlL+PhUVQfX00/DNy2vY3ysHBwdtUlJSQmpqarydnR2vXLmyTi0yU1NTHTZt2tSgZF5WVufidS3Cpk2bzvft27cYAD777LN2H3744fkjR46k3HPPPYXr1q0zutBNVe/vu+++227MmDE39HPefXx8SvRlYZOSkhIscZrhE088cXXZsmUd67KNJPMmIN3SLIyUbrVoixahc3HxnX+biouheuYZdG6s1xg0aFBBWlqaw5UrV2zuvfde/6CgoLCePXuGHDlyxAkAdu/e7aI/cwsNDQ27ceOG6oUXXvCKjY11CQkJCXvllVfaV9xnVFRU8GOPPeajP/s/cOCAM6C0Gx07dmzXPn36hIwfP75rcnKyff/+/YOCgoLCBgwYEJSammoPVN/69IMPPvCIiIgIDQkJCZs6daqvRqOBRqNBTEyMX2BgYPegoKDb8fzzn/9sr293Onr06G6A0hp14sSJfhEREaGhoaG3W6YWFBTQ6NGju3Xr1q37yJEj/as7Qz106JBz7969Q4KDg8MiIiJCb9y4cce/zYEDB5x79eoVEhoaGta7d++QU6dOOQBKX3N93EFBQWFnzpxxqK5Nq/6qwKJFizodO3bMZfbs2X6zZ8/2NrwCUN1xVGxfWzH+zZs3t33ooYdu1vT7sH379ta9evUKCQsLCx01alQ3fQlbLy+viKeeesorJCQkLDw8PPTnn392HjRoUKCPj0/4W2+95QlU35K2oqrayVb3fkRHRxf89NNPrevy4a/GqWlEZANgPzMPM3qPohK1Guhdr4stwiSkdKvFOnECjl98gfYlJbgjsZSUQPXFF2g/dy6ye/ZEg+ptl5WVYd++fa3vu+++vGeffbZzz549C/fv3392x44drtOnT++alJSUsHLlyo6rVq06f999993Kzc1VOTs7a5ctW3Zp5cqVHQ4cOJBW3b6LiopUSUlJCXv27HGZNWtW19TU1HgASE1NdTxy5EiSi4sLDx8+PGDatGnX//rXv15/55132j7xxBM++/fvP6tvffryyy+f1Wg0yM3NtTl+/Ljj1q1bPWJjY5McHBz4z3/+c5fVq1e37dmzZ1FmZqadfv/Xrl2zAYBVq1Z1PH/+/BknJyfWL/v73//eadiwYXlbtmxJv3btmk1kZGTomDFj8t5++21PJycn7blz5+KPHDniNHDgwLCKx1NcXEzTpk3z37Bhw9khQ4YU5uTkqAw7ngFAz549i48ePZpkZ2eHb7/91vXZZ5/13rdv39l3333X88knn7zyxBNP5BQXF5NGo8HWrVvdOnbsWHbw4ME0AJXqzq9YsSLz8OHDrVesWJFxzz33FO7atctV/1x1xwEo7WtPnz4dX7HrXXFxMWVkZDgEBwffLl6TkZHhEBISEgYAd911V8Fbb711+bXXXut0+PDhlNatW2tfeOGFjq+++mqHFStWZAJAly5dSpOSkhIef/xxnxkzZvgdOXIkqaioSBUREdH92Wefzda3pPXw8NBmZmba9uvXL2Tq1Kk3VQZ1EQzbyTIz7r333oA9e/a4XLlyxbaq98PGxga+vr7Fv//+u/PgwYOr7NJXUY3JnJnLiUhLRG7M3OSVl5qD8nLg/Hlg/HhzRyIAKJ+stmxRRrC7uta+vmgy+kFv1Z2MlJUBjz4Kv2PHkFyf+jElJSUq/R/xfv365c+fP/9a7969Q7dt25YGKM1YZs2aZZuTk6Pq379/waJFi3weeuihnClTptzw9/c3agje1KlTcwBg1KhRBQUFBSp9Qo2Ojr7p4uLCAHDixIlWe/bsOQsATzzxRM4rr7ziDVTd+nTNmjUecXFxzj179gwFgOLiYlX79u01kyZNupmRkeEwffp0nwceeCB33LhxeQAQHBxcNG7cuK5jxoy5OW3atJsAcPDgwdb79u1rs2rVqo6694HS0tLsf/75Z5d58+Zd1b0fRVW1dj19+rRj+/bty4YMGVIIKD3gK66ja+TSNT093ZGIuKysjABgwIABt1asWNHp4sWL9pMnT74RERFRUlWbVqP+8Wo4DqD69rVZWVm2rq6ud0xw1F9m1z/++uuv3c6ePesYFRUVAgBlZWXUt2/f23Hpz+ojIiIKb926pXJ3d9e6u7tr7e3ttdeuXbNxdXXVVtWStkuXLrdft7p2siNGjMiv7v1o166dJiMjw+hLh8YUjSkAcIaIfgRwu4gDM0vtSyNcvqz8EZLL7BZCSrdarIQEOMTFoVV1I9e1WtCZM3BJTIRD9+51PzvX3zM3Zt3XXnsta+zYsbnfffed2+DBg0N2796dWnGdCRMm+MXFxTl36NCh9NChQ2kAQBUGU+oft2rVql7j8ZmZJk6ceP3999+v1BAmLi4u4Ztvvmm9evVqz02bNnls2bIl/cCBA6l79uxx/e6779xWrFjRKTk5OZ6ZsXXr1rSePXuapIPYc8895zVkyJD8H3/88WxycrL98OHDgwFgzpw5OYMHD771zTffuI0ePTrw3XffPT9mzJj848ePJ2zbts3tpZde8tq/f3+e/gy4NtUdx88//9yqulanrVq10paWltb40Y+ZMWjQoLydO3eqq3pef09dpVLd0eZWpVKhrKyMDFvSOjg4sJeXV0TFFrD6drKLFy+uVJmuuvejpKREVZcWrsZ8vt0O4CUAhwEcM/gSRpBpaRZESrdatLAwlISH45ZKhSoHJKlU4IgIFISGNuwyu6F+/frlf/bZZ20BZYS2u7u7xsPDQxsfH+8QFRVVtGzZsqwePXrciouLc3Rzcys37Hi2devW9KSkpAR9IgeAr7/+2h0A9u3b5+Lq6lretm3bSmeLvXv3vvXxxx+7A0pv8sjIyAKg6tan0dHRebt27XLXtx69cuWKTUpKin1mZqZteXk5Hn300Zuvv/76pTNnzjiXl5fj7Nmz9g888ED++++/f6mgoMAmNzfXZtiwYXkrV67soNV9Svrll1+cAGXcwIYNGzwA4OjRo44pKSmVRrL36NGj+OrVq3aHDh1yBoAbN26oKt7HzcvLs/H29i7VHU87/fKEhAT70NDQkhdffPHq/ffff/PkyZNOVbVpNfbfqrrjqImnp2d5eXk5FRYWVjtifejQobdiY2Nd4uLiHHTHozp9+rTRI8mra0lrqLp2sjW9H2q12qFPnz5FxsZR65k5M39u7M5EZTItzYJI6VaLplIBn32G9AEDEFZSRbq2swPWrUN6Y5Zof/PNNy9PmzbNLygoKMzJyUm7bt06NQC89dZb7X/99dfWRMTBwcFFEyZMyFWpVLCxseHg4OCwqVOnXluyZMnVivtzdHTk0NDQMI1GQ2vXrq3yTG/16tUXHnnkEb9///vfHdu2batZv359OlB969MXX3zx0ogRI4K0Wi3s7Ox41apVF5ydnbWPP/64n1bX/33p0qUXNRoNTZ06tWt+fr4NM9PMmTOvtmvXrvyNN964PGvWrC4hISFhWq2WfHx8Sg4cOJC2aNGiq5MnT+7arVu37gEBAcVhYWGVyuc6Ojryhg0bzs6bN69LcXGxytHRUXv48OEUw3Wee+65rJkzZ3Z98803O48cOfKmfvmXX37psXnz5ra2trbs6elZ9uqrr2b+/PPPrSq2aTX236q646htu3vuuSf3hx9+cKnYklWvc+fOmjVr1qRPnjy5W2lpKQHAkiVLLvXo0cOoD43VtaQ1VF072aSkJIeq3o+MjAxbBwcHNrxUX5taW6AS0UAA/wDgCyX5EwBm5m7GvkhTsNQWqK+8onwVFQEOUsfKfEpKAF9fZSTinj3mjsZqmbwFKoC//AXeX3wBz5KS/105dHCA9uGHkf3RR7hY19duKlFRUcH6gVvmjkX8z88//+y8YsWKDt9++22VH64s0SuvvNK+devW2gULFlT6P1NdC1RjPuN+AuBtAIMA3AUgUve9VkQUTUTJRJRGRM9X8fyjRJRNRCd1XzMNnis3WL7DmNezRGo10LmzJHKzk9KtVmPlSlx2dMQd9wodHaFduRKXzRWTsF6DBg0qHDp0aJ7Gigr9t2nTpnzu3Ll16vxmzAC4XGau86mMblrb+wBGArgI4CgR7WDmigNQNjHz3Cp2UcTMver6upZGrQb8/MwdRQsnpVutSuvW0P7rX7jw1FPwKyqCyskJ2nfewfnWrWHRRV3/+OOPZHPHIKr29NNPXzd3DHUxf/78OsdrzJn5ASJaTkQDiKiP/suI7aIApDHzOWYuBbARwIN1DdDaScEYCyClW63O9Om4ERSEIiIgOBiFjzyCm+aOSQhLZsyZeT/dd8P7ZAygtlMcLwCGZfguGuzLUAwR3QMgBcACZtZv40hEsQA0AN5g5m8rbkhEswDMAoAuXbrUEk7TKy0FLl6UZG52UrrV6ugHww27RxOybp1tow56E6I5qva/CBHN1/34EjMPq/DVWNcqdwLwY+YeAH4EYDhy3lc30GYqgHeIyL/ixsy8lpkjmTnS07NOZZabxIULALMkc7OS0q1WqzdO4AZ5oCdOmTsUISxeTZ93H9N9X1XPfV8CYNjtxlu37DZmvs7M+uH/HwPoa/DcJd33cwAOArC6gqgyLc0CSOlW66SUg/OjggIbPPqoH6QHqhA1qimZJxJRKoBgIjpt8HWGiE4bse+jAAKJqCsR2QOYDOCOUelE1Mng4RgAibrl7kTkoPu5HYCBAIyq3GRJpGCMmelLt86eLaVbrc3nn7sjJcUJSg9UZ6xf36ahu5QWqDVrKS1QiajvvHnzbjftyczMtLW1te3zyCOP1HivdtWqVW1rW6cqly9fth08eHBgfWKvi2p/mZl5CoDBANIAPGDwNVr3vUbMrAEwF8A+KEl6MzPHE9FSIhqjW20eEcUT0SkA8wA8qlseCiBWt/wAlHvmVpnMbW0Bb29zR9JCSelW65Sbq8KCBV2gL4lZVKTC00/7Ii9PWqA2U03ZAtXLy6v0xx9/bKN/fv369e4BAQGVCr00ls6dO2s6dOhQ9sMPP7Qy1WsAtYxmZ+YsZu7JzOcrfhmzc2b+npmDmNmfmZfplr3MzDt0P/+NmbvrXmMYMyfplv/KzBG65RHM/ElDD9Qc1GqgSxcln4gmJqVbrdeiRZ1RXHzn36biYhWeeUZaoEoL1Aa3QHVyctIGBAQU6a9CbNu2zWPs2LE5+ue/+uortx49eoSEhoaG3X333UEZGRmVBopfvnzZ9v777/cPDw8PDQ8PD9Un6qp+bwBg7NixN9evX9+2qve3sRgzml3Uk8wxNyMp3WqdTpxwxBdftEdJyZ2JpaREhS++aI+5c7PRwIYh0gK1ZbdABYDJkyfnfPnllx6dO3cus7Gx4c6dO5fpa6qPHDmyYPLkyUkqlQpvv/12u6VLl3b86KOP7qg8OHv2bJ+FCxdeuf/++wtSU1Pt77///sBz587FV/V7AwADBw68tXTp0kb7MFoVSeYmpFYDD9R6Q0I0upISYNUqIDoaiIgwdzTCWLpBb6i5B6ofjh1LRj3mqkkLVGmBqhcTE5O3dOlSrw4dOpTFxMTkGD6nVqvtx44d652dnW1XWlqq8vHxqfTh8Zdffmmdmpp6u9GLrqlNtb83nTt31ly9erVSA5bGJLM3TeTWLeDqVRn8ZhZSutU6JSQ4IC6uFXTNQyrRaglnzrggMbFexZH198yTkpISPv/88wx9a8uqvPbaa1kff/zx+aKiItXgwYNDTpw44VhxnQkTJviFhISEDRky5PZAMFO1QNXHnZ6eHvf2229f9vT0LI+Li0sYNmxY/urVqz0nT57sBwAHDhxIfeqpp7KPHz/u3Lt379CysrLbrUP1+8jMzDzTp0+fRrtHrG+BmpqaGr9z5840fcvROXPm5Hz33XdpTk5O2tGjRwfu2LHDtUePHiXHjx9PiIiIKHrppZe8Fi1a1Km2/Ru8F9UeR11boDo6OnKPHj0KP/zww44PP/zwDcPn5s6d2+XJJ5+8mpKSkvDee++dLykpqbQ9M+P48eOJ+liuXr162s3NTVvd701hYSE5ODiYdEpGTfPMdxLRjuq+TBlUc3BeN6pAknkTk9Kt1issrATh4begUlWdZFUqRkREAUJDpQWqtEBtcAvU5557LmvJkiUXK57R5+fn23Tp0qUMANatW1flfe5Bgwblvf7667fHTfz6669OAFDV7w0AxMXFOQYFBRndzrQ+arrMvkL3fTyAjgC+1D2eAuCKKYNqDmRampnoS7du2CClW62NUvYtHQMGhKH6Hqjp9bnEXh1pgdpyW6BGRkYWR0ZGVrpC8cILL1yeMmWKv5ubm2bQoEH5Fy5cqHQlaO3atRkzZ87sEhQUFFZeXk79+vXLv/vuuy9U9XsDAD/++KNrdHR0rrHHWh/GtECNrdjysKpl5mZpLVDfew/461+BzEygY0dzR9OCDB0KnDsHnD0rFd9MoClaoOIvf/HGF194wvDypoODFg8/nI0KA5EsibRAtUyW0AI1MjIyeM+ePWmenp6VrtTUVUNaoLYiotu9y4moKwCTzpdrDtRqwNER6NDB3JG0IFK6tXlYufIyHB3vvL/o6KjFypXSAlXUmblboF6+fNl2/vz5VxojkdfEmNHsCwAcJKJzAAiAL4DZpgyqOdBPS5MrvU1ISrc2D61ba/Gvf13AU0/5oahIBScnLd555zxat7bomq7SAtVymbMFaufOnTUPP/zwTVO/Tq1n5sy8F0AggPlQqrQFM/M+Uwdm7aT1aROT0q2WTqutbpR6VaZPv4GgoCIoPVAL8cgjN00XmhDWQfd/qMoPtbUmcyJyBrAYwFxmPgWgCxGNbtwQmx9J5k1MSrdaurjs7Gw3oxO6fjCci0t5Yw96E8IaabVays7OdgMQV9Xzxlxm/wzAMQADdI8vAdgCYFejRNgM3bwJ5OZKMm8yOTnAJ59I6VYLptFoZmZlZX2clZUVDmPrW6hUwMGDFwG44tQpudwiWjotgDiNRlPlfURjkrk/M08ioikAwMyFVLEygriDTEtrYqtXK1V6pHSrxerbt+9VKJ0RhRAmYMwn5FIicgLAAEBE/gAarWhDcyTJvAlJ6VYhhDDqzPwfAPYC8CGiDVB6iz9qwpisWnm5JHOTKS+v3IJOSrcKIYRRo9l/gFIF7lEAXwOIZOaDpg3LOiUlAe3aAcePA61bA23amDuiZkT/5iYbzP6R0q1CCAHAuNHs/wHQj5l3M/MuZr5GRGubIDarwgzMmAHk5QHff6+clcvIgkZi+ObOmKE8Bv5XunXxYnmzhRAtmjH3zLsCeI6Ilhgss6hSrpZg61bgxAnlZDE3F7A3abO7FmbbNuD0aeXNPXUK2L5dWb58OeDjA0ycaN74hBDCzIxJ5jcBjADQQddJzc20IVmfbduAyZOBYl3Jfmalsqg+54gGKCgA5sxRRqsDyvfZs4HDh6V0qxBC6BiTzImZNcz8JIBtAH4G0L6WbZQNiaKJKJmI0ojo+Sqef5SIsonopO5rpsFz04koVfc13dgDamrffw9MmqScNFb00EPK86IBliwBCiv0rSgsBGbNktKtQgihY8xo9tX6H5h5HRGdAfBUbRsRkQ2A9wGMBHARwFEi2sHMCRVW3cTMcyts6wFgCZTL+QzgmG7bG7AgzMBjjymDrKtSXq7c4s3MlFu69ZKUBHz4IVBUoQ1wUZEyEG7mTCndKoQQqCGZE1FrZs4DsEWXXPXUAIyZBxQFII2Zz+n2txHAgwAqJvOq3A/gR2bO0W37I4BoKKPpq3T9+nWsW7fujmXdu3fHXXfdhbKyMmzYsKHSNr169UKvXr1QWFiIzZs3V3o+MjIS4eHhyM3NxTfffFPpeXf3Abh2LRht217DAw9ULoh3+PA9SE/vhh9/zMLly3srPT9ixAj4+PggIyMD//nPfyo9Hx0djY4dO+LcuXM4fPhwpedHjx6Ndu3aITk5Gb/99lul58eNGwc3NzfExcWhqvawDz30EJydnXHy5EmcPHmy0vPTpk2DnZ0djh49ivj4+ErPP/roowCAX3/9FSkpd7Q4hp2dHaZNmwYAOHToENTqO7sPOjs746GHHgIA7N+/Hxcv3tnZsnXr1hi/YgVQUoK90dHIqtBHtu3163jgxAmAGTt37cL163f2UejYsSOio6MBANu3b0deXt4dz3t7e+Pee+8FAGzevBmFFc7+u3btiiFDhgAANmzYgLKysjueDwoKwt133w0AlX7vANP/7g0YMADBwcG4du0adu2q/Lt3zz33oFu3bsjKysLevY3/uyeEsCw1XWb/Svf9GIBY3fdjBo9r4wUgw+DxRd2yimKI6DQRbSUin7psS0SziCiWiGIr/rFtCufOVX153ZBWq5xgijrKz1cGHtT0Bp86pfwjCCFEC0esn+bT2DsmmgAgmpln6h4/DGWK21yDddoCKGDmEiKaDWASMw8nokUAHJn5n7r1XgJQxMwrqnu9yMhIrurs05R++w0YNKjmfKNSAb/8AvTv33RxNQvMwMCBwJEjVb/BRMCAAcDPP8s9jCZERMeYWWazCGFharrM3qemDZn5eC37vgTAx+Cxt26Z4T4Mr41+DOAtg22HVtj2YC2v1+T691fqmFy9Wv06np5Av35NF1OzQQR8+inQp0/le+YA4OioPC+JXAghahwAt7KG5xhAbSW3jgIIJKKuUJLzZABTDVcgok7MnKl7OAZAou7nfQBeIyJ33eP7APytltdrckTAZ58BY8ZUPQjOxkbyTYOEhABPPFF5EJy9vbI8ONh8sQkhhAUx2WV2ACCiPwF4B4ANgE+ZeRkRLQUQy8w7iOh1KElcAyAHwBPMnKTbdgaAv+t2tYyZP6vptcxxmV1v+3albonh1WCVCtiyBRg/3iwhNR+3bgG+voDhADcPD+DCBaBVK/PF1ULJZXYhLJNRyZyIwgGEAXDUL2Pm9SaMq87MmcwBJXE//LDSxMvWFvj6a2DCBLOF07xs3Qr8+c/Km+vgAGzYAMTEmDuqFkmSuRCWyZja7EsAvKv7Ggblvrb0Ja5gwgTA31/5OShIck2jiolRkjgA9O4tlzuEEKICYyrATYBSzjWLmR8D0BOAlHStgEg5eQSAf/1L7pM3qthYpcmKoyOwbp28uUIIUYExFeCKmFlLRBoiag3gKu4cpS50iouVPKOrNSIay8qVSunW9HTpKyuEEFUwJpnHElEbAB9BKRhTAKByuTEBtRrw8vrfFWHRCNRqZUDCokWSyIUQoho1JnMiIgCvM/NNAKuJaC+A1sx8uimCszbp6Uofc9GI3nlHmeM3b565IxFCCItV4z1zVoa6f2/wOF0SefXUaknmjSonB/jkE2DqVOWShxBCiCoZMwDuOBHdZfJIrFxJCXDpkiTzRrV6tTLP/JlnzB2JEEJYNGPumfcDMI2IzgO4BYCgnLT3MGlkVubCBaWcuCTzRlJSAqxaBURHAxER5o5GCCEsmjHJ/H6TR9EM6Dt8SjJvJF9+CVy5ogx8E0IIUaNaL7Mz83koU9GG634uNGa7lkafzP38zBpG86DVKtPRevUChtfWAkAIIUStZ+a6CnCRAIIBfAbADsCXAAaaNjTrolYDdnYyTqtR7NkDJCYqZVulQIwQQtTKmDPscVDKt94CAGa+DMDVlEFZo/R0oEsXZRaVaKDlywEfH6V7jRBCiFoZk8xLdVPUGACISFpVVUGmpTWSo0eBQ4eAp59WLnUIIYSolTHJfDMRrQHQhoj+AmA/lGpwwoAk80aiL936l7+YOxIhhLAatd4zZ+YVRDQSQB6U++YvM/OPJo/MihQUANnZkswbzLB0q6vcyRFCCGMZMzUNuuQtCbwa6enKd0nmDSSlW4UQol6M6Wc+nohSiSiXiPKIKJ+I8poiOGsh09IagZRuFUKIejPmzPwtAA8wc6Kpg7FWUjCmEUjpViGEqDdjBsBdqW8iJ6JoIkomojQier6G9WKIiIkoUvfYj4iKiOik7mt1fV6/qaSnA87OQPv25o7ESknpViGEaBBj+5lvAvAtgBL9QmbeXtNGRGQD4H0AIwFcBHCUiHYwc0KF9VwBzAdwpMIuzjJzLyPiMzu1WrnELvVN6mnDBindKoQQDWBMMm8NpYTrfQbLGECNyRxAFIA0Zj4HAES0EcCDABIqrPcqgDcBLDYmYEsk09IaQKsFVqyQ0q1CCNEAxkxNe6ye+/YCkGHw+CKUDmy3EVEfAD7MvJuIKibzrkR0AsqUuBeZ+aeKL0BEswDMAoAuXbrUM8yGYVaS+eDBZnl56yelW4UQosGMGc0eRET/IaI43eMeRPRiQ1+YiFQA3gZQ1YinTABdmLk3gIUAviKi1hVXYua1zBzJzJGenp4NDalebtwA8vLkzLzepHSrEEI0mDED4D4C8DcAZQDAzKcBTDZiu0tQuq3peeuW6bkCCAdwkIjSAfQHsIOIIpm5hJmv617vGICzAIKMeM0mJyPZG0BKtwohRKMwJpk7M/MfFZZpjNjuKIBAIupKRPZQPgDs0D/JzLnM3I6Z/ZjZD8DvAMYwcywReeoG0IGIugEIBHDOiNdscjLHvAGkdKsQQjQKY5L5NSLyx/8arUyAchm8RsysATAXwD4AiQA2M3M8ES0lojG1bH4PgNNEdBLAVgBzmDnHiFibnFR/qyd96dbZs6V0qxBCNJAxo9mfArAWQAgRXQKgBvBnY3bOzN8D+L7CsperWXeowc/bAGwz5jXMTa0G2rRRvkQdSOlWIYRoNMaMZj8H4F5d61MVM+ebPizrIdPS6kFKtwohRKOqNZkT0cIKjwEgF8AxZj5pmrCsh1oNhIWZOworI6VbhRCiURlzzzwSwBwo88a9AMwGEA3gIyJ61oSxWTxm5Z65nJnXgZRuFUKIRmfMPXNvAH2YuQAAiGgJgN1QBqkdg9KIpUXKygKKiyWZ14mUbhVCiEZnzJl5exjUZIcy37wDMxdVWN7iyLS0OpLSrUIIYRLGnJlvAHCEiL7TPX4ASkW2VqhcZ71FkWlpdSSlW4UQwiSMGc3+KhHtATBQt2gOM8fqfp5mssisgJyZ15GUbhVCCJMw5swcuuQdW+uKLYxaDXTooPQyF7XQl25duVJKtwohRCMzKpmLqskc82owA0eOAJmZQKdOQL9+UrpVCCFMSJJ5A6jVQP/+5o7Cwnz/vVKi9eZNQKVSBr25uABXryoj2KV0qxBCNDpJ5vWk0QAXLgBTppg7Egvy/ffAhAlAUdGdywsKlO/duzd9TEII0QIYMzVNVOHiRaC8XAa/3cYMzJpVOZEbeuEFZT0hhBCNSpJ5Pcm0tAqOHAFyc2te5+ZN4I+K3XSFEEI0lCTzetJPS5NkrpOZqdwjr4lKBVy+3DTxCCFECyLJvJ7UaiU3deli7kgsRKdOymC3mmi1QOfOTROPEEK0IJLM60mtBry9Zcr0bf36KVPPatKmDRAV1SThCCFESyLJvJ5kjnkFRMDatYCTU9XPOzkBa9ZIGVchhDABSeb1JMm8Cn/6E7B1q3LJQqVSEnerVsrjrVuV54UQQjQ6kyZzIoomomQiSiOi52tYL4aImIgiDZb9TbddMhHdb8o466q4WBnHJdPSqvCnPwHbtin3x6dPB/7zH2VCviRyIYQwGZMVjSEiGwDvAxgJ4CKAo0S0g5kTKqznCmA+gCMGy8IATAbQHUBnAPuJKIiZy00Vb11cuKB8lzPzarz9tnL/fNUqqfgmhBBNwJRn5lEA0pj5HDOXAtgI4MEq1nsVwJsAig2WPQhgIzOXMLMaQJpufxZBpqXVQK0GtmxRSrpKIhdCiCZhymTuBSDD4PFF3bLbiKgPAB9m3l3XbXXbzyKiWCKKzc7ObpyojSDJvAbvvAPY2ADz5pk7EiGEaDHMNgCOiFQA3gbwTH33wcxrmTmSmSM9PT0bL7haqNWAvb1Mma4kJwf45BNg6lTAq9JnLyGEECZiykYrlwD4GDz21i3TcwUQDuAgKdOVOgLYQURjjNjWrNRqwNe39oJnLc7q1cCtW8Az9f58JoQQoh5MmY6OAggkoq5EZA9lQNsO/ZPMnMvM7ZjZj5n9APwOYAwzx+rWm0xEDkTUFUAgAIsp6i3T0qpQUqIMeIuOBiIizB2NEEK0KCZL5sysATAXwD4AiQA2M3M8ES3VnX3XtG08gM0AEgDsBfCUpYxkBySZV2nDBuDKFaVnuRBCiCZF3ExaUkZGRnJsbKzJXyc/H2jdGnj9deD5amfOtzBaLRAeDjg4AMePS5W3ZoyIjjFzZO1rCiGakinvmTdL0vq0Cnv2AImJytm5JHIhhGhyMoSrjmRaWhWWLwd8fICJE80diRBCtEiSzOtIknkFR48Chw4BTz8tLeSEEMJMJJnXkVqt9A5p187ckViIlSuV0q1/+Yu5IxFCiBZLknkd6Ueyy61hSOlWIYSwEJLM60impRmQ0q1CCGERJJnXAbMyml1an0JKtwohhAWRZF4HOTnKPHM5M4eUbhVCCAsiybwOZCS7jpRuFUIIiyLJvA4kmetI6VYhhLAokszrQJI5lNKtK1YAvXoBw4ebOxohhBCQcq51olYDHh5KbfYWS0q3CiGExZEz8zqQaWlQzsqldKsQQlgUSeZ10OKnpcXGAgcPSulWIYSwMJLMjaTVKsm8RZ+Zr1ghpVuFEMICSTI3UlaWMiOrxSZzKd0qhBAWS5K5kVr8SHYp3SqEEBZLkrmRWnQyl9KtQghh0UyazIkomoiSiSiNiJ6v4vk5RHSGiE4S0c9EFKZb7kdERbrlJ4lotSnjNIY+mbfIAXBSulUIISyayeaZE5ENgPcBjARwEcBRItrBzAkGq33FzKt1648B8DaAaN1zZ5m5l6niqyu1GujUCXB0NHckTUxKtwohhMUz5Zl5FIA0Zj7HzKUANgJ40HAFZs4zeNgKAJswngZpsdPSpHSrEEJYPFMmcy8AGQaPL+qW3YGIniKiswDeAmA4uqorEZ0gokNENNiEcRqlRRaMkdKtQghhFcw+AI6Z32dmfwDPAXhRtzgTQBdm7g1gIYCviKhSEVUimkVEsUQUm52dbbIYNRogI6MFJnN96dbFi6V0qxBCWDBTJvNLAHwMHnvrllVnI4CxAMDMJcx8XffzMQBnAQRV3ICZ1zJzJDNHenp6NlbclWRkAOXlLTCZS+lWIYSwCqZM5kcBBBJRVyKyBzAZwA7DFYgo0ODh/wFI1S331A2gAxF1AxAI4JwJY61Ri5yWJqVbhRDCaphsNDsza4hoLoB9AGwAfMrM8US0FEAsM+8AMJeI7gVQBuAGgOm6ze8BsJSIygBoAcxh5hxTxVqbFpnMpXSrEEJYDZO2QGXm7wF8X2HZywY/z69mu20AtpkytrpQq5XiZz4+ta/bLOhLty5aJKVbhRDCCph9AJw1SE9XErltS+n+LqVbhRDCqkgyN4Ja3YLmmEvpViGEsDqSzI3QouaYS+lWIYSwOpLMa1FUBGRmtpBkLqVbhRDCKkkyr8X588r3FpHMpXSrEEJYJUnmtWgx09KkdKsQQlitljI+u95aTDLXl27dsEFKtwohhJWRM/NapKcDDg5Ax47mjsTEpHSrEEJYLUnmtVCrAV9fQNWc3ykp3SqEEFatOaeoRtEipqVJ6VYhhLBqksxr0eyTub506+zZUrpVCCGslCTzGuTlKQXRmnUyl9KtQghh9SSZ16DZj2SX0q1CCNEsSDKvQbNP5lK6VQghmgVJ5jVIT1e+N8tkLqVbhRCi2ZBkXgO1GnBxATw8zB2JCUjpViGEaDYkmddAP5K92RVEk9KtQgjRrEg51xqo1UC3buaOwgSkdKsQQjQrcmZeDeZmPMdcSrcKIUSzYtJkTkTRRJRMRGlE9HwVz88hojNEdJKIfiaiMIPn/qbbLpmI7jdlnFW5dk0Z6N3skrmUbhVCiGbHZMmciGwAvA9gFIAwAFMMk7XOV8wcwcy9ALwF4G3dtmEAJgPoDiAawAe6/TWZZjstTUq3CiFEs2PKM/MoAGnMfI6ZSwFsBPCg4QrMnGfwsBUA1v38IICNzFzCzGoAabr9NZlmOS0tPV1KtwohRDNkygFwXgAyDB5fBNCv4kpE9BSAhQDsAeiHVnsB+L3CtpVKlBHRLACzdA9LiCiu4WHfqUePxt5jvbQDcK3R9vbWW8qX+TXucVmW5npsweYOQAhRmdlHszPz+wDeJ6KpAF4EML0O264FsBYAiCiWmSNNE6V5Nddja67HBTTfYyOiWHPHIISozJSX2S8B8DF47K1bVp2NAMbWc1shhBCixTJlMj8KIJCIuhKRPZQBbTsMVyCiQIOH/wcgVffzDgCTiciBiLoCCATwhwljFUIIIayWyS6zM7OGiOYC2AfABsCnzBxPREsBxDLzDgBzieheAGUAbkB3iV233mYACQA0AJ5i5vJaXnKtqY7FAjTXY2uuxwU032NrrsclhFUjZq59LSGEEEJYLKkAJ4QQQlg5SeZCCCGElbO6ZG5EiVgHItqke/4IEfmZIcw6M+K47iGi40SkIaIJ5oixvow4toVElEBEp4noP0Tka44466oh5YotXW3HZrBeDBExETW7aXhCWBOrSuZGloh9HMANZg4A8C8AbzZtlHVn5HFdAPAogK+aNrqGMfLYTgCIZOYeALZCKe1r0RpSrtjSGXlsICJXAPMBHGnaCIUQFVlVMocRJWJ1jz/X/bwVwAgii+/zaUzp23RmPg1Aa44AG8CYYzvAzIW6h79DqStg6RpSrtjSGfP/DABehfJhubgpgxNCVGZtybyqErEVy7zeXoeZNQByAbRtkujqz5jjslZ1PbbHAewxaUSNw6jjIqKniOgslDPzeU0UW0PVemxE1AeADzPvbsrAhBBVs7ZkLpoxIvozgEgAy80dS2Nh5veZ2R/Ac1DKFVs9IlJBuWXwjLljEUIorC2ZG1Pm9fY6RGQLwA3A9SaJrv6ac/lao45NVzzoBQBjmLmkiWJriIaUK7Z0tR2bK4BwAAeJKB1AfwA7ZBCcEOZjbcm81hKxusf6Zi0TAPyXLb8yjjHHZa2MKevbG8AaKIn8qhlirI+GlCu2dDUeGzPnMnM7ZvZjZj8o4xzGMLM0YRHCTKwqmevugetLxCYC2KwvEUtEY3SrfQKgLRGlQWmtWu20GkthzHER0V1EdBHARABriCjefBEbz8h/s+UAXABs0U3jsvgPMkYe11wiiieik1B+F43uCGhORh6bEMKCSDlXIYQQwspZ1Zm5EEIIISqTZC6EEEJYOUnmQgghhJWTZC6EEEJYOUnmQgghhJWTZC6aHBEd1BcYIaLviahNA/c3lIh2VfPc17pubAsa8hpCCGHJbM0dgGh+dI1tiJlrbQrDzH8yYRwdAdyl66Bn7Da2unnWQghhNeTMvIUgopd0/al/1p2tLtIt9yeivUR0jIh+IqIQ3fJ1RLSKiH4lonOGPdSJaDERHdWd8b6iW+an2/96AHEAfIjoQyKK1RVOeaWauNKJqJ2u9/dJ3ZeaiA7onr+PiH4jpZf7FiJy0S2PJqIkIjoOYHw1h/0DAC/dPgfrrgj8W/c4joiidPv6BxF9QUS/APiiMd5vIYRoSpLMWwAiugtADICeUHpUG9bQXgvgr8zcF8AiAB8YPNcJwCAAowG8odvXfQACobTJ7AWgLxHdo1s/EMAHzNydmc8DeIGZIwH0ADCEiHpUFyMzr9b1/b4LSpeut4moHZTmJPcycx8AsQAWEpEjgI8APACgL4CO1ex2DICzzNyLmX/SLXPWvc6TAD41WDdM9zpTqotRCCEslVxmbxkGAviOmYsBFBPRTgDQneXeDaWMqn5dB4PtvtVdKk8gog66Zffpvk7oHrtASeIXAJxn5t8Ntn+IiGZB+T3rBCVhnq4l1n9Dqae/k4hG67b5RRefPYDfAIQAUDNzqu44vgQwy8j34msAYObDRNTa4H79DmYuMnIfQghhUSSZt2wqADd1Z6pVMexeRgbfX2fmNYYrEpEfgFsGj7tCOdO/i5lvENE6AI41BUNEjwLwhVIXXP9aP1Y8Wyai6uI1RsX6xfrHtyquKIQQ1kIus7cMvwB4gIgcdWfjowGAmfMAqIloIqAMXCOinrXsax+AGQb3rr2IqH0V67WGkiBzdWf1o2raKRHpL/P/2WDg3O8ABhJRgG6dVkQUBCAJgB8R+evWq8ul8Um6fQ0CkMvMuXXYVgghLJKcmbcAzHxU14nsNIArAM4A0CexaQA+JKIXAdhB6bt9qoZ9/UBEoQB+0136LgDwZwDlFdY7RUQnoCTeDCgfKGoyF4AHgAO6/cYy80zd2frXRKS//P8iM6foLt/vJqJCAD9B6bFtjGJdXHYAZhi5jRBCWDTpmtZCEJELMxcQkTOAwwBmMfNxc8fVlIjoIIBF0ndbCNHcyJl5y7GWiMKg3Lf+vKUlciGEaM7kzFwIIYSwcjIATgghhLByksyFEEIIKyfJXAghhLByksyFEEIIKyfJXAghhLBy/w/hhGFuEAMUnAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred = postproc.predict_proba(X_test)[:, 1]\n",
    "y_lr = postproc.estimator_.predict_proba(X_test)[:, 1]\n",
    "br = postproc.postprocessor_.base_rates_\n",
    "i = X_test.index.get_level_values('sex') == 1\n",
    "\n",
    "plt.plot([0, br[0]], [0, 1-br[0]], '-b', label='All calibrated classifiers (Females)')\n",
    "plt.plot([0, br[1]], [0, 1-br[1]], '-r', label='All calibrated classifiers (Males)')\n",
    "\n",
    "plt.scatter(generalized_fpr(y_test[~i], y_lr[~i]),\n",
    "            generalized_fnr(y_test[~i], y_lr[~i]),\n",
    "            300, c='b', marker='.', label='Original classifier (Females)')\n",
    "plt.scatter(generalized_fpr(y_test[i], y_lr[i]),\n",
    "            generalized_fnr(y_test[i], y_lr[i]),\n",
    "            300, c='r', marker='.', label='Original classifier (Males)')\n",
    "\n",
    "plt.scatter(generalized_fpr(y_test[~i], y_pred[~i]),\n",
    "            generalized_fnr(y_test[~i], y_pred[~i]),\n",
    "            100, c='b', marker='d', label='Post-processed classifier (Females)')\n",
    "plt.scatter(generalized_fpr(y_test[i], y_pred[i]),\n",
    "            generalized_fnr(y_test[i], y_pred[i]),\n",
    "            100, c='r', marker='d', label='Post-processed classifier (Males)')\n",
    "\n",
    "plt.plot([0, 1], [generalized_fnr(y_test, y_pred)]*2, '--', c='0.5')\n",
    "\n",
    "plt.axis('square')\n",
    "plt.xlim([0.0, 0.4])\n",
    "plt.ylim([0.3, 0.7])\n",
    "plt.xlabel('generalized fpr');\n",
    "plt.ylabel('generalized fnr');\n",
    "plt.legend(bbox_to_anchor=(1.04,1), loc='upper left');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "na2YWDICQKUg"
   },
   "source": [
    "We can see the generalized false negative rate is approximately equalized and the classifiers remain close to the calibration lines.\n",
    "\n",
    "We can quanitify the discrepancy between protected groups using the `difference` operator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "smrTomVyQKUg",
    "outputId": "6add1353-fae6-474a-b9e8-5d1be8cd8a39"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0006901337553552045"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "difference(generalized_fnr, y_test, y_pred, prot_attr='sex')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "vscode": {
   "interpreter": {
    "hash": "d0c5ced7753e77a483fec8ff7063075635521cce6e0bd54998c8f174742209dd"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}