05-UCI with pipelines_clean.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " --- \n",
    "# UCI - Default from Credit Card Clients\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": true
   },
   "source": [
    "# Dataset presentation\n",
    "\n",
    "This dataset contains information on default payments, demographic factors, credit data, history of payment, and bill statements of credit card clients in Taiwan from April 2005 to September 2005.\n",
    "\n",
    "It can be found here:\n",
    "https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variables\n",
    "There are 25 variables:\n",
    "\n",
    "* ID: ID of each client\n",
    "* LIMIT_BAL: Amount of given credit in NT dollars (includes individual and family/supplementary credit\n",
    "* SEX: Gender (1=male, 2=female)\n",
    "* EDUCATION: (1=graduate school, 2=university, 3=high school, 4=others, 5=unknown, 6=unknown)\n",
    "* MARRIAGE: Marital status (1=married, 2=single, 3=others)\n",
    "* AGE: Age in years\n",
    "* PAY_0: Repayment status in September, 2005 (-1=pay duly, 1=payment delay for one month, 2=payment delay for two months, ... 8=payment delay for eight months, 9=payment delay for nine months and above)\n",
    "* PAY_2: Repayment status in August, 2005 (scale same as above)\n",
    "* PAY_3: Repayment status in July, 2005 (scale same as above)\n",
    "* PAY_4: Repayment status in June, 2005 (scale same as above)\n",
    "* PAY_5: Repayment status in May, 2005 (scale same as above)\n",
    "* PAY_6: Repayment status in April, 2005 (scale same as above)\n",
    "* BILL_AMT1: Amount of bill statement in September, 2005 (NT dollar)\n",
    "* BILL_AMT2: Amount of bill statement in August, 2005 (NT dollar)\n",
    "* BILL_AMT3: Amount of bill statement in July, 2005 (NT dollar)\n",
    "* BILL_AMT4: Amount of bill statement in June, 2005 (NT dollar)\n",
    "* BILL_AMT5: Amount of bill statement in May, 2005 (NT dollar)\n",
    "* BILL_AMT6: Amount of bill statement in April, 2005 (NT dollar)\n",
    "* PAY_AMT1: Amount of previous payment in September, 2005 (NT dollar)\n",
    "* PAY_AMT2: Amount of previous payment in August, 2005 (NT dollar)\n",
    "* PAY_AMT3: Amount of previous payment in July, 2005 (NT dollar)\n",
    "* PAY_AMT4: Amount of previous payment in June, 2005 (NT dollar)\n",
    "* PAY_AMT5: Amount of previous payment in May, 2005 (NT dollar)\n",
    "* PAY_AMT6: Amount of previous payment in April, 2005 (NT dollar)\n",
    "* default.payment.next.month: Default payment (1=yes, 0=no)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": true
   },
   "source": [
    "# Useful imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "import sklearn\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.set_option('display.max_columns', 120)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.set_style(\"darkgrid\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xgboost as xgb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightgbm as lgb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import catboost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preproecssing imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Metrics imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import recall_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import roc_auc_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Library\\\\mingw-w64\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Library\\\\usr\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Library\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Scripts',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\condabin',\n",
       " 'C:\\\\Program Files\\\\Docker\\\\Docker\\\\Resources\\\\bin',\n",
       " 'C:\\\\ProgramData\\\\Oracle\\\\Java\\\\javapath',\n",
       " 'C:\\\\WINDOWS\\\\system32',\n",
       " 'C:\\\\WINDOWS',\n",
       " 'C:\\\\WINDOWS\\\\System32\\\\Wbem',\n",
       " 'C:\\\\WINDOWS\\\\System32\\\\WindowsPowerShell\\\\v1.0',\n",
       " 'C:\\\\Program Files\\\\PuTTY',\n",
       " 'C:\\\\Program Files\\\\Intel\\\\WiFi\\\\bin',\n",
       " 'C:\\\\Program Files\\\\Common Files\\\\Intel\\\\WirelessCommon',\n",
       " 'C:\\\\Program Files\\\\Git LFS',\n",
       " 'C:\\\\Program Files\\\\Pandoc',\n",
       " 'C:\\\\Program Files\\\\MiKTeX 2.9\\\\miktex\\\\bin\\\\x64',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Library\\\\mingw-w64\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Library\\\\usr\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Library\\\\bin',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Continuum\\\\anaconda3\\\\Scripts',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Microsoft\\\\WindowsApps',\n",
       " 'C:\\\\Users\\\\twang\\\\AppData\\\\Local\\\\Programs\\\\Git\\\\cmd',\n",
       " 'C:\\\\Program Files\\\\Intel\\\\WiFi\\\\bin',\n",
       " 'C:\\\\Program Files\\\\Common Files\\\\Intel\\\\WirelessCommon',\n",
       " 'C:\\\\Users\\\\twang\\\\Downloads\\\\Portable\\\\npp.7.7.1.bin.x64',\n",
       " 'C:\\\\Program Files\\\\MiKTeX 2.9\\\\miktex\\\\bin\\\\x64',\n",
       " '.']"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "os.environ['PATH'].split(';')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing our Data easily!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = 'UCI_Credit_Card.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataprep.load import load_raw_data\n",
    "from dataprep.load import load_data\n",
    "from dataprep.load import load_data_xy\n",
    "\n",
    "\n",
    "df = load_data(path)\n",
    "X_raw, y_raw = load_data_xy(path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Defining our train/test sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting our Df into test/train/val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "df_train, df_val = train_test_split(df, \n",
    "                                    test_size=6000, \n",
    "                                    stratify=df.default, \n",
    "                                    random_state=42)\n",
    "df_train, df_test = train_test_split(df_train, \n",
    "                                     test_size=6000, \n",
    "                                     stratify=df_train.default, \n",
    "                                     random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting into (X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataprep.load import df2xy\n",
    "\n",
    "X_train, y_train = df2xy(df_train, 'default')\n",
    "X_test, y_test = df2xy(df_test, 'default')\n",
    "X_val, y_val = df2xy(df_val, 'default')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Protocol for pipelined workflow"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "# * Pipeline for lowercasing\n",
    "* pipeline for re-labelling the mislabelled data\n",
    "\n",
    "* pipe for ageBins\n",
    "* pipe for gender x Marriage\n",
    "* pipe for gender x age\n",
    "* pipe for next month's bill\n",
    "---> One output: X_cat, que l'on normalise avec pipe=StandardScaler\n",
    "* pipe for 1-hot\n",
    "* pipe for scaling\n",
    "---> We get our second output X_1hot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Puis on se doit de définir les dictionnaires / classes que l'on va utiliser pour stocker les modèles, leurs scores selon toutes les métriques"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "models = []\n",
    "models.append(LogisticReg...)\n",
    "\n",
    "scores_accuracy={}\n",
    "scores_recall\n",
    "scores_auc\n",
    "features_importances={}\n",
    "chacun de ces dictionnaires contenant les performances du modèle: {'test': score_test, 'train': score_train}\n",
    "for model in models:\n",
    "    scores_accuracy{[model][train]} = accuracy_score(model.predict(X_test), y_test)\n",
    "    scores_accuracy{[model][test]} = accuracy(model.predict(X_train), y_train)\n",
    "    idem for recall, auc...\n",
    "    features_importances[model] = features_importanes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ainsi, on pourra les capitalizer facilement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Elementary Pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Elementary Pipelines are Pipelines that only do a little processing, such as adding / removing a single feature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Elementary Transformers from Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Transformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Gather data by age group\n",
    "from dataprep.pipelines import AgeBinAdder\n",
    "# Gender x Marriage new category\n",
    "from dataprep.pipelines import GenderXMarriageAdder\n",
    "# Gender x AgeBin new category\n",
    "from dataprep.pipelines import GenderXAgeBinAdder\n",
    "# Predict next month's bill statement\n",
    "from dataprep.pipelines import NextBillAdder\n",
    "# Get_dummies to Df\n",
    "from dataprep.pipelines import CategoricalWarrior\n",
    "# Drop a column\n",
    "from dataprep.pipelines import ColumnDropper\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Complex Pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complex Pipelines are combinations of multiple Elementary Pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will define them, and check that they work well below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "complex_pipelines = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "def show_pipe_result(pipeline, df, n=2):\n",
    "    result = pipeline.fit_transform(df)\n",
    "    print(\"Shape: \", result.shape)\n",
    "    if isinstance(result, pd.DataFrame):\n",
    "        # We're using display, because the df doesn't show by itself when it's encapsulated in a function\n",
    "        display(result.head())\n",
    "    elif isinstance(result, np.ndarray):\n",
    "        print(result[:n])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining our Pipelines manually"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With Scaler only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's scale the data, as our only data modification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pipeline with a StandardScaler\n",
    "complex_pipe1 = Pipeline([\n",
    "    (\"scaler\", StandardScaler())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 23)\n",
      "[[-1.13672015  0.81016074  0.21186989 -1.06879721 -1.24601985  1.84394071\n",
      "   2.39995869 -0.40419908 -0.36400938 -0.33135449 -0.33818882 -0.64250107\n",
      "  -0.64739923 -0.66799331 -0.67249727 -0.66305853 -0.65272422 -0.34194162\n",
      "  -0.22708564 -0.29680127 -0.30806256 -0.31413612 -0.29338206]\n",
      " [-0.3659805   0.81016074  0.21186989  0.84913055 -1.02904717 -0.54231679\n",
      "   2.39995869 -0.40419908 -0.36400938 -0.33135449  2.956928   -0.65921875\n",
      "  -0.66674657 -0.63925429 -0.62163594 -0.60622927 -0.59796638 -0.34194162\n",
      "  -0.21358766 -0.24000461 -0.24422965 -0.31413612 -0.18087821]]\n"
     ]
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe1, X_raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With Next bill & Scaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add the Next bill prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pipeline with a StandardScaler\n",
    "complex_pipe2 = Pipeline([\n",
    "    (\"next_bill\", NextBillAdder()),\n",
    "    (\"scaler\", StandardScaler())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 24)\n",
      "[[-1.13672015  0.81016074  0.21186989 -1.06879721 -1.24601985  1.84394071\n",
      "   2.39995869 -0.40419908 -0.36400938 -0.33135449 -0.33818882 -0.64250107\n",
      "  -0.64739923 -0.66799331 -0.67249727 -0.66305853 -0.65272422 -0.34194162\n",
      "  -0.22708564 -0.29680127 -0.30806256 -0.31413612 -0.29338206 -0.64264241]\n",
      " [-0.3659805   0.81016074  0.21186989  0.84913055 -1.02904717 -0.54231679\n",
      "   2.39995869 -0.40419908 -0.36400938 -0.33135449  2.956928   -0.65921875\n",
      "  -0.66674657 -0.63925429 -0.62163594 -0.60622927 -0.59796638 -0.34194162\n",
      "  -0.21358766 -0.24000461 -0.24422965 -0.31413612 -0.18087821 -0.65937279]]\n"
     ]
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe2, X_raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With our 4 engineered features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's add our 4 feature engineering Elementary Pipelines:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Categorical version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [],
   "source": [
    "complex_pipe3_cat = Pipeline([\n",
    "    (\"age_bin\", AgeBinAdder()),\n",
    "    (\"gender_age_bin\", GenderXAgeBinAdder()),\n",
    "    (\"gender_marriage\", GenderXMarriageAdder()),\n",
    "    (\"next_bill\", NextBillAdder())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 27)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>limit_bal</th>\n",
       "      <th>gender</th>\n",
       "      <th>education</th>\n",
       "      <th>marriage</th>\n",
       "      <th>age</th>\n",
       "      <th>pay_1</th>\n",
       "      <th>pay_2</th>\n",
       "      <th>pay_3</th>\n",
       "      <th>pay_4</th>\n",
       "      <th>pay_5</th>\n",
       "      <th>pay_6</th>\n",
       "      <th>bill_amt1</th>\n",
       "      <th>bill_amt2</th>\n",
       "      <th>bill_amt3</th>\n",
       "      <th>bill_amt4</th>\n",
       "      <th>bill_amt5</th>\n",
       "      <th>bill_amt6</th>\n",
       "      <th>pay_amt1</th>\n",
       "      <th>pay_amt2</th>\n",
       "      <th>pay_amt3</th>\n",
       "      <th>pay_amt4</th>\n",
       "      <th>pay_amt5</th>\n",
       "      <th>pay_amt6</th>\n",
       "      <th>age_bin</th>\n",
       "      <th>gen_ageBin</th>\n",
       "      <th>gen_mar</th>\n",
       "      <th>pred_bill_amt0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3913.0</td>\n",
       "      <td>3102.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20s</td>\n",
       "      <td>(2, 20s)</td>\n",
       "      <td>(2, 1)</td>\n",
       "      <td>3909.683795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>120000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>1725.0</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>3272.0</td>\n",
       "      <td>3455.0</td>\n",
       "      <td>3261.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>20s</td>\n",
       "      <td>(2, 20s)</td>\n",
       "      <td>(2, 2)</td>\n",
       "      <td>2678.148995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29239.0</td>\n",
       "      <td>14027.0</td>\n",
       "      <td>13559.0</td>\n",
       "      <td>14331.0</td>\n",
       "      <td>14948.0</td>\n",
       "      <td>15549.0</td>\n",
       "      <td>1518.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>30s</td>\n",
       "      <td>(2, 30s)</td>\n",
       "      <td>(2, 2)</td>\n",
       "      <td>29177.529785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>46990.0</td>\n",
       "      <td>48233.0</td>\n",
       "      <td>49291.0</td>\n",
       "      <td>28314.0</td>\n",
       "      <td>28959.0</td>\n",
       "      <td>29547.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1100.0</td>\n",
       "      <td>1069.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>30s</td>\n",
       "      <td>(2, 30s)</td>\n",
       "      <td>(2, 1)</td>\n",
       "      <td>46995.038110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8617.0</td>\n",
       "      <td>5670.0</td>\n",
       "      <td>35835.0</td>\n",
       "      <td>20940.0</td>\n",
       "      <td>19146.0</td>\n",
       "      <td>19131.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>36681.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9000.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>679.0</td>\n",
       "      <td>50s</td>\n",
       "      <td>(1, 50s)</td>\n",
       "      <td>(1, 1)</td>\n",
       "      <td>8605.584386</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   limit_bal  gender  education  marriage  age  pay_1  pay_2  pay_3  pay_4  \\\n",
       "0    20000.0       2          2         1   24      1      1      0      0   \n",
       "1   120000.0       2          2         2   26      0      1      0      0   \n",
       "2    90000.0       2          2         2   34      0      0      0      0   \n",
       "3    50000.0       2          2         1   37      0      0      0      0   \n",
       "4    50000.0       1          2         1   57      0      0      0      0   \n",
       "\n",
       "   pay_5  pay_6  bill_amt1  bill_amt2  bill_amt3  bill_amt4  bill_amt5  \\\n",
       "0      0      0     3913.0     3102.0      689.0        0.0        0.0   \n",
       "1      0      1     2682.0     1725.0     2682.0     3272.0     3455.0   \n",
       "2      0      0    29239.0    14027.0    13559.0    14331.0    14948.0   \n",
       "3      0      0    46990.0    48233.0    49291.0    28314.0    28959.0   \n",
       "4      0      0     8617.0     5670.0    35835.0    20940.0    19146.0   \n",
       "\n",
       "   bill_amt6  pay_amt1  pay_amt2  pay_amt3  pay_amt4  pay_amt5  pay_amt6  \\\n",
       "0        0.0       0.0     689.0       0.0       0.0       0.0       0.0   \n",
       "1     3261.0       0.0    1000.0    1000.0    1000.0       0.0    2000.0   \n",
       "2    15549.0    1518.0    1500.0    1000.0    1000.0    1000.0    5000.0   \n",
       "3    29547.0    2000.0    2019.0    1200.0    1100.0    1069.0    1000.0   \n",
       "4    19131.0    2000.0   36681.0   10000.0    9000.0     689.0     679.0   \n",
       "\n",
       "  age_bin gen_ageBin gen_mar  pred_bill_amt0  \n",
       "0     20s   (2, 20s)  (2, 1)     3909.683795  \n",
       "1     20s   (2, 20s)  (2, 2)     2678.148995  \n",
       "2     30s   (2, 30s)  (2, 2)    29177.529785  \n",
       "3     30s   (2, 30s)  (2, 1)    46995.038110  \n",
       "4     50s   (1, 50s)  (1, 1)     8605.584386  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe3_cat, X_raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Numerical version (with One-hot)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "complex_pipe3_num = Pipeline([\n",
    "    (\"age_bin\", AgeBinAdder()),\n",
    "    (\"gender_age_bin\", GenderXAgeBinAdder()),\n",
    "    (\"gender_marriage\", GenderXMarriageAdder()),\n",
    "    (\"next_bill\", NextBillAdder()),\n",
    "    (\"1_hot\", CategoricalWarrior(['gender', 'education', 'marriage']))\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 51)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>limit_bal</th>\n",
       "      <th>age</th>\n",
       "      <th>pay_1</th>\n",
       "      <th>pay_2</th>\n",
       "      <th>pay_3</th>\n",
       "      <th>pay_4</th>\n",
       "      <th>pay_5</th>\n",
       "      <th>pay_6</th>\n",
       "      <th>bill_amt1</th>\n",
       "      <th>bill_amt2</th>\n",
       "      <th>bill_amt3</th>\n",
       "      <th>bill_amt4</th>\n",
       "      <th>bill_amt5</th>\n",
       "      <th>bill_amt6</th>\n",
       "      <th>pay_amt1</th>\n",
       "      <th>pay_amt2</th>\n",
       "      <th>pay_amt3</th>\n",
       "      <th>pay_amt4</th>\n",
       "      <th>pay_amt5</th>\n",
       "      <th>pay_amt6</th>\n",
       "      <th>pred_bill_amt0</th>\n",
       "      <th>gender_2</th>\n",
       "      <th>gender_1</th>\n",
       "      <th>education_2</th>\n",
       "      <th>education_1</th>\n",
       "      <th>education_3</th>\n",
       "      <th>education_4</th>\n",
       "      <th>marriage_1</th>\n",
       "      <th>marriage_2</th>\n",
       "      <th>marriage_3</th>\n",
       "      <th>age_bin_20s</th>\n",
       "      <th>age_bin_30s</th>\n",
       "      <th>age_bin_40s</th>\n",
       "      <th>age_bin_50s</th>\n",
       "      <th>age_bin_60+</th>\n",
       "      <th>gen_ageBin_(1, '20s')</th>\n",
       "      <th>gen_ageBin_(1, '30s')</th>\n",
       "      <th>gen_ageBin_(1, '40s')</th>\n",
       "      <th>gen_ageBin_(1, '50s')</th>\n",
       "      <th>gen_ageBin_(1, '60+')</th>\n",
       "      <th>gen_ageBin_(2, '20s')</th>\n",
       "      <th>gen_ageBin_(2, '30s')</th>\n",
       "      <th>gen_ageBin_(2, '40s')</th>\n",
       "      <th>gen_ageBin_(2, '50s')</th>\n",
       "      <th>gen_ageBin_(2, '60+')</th>\n",
       "      <th>gen_mar_(1, 1)</th>\n",
       "      <th>gen_mar_(1, 2)</th>\n",
       "      <th>gen_mar_(1, 3)</th>\n",
       "      <th>gen_mar_(2, 1)</th>\n",
       "      <th>gen_mar_(2, 2)</th>\n",
       "      <th>gen_mar_(2, 3)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3913.0</td>\n",
       "      <td>3102.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3909.683795</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>120000.0</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>1725.0</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>3272.0</td>\n",
       "      <td>3455.0</td>\n",
       "      <td>3261.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2678.148995</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29239.0</td>\n",
       "      <td>14027.0</td>\n",
       "      <td>13559.0</td>\n",
       "      <td>14331.0</td>\n",
       "      <td>14948.0</td>\n",
       "      <td>15549.0</td>\n",
       "      <td>1518.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>29177.529785</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>46990.0</td>\n",
       "      <td>48233.0</td>\n",
       "      <td>49291.0</td>\n",
       "      <td>28314.0</td>\n",
       "      <td>28959.0</td>\n",
       "      <td>29547.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1100.0</td>\n",
       "      <td>1069.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>46995.038110</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8617.0</td>\n",
       "      <td>5670.0</td>\n",
       "      <td>35835.0</td>\n",
       "      <td>20940.0</td>\n",
       "      <td>19146.0</td>\n",
       "      <td>19131.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>36681.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9000.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>679.0</td>\n",
       "      <td>8605.584386</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   limit_bal  age  pay_1  pay_2  pay_3  pay_4  pay_5  pay_6  bill_amt1  \\\n",
       "0    20000.0   24      1      1      0      0      0      0     3913.0   \n",
       "1   120000.0   26      0      1      0      0      0      1     2682.0   \n",
       "2    90000.0   34      0      0      0      0      0      0    29239.0   \n",
       "3    50000.0   37      0      0      0      0      0      0    46990.0   \n",
       "4    50000.0   57      0      0      0      0      0      0     8617.0   \n",
       "\n",
       "   bill_amt2  bill_amt3  bill_amt4  bill_amt5  bill_amt6  pay_amt1  pay_amt2  \\\n",
       "0     3102.0      689.0        0.0        0.0        0.0       0.0     689.0   \n",
       "1     1725.0     2682.0     3272.0     3455.0     3261.0       0.0    1000.0   \n",
       "2    14027.0    13559.0    14331.0    14948.0    15549.0    1518.0    1500.0   \n",
       "3    48233.0    49291.0    28314.0    28959.0    29547.0    2000.0    2019.0   \n",
       "4     5670.0    35835.0    20940.0    19146.0    19131.0    2000.0   36681.0   \n",
       "\n",
       "   pay_amt3  pay_amt4  pay_amt5  pay_amt6  pred_bill_amt0  gender_2  gender_1  \\\n",
       "0       0.0       0.0       0.0       0.0     3909.683795         1         0   \n",
       "1    1000.0    1000.0       0.0    2000.0     2678.148995         1         0   \n",
       "2    1000.0    1000.0    1000.0    5000.0    29177.529785         1         0   \n",
       "3    1200.0    1100.0    1069.0    1000.0    46995.038110         1         0   \n",
       "4   10000.0    9000.0     689.0     679.0     8605.584386         0         1   \n",
       "\n",
       "   education_2  education_1  education_3  education_4  marriage_1  marriage_2  \\\n",
       "0            1            0            0            0           1           0   \n",
       "1            1            0            0            0           0           1   \n",
       "2            1            0            0            0           0           1   \n",
       "3            1            0            0            0           1           0   \n",
       "4            1            0            0            0           1           0   \n",
       "\n",
       "   marriage_3  age_bin_20s  age_bin_30s  age_bin_40s  age_bin_50s  \\\n",
       "0           0            1            0            0            0   \n",
       "1           0            1            0            0            0   \n",
       "2           0            0            1            0            0   \n",
       "3           0            0            1            0            0   \n",
       "4           0            0            0            0            1   \n",
       "\n",
       "   age_bin_60+  gen_ageBin_(1, '20s')  gen_ageBin_(1, '30s')  \\\n",
       "0            0                      0                      0   \n",
       "1            0                      0                      0   \n",
       "2            0                      0                      0   \n",
       "3            0                      0                      0   \n",
       "4            0                      0                      0   \n",
       "\n",
       "   gen_ageBin_(1, '40s')  gen_ageBin_(1, '50s')  gen_ageBin_(1, '60+')  \\\n",
       "0                      0                      0                      0   \n",
       "1                      0                      0                      0   \n",
       "2                      0                      0                      0   \n",
       "3                      0                      0                      0   \n",
       "4                      0                      1                      0   \n",
       "\n",
       "   gen_ageBin_(2, '20s')  gen_ageBin_(2, '30s')  gen_ageBin_(2, '40s')  \\\n",
       "0                      1                      0                      0   \n",
       "1                      1                      0                      0   \n",
       "2                      0                      1                      0   \n",
       "3                      0                      1                      0   \n",
       "4                      0                      0                      0   \n",
       "\n",
       "   gen_ageBin_(2, '50s')  gen_ageBin_(2, '60+')  gen_mar_(1, 1)  \\\n",
       "0                      0                      0               0   \n",
       "1                      0                      0               0   \n",
       "2                      0                      0               0   \n",
       "3                      0                      0               0   \n",
       "4                      0                      0               1   \n",
       "\n",
       "   gen_mar_(1, 2)  gen_mar_(1, 3)  gen_mar_(2, 1)  gen_mar_(2, 2)  \\\n",
       "0               0               0               1               0   \n",
       "1               0               0               0               1   \n",
       "2               0               0               0               1   \n",
       "3               0               0               1               0   \n",
       "4               0               0               0               0   \n",
       "\n",
       "   gen_mar_(2, 3)  \n",
       "0               0  \n",
       "1               0  \n",
       "2               0  \n",
       "3               0  \n",
       "4               0  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe3_num, X_raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With our 4 engineered features, but without 'age'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's remove the 'age' feature:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Categorical version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [],
   "source": [
    "complex_pipe4_cat = Pipeline([\n",
    "    (\"age_bin\", AgeBinAdder()),\n",
    "    (\"gender_age_bin\", GenderXAgeBinAdder()),\n",
    "    (\"gender_marriage\", GenderXMarriageAdder()),\n",
    "    (\"next_bill\", NextBillAdder()),\n",
    "    (\"age_remove\", ColumnDropper('age'))\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 26)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>limit_bal</th>\n",
       "      <th>gender</th>\n",
       "      <th>education</th>\n",
       "      <th>marriage</th>\n",
       "      <th>pay_1</th>\n",
       "      <th>pay_2</th>\n",
       "      <th>pay_3</th>\n",
       "      <th>pay_4</th>\n",
       "      <th>pay_5</th>\n",
       "      <th>pay_6</th>\n",
       "      <th>bill_amt1</th>\n",
       "      <th>bill_amt2</th>\n",
       "      <th>bill_amt3</th>\n",
       "      <th>bill_amt4</th>\n",
       "      <th>bill_amt5</th>\n",
       "      <th>bill_amt6</th>\n",
       "      <th>pay_amt1</th>\n",
       "      <th>pay_amt2</th>\n",
       "      <th>pay_amt3</th>\n",
       "      <th>pay_amt4</th>\n",
       "      <th>pay_amt5</th>\n",
       "      <th>pay_amt6</th>\n",
       "      <th>age_bin</th>\n",
       "      <th>gen_ageBin</th>\n",
       "      <th>gen_mar</th>\n",
       "      <th>pred_bill_amt0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3913.0</td>\n",
       "      <td>3102.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20s</td>\n",
       "      <td>(2, 20s)</td>\n",
       "      <td>(2, 1)</td>\n",
       "      <td>3909.683795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>120000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>1725.0</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>3272.0</td>\n",
       "      <td>3455.0</td>\n",
       "      <td>3261.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>20s</td>\n",
       "      <td>(2, 20s)</td>\n",
       "      <td>(2, 2)</td>\n",
       "      <td>2678.148995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29239.0</td>\n",
       "      <td>14027.0</td>\n",
       "      <td>13559.0</td>\n",
       "      <td>14331.0</td>\n",
       "      <td>14948.0</td>\n",
       "      <td>15549.0</td>\n",
       "      <td>1518.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>30s</td>\n",
       "      <td>(2, 30s)</td>\n",
       "      <td>(2, 2)</td>\n",
       "      <td>29177.529785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>46990.0</td>\n",
       "      <td>48233.0</td>\n",
       "      <td>49291.0</td>\n",
       "      <td>28314.0</td>\n",
       "      <td>28959.0</td>\n",
       "      <td>29547.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1100.0</td>\n",
       "      <td>1069.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>30s</td>\n",
       "      <td>(2, 30s)</td>\n",
       "      <td>(2, 1)</td>\n",
       "      <td>46995.038110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8617.0</td>\n",
       "      <td>5670.0</td>\n",
       "      <td>35835.0</td>\n",
       "      <td>20940.0</td>\n",
       "      <td>19146.0</td>\n",
       "      <td>19131.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>36681.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9000.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>679.0</td>\n",
       "      <td>50s</td>\n",
       "      <td>(1, 50s)</td>\n",
       "      <td>(1, 1)</td>\n",
       "      <td>8605.584386</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   limit_bal  gender  education  marriage  pay_1  pay_2  pay_3  pay_4  pay_5  \\\n",
       "0    20000.0       2          2         1      1      1      0      0      0   \n",
       "1   120000.0       2          2         2      0      1      0      0      0   \n",
       "2    90000.0       2          2         2      0      0      0      0      0   \n",
       "3    50000.0       2          2         1      0      0      0      0      0   \n",
       "4    50000.0       1          2         1      0      0      0      0      0   \n",
       "\n",
       "   pay_6  bill_amt1  bill_amt2  bill_amt3  bill_amt4  bill_amt5  bill_amt6  \\\n",
       "0      0     3913.0     3102.0      689.0        0.0        0.0        0.0   \n",
       "1      1     2682.0     1725.0     2682.0     3272.0     3455.0     3261.0   \n",
       "2      0    29239.0    14027.0    13559.0    14331.0    14948.0    15549.0   \n",
       "3      0    46990.0    48233.0    49291.0    28314.0    28959.0    29547.0   \n",
       "4      0     8617.0     5670.0    35835.0    20940.0    19146.0    19131.0   \n",
       "\n",
       "   pay_amt1  pay_amt2  pay_amt3  pay_amt4  pay_amt5  pay_amt6 age_bin  \\\n",
       "0       0.0     689.0       0.0       0.0       0.0       0.0     20s   \n",
       "1       0.0    1000.0    1000.0    1000.0       0.0    2000.0     20s   \n",
       "2    1518.0    1500.0    1000.0    1000.0    1000.0    5000.0     30s   \n",
       "3    2000.0    2019.0    1200.0    1100.0    1069.0    1000.0     30s   \n",
       "4    2000.0   36681.0   10000.0    9000.0     689.0     679.0     50s   \n",
       "\n",
       "  gen_ageBin gen_mar  pred_bill_amt0  \n",
       "0   (2, 20s)  (2, 1)     3909.683795  \n",
       "1   (2, 20s)  (2, 2)     2678.148995  \n",
       "2   (2, 30s)  (2, 2)    29177.529785  \n",
       "3   (2, 30s)  (2, 1)    46995.038110  \n",
       "4   (1, 50s)  (1, 1)     8605.584386  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe4_cat, X_raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Numerical version (with One-hot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add a CategoricalWarrior to 1-hot encode our categorical features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pipeline with 1-hot encoded\n",
    "complex_pipe4_num = Pipeline([\n",
    "    (\"age_bin\", AgeBinAdder()),\n",
    "    (\"gender_age_bin\", GenderXAgeBinAdder()),\n",
    "    (\"gender_marriage\", GenderXMarriageAdder()),\n",
    "    (\"next_bill\", NextBillAdder()),\n",
    "    (\"age_remove\", ColumnDropper('age')),\n",
    "    (\"1_hot\", CategoricalWarrior(['gender', 'education', 'marriage']))\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 50)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>limit_bal</th>\n",
       "      <th>pay_1</th>\n",
       "      <th>pay_2</th>\n",
       "      <th>pay_3</th>\n",
       "      <th>pay_4</th>\n",
       "      <th>pay_5</th>\n",
       "      <th>pay_6</th>\n",
       "      <th>bill_amt1</th>\n",
       "      <th>bill_amt2</th>\n",
       "      <th>bill_amt3</th>\n",
       "      <th>bill_amt4</th>\n",
       "      <th>bill_amt5</th>\n",
       "      <th>bill_amt6</th>\n",
       "      <th>pay_amt1</th>\n",
       "      <th>pay_amt2</th>\n",
       "      <th>pay_amt3</th>\n",
       "      <th>pay_amt4</th>\n",
       "      <th>pay_amt5</th>\n",
       "      <th>pay_amt6</th>\n",
       "      <th>pred_bill_amt0</th>\n",
       "      <th>gender_2</th>\n",
       "      <th>gender_1</th>\n",
       "      <th>education_2</th>\n",
       "      <th>education_1</th>\n",
       "      <th>education_3</th>\n",
       "      <th>education_4</th>\n",
       "      <th>marriage_1</th>\n",
       "      <th>marriage_2</th>\n",
       "      <th>marriage_3</th>\n",
       "      <th>age_bin_20s</th>\n",
       "      <th>age_bin_30s</th>\n",
       "      <th>age_bin_40s</th>\n",
       "      <th>age_bin_50s</th>\n",
       "      <th>age_bin_60+</th>\n",
       "      <th>gen_ageBin_(1, '20s')</th>\n",
       "      <th>gen_ageBin_(1, '30s')</th>\n",
       "      <th>gen_ageBin_(1, '40s')</th>\n",
       "      <th>gen_ageBin_(1, '50s')</th>\n",
       "      <th>gen_ageBin_(1, '60+')</th>\n",
       "      <th>gen_ageBin_(2, '20s')</th>\n",
       "      <th>gen_ageBin_(2, '30s')</th>\n",
       "      <th>gen_ageBin_(2, '40s')</th>\n",
       "      <th>gen_ageBin_(2, '50s')</th>\n",
       "      <th>gen_ageBin_(2, '60+')</th>\n",
       "      <th>gen_mar_(1, 1)</th>\n",
       "      <th>gen_mar_(1, 2)</th>\n",
       "      <th>gen_mar_(1, 3)</th>\n",
       "      <th>gen_mar_(2, 1)</th>\n",
       "      <th>gen_mar_(2, 2)</th>\n",
       "      <th>gen_mar_(2, 3)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3913.0</td>\n",
       "      <td>3102.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3909.683795</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>120000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>1725.0</td>\n",
       "      <td>2682.0</td>\n",
       "      <td>3272.0</td>\n",
       "      <td>3455.0</td>\n",
       "      <td>3261.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2678.148995</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29239.0</td>\n",
       "      <td>14027.0</td>\n",
       "      <td>13559.0</td>\n",
       "      <td>14331.0</td>\n",
       "      <td>14948.0</td>\n",
       "      <td>15549.0</td>\n",
       "      <td>1518.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>29177.529785</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>46990.0</td>\n",
       "      <td>48233.0</td>\n",
       "      <td>49291.0</td>\n",
       "      <td>28314.0</td>\n",
       "      <td>28959.0</td>\n",
       "      <td>29547.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1100.0</td>\n",
       "      <td>1069.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>46995.038110</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8617.0</td>\n",
       "      <td>5670.0</td>\n",
       "      <td>35835.0</td>\n",
       "      <td>20940.0</td>\n",
       "      <td>19146.0</td>\n",
       "      <td>19131.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>36681.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9000.0</td>\n",
       "      <td>689.0</td>\n",
       "      <td>679.0</td>\n",
       "      <td>8605.584386</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   limit_bal  pay_1  pay_2  pay_3  pay_4  pay_5  pay_6  bill_amt1  bill_amt2  \\\n",
       "0    20000.0      1      1      0      0      0      0     3913.0     3102.0   \n",
       "1   120000.0      0      1      0      0      0      1     2682.0     1725.0   \n",
       "2    90000.0      0      0      0      0      0      0    29239.0    14027.0   \n",
       "3    50000.0      0      0      0      0      0      0    46990.0    48233.0   \n",
       "4    50000.0      0      0      0      0      0      0     8617.0     5670.0   \n",
       "\n",
       "   bill_amt3  bill_amt4  bill_amt5  bill_amt6  pay_amt1  pay_amt2  pay_amt3  \\\n",
       "0      689.0        0.0        0.0        0.0       0.0     689.0       0.0   \n",
       "1     2682.0     3272.0     3455.0     3261.0       0.0    1000.0    1000.0   \n",
       "2    13559.0    14331.0    14948.0    15549.0    1518.0    1500.0    1000.0   \n",
       "3    49291.0    28314.0    28959.0    29547.0    2000.0    2019.0    1200.0   \n",
       "4    35835.0    20940.0    19146.0    19131.0    2000.0   36681.0   10000.0   \n",
       "\n",
       "   pay_amt4  pay_amt5  pay_amt6  pred_bill_amt0  gender_2  gender_1  \\\n",
       "0       0.0       0.0       0.0     3909.683795         1         0   \n",
       "1    1000.0       0.0    2000.0     2678.148995         1         0   \n",
       "2    1000.0    1000.0    5000.0    29177.529785         1         0   \n",
       "3    1100.0    1069.0    1000.0    46995.038110         1         0   \n",
       "4    9000.0     689.0     679.0     8605.584386         0         1   \n",
       "\n",
       "   education_2  education_1  education_3  education_4  marriage_1  marriage_2  \\\n",
       "0            1            0            0            0           1           0   \n",
       "1            1            0            0            0           0           1   \n",
       "2            1            0            0            0           0           1   \n",
       "3            1            0            0            0           1           0   \n",
       "4            1            0            0            0           1           0   \n",
       "\n",
       "   marriage_3  age_bin_20s  age_bin_30s  age_bin_40s  age_bin_50s  \\\n",
       "0           0            1            0            0            0   \n",
       "1           0            1            0            0            0   \n",
       "2           0            0            1            0            0   \n",
       "3           0            0            1            0            0   \n",
       "4           0            0            0            0            1   \n",
       "\n",
       "   age_bin_60+  gen_ageBin_(1, '20s')  gen_ageBin_(1, '30s')  \\\n",
       "0            0                      0                      0   \n",
       "1            0                      0                      0   \n",
       "2            0                      0                      0   \n",
       "3            0                      0                      0   \n",
       "4            0                      0                      0   \n",
       "\n",
       "   gen_ageBin_(1, '40s')  gen_ageBin_(1, '50s')  gen_ageBin_(1, '60+')  \\\n",
       "0                      0                      0                      0   \n",
       "1                      0                      0                      0   \n",
       "2                      0                      0                      0   \n",
       "3                      0                      0                      0   \n",
       "4                      0                      1                      0   \n",
       "\n",
       "   gen_ageBin_(2, '20s')  gen_ageBin_(2, '30s')  gen_ageBin_(2, '40s')  \\\n",
       "0                      1                      0                      0   \n",
       "1                      1                      0                      0   \n",
       "2                      0                      1                      0   \n",
       "3                      0                      1                      0   \n",
       "4                      0                      0                      0   \n",
       "\n",
       "   gen_ageBin_(2, '50s')  gen_ageBin_(2, '60+')  gen_mar_(1, 1)  \\\n",
       "0                      0                      0               0   \n",
       "1                      0                      0               0   \n",
       "2                      0                      0               0   \n",
       "3                      0                      0               0   \n",
       "4                      0                      0               1   \n",
       "\n",
       "   gen_mar_(1, 2)  gen_mar_(1, 3)  gen_mar_(2, 1)  gen_mar_(2, 2)  \\\n",
       "0               0               0               1               0   \n",
       "1               0               0               0               1   \n",
       "2               0               0               0               1   \n",
       "3               0               0               1               0   \n",
       "4               0               0               0               0   \n",
       "\n",
       "   gen_mar_(2, 3)  \n",
       "0               0  \n",
       "1               0  \n",
       "2               0  \n",
       "3               0  \n",
       "4               0  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe4_num, X_raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One-hot encoding & Scaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add a StandardScaler to normalize our inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pipeline with 1-hot encoded\n",
    "# Pipeline with a StandardScaler\n",
    "complex_pipe5 = Pipeline([\n",
    "    (\"age_bin\", AgeBinAdder()),\n",
    "    (\"gender_age_bin\", GenderXAgeBinAdder()),\n",
    "    (\"gender_marriage\", GenderXMarriageAdder()),\n",
    "    (\"next_bill\", NextBillAdder()),\n",
    "    (\"age_remove\", ColumnDropper('age')),\n",
    "    (\"1_hot\", CategoricalWarrior(['gender', 'education', 'marriage'])),\n",
    "    (\"scaler\", StandardScaler())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (30000, 50)\n",
      "[[-1.13672015  1.84394071  2.39995869 -0.40419908 -0.36400938 -0.33135449\n",
      "  -0.33818882 -0.64250107 -0.64739923 -0.66799331 -0.67249727 -0.66305853\n",
      "  -0.65272422 -0.34194162 -0.22708564 -0.29680127 -0.30806256 -0.31413612\n",
      "  -0.29338206 -0.64264241  0.81016074 -0.81016074  1.06689977 -0.73837457\n",
      "  -0.44275183 -0.12588573  1.09377971 -1.06647132 -0.11281222  1.31303214\n",
      "  -0.74528643 -0.5002604  -0.26221495 -0.1069072  -0.38324492 -0.41253329\n",
      "  -0.30719909 -0.17756815 -0.07747568  1.78424128 -0.51816884 -0.35858507\n",
      "  -0.18694037 -0.0732252  -0.45737276 -0.52865999 -0.0696908   1.59446883\n",
      "  -0.67608338 -0.08828139]\n",
      " [-0.3659805  -0.54231679  2.39995869 -0.40419908 -0.36400938 -0.33135449\n",
      "   2.956928   -0.65921875 -0.66674657 -0.63925429 -0.62163594 -0.60622927\n",
      "  -0.59796638 -0.34194162 -0.21358766 -0.24000461 -0.24422965 -0.31413612\n",
      "  -0.18087821 -0.65937279  0.81016074 -0.81016074  1.06689977 -0.73837457\n",
      "  -0.44275183 -0.12588573 -0.91426088  0.93767172 -0.11281222  1.31303214\n",
      "  -0.74528643 -0.5002604  -0.26221495 -0.1069072  -0.38324492 -0.41253329\n",
      "  -0.30719909 -0.17756815 -0.07747568  1.78424128 -0.51816884 -0.35858507\n",
      "  -0.18694037 -0.0732252  -0.45737276 -0.52865999 -0.0696908  -0.62716811\n",
      "   1.4791075  -0.08828139]]\n"
     ]
    }
   ],
   "source": [
    "show_pipe_result(complex_pipe5, X_raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [],
   "source": [
    "complex_pipelines_cat = [complex_pipe3_cat,\n",
    "                        complex_pipe4_cat]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [],
   "source": [
    "complex_pipelines_num = [complex_pipe1,\n",
    "                    complex_pipe2,\n",
    "                    complex_pipe3_num,\n",
    "                    complex_pipe4_num,\n",
    "                    complex_pipe5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining our Pipelines programmatically"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_pipelines = {\n",
    "    \"age_bin\": AgeBinAdder(),\n",
    "    \"gender_age_bin\": GenderXAgeBinAdder(),\n",
    "    \"gender_marriage\": GenderXMarriageAdder(),\n",
    "    \"next_bill\": NextBillAdder(),\n",
    "    \"age_remove\": ColumnDropper('age'),\n",
    "    \"1_hot\": CategoricalWarrior(['gender', 'education', 'marriage']),\n",
    "    \"scaler\": StandardScaler()\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_complex_pipe(pipeline_names, dico_pipelines=dico_pipelines):\n",
    "    pipe = Pipeline([(name, dico_pipelines[name]) for name in pipeline_names])\n",
    "    return pipe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('age_bin', AgeBinAdder()),\n",
       "                ('1_hot',\n",
       "                 CategoricalWarrior(attribute_names=['gender', 'education',\n",
       "                                                     'marriage'])),\n",
       "                ('scaler',\n",
       "                 StandardScaler(copy=True, with_mean=True, with_std=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 233,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "create_complex_pipe([\"age_bin\", \"1_hot\", \"scaler\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.13746157,  1.69528941, -0.54773574, ..., -0.49304556,\n",
       "         3.2349603 , -0.13367595],\n",
       "       [ 1.02364663, -0.47701361, -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       [ 2.56729535,  0.50052275, -0.54773574, ...,  2.02821014,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       ...,\n",
       "       [ 0.02027497, -0.80285906, -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       [-0.52000208, -0.2597833 , -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       [-0.82873183, -1.12870451, -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595]])"
      ]
     },
     "execution_count": 222,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Test\n",
    "create_complex_pipe([\"age_bin\", \"1_hot\", \"scaler\"]).fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "pipe2 = make_pipeline(AgeBinAdder(), CategoricalWarrior(['gender', 'education', 'marriage']), StandardScaler())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('agebinadder', AgeBinAdder()),\n",
       "                ('categoricalwarrior',\n",
       "                 CategoricalWarrior(attribute_names=['gender', 'education',\n",
       "                                                     'marriage'])),\n",
       "                ('standardscaler',\n",
       "                 StandardScaler(copy=True, with_mean=True, with_std=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 235,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.13746157,  1.69528941, -0.54773574, ..., -0.49304556,\n",
       "         3.2349603 , -0.13367595],\n",
       "       [ 1.02364663, -0.47701361, -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       [ 2.56729535,  0.50052275, -0.54773574, ...,  2.02821014,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       ...,\n",
       "       [ 0.02027497, -0.80285906, -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       [-0.52000208, -0.2597833 , -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595],\n",
       "       [-0.82873183, -1.12870451, -0.54773574, ..., -0.49304556,\n",
       "        -0.3091228 , -0.13367595]])"
      ]
     },
     "execution_count": 236,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe2.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>limit_bal</th>\n",
       "      <th>gender</th>\n",
       "      <th>education</th>\n",
       "      <th>marriage</th>\n",
       "      <th>age</th>\n",
       "      <th>pay_1</th>\n",
       "      <th>pay_2</th>\n",
       "      <th>pay_3</th>\n",
       "      <th>pay_4</th>\n",
       "      <th>pay_5</th>\n",
       "      <th>pay_6</th>\n",
       "      <th>bill_amt1</th>\n",
       "      <th>bill_amt2</th>\n",
       "      <th>bill_amt3</th>\n",
       "      <th>bill_amt4</th>\n",
       "      <th>bill_amt5</th>\n",
       "      <th>bill_amt6</th>\n",
       "      <th>pay_amt1</th>\n",
       "      <th>pay_amt2</th>\n",
       "      <th>pay_amt3</th>\n",
       "      <th>pay_amt4</th>\n",
       "      <th>pay_amt5</th>\n",
       "      <th>pay_amt6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10574</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19255.0</td>\n",
       "      <td>20650.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11330</th>\n",
       "      <td>300000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>285038.0</td>\n",
       "      <td>291480.0</td>\n",
       "      <td>298378.0</td>\n",
       "      <td>131103.0</td>\n",
       "      <td>128742.0</td>\n",
       "      <td>124882.0</td>\n",
       "      <td>12507.0</td>\n",
       "      <td>15056.0</td>\n",
       "      <td>5027.0</td>\n",
       "      <td>5007.0</td>\n",
       "      <td>5063.0</td>\n",
       "      <td>3039.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7127</th>\n",
       "      <td>500000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>40</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>33179.0</td>\n",
       "      <td>97882.0</td>\n",
       "      <td>48017.0</td>\n",
       "      <td>49403.0</td>\n",
       "      <td>40018.0</td>\n",
       "      <td>54109.0</td>\n",
       "      <td>99159.0</td>\n",
       "      <td>48276.0</td>\n",
       "      <td>49450.0</td>\n",
       "      <td>201.0</td>\n",
       "      <td>54379.0</td>\n",
       "      <td>9905.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25150</th>\n",
       "      <td>70000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>70349.0</td>\n",
       "      <td>70068.0</td>\n",
       "      <td>71318.0</td>\n",
       "      <td>59665.0</td>\n",
       "      <td>29510.0</td>\n",
       "      <td>26738.0</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>3700.0</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6000</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13538.0</td>\n",
       "      <td>9199.0</td>\n",
       "      <td>6879.0</td>\n",
       "      <td>6753.0</td>\n",
       "      <td>5440.0</td>\n",
       "      <td>5440.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>1284.0</td>\n",
       "      <td>264.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20951</th>\n",
       "      <td>340000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11379.0</td>\n",
       "      <td>11086.0</td>\n",
       "      <td>3112.0</td>\n",
       "      <td>3351.0</td>\n",
       "      <td>1259.0</td>\n",
       "      <td>19271.0</td>\n",
       "      <td>11258.0</td>\n",
       "      <td>3112.0</td>\n",
       "      <td>3351.0</td>\n",
       "      <td>1259.0</td>\n",
       "      <td>19271.0</td>\n",
       "      <td>16992.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13894</th>\n",
       "      <td>300000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>145138.0</td>\n",
       "      <td>148586.0</td>\n",
       "      <td>151878.0</td>\n",
       "      <td>123417.0</td>\n",
       "      <td>104922.0</td>\n",
       "      <td>107005.0</td>\n",
       "      <td>5785.0</td>\n",
       "      <td>6163.0</td>\n",
       "      <td>4728.0</td>\n",
       "      <td>3811.0</td>\n",
       "      <td>3830.0</td>\n",
       "      <td>3970.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15285</th>\n",
       "      <td>230000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>970.0</td>\n",
       "      <td>1475.0</td>\n",
       "      <td>1636.0</td>\n",
       "      <td>1149.0</td>\n",
       "      <td>334.0</td>\n",
       "      <td>459.0</td>\n",
       "      <td>1483.0</td>\n",
       "      <td>1640.0</td>\n",
       "      <td>1152.0</td>\n",
       "      <td>335.0</td>\n",
       "      <td>460.0</td>\n",
       "      <td>553.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6881</th>\n",
       "      <td>200000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>26344.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>320.0</td>\n",
       "      <td>170.0</td>\n",
       "      <td>2865.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>329.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2865.0</td>\n",
       "      <td>801.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22145</th>\n",
       "      <td>60000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23304.0</td>\n",
       "      <td>20517.0</td>\n",
       "      <td>20261.0</td>\n",
       "      <td>22832.0</td>\n",
       "      <td>24234.0</td>\n",
       "      <td>24928.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10270</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>49423.0</td>\n",
       "      <td>48332.0</td>\n",
       "      <td>46043.0</td>\n",
       "      <td>18759.0</td>\n",
       "      <td>19161.0</td>\n",
       "      <td>19516.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>655.0</td>\n",
       "      <td>688.0</td>\n",
       "      <td>666.0</td>\n",
       "      <td>649.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28740</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4145.0</td>\n",
       "      <td>3930.0</td>\n",
       "      <td>5859.0</td>\n",
       "      <td>10299.0</td>\n",
       "      <td>11825.0</td>\n",
       "      <td>13121.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>5008.0</td>\n",
       "      <td>3007.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19310</th>\n",
       "      <td>360000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4087</th>\n",
       "      <td>150000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>135587.0</td>\n",
       "      <td>138473.0</td>\n",
       "      <td>141297.0</td>\n",
       "      <td>144605.0</td>\n",
       "      <td>148420.0</td>\n",
       "      <td>150500.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>4500.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13760</th>\n",
       "      <td>480000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7174.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27714</th>\n",
       "      <td>130000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>44930.0</td>\n",
       "      <td>40688.0</td>\n",
       "      <td>40880.0</td>\n",
       "      <td>37851.0</td>\n",
       "      <td>37816.0</td>\n",
       "      <td>39849.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>8002.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7584</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>83480.0</td>\n",
       "      <td>89264.0</td>\n",
       "      <td>89896.0</td>\n",
       "      <td>86706.0</td>\n",
       "      <td>87385.0</td>\n",
       "      <td>88351.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>5000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29650</th>\n",
       "      <td>260000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>36</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>334214.0</td>\n",
       "      <td>310546.0</td>\n",
       "      <td>288566.0</td>\n",
       "      <td>291597.0</td>\n",
       "      <td>214353.0</td>\n",
       "      <td>216238.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11000.0</td>\n",
       "      <td>11000.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>8500.0</td>\n",
       "      <td>8300.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9472</th>\n",
       "      <td>440000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>225416.0</td>\n",
       "      <td>220405.0</td>\n",
       "      <td>226731.0</td>\n",
       "      <td>231456.0</td>\n",
       "      <td>237707.0</td>\n",
       "      <td>243555.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7022</th>\n",
       "      <td>100000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>59586.0</td>\n",
       "      <td>60842.0</td>\n",
       "      <td>61749.0</td>\n",
       "      <td>65130.0</td>\n",
       "      <td>63816.0</td>\n",
       "      <td>65011.0</td>\n",
       "      <td>2800.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2400.0</td>\n",
       "      <td>2400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21323</th>\n",
       "      <td>250000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>900.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>900.0</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23409</th>\n",
       "      <td>200000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>43</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>198595.0</td>\n",
       "      <td>194734.0</td>\n",
       "      <td>190447.0</td>\n",
       "      <td>187224.0</td>\n",
       "      <td>183641.0</td>\n",
       "      <td>178239.0</td>\n",
       "      <td>7003.0</td>\n",
       "      <td>7000.0</td>\n",
       "      <td>7000.0</td>\n",
       "      <td>6510.0</td>\n",
       "      <td>6550.0</td>\n",
       "      <td>6001.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3884</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9456</th>\n",
       "      <td>310000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2948.0</td>\n",
       "      <td>62593.0</td>\n",
       "      <td>152141.0</td>\n",
       "      <td>125325.0</td>\n",
       "      <td>109142.0</td>\n",
       "      <td>103020.0</td>\n",
       "      <td>62635.0</td>\n",
       "      <td>100042.0</td>\n",
       "      <td>4017.0</td>\n",
       "      <td>9184.0</td>\n",
       "      <td>4081.0</td>\n",
       "      <td>6042.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15820</th>\n",
       "      <td>30000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23575.0</td>\n",
       "      <td>24584.0</td>\n",
       "      <td>25885.0</td>\n",
       "      <td>26851.0</td>\n",
       "      <td>26808.0</td>\n",
       "      <td>27599.0</td>\n",
       "      <td>1400.0</td>\n",
       "      <td>1700.0</td>\n",
       "      <td>1900.0</td>\n",
       "      <td>557.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13152</th>\n",
       "      <td>180000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>43507.0</td>\n",
       "      <td>44282.0</td>\n",
       "      <td>23946.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10707</th>\n",
       "      <td>80000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>34</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17294.0</td>\n",
       "      <td>22256.0</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>22055.0</td>\n",
       "      <td>22603.0</td>\n",
       "      <td>23133.0</td>\n",
       "      <td>5210.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1541</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23062</th>\n",
       "      <td>150000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23364.0</td>\n",
       "      <td>22846.0</td>\n",
       "      <td>25779.0</td>\n",
       "      <td>22303.0</td>\n",
       "      <td>16981.0</td>\n",
       "      <td>16837.0</td>\n",
       "      <td>1450.0</td>\n",
       "      <td>5779.0</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>700.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11118</th>\n",
       "      <td>200000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>1879.0</td>\n",
       "      <td>1707.0</td>\n",
       "      <td>617.0</td>\n",
       "      <td>620.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1879.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>617.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3063</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>43685.0</td>\n",
       "      <td>44694.0</td>\n",
       "      <td>43673.0</td>\n",
       "      <td>44659.0</td>\n",
       "      <td>49512.0</td>\n",
       "      <td>48454.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>300.0</td>\n",
       "      <td>3000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21234</th>\n",
       "      <td>300000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50281.0</td>\n",
       "      <td>51441.0</td>\n",
       "      <td>54107.0</td>\n",
       "      <td>55111.0</td>\n",
       "      <td>56688.0</td>\n",
       "      <td>58272.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2459</th>\n",
       "      <td>250000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>37137.0</td>\n",
       "      <td>51803.0</td>\n",
       "      <td>4997.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>51811.0</td>\n",
       "      <td>9544.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24360</th>\n",
       "      <td>100000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>1158.0</td>\n",
       "      <td>2485.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9048</th>\n",
       "      <td>140000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>854.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25080</th>\n",
       "      <td>30000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14476.0</td>\n",
       "      <td>15297.0</td>\n",
       "      <td>8531.0</td>\n",
       "      <td>8020.0</td>\n",
       "      <td>-620.0</td>\n",
       "      <td>-620.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>1152.0</td>\n",
       "      <td>1081.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6886</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>55</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>390.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>911.0</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>930.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>911.0</td>\n",
       "      <td>559.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>390.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18264</th>\n",
       "      <td>180000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>77408.0</td>\n",
       "      <td>74156.0</td>\n",
       "      <td>75659.0</td>\n",
       "      <td>72046.0</td>\n",
       "      <td>70355.0</td>\n",
       "      <td>71208.0</td>\n",
       "      <td>3293.0</td>\n",
       "      <td>3350.0</td>\n",
       "      <td>3111.0</td>\n",
       "      <td>2615.0</td>\n",
       "      <td>2689.0</td>\n",
       "      <td>2766.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17546</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>31422.0</td>\n",
       "      <td>32420.0</td>\n",
       "      <td>33462.0</td>\n",
       "      <td>36199.0</td>\n",
       "      <td>36803.0</td>\n",
       "      <td>37566.0</td>\n",
       "      <td>1521.0</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>3300.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12326</th>\n",
       "      <td>240000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>133230.0</td>\n",
       "      <td>136579.0</td>\n",
       "      <td>139281.0</td>\n",
       "      <td>132487.0</td>\n",
       "      <td>107437.0</td>\n",
       "      <td>109237.0</td>\n",
       "      <td>5500.0</td>\n",
       "      <td>5036.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>6200.0</td>\n",
       "      <td>5500.0</td>\n",
       "      <td>5000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2440</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>40</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>67925.0</td>\n",
       "      <td>69771.0</td>\n",
       "      <td>73233.0</td>\n",
       "      <td>72328.0</td>\n",
       "      <td>74234.0</td>\n",
       "      <td>75500.0</td>\n",
       "      <td>3509.0</td>\n",
       "      <td>5824.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2204.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8905</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>42883.0</td>\n",
       "      <td>47887.0</td>\n",
       "      <td>46836.0</td>\n",
       "      <td>45785.0</td>\n",
       "      <td>47003.0</td>\n",
       "      <td>48085.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5739</th>\n",
       "      <td>150000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112391.0</td>\n",
       "      <td>98583.0</td>\n",
       "      <td>95810.0</td>\n",
       "      <td>87216.0</td>\n",
       "      <td>88201.0</td>\n",
       "      <td>87483.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>3290.0</td>\n",
       "      <td>3200.0</td>\n",
       "      <td>3215.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>360</th>\n",
       "      <td>500000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>507726.0</td>\n",
       "      <td>509229.0</td>\n",
       "      <td>499936.0</td>\n",
       "      <td>505507.0</td>\n",
       "      <td>484612.0</td>\n",
       "      <td>419643.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>16500.0</td>\n",
       "      <td>16000.0</td>\n",
       "      <td>19000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4816</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40019.0</td>\n",
       "      <td>20394.0</td>\n",
       "      <td>19597.0</td>\n",
       "      <td>17830.0</td>\n",
       "      <td>18397.0</td>\n",
       "      <td>19064.0</td>\n",
       "      <td>2006.0</td>\n",
       "      <td>1519.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15178</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28276.0</td>\n",
       "      <td>29305.0</td>\n",
       "      <td>30336.0</td>\n",
       "      <td>28800.0</td>\n",
       "      <td>9852.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>1505.0</td>\n",
       "      <td>1536.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>9852.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18321</th>\n",
       "      <td>480000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3655.0</td>\n",
       "      <td>38022.0</td>\n",
       "      <td>5785.0</td>\n",
       "      <td>9233.0</td>\n",
       "      <td>4893.0</td>\n",
       "      <td>2168.0</td>\n",
       "      <td>38024.0</td>\n",
       "      <td>5785.0</td>\n",
       "      <td>9233.0</td>\n",
       "      <td>4893.0</td>\n",
       "      <td>2168.0</td>\n",
       "      <td>3625.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23609</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>43886.0</td>\n",
       "      <td>50411.0</td>\n",
       "      <td>9688.0</td>\n",
       "      <td>857.0</td>\n",
       "      <td>3810.0</td>\n",
       "      <td>4741.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>7026.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16230</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>49450.0</td>\n",
       "      <td>50067.0</td>\n",
       "      <td>69594.0</td>\n",
       "      <td>46181.0</td>\n",
       "      <td>16770.0</td>\n",
       "      <td>19890.0</td>\n",
       "      <td>1743.0</td>\n",
       "      <td>2236.0</td>\n",
       "      <td>2807.0</td>\n",
       "      <td>629.0</td>\n",
       "      <td>3810.0</td>\n",
       "      <td>719.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21927</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3849.0</td>\n",
       "      <td>6393.0</td>\n",
       "      <td>6969.0</td>\n",
       "      <td>9044.0</td>\n",
       "      <td>2325.0</td>\n",
       "      <td>3998.0</td>\n",
       "      <td>6458.0</td>\n",
       "      <td>6980.0</td>\n",
       "      <td>9044.0</td>\n",
       "      <td>2325.0</td>\n",
       "      <td>4997.0</td>\n",
       "      <td>9947.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27267</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>14137.0</td>\n",
       "      <td>13613.0</td>\n",
       "      <td>14634.0</td>\n",
       "      <td>16532.0</td>\n",
       "      <td>15969.0</td>\n",
       "      <td>17701.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1247.0</td>\n",
       "      <td>2145.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11593</th>\n",
       "      <td>170000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-1020.0</td>\n",
       "      <td>25607.0</td>\n",
       "      <td>23922.0</td>\n",
       "      <td>24399.0</td>\n",
       "      <td>24919.0</td>\n",
       "      <td>26884.0</td>\n",
       "      <td>27107.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>863.0</td>\n",
       "      <td>903.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>6186.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15732</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27369.0</td>\n",
       "      <td>6425.0</td>\n",
       "      <td>4580.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1050.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25950</th>\n",
       "      <td>410000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>110345.0</td>\n",
       "      <td>116128.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>110345.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>4200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5863</th>\n",
       "      <td>320000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>44</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>36831.0</td>\n",
       "      <td>28785.0</td>\n",
       "      <td>13104.0</td>\n",
       "      <td>2621.0</td>\n",
       "      <td>9075.0</td>\n",
       "      <td>6564.0</td>\n",
       "      <td>28791.0</td>\n",
       "      <td>13227.0</td>\n",
       "      <td>2621.0</td>\n",
       "      <td>9075.0</td>\n",
       "      <td>6564.0</td>\n",
       "      <td>12767.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22847</th>\n",
       "      <td>310000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>67940.0</td>\n",
       "      <td>45746.0</td>\n",
       "      <td>46979.0</td>\n",
       "      <td>42045.0</td>\n",
       "      <td>40334.0</td>\n",
       "      <td>49542.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>13000.0</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17496</th>\n",
       "      <td>80000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>52062.0</td>\n",
       "      <td>49884.0</td>\n",
       "      <td>22348.0</td>\n",
       "      <td>22063.0</td>\n",
       "      <td>8723.0</td>\n",
       "      <td>7254.0</td>\n",
       "      <td>2158.0</td>\n",
       "      <td>1708.0</td>\n",
       "      <td>1411.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>220.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2145</th>\n",
       "      <td>170000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>171098.0</td>\n",
       "      <td>166176.0</td>\n",
       "      <td>169759.0</td>\n",
       "      <td>11760.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4902.0</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>5695.0</td>\n",
       "      <td>11760.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4902.0</td>\n",
       "      <td>6000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15305</th>\n",
       "      <td>100000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>83013.0</td>\n",
       "      <td>89589.0</td>\n",
       "      <td>92887.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25509</th>\n",
       "      <td>60000.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>56136.0</td>\n",
       "      <td>57280.0</td>\n",
       "      <td>56969.0</td>\n",
       "      <td>58695.0</td>\n",
       "      <td>57453.0</td>\n",
       "      <td>61153.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>2685.0</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>4800.0</td>\n",
       "      <td>3400.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>18000 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       limit_bal  gender  education  marriage  age  pay_1  pay_2  pay_3  \\\n",
       "10574    20000.0       2          3         1   51      0      0      0   \n",
       "11330   300000.0       1          2         2   31      0      0      0   \n",
       "7127    500000.0       1          2         2   40      0      0      0   \n",
       "25150    70000.0       2          2         1   46      0      0      0   \n",
       "6000     50000.0       1          2         1   33      0      0      0   \n",
       "20951   340000.0       1          1         1   41      0      0      0   \n",
       "13894   300000.0       2          2         1   39      0      0      0   \n",
       "15285   230000.0       2          1         1   39      0      0      0   \n",
       "6881    200000.0       2          1         1   38      0      0      0   \n",
       "22145    60000.0       2          1         2   29      0      0      0   \n",
       "10270    50000.0       1          2         2   46      1      1      0   \n",
       "28740   160000.0       2          1         2   28      0      1      0   \n",
       "19310   360000.0       2          1         2   33      0      0      0   \n",
       "4087    150000.0       1          2         2   34      0      0      0   \n",
       "13760   480000.0       1          4         2   49      0      0      0   \n",
       "27714   130000.0       2          2         2   23      0      0      0   \n",
       "7584     90000.0       2          2         2   28      1      1      1   \n",
       "29650   260000.0       1          2         2   36      1      0      0   \n",
       "9472    440000.0       2          2         2   25      1      1      0   \n",
       "7022    100000.0       1          2         1   44      1      1      1   \n",
       "21323   250000.0       1          1         1   50      1      0      0   \n",
       "23409   200000.0       2          4         1   43      0      0      0   \n",
       "3884     20000.0       1          2         2   24      0      0      1   \n",
       "9456    310000.0       2          1         1   35      0      0      0   \n",
       "15820    30000.0       2          1         2   27      1      0      0   \n",
       "13152   180000.0       2          1         1   39      0      0      0   \n",
       "10707    80000.0       2          2         1   34      1      1      1   \n",
       "1541     50000.0       1          3         2   22      1      0      0   \n",
       "23062   150000.0       2          1         1   32      0      0      0   \n",
       "11118   200000.0       1          2         1   39      0      0      1   \n",
       "...          ...     ...        ...       ...  ...    ...    ...    ...   \n",
       "3063     50000.0       1          3         2   25      1      1      1   \n",
       "21234   300000.0       1          2         2   34      0      0      0   \n",
       "2459    250000.0       2          2         1   41      0      0      0   \n",
       "24360   100000.0       1          2         2   34      1      0      0   \n",
       "9048    140000.0       2          2         2   30      0      0      0   \n",
       "25080    30000.0       1          1         1   52      0      0      0   \n",
       "6886     50000.0       2          3         3   55      0      0      0   \n",
       "18264   180000.0       1          3         2   52      0      0      0   \n",
       "17546    50000.0       2          3         1   42      1      0      0   \n",
       "12326   240000.0       2          1         2   28      0      0      0   \n",
       "2440    160000.0       2          1         2   40      0      0      0   \n",
       "8905     50000.0       1          2         2   25      1      1      1   \n",
       "5739    150000.0       2          3         1   34      0      0      0   \n",
       "360     500000.0       2          2         2   23      1      1      1   \n",
       "4816     50000.0       2          2         2   31      0      0      0   \n",
       "15178    90000.0       2          2         2   25      0      0      0   \n",
       "18321   480000.0       2          1         2   32      0      0      0   \n",
       "23609    50000.0       2          1         2   37      0      0      0   \n",
       "16230    50000.0       2          2         1   33      0      0      0   \n",
       "21927   160000.0       2          1         2   28      0      0      0   \n",
       "27267   160000.0       1          1         1   42      1      1      0   \n",
       "11593   170000.0       2          2         2   27      1      0      0   \n",
       "15732    50000.0       2          2         1   30      1      1      0   \n",
       "25950   410000.0       2          3         1   32      0      0      0   \n",
       "5863    320000.0       1          2         2   44      0      0      0   \n",
       "22847   310000.0       2          1         2   36      0      0      0   \n",
       "17496    80000.0       2          2         1   28      1      0      0   \n",
       "2145    170000.0       1          4         1   28      0      0      0   \n",
       "15305   100000.0       2          2         1   33      0      0      0   \n",
       "25509    60000.0       2          1         2   25      0      0      0   \n",
       "\n",
       "       pay_4  pay_5  pay_6  bill_amt1  bill_amt2  bill_amt3  bill_amt4  \\\n",
       "10574      0      0      0    19255.0    20650.0        0.0        0.0   \n",
       "11330      0      0      0   285038.0   291480.0   298378.0   131103.0   \n",
       "7127       0      0      0    33179.0    97882.0    48017.0    49403.0   \n",
       "25150      0      0      0    70349.0    70068.0    71318.0    59665.0   \n",
       "6000       0      0      0    13538.0     9199.0     6879.0     6753.0   \n",
       "20951      0      0      0    11379.0    11086.0     3112.0     3351.0   \n",
       "13894      0      0      0   145138.0   148586.0   151878.0   123417.0   \n",
       "15285      0      0      0      970.0     1475.0     1636.0     1149.0   \n",
       "6881       0      1      0    26344.0        0.0        0.0      320.0   \n",
       "22145      0      0      0    23304.0    20517.0    20261.0    22832.0   \n",
       "10270      0      0      0    49423.0    48332.0    46043.0    18759.0   \n",
       "28740      0      0      0     4145.0     3930.0     5859.0    10299.0   \n",
       "19310      0      0      0        0.0        0.0        0.0        0.0   \n",
       "4087       0      0      0   135587.0   138473.0   141297.0   144605.0   \n",
       "13760      0      0      0     7174.0        0.0        0.0        0.0   \n",
       "27714      0      0      0    44930.0    40688.0    40880.0    37851.0   \n",
       "7584       1      0      0    83480.0    89264.0    89896.0    86706.0   \n",
       "29650      0      0      0   334214.0   310546.0   288566.0   291597.0   \n",
       "9472       0      0      0   225416.0   220405.0   226731.0   231456.0   \n",
       "7022       1      1      1    59586.0    60842.0    61749.0    65130.0   \n",
       "21323      0      0      0        0.0        0.0      191.0     2160.0   \n",
       "23409      0      0      0   198595.0   194734.0   190447.0   187224.0   \n",
       "3884       0      0      0      390.0      780.0      390.0      780.0   \n",
       "9456       0      0      0     2948.0    62593.0   152141.0   125325.0   \n",
       "15820      0      0      0    23575.0    24584.0    25885.0    26851.0   \n",
       "13152      0      0      0    43507.0    44282.0    23946.0        0.0   \n",
       "10707      0      0      0    17294.0    22256.0    21660.0    22055.0   \n",
       "1541       0      0      0        0.0        0.0        0.0        0.0   \n",
       "23062      0      0      0    23364.0    22846.0    25779.0    22303.0   \n",
       "11118      0      0      0      162.0     1879.0     1707.0      617.0   \n",
       "...      ...    ...    ...        ...        ...        ...        ...   \n",
       "3063       0      0      1    43685.0    44694.0    43673.0    44659.0   \n",
       "21234      0      0      0    50281.0    51441.0    54107.0    55111.0   \n",
       "2459       0      0      0    37137.0    51803.0     4997.0        0.0   \n",
       "24360      0      0      0     -342.0     -342.0     -342.0     -342.0   \n",
       "9048       0      0      0      854.0        0.0        0.0        0.0   \n",
       "25080      0      0      0    14476.0    15297.0     8531.0     8020.0   \n",
       "6886       0      0      1      390.0      390.0      911.0     1080.0   \n",
       "18264      0      0      0    77408.0    74156.0    75659.0    72046.0   \n",
       "17546      0      1      1    31422.0    32420.0    33462.0    36199.0   \n",
       "12326      0      0      0   133230.0   136579.0   139281.0   132487.0   \n",
       "2440       0      0      0    67925.0    69771.0    73233.0    72328.0   \n",
       "8905       1      1      1    42883.0    47887.0    46836.0    45785.0   \n",
       "5739       0      0      0   112391.0    98583.0    95810.0    87216.0   \n",
       "360        0      0      0   507726.0   509229.0   499936.0   505507.0   \n",
       "4816       0      0      0    40019.0    20394.0    19597.0    17830.0   \n",
       "15178      0      0      0    28276.0    29305.0    30336.0    28800.0   \n",
       "18321      0      0      0     3655.0    38022.0     5785.0     9233.0   \n",
       "23609      0      0      0    43886.0    50411.0     9688.0      857.0   \n",
       "16230      0      0      0    49450.0    50067.0    69594.0    46181.0   \n",
       "21927      0      0      0     3849.0     6393.0     6969.0     9044.0   \n",
       "27267      0      1      1    14137.0    13613.0    14634.0    16532.0   \n",
       "11593      0      0      0    -1020.0    25607.0    23922.0    24399.0   \n",
       "15732      0      0      0    27369.0     6425.0     4580.0        0.0   \n",
       "25950      0      0      0      388.0      388.0      388.0      388.0   \n",
       "5863       0      0      0    36831.0    28785.0    13104.0     2621.0   \n",
       "22847      0      0      0    67940.0    45746.0    46979.0    42045.0   \n",
       "17496      0      0      0    52062.0    49884.0    22348.0    22063.0   \n",
       "2145       0      0      0   171098.0   166176.0   169759.0    11760.0   \n",
       "15305      0      0      0    83013.0    89589.0    92887.0        0.0   \n",
       "25509      0      0      0    56136.0    57280.0    56969.0    58695.0   \n",
       "\n",
       "       bill_amt5  bill_amt6  pay_amt1  pay_amt2  pay_amt3  pay_amt4  pay_amt5  \\\n",
       "10574        0.0        0.0    2000.0       0.0       0.0       0.0       0.0   \n",
       "11330   128742.0   124882.0   12507.0   15056.0    5027.0    5007.0    5063.0   \n",
       "7127     40018.0    54109.0   99159.0   48276.0   49450.0     201.0   54379.0   \n",
       "25150    29510.0    26738.0    2600.0    3700.0    2600.0    1200.0    1200.0   \n",
       "6000      5440.0     5440.0    2000.0    1284.0     264.0     109.0     390.0   \n",
       "20951     1259.0    19271.0   11258.0    3112.0    3351.0    1259.0   19271.0   \n",
       "13894   104922.0   107005.0    5785.0    6163.0    4728.0    3811.0    3830.0   \n",
       "15285      334.0      459.0    1483.0    1640.0    1152.0     335.0     460.0   \n",
       "6881       170.0     2865.0       0.0       0.0     329.0       0.0    2865.0   \n",
       "22145    24234.0    24928.0    3000.0    3000.0    3000.0    2000.0    2000.0   \n",
       "10270    19161.0    19516.0      10.0    2000.0     655.0     688.0     666.0   \n",
       "28740    11825.0    13121.0       0.0    2000.0    5008.0    3007.0    2000.0   \n",
       "19310        0.0        0.0       0.0       0.0       0.0       0.0       0.0   \n",
       "4087    148420.0   150500.0    4000.0    4000.0    4500.0    5000.0    3500.0   \n",
       "13760        0.0        0.0       0.0       0.0       0.0       0.0       0.0   \n",
       "27714    37816.0    39849.0    6000.0    6000.0    4000.0   10000.0    5000.0   \n",
       "7584     87385.0    88351.0    8000.0    3000.0       0.0    3000.0    5000.0   \n",
       "29650   214353.0   216238.0   12000.0   11000.0   11000.0    8000.0    8500.0   \n",
       "9472    237707.0   243555.0       0.0   10000.0   10000.0   10000.0   10000.0   \n",
       "7022     63816.0    65011.0    2800.0    2500.0    5000.0       0.0    2400.0   \n",
       "21323        0.0      900.0       0.0     191.0    2160.0       0.0     900.0   \n",
       "23409   183641.0   178239.0    7003.0    7000.0    7000.0    6510.0    6550.0   \n",
       "3884       780.0        0.0     780.0       0.0     780.0       0.0       0.0   \n",
       "9456    109142.0   103020.0   62635.0  100042.0    4017.0    9184.0    4081.0   \n",
       "15820    26808.0    27599.0    1400.0    1700.0    1900.0     557.0    1000.0   \n",
       "13152        0.0        0.0    2000.0    2000.0       0.0       0.0       0.0   \n",
       "10707    22603.0    23133.0    5210.0       0.0    1000.0    1000.0    1000.0   \n",
       "1541         0.0        0.0       0.0       0.0       0.0       0.0       0.0   \n",
       "23062    16981.0    16837.0    1450.0    5779.0    1300.0     700.0    1000.0   \n",
       "11118      620.0        0.0    1879.0       8.0     617.0       3.0       0.0   \n",
       "...          ...        ...       ...       ...       ...       ...       ...   \n",
       "3063     49512.0    48454.0    2000.0       0.0    2000.0    6000.0     300.0   \n",
       "21234    56688.0    58272.0    2000.0    3500.0    3000.0    2500.0    2500.0   \n",
       "2459         0.0        0.0   51811.0    9544.0       0.0       0.0       0.0   \n",
       "24360     1158.0     2485.0       0.0       0.0       0.0    1500.0    1500.0   \n",
       "9048         0.0        0.0       0.0       0.0       0.0       0.0       0.0   \n",
       "25080     -620.0     -620.0    1129.0    1152.0    1081.0       0.0       0.0   \n",
       "6886       930.0      390.0     390.0     911.0     559.0     390.0       0.0   \n",
       "18264    70355.0    71208.0    3293.0    3350.0    3111.0    2615.0    2689.0   \n",
       "17546    36803.0    37566.0    1521.0    1563.0    3300.0    1500.0    1500.0   \n",
       "12326   107437.0   109237.0    5500.0    5036.0    4000.0    6200.0    5500.0   \n",
       "2440     74234.0    75500.0    3509.0    5824.0       0.0    2204.0    2000.0   \n",
       "8905     47003.0    48085.0    6000.0       0.0       0.0    2100.0    2000.0   \n",
       "5739     88201.0    87483.0    5000.0    4000.0    4000.0    3290.0    3200.0   \n",
       "360     484612.0   419643.0   10000.0       0.0   14000.0   16500.0   16000.0   \n",
       "4816     18397.0    19064.0    2006.0    1519.0    1000.0    1000.0    1000.0   \n",
       "15178     9852.0     2160.0    1505.0    1536.0    1000.0    9852.0    2160.0   \n",
       "18321     4893.0     2168.0   38024.0    5785.0    9233.0    4893.0    2168.0   \n",
       "23609     3810.0     4741.0    2500.0    7026.0    1000.0    3000.0    1000.0   \n",
       "16230    16770.0    19890.0    1743.0    2236.0    2807.0     629.0    3810.0   \n",
       "21927     2325.0     3998.0    6458.0    6980.0    9044.0    2325.0    4997.0   \n",
       "27267    15969.0    17701.0       0.0    1247.0    2145.0       0.0    2000.0   \n",
       "11593    24919.0    26884.0   27107.0    1500.0     863.0     903.0    3000.0   \n",
       "15732        0.0        0.0       0.0    1050.0       0.0       0.0       0.0   \n",
       "25950   110345.0   116128.0     388.0     388.0     388.0  110345.0    8000.0   \n",
       "5863      9075.0     6564.0   28791.0   13227.0    2621.0    9075.0    6564.0   \n",
       "22847    40334.0    49542.0    2000.0    2000.0    2000.0    2000.0   13000.0   \n",
       "17496     8723.0     7254.0    2158.0    1708.0    1411.0       4.0      21.0   \n",
       "2145         0.0     4902.0   14000.0    5695.0   11760.0       0.0    4902.0   \n",
       "15305        0.0        0.0    8000.0    5000.0       0.0       0.0       0.0   \n",
       "25509    57453.0    61153.0    2100.0    2100.0    2685.0    2300.0    4800.0   \n",
       "\n",
       "       pay_amt6  \n",
       "10574       0.0  \n",
       "11330    3039.0  \n",
       "7127     9905.0  \n",
       "25150    1000.0  \n",
       "6000        0.0  \n",
       "20951   16992.0  \n",
       "13894    3970.0  \n",
       "15285     553.0  \n",
       "6881      801.0  \n",
       "22145    2000.0  \n",
       "10270     649.0  \n",
       "28740    2000.0  \n",
       "19310       0.0  \n",
       "4087        0.0  \n",
       "13760       0.0  \n",
       "27714    8002.0  \n",
       "7584     5000.0  \n",
       "29650    8300.0  \n",
       "9472    10000.0  \n",
       "7022     2400.0  \n",
       "21323    2000.0  \n",
       "23409    6001.0  \n",
       "3884        0.0  \n",
       "9456     6042.0  \n",
       "15820    2000.0  \n",
       "13152       0.0  \n",
       "10707    1000.0  \n",
       "1541        0.0  \n",
       "23062    1000.0  \n",
       "11118       0.0  \n",
       "...         ...  \n",
       "3063     3000.0  \n",
       "21234    2500.0  \n",
       "2459        0.0  \n",
       "24360       0.0  \n",
       "9048        0.0  \n",
       "25080       0.0  \n",
       "6886      390.0  \n",
       "18264    2766.0  \n",
       "17546    1600.0  \n",
       "12326    5000.0  \n",
       "2440        0.0  \n",
       "8905        0.0  \n",
       "5739     3215.0  \n",
       "360     19000.0  \n",
       "4816     1000.0  \n",
       "15178     400.0  \n",
       "18321    3625.0  \n",
       "23609    1000.0  \n",
       "16230     719.0  \n",
       "21927    9947.0  \n",
       "27267       0.0  \n",
       "11593    6186.0  \n",
       "15732       0.0  \n",
       "25950    4200.0  \n",
       "5863    12767.0  \n",
       "22847    2000.0  \n",
       "17496     220.0  \n",
       "2145     6000.0  \n",
       "15305       0.0  \n",
       "25509    3400.0  \n",
       "\n",
       "[18000 rows x 23 columns]"
      ]
     },
     "execution_count": 230,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_pipeline(None).fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction Classifiers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define the different classifiers that we will be using"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import AdaBoostClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_models = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_models['log'] = log_clf = LogisticRegression(C=0.1, \n",
    "                             solver='liblinear',\n",
    "                             penalty='l2',\n",
    "                             class_weight='balanced', \n",
    "                             random_state=42, \n",
    "                             n_jobs=-1)\n",
    "dico_models['svm'] =  SVC(gamma='auto', C=1, class_weight='balanced')\n",
    "dico_models['tree'] = DecisionTreeClassifier(criterion='entropy', \n",
    "                                  random_state=42, \n",
    "                                 max_leaf_nodes=5,\n",
    "                                 class_weight='balanced')\n",
    "dico_models['forest'] = RandomForestClassifier(n_estimators=500, \n",
    "                                 max_leaf_nodes=10, \n",
    "                                 n_jobs=-1, \n",
    "                                 class_weight='balanced',\n",
    "                                random_state=42)\n",
    "\n",
    "dico_models['adaboost'] = AdaBoostClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_models['xgboost'] = xgb.XGBClassifier(scale_pos_weight=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction Pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prediction Pipelines consists of the following architecture:  \n",
    "  [Complex Pipeline, Classifier]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "\n",
    "dico_pipelined_models_num = defaultdict(dict)\n",
    "dico_pipelined_models_cat = defaultdict(dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "for i in range(len(complex_pipelines_num)):\n",
    "    print(i)\n",
    "    for (name, model) in dico_models.items():\n",
    "        dico_pipelined_models_num[i][name] = Pipeline([\n",
    "            ('preproc', complex_pipelines_num[i]),\n",
    "            ('classifier', model)\n",
    "        ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'log': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('scaler',\n",
       "                                   StandardScaler(copy=True, with_mean=True,\n",
       "                                                  with_std=True))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                     fit_intercept=True, intercept_scaling=1,\n",
       "                                     l1_ratio=None, max_iter=100,\n",
       "                                     multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                     random_state=42, solver='liblinear',\n",
       "                                     tol=0.0001, verbose=0, warm_start=False))],\n",
       "          verbose=False), 'svm': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('scaler',\n",
       "                                   StandardScaler(copy=True, with_mean=True,\n",
       "                                                  with_std=True))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  SVC(C=1, cache_size=200, class_weight='balanced', coef0=0.0,\n",
       "                      decision_function_shape='ovr', degree=3, gamma='auto',\n",
       "                      kernel='rbf', max_iter=-1, probability=False,\n",
       "                      random_state=None, shrinking=True, tol=0.001,\n",
       "                      verbose=False))],\n",
       "          verbose=False), 'tree': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('scaler',\n",
       "                                   StandardScaler(copy=True, with_mean=True,\n",
       "                                                  with_std=True))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  DecisionTreeClassifier(class_weight='balanced',\n",
       "                                         criterion='entropy', max_depth=None,\n",
       "                                         max_features=None, max_leaf_nodes=5,\n",
       "                                         min_impurity_decrease=0.0,\n",
       "                                         min_impurity_split=None,\n",
       "                                         min_samples_leaf=1, min_samples_split=2,\n",
       "                                         min_weight_fraction_leaf=0.0,\n",
       "                                         presort=False, random_state=42,\n",
       "                                         splitter='best'))],\n",
       "          verbose=False), 'forest': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('scaler',\n",
       "                                   StandardScaler(copy=True, with_mean=True,\n",
       "                                                  with_std=True))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                         criterion='gini', max_depth=None,\n",
       "                                         max_features='auto', max_leaf_nodes=10,\n",
       "                                         min_impurity_decrease=0.0,\n",
       "                                         min_impurity_split=None,\n",
       "                                         min_samples_leaf=1, min_samples_split=2,\n",
       "                                         min_weight_fraction_leaf=0.0,\n",
       "                                         n_estimators=500, n_jobs=-1,\n",
       "                                         oob_score=False, random_state=42,\n",
       "                                         verbose=0, warm_start=False))],\n",
       "          verbose=False), 'adaboost': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('scaler',\n",
       "                                   StandardScaler(copy=True, with_mean=True,\n",
       "                                                  with_std=True))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  AdaBoostClassifier(algorithm='SAMME.R', base_estimator=None,\n",
       "                                     learning_rate=1.0, n_estimators=50,\n",
       "                                     random_state=None))],\n",
       "          verbose=False)}"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dico_pipelined_models_num[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'log': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('age_bin', AgeBinAdder()),\n",
       "                                  ('gender_age_bin', GenderXAgeBinAdder()),\n",
       "                                  ('gender_marriage', GenderXMarriageAdder()),\n",
       "                                  ('next_bill', NextBillAdder()),\n",
       "                                  ('age_remove', ColumnDropper(column='age')),\n",
       "                                  ('1_hot',\n",
       "                                   CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                       'education',\n",
       "                                                                       'marriage']))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                     fit_intercept=True, intercept_scaling=1,\n",
       "                                     l1_ratio=None, max_iter=100,\n",
       "                                     multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                     random_state=42, solver='liblinear',\n",
       "                                     tol=0.0001, verbose=0, warm_start=False))],\n",
       "          verbose=False), 'svm': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('age_bin', AgeBinAdder()),\n",
       "                                  ('gender_age_bin', GenderXAgeBinAdder()),\n",
       "                                  ('gender_marriage', GenderXMarriageAdder()),\n",
       "                                  ('next_bill', NextBillAdder()),\n",
       "                                  ('age_remove', ColumnDropper(column='age')),\n",
       "                                  ('1_hot',\n",
       "                                   CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                       'education',\n",
       "                                                                       'marriage']))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  SVC(C=1, cache_size=200, class_weight='balanced', coef0=0.0,\n",
       "                      decision_function_shape='ovr', degree=3, gamma='auto',\n",
       "                      kernel='rbf', max_iter=-1, probability=False,\n",
       "                      random_state=None, shrinking=True, tol=0.001,\n",
       "                      verbose=False))],\n",
       "          verbose=False), 'tree': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('age_bin', AgeBinAdder()),\n",
       "                                  ('gender_age_bin', GenderXAgeBinAdder()),\n",
       "                                  ('gender_marriage', GenderXMarriageAdder()),\n",
       "                                  ('next_bill', NextBillAdder()),\n",
       "                                  ('age_remove', ColumnDropper(column='age')),\n",
       "                                  ('1_hot',\n",
       "                                   CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                       'education',\n",
       "                                                                       'marriage']))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  DecisionTreeClassifier(class_weight='balanced',\n",
       "                                         criterion='entropy', max_depth=None,\n",
       "                                         max_features=None, max_leaf_nodes=5,\n",
       "                                         min_impurity_decrease=0.0,\n",
       "                                         min_impurity_split=None,\n",
       "                                         min_samples_leaf=1, min_samples_split=2,\n",
       "                                         min_weight_fraction_leaf=0.0,\n",
       "                                         presort=False, random_state=42,\n",
       "                                         splitter='best'))],\n",
       "          verbose=False), 'forest': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('age_bin', AgeBinAdder()),\n",
       "                                  ('gender_age_bin', GenderXAgeBinAdder()),\n",
       "                                  ('gender_marriage', GenderXMarriageAdder()),\n",
       "                                  ('next_bill', NextBillAdder()),\n",
       "                                  ('age_remove', ColumnDropper(column='age')),\n",
       "                                  ('1_hot',\n",
       "                                   CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                       'education',\n",
       "                                                                       'marriage']))],\n",
       "                           verbose=False)),\n",
       "                 ('cl...\n",
       "                  RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                         criterion='gini', max_depth=None,\n",
       "                                         max_features='auto', max_leaf_nodes=10,\n",
       "                                         min_impurity_decrease=0.0,\n",
       "                                         min_impurity_split=None,\n",
       "                                         min_samples_leaf=1, min_samples_split=2,\n",
       "                                         min_weight_fraction_leaf=0.0,\n",
       "                                         n_estimators=500, n_jobs=-1,\n",
       "                                         oob_score=False, random_state=42,\n",
       "                                         verbose=0, warm_start=False))],\n",
       "          verbose=False), 'adaboost': Pipeline(memory=None,\n",
       "          steps=[('preproc',\n",
       "                  Pipeline(memory=None,\n",
       "                           steps=[('age_bin', AgeBinAdder()),\n",
       "                                  ('gender_age_bin', GenderXAgeBinAdder()),\n",
       "                                  ('gender_marriage', GenderXMarriageAdder()),\n",
       "                                  ('next_bill', NextBillAdder()),\n",
       "                                  ('age_remove', ColumnDropper(column='age')),\n",
       "                                  ('1_hot',\n",
       "                                   CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                       'education',\n",
       "                                                                       'marriage']))],\n",
       "                           verbose=False)),\n",
       "                 ('classifier',\n",
       "                  AdaBoostClassifier(algorithm='SAMME.R', base_estimator=None,\n",
       "                                     learning_rate=1.0, n_estimators=50,\n",
       "                                     random_state=None))],\n",
       "          verbose=False)}"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dico_pipelined_models_num[4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DEPRECATE: we explore all combinations of preproc x SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_classification_reports = defaultdict(dict)\n",
    "dico_confusion_matrices = defaultdict(dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 0, Log\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.48      0.57      0.52      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.78      0.77      0.78      6000\n",
      "\n",
      "Pipe 0, Svm\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.80      0.84      4673\n",
      "           1       0.46      0.59      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 0, Tree\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 0, Forest\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.60      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 0, Adaboost\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.94      0.88      4673\n",
      "           1       0.62      0.33      0.43      1327\n",
      "\n",
      "    accuracy                           0.81      6000\n",
      "   macro avg       0.73      0.64      0.66      6000\n",
      "weighted avg       0.78      0.81      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 1, Log\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.48      0.57      0.52      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.78      0.77      0.78      6000\n",
      "\n",
      "Pipe 1, Svm\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.80      0.84      4673\n",
      "           1       0.46      0.59      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 1, Tree\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 1, Forest\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.61      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "Pipe 1, Adaboost\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.95      0.88      4673\n",
      "           1       0.62      0.31      0.42      1327\n",
      "\n",
      "    accuracy                           0.81      6000\n",
      "   macro avg       0.73      0.63      0.65      6000\n",
      "weighted avg       0.78      0.81      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 2, Log\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.51      0.64      4673\n",
      "           1       0.29      0.71      0.41      1327\n",
      "\n",
      "    accuracy                           0.56      6000\n",
      "   macro avg       0.58      0.61      0.53      6000\n",
      "weighted avg       0.74      0.56      0.59      6000\n",
      "\n",
      "Pipe 2, Svm\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.78      0.99      0.87      4673\n",
      "           1       0.39      0.03      0.06      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.59      0.51      0.47      6000\n",
      "weighted avg       0.70      0.77      0.69      6000\n",
      "\n",
      "Pipe 2, Tree\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 2, Forest\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.79      0.83      4673\n",
      "           1       0.46      0.62      0.53      1327\n",
      "\n",
      "    accuracy                           0.75      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.75      0.77      6000\n",
      "\n",
      "Pipe 2, Adaboost\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.95      0.88      4673\n",
      "           1       0.62      0.30      0.41      1327\n",
      "\n",
      "    accuracy                           0.81      6000\n",
      "   macro avg       0.73      0.63      0.65      6000\n",
      "weighted avg       0.78      0.81      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 3, Log\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.81      0.78      0.80      4673\n",
      "           1       0.33      0.37      0.34      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.57      0.58      0.57      6000\n",
      "weighted avg       0.71      0.69      0.70      6000\n",
      "\n",
      "Pipe 3, Svm\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.78      0.98      0.87      4673\n",
      "           1       0.40      0.05      0.09      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.59      0.51      0.48      6000\n",
      "weighted avg       0.70      0.77      0.70      6000\n",
      "\n",
      "Pipe 3, Tree\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 3, Forest\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.62      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "Pipe 3, Adaboost\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.95      0.88      4673\n",
      "           1       0.62      0.30      0.41      1327\n",
      "\n",
      "    accuracy                           0.81      6000\n",
      "   macro avg       0.73      0.63      0.65      6000\n",
      "weighted avg       0.78      0.81      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 4, Log\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.81      0.78      0.80      4673\n",
      "           1       0.33      0.37      0.34      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.57      0.58      0.57      6000\n",
      "weighted avg       0.71      0.69      0.70      6000\n",
      "\n",
      "Pipe 4, Svm\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.78      0.98      0.87      4673\n",
      "           1       0.40      0.05      0.09      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.59      0.51      0.48      6000\n",
      "weighted avg       0.70      0.77      0.70      6000\n",
      "\n",
      "Pipe 4, Tree\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 4, Forest\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.62      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "Pipe 4, Adaboost\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.95      0.88      4673\n",
      "           1       0.62      0.30      0.41      1327\n",
      "\n",
      "    accuracy                           0.81      6000\n",
      "   macro avg       0.73      0.63      0.65      6000\n",
      "weighted avg       0.78      0.81      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 5, Log\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.49      0.57      0.53      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.79      0.77      0.78      6000\n",
      "\n",
      "Pipe 5, Svm\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.81      0.84      4673\n",
      "           1       0.46      0.58      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 5, Tree\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 5, Forest\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.61      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "Pipe 5, Adaboost\n",
      "--------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.95      0.88      4673\n",
      "           1       0.62      0.31      0.41      1327\n",
      "\n",
      "    accuracy                           0.81      6000\n",
      "   macro avg       0.73      0.63      0.65      6000\n",
      "weighted avg       0.78      0.81      0.78      6000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(len(complex_pipelines_num)):\n",
    "    for (name, model) in dico_models.items():\n",
    "        pipe = dico_pipelined_models[i][name]\n",
    "        pipe.fit(X_train, np.ravel(y_train))\n",
    "        print('Pipe {}, {}'.format(i, name.capitalize()))\n",
    "        print('--------')\n",
    "        \n",
    "        y_pred = pipe.predict(X_test)\n",
    "        dico_classification_reports[i][name] = classification_report(y_test, y_pred)\n",
    "        dico_confusion_matrices[i][name] = confusion_matrix(y_test, y_pred)\n",
    "        print(dico_classification_reports[i][name])\n",
    "        #print(dico_confusion_matrices[i][name])\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "for i in range(len(complex_pipelines)):\n",
    "    for (name, model) in dico_models.items():\n",
    "        pipe = dico_pipelined_models[i][name]\n",
    "        pipe.fit(X_train, np.ravel(y_train))\n",
    "        print(i, name.capitalize())\n",
    "        print('--------')\n",
    "        \n",
    "        y_pred = pipe.predict(X_test)\n",
    "        dico_classification_reports[i][name] = classification_report(y_test, y_pred)\n",
    "        dico_confusion_matrices[i][name] = confusion_matrix(y_test, y_pred)\n",
    "        print(dico_classification_reports[i][name])\n",
    "        #print(dico_confusion_matrices[i][name])\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What can we conclude from all these pipelines?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* It appears that the next-bill amount alone doesn't do much: the results are exactly the same with or without it.  \n",
    "This may be because, even though it makes sense from a business point of view, its predicted value is very close to bill_amt1  \n",
    "(as our best $\\alpha$ coefficient from our exponential smoothing modelisation was ~1 )\n",
    "* Our adding our cross-categorical features (the 3rd pipeline) is very useful for better results with Logistic Regresseion, but doesn't seem to impact our other models much\n",
    "* SVM overfits a lot if our features are now normalized  \n",
    "Plus, it takes a lot of time to train, so we shouldn't use it with non-normalizing pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining which preprocessing pipelines we should use, for each model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 309,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_appropriate_preproc = defaultdict(list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_appropriate_preproc['log'] = []\n",
    "\n",
    "dico_appropriate_preproc['log'].append( make_pipeline(StandardScaler()) )\n",
    "dico_appropriate_preproc['log'].append( make_pipeline(NextBillAdder(), \n",
    "                                                      StandardScaler()) )\n",
    "dico_appropriate_preproc['log'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) )\n",
    "dico_appropriate_preproc['log'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']),\n",
    "                                                      StandardScaler()) )\n",
    "dico_appropriate_preproc['log'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) ) \n",
    "dico_appropriate_preproc['log'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']), \n",
    "                                                      StandardScaler()) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 321,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_appropriate_preproc['svm'] = []\n",
    "\n",
    "dico_appropriate_preproc['svm'].append( make_pipeline(StandardScaler()) )\n",
    "dico_appropriate_preproc['svm'].append( make_pipeline(NextBillAdder(), \n",
    "                                                      StandardScaler()) )\n",
    "dico_appropriate_preproc['svm'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']), \n",
    "                                                      StandardScaler()) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 322,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_appropriate_preproc['tree'] = []\n",
    "\n",
    "dico_appropriate_preproc['tree'].append( make_pipeline(StandardScaler()) )\n",
    "dico_appropriate_preproc['tree'].append( make_pipeline(NextBillAdder(), \n",
    "                                                      StandardScaler()) )\n",
    "dico_appropriate_preproc['tree'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) )\n",
    "dico_appropriate_preproc['tree'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) ) \n",
    "dico_appropriate_preproc['tree'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']), \n",
    "                                                      StandardScaler()) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 323,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_appropriate_preproc['forest'] = []\n",
    "\n",
    "dico_appropriate_preproc['forest'].append( make_pipeline(StandardScaler()) )\n",
    "dico_appropriate_preproc['forest'].append( make_pipeline(NextBillAdder(), \n",
    "                                                      StandardScaler()) )\n",
    "dico_appropriate_preproc['forest'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) )\n",
    "dico_appropriate_preproc['forest'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) ) \n",
    "dico_appropriate_preproc['forest'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']), \n",
    "                                                      StandardScaler()) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adaboost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It overfits the majority class 'default' too much, let's stop experimenting on it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### XGBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 324,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_appropriate_preproc['xgboost'] = []\n",
    "\n",
    "dico_appropriate_preproc['xgboost'].append( make_pipeline(StandardScaler()) )\n",
    "dico_appropriate_preproc['xgboost'].append( make_pipeline(NextBillAdder(), \n",
    "                                                      StandardScaler()) )\n",
    "dico_appropriate_preproc['xgboost'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) )\n",
    "dico_appropriate_preproc['xgboost'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']) ) ) \n",
    "dico_appropriate_preproc['xgboost'].append( make_pipeline(AgeBinAdder(), \n",
    "                                                      GenderXAgeBinAdder(), \n",
    "                                                      GenderXMarriageAdder(), \n",
    "                                                      NextBillAdder(), \n",
    "                                                      ColumnDropper('age'), \n",
    "                                                      CategoricalWarrior(['gender', 'education', 'marriage']), \n",
    "                                                      StandardScaler()) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CatBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining all the Prediction Pipelines to train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 316,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_pipelined_models_num = defaultdict(dict)\n",
    "\n",
    "for (name, model) in dico_models.items():\n",
    "#     print(name)\n",
    "    for (i, complex_pipe) in enumerate(dico_appropriate_preproc[name]):\n",
    "#         print(i)\n",
    "        dico_pipelined_models_num[name][i] = Pipeline([\n",
    "            ('preproc', complex_pipe),\n",
    "            ('classifier', model)\n",
    "        ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 317,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defaultdict(dict, {'log': {0: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                                  fit_intercept=True, intercept_scaling=1,\n",
       "                                                  l1_ratio=None, max_iter=100,\n",
       "                                                  multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                                  random_state=42, solver='liblinear',\n",
       "                                                  tol=0.0001, verbose=0, warm_start=False))],\n",
       "                       verbose=False), 1: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('nextbilladder', NextBillAdder()),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                                  fit_intercept=True, intercept_scaling=1,\n",
       "                                                  l1_ratio=None, max_iter=100,\n",
       "                                                  multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                                  random_state=42, solver='liblinear',\n",
       "                                                  tol=0.0001, verbose=0, warm_start=False))],\n",
       "                       verbose=False), 2: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage']))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                                  fit_intercept=True, intercept_scaling=1,\n",
       "                                                  l1_ratio=None, max_iter=100,\n",
       "                                                  multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                                  random_state=42, solver='liblinear',\n",
       "                                                  tol=0.0001, verbose=0, warm_start=False))],\n",
       "                       verbose=False), 3: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage'])),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                                  fit_intercept=True, intercept_scaling=1,\n",
       "                                                  l1_ratio=None, max_iter=100,\n",
       "                                                  multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                                  random_state=42, solver='liblinear',\n",
       "                                                  tol=0.0001, verbose=0, warm_start=False))],\n",
       "                       verbose=False), 4: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage']))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                                  fit_intercept=True, intercept_scaling=1,\n",
       "                                                  l1_ratio=None, max_iter=100,\n",
       "                                                  multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                                  random_state=42, solver='liblinear',\n",
       "                                                  tol=0.0001, verbose=0, warm_start=False))],\n",
       "                       verbose=False), 5: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'm...\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                                  fit_intercept=True, intercept_scaling=1,\n",
       "                                                  l1_ratio=None, max_iter=100,\n",
       "                                                  multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                                  random_state=42, solver='liblinear',\n",
       "                                                  tol=0.0001, verbose=0, warm_start=False))],\n",
       "                       verbose=False)}, 'svm': {0: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               SVC(C=1, cache_size=200, class_weight='balanced', coef0=0.0,\n",
       "                                   decision_function_shape='ovr', degree=3, gamma='auto',\n",
       "                                   kernel='rbf', max_iter=-1, probability=False,\n",
       "                                   random_state=None, shrinking=True, tol=0.001,\n",
       "                                   verbose=False))],\n",
       "                       verbose=False), 1: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('nextbilladder', NextBillAdder()),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               SVC(C=1, cache_size=200, class_weight='balanced', coef0=0.0,\n",
       "                                   decision_function_shape='ovr', degree=3, gamma='auto',\n",
       "                                   kernel='rbf', max_iter=-1, probability=False,\n",
       "                                   random_state=None, shrinking=True, tol=0.001,\n",
       "                                   verbose=False))],\n",
       "                       verbose=False), 2: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage'])),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               SVC(C=1, cache_size=200, class_weight='balanced', coef0=0.0,\n",
       "                                   decision_function_shape='ovr', degree=3, gamma='auto',\n",
       "                                   kernel='rbf', max_iter=-1, probability=False,\n",
       "                                   random_state=None, shrinking=True, tol=0.001,\n",
       "                                   verbose=False))],\n",
       "                       verbose=False)}, 'tree': {0: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               DecisionTreeClassifier(class_weight='balanced',\n",
       "                                                      criterion='entropy', max_depth=None,\n",
       "                                                      max_features=None, max_leaf_nodes=5,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      presort=False, random_state=42,\n",
       "                                                      splitter='best'))],\n",
       "                       verbose=False), 1: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('nextbilladder', NextBillAdder()),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               DecisionTreeClassifier(class_weight='balanced',\n",
       "                                                      criterion='entropy', max_depth=None,\n",
       "                                                      max_features=None, max_leaf_nodes=5,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      presort=False, random_state=42,\n",
       "                                                      splitter='best'))],\n",
       "                       verbose=False), 2: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage']))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               DecisionTreeClassifier(class_weight='balanced',\n",
       "                                                      criterion='entropy', max_depth=None,\n",
       "                                                      max_features=None, max_leaf_nodes=5,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      presort=False, random_state=42,\n",
       "                                                      splitter='best'))],\n",
       "                       verbose=False), 3: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage']))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               DecisionTreeClassifier(class_weight='balanced',\n",
       "                                                      criterion='entropy', max_depth=None,\n",
       "                                                      max_features=None, max_leaf_nodes=5,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      presort=False, random_state=42,\n",
       "                                                      splitter='best'))],\n",
       "                       verbose=False), 4: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'm...\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               DecisionTreeClassifier(class_weight='balanced',\n",
       "                                                      criterion='entropy', max_depth=None,\n",
       "                                                      max_features=None, max_leaf_nodes=5,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      presort=False, random_state=42,\n",
       "                                                      splitter='best'))],\n",
       "                       verbose=False)}, 'forest': {0: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                                      criterion='gini', max_depth=None,\n",
       "                                                      max_features='auto', max_leaf_nodes=10,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      n_estimators=500, n_jobs=-1,\n",
       "                                                      oob_score=False, random_state=42,\n",
       "                                                      verbose=0, warm_start=False))],\n",
       "                       verbose=False), 1: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('nextbilladder', NextBillAdder()),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                                      criterion='gini', max_depth=None,\n",
       "                                                      max_features='auto', max_leaf_nodes=10,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      n_estimators=500, n_jobs=-1,\n",
       "                                                      oob_score=False, random_state=42,\n",
       "                                                      verbose=0, warm_start=False))],\n",
       "                       verbose=False), 2: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage']))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                                      criterion='gini', max_depth=None,\n",
       "                                                      max_features='auto', max_leaf_nodes=10,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      n_estimators=500, n_jobs=-1,\n",
       "                                                      oob_score=False, random_state=42,\n",
       "                                                      verbose=0, warm_start=False))],\n",
       "                       verbose=False), 3: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'm...\n",
       "                               RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                                      criterion='gini', max_depth=None,\n",
       "                                                      max_features='auto', max_leaf_nodes=10,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      n_estimators=500, n_jobs=-1,\n",
       "                                                      oob_score=False, random_state=42,\n",
       "                                                      verbose=0, warm_start=False))],\n",
       "                       verbose=False), 4: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'm...\n",
       "                               RandomForestClassifier(bootstrap=True, class_weight='balanced',\n",
       "                                                      criterion='gini', max_depth=None,\n",
       "                                                      max_features='auto', max_leaf_nodes=10,\n",
       "                                                      min_impurity_decrease=0.0,\n",
       "                                                      min_impurity_split=None,\n",
       "                                                      min_samples_leaf=1, min_samples_split=2,\n",
       "                                                      min_weight_fraction_leaf=0.0,\n",
       "                                                      n_estimators=500, n_jobs=-1,\n",
       "                                                      oob_score=False, random_state=42,\n",
       "                                                      verbose=0, warm_start=False))],\n",
       "                       verbose=False)}, 'xgboost': {0: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               XGBClassifier(base_score=0.5, booster='gbtree',\n",
       "                                             colsample_bylevel=1, colsample_bynode=1,\n",
       "                                             colsample_bytree=1, gamma=0, learning_rate=0.1,\n",
       "                                             max_delta_step=0, max_depth=3,\n",
       "                                             min_child_weight=1, missing=None,\n",
       "                                             n_estimators=100, n_jobs=1, nthread=None,\n",
       "                                             objective='binary:logistic', random_state=0,\n",
       "                                             reg_alpha=0, reg_lambda=1, scale_pos_weight=5,\n",
       "                                             seed=None, silent=None, subsample=1,\n",
       "                                             verbosity=1))],\n",
       "                       verbose=False), 1: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('nextbilladder', NextBillAdder()),\n",
       "                                               ('standardscaler',\n",
       "                                                StandardScaler(copy=True, with_mean=True,\n",
       "                                                               with_std=True))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               XGBClassifier(base_score=0.5, booster='gbtree',\n",
       "                                             colsample_bylevel=1, colsample_bynode=1,\n",
       "                                             colsample_bytree=1, gamma=0, learning_rate=0.1,\n",
       "                                             max_delta_step=0, max_depth=3,\n",
       "                                             min_child_weight=1, missing=None,\n",
       "                                             n_estimators=100, n_jobs=1, nthread=None,\n",
       "                                             objective='binary:logistic', random_state=0,\n",
       "                                             reg_alpha=0, reg_lambda=1, scale_pos_weight=5,\n",
       "                                             seed=None, silent=None, subsample=1,\n",
       "                                             verbosity=1))],\n",
       "                       verbose=False), 2: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'marriage']))],\n",
       "                                        verbose=False)),\n",
       "                              ('classifier',\n",
       "                               XGB...ooster='gbtree',\n",
       "                                             colsample_bylevel=1, colsample_bynode=1,\n",
       "                                             colsample_bytree=1, gamma=0, learning_rate=0.1,\n",
       "                                             max_delta_step=0, max_depth=3,\n",
       "                                             min_child_weight=1, missing=None,\n",
       "                                             n_estimators=100, n_jobs=1, nthread=None,\n",
       "                                             objective='binary:logistic', random_state=0,\n",
       "                                             reg_alpha=0, reg_lambda=1, scale_pos_weight=5,\n",
       "                                             seed=None, silent=None, subsample=1,\n",
       "                                             verbosity=1))],\n",
       "                       verbose=False), 3: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'm...\n",
       "                               XGBClassifier(base_score=0.5, booster='gbtree',\n",
       "                                             colsample_bylevel=1, colsample_bynode=1,\n",
       "                                             colsample_bytree=1, gamma=0, learning_rate=0.1,\n",
       "                                             max_delta_step=0, max_depth=3,\n",
       "                                             min_child_weight=1, missing=None,\n",
       "                                             n_estimators=100, n_jobs=1, nthread=None,\n",
       "                                             objective='binary:logistic', random_state=0,\n",
       "                                             reg_alpha=0, reg_lambda=1, scale_pos_weight=5,\n",
       "                                             seed=None, silent=None, subsample=1,\n",
       "                                             verbosity=1))],\n",
       "                       verbose=False), 4: Pipeline(memory=None,\n",
       "                       steps=[('preproc',\n",
       "                               Pipeline(memory=None,\n",
       "                                        steps=[('agebinadder', AgeBinAdder()),\n",
       "                                               ('genderxagebinadder', GenderXAgeBinAdder()),\n",
       "                                               ('genderxmarriageadder',\n",
       "                                                GenderXMarriageAdder()),\n",
       "                                               ('nextbilladder', NextBillAdder()),\n",
       "                                               ('columndropper', ColumnDropper(column='age')),\n",
       "                                               ('categoricalwarrior',\n",
       "                                                CategoricalWarrior(attribute_names=['gender',\n",
       "                                                                                    'education',\n",
       "                                                                                    'm...\n",
       "                               XGBClassifier(base_score=0.5, booster='gbtree',\n",
       "                                             colsample_bylevel=1, colsample_bynode=1,\n",
       "                                             colsample_bytree=1, gamma=0, learning_rate=0.1,\n",
       "                                             max_delta_step=0, max_depth=3,\n",
       "                                             min_child_weight=1, missing=None,\n",
       "                                             n_estimators=100, n_jobs=1, nthread=None,\n",
       "                                             objective='binary:logistic', random_state=0,\n",
       "                                             reg_alpha=0, reg_lambda=1, scale_pos_weight=5,\n",
       "                                             seed=None, silent=None, subsample=1,\n",
       "                                             verbosity=1))],\n",
       "                       verbose=False)}})"
      ]
     },
     "execution_count": 317,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dico_pipelined_models_num"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training all the Prediction Pipelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 318,
   "metadata": {},
   "outputs": [],
   "source": [
    "dico_classification_reports = defaultdict(dict)\n",
    "dico_confusion_matrices = defaultdict(dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 319,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LOG\n",
      "-------------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 0, Log\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.48      0.57      0.52      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.78      0.77      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 1, Log\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.48      0.57      0.52      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.78      0.77      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 2, Log\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.51      0.64      4673\n",
      "           1       0.29      0.71      0.41      1327\n",
      "\n",
      "    accuracy                           0.56      6000\n",
      "   macro avg       0.58      0.61      0.53      6000\n",
      "weighted avg       0.74      0.56      0.59      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 3, Log\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.49      0.57      0.53      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.79      0.77      0.78      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 4, Log\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.81      0.78      0.80      4673\n",
      "           1       0.33      0.37      0.34      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.57      0.58      0.57      6000\n",
      "weighted avg       0.71      0.69      0.70      6000\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\twang\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:1544: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = 8.\n",
      "  \" = {}.\".format(effective_n_jobs(self.n_jobs)))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipe 5, Log\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.83      0.85      4673\n",
      "           1       0.49      0.57      0.53      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.68      0.70      0.69      6000\n",
      "weighted avg       0.79      0.77      0.78      6000\n",
      "\n",
      "SVM\n",
      "-------------\n",
      "Pipe 0, Svm\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.80      0.84      4673\n",
      "           1       0.46      0.59      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 1, Svm\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.80      0.84      4673\n",
      "           1       0.46      0.59      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 2, Svm\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.81      0.84      4673\n",
      "           1       0.46      0.58      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "TREE\n",
      "-------------\n",
      "Pipe 0, Tree\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 1, Tree\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 2, Tree\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 3, Tree\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "Pipe 4, Tree\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.83      0.85      4673\n",
      "           1       0.48      0.54      0.51      1327\n",
      "\n",
      "    accuracy                           0.77      6000\n",
      "   macro avg       0.67      0.69      0.68      6000\n",
      "weighted avg       0.78      0.77      0.77      6000\n",
      "\n",
      "FOREST\n",
      "-------------\n",
      "Pipe 0, Forest\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.60      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "Pipe 1, Forest\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.61      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "Pipe 2, Forest\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.79      0.83      4673\n",
      "           1       0.46      0.62      0.53      1327\n",
      "\n",
      "    accuracy                           0.75      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.75      0.77      6000\n",
      "\n",
      "Pipe 3, Forest\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.62      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "Pipe 4, Forest\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.61      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "ADABOOST\n",
      "-------------\n",
      "XGBOOST\n",
      "-------------\n",
      "Pipe 0, Xgboost\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.69      0.78      4673\n",
      "           1       0.40      0.73      0.51      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.65      0.71      0.65      6000\n",
      "weighted avg       0.79      0.69      0.72      6000\n",
      "\n",
      "Pipe 1, Xgboost\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.68      0.78      4673\n",
      "           1       0.39      0.72      0.51      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.64      0.70      0.64      6000\n",
      "weighted avg       0.79      0.69      0.72      6000\n",
      "\n",
      "Pipe 2, Xgboost\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.68      0.78      4673\n",
      "           1       0.39      0.72      0.51      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.65      0.70      0.64      6000\n",
      "weighted avg       0.79      0.69      0.72      6000\n",
      "\n",
      "Pipe 3, Xgboost\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.69      0.78      4673\n",
      "           1       0.39      0.72      0.51      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.64      0.70      0.64      6000\n",
      "weighted avg       0.78      0.69      0.72      6000\n",
      "\n",
      "Pipe 4, Xgboost\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.68      0.77      4673\n",
      "           1       0.39      0.72      0.51      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.64      0.70      0.64      6000\n",
      "weighted avg       0.78      0.69      0.72      6000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for (name, model) in dico_models.items():\n",
    "    print(name.upper())\n",
    "    print('-------------')\n",
    "    for (i, complex_pipe) in enumerate(dico_pipelined_models_num[name]):\n",
    "#         print('!!! POURQUOI CA MARCHE PAS !!!')\n",
    "        pipe = dico_pipelined_models_num[name][i]\n",
    "        pipe.fit(X_train, np.ravel(y_train))\n",
    "        print('Pipe {}, {}'.format(i, name.capitalize()))\n",
    "        print('----------------')\n",
    "        \n",
    "        y_pred = pipe.predict(X_test)\n",
    "        dico_classification_reports[name][i] = classification_report(y_test, y_pred)\n",
    "        dico_confusion_matrices[name][i] = confusion_matrix(y_test, y_pred)\n",
    "        print(dico_classification_reports[name][i])\n",
    "        #print(dico_confusion_matrices[i][name])\n",
    "\n",
    "#         print('!!! POURQUOI CA MARCHE PAS !!!')'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Our takeaways"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Logistic regression:  \n",
    "We have a great recall of our default class, using all our engineered features, as well as keeping our 'age' feature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* SVM:  \n",
    "Adding our previously engineered features leads to no change"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Decision Tree:  \n",
    "All the different preprocessing pipelines lead to similar results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Random Forest:  \n",
    "All the different preprocessing pipelines lead to similar results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* XGBoost:  \n",
    "All the different preprocessing pipelines lead to similar results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In conclusion, it seems that all our new features aren't too useful, excepted for the Logistic Regression model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at the best (preprocessing, model) couples: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 325,
   "metadata": {},
   "outputs": [],
   "source": [
    "def show_classification_report(model_name, pipe_number):\n",
    "    print('{}, Pipe {}'.format(model_name.capitalize(), pipe_number))\n",
    "    print('----------------')\n",
    "    print(dico_classification_reports[model_name][pipe_number])\n",
    "    print('------------------------------------------------------')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 302,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log, Pipe 2\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.86      0.51      0.64      4673\n",
      "           1       0.29      0.71      0.41      1327\n",
      "\n",
      "    accuracy                           0.56      6000\n",
      "   macro avg       0.58      0.61      0.53      6000\n",
      "weighted avg       0.74      0.56      0.59      6000\n",
      "\n",
      "------------------------------------------------------\n",
      "Svm, Pipe 0\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.87      0.80      0.84      4673\n",
      "           1       0.46      0.59      0.52      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.70      0.68      6000\n",
      "weighted avg       0.78      0.76      0.77      6000\n",
      "\n",
      "------------------------------------------------------\n",
      "Forest, Pipe 3\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.80      0.84      4673\n",
      "           1       0.46      0.62      0.53      1327\n",
      "\n",
      "    accuracy                           0.76      6000\n",
      "   macro avg       0.67      0.71      0.68      6000\n",
      "weighted avg       0.79      0.76      0.77      6000\n",
      "\n",
      "------------------------------------------------------\n",
      "Xgboost, Pipe 1\n",
      "----------------\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.90      0.68      0.78      4673\n",
      "           1       0.39      0.72      0.51      1327\n",
      "\n",
      "    accuracy                           0.69      6000\n",
      "   macro avg       0.64      0.70      0.64      6000\n",
      "weighted avg       0.79      0.69      0.72      6000\n",
      "\n",
      "------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "show_classification_report('log', 2)\n",
    "show_classification_report('svm', 0)\n",
    "show_classification_report('forest', 3)\n",
    "show_classification_report('xgboost', 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Meilleurs modèles (en comparant selon le recall des default):\n",
    "* XGBoost, avec la prédiction du Next Bill\n",
    "* Logistic Regression, avec toutes les features engineerées et sans Scaling (!)\n",
    "* Random Forest (peu importe le type de preprocessing\n",
    "* SVM (scalé absolument, avec ou sans Next Bill ^^ )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TO DO Next"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "OK - SOVLED\n",
    "Careful!\n",
    "Log doesn't take categorical: complex_pipeline2"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "OK - DONE\n",
    "* Find a way to not explore all preprocessing pipelines for some models"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "OK - DONE\n",
    "* Use pipelines with XGBoost model (only the relevant Complex preprocessing pipelines ones!)\n",
    "ie: no need to scale"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "* Use pipelines with LightGBM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "setting an array element with a sequence.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-182-3e654d667441>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mdico_pipelined_models\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m2\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'log'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mravel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my_train\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\pipeline.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, **fit_params)\u001b[0m\n\u001b[0;32m    354\u001b[0m                                  self._log_message(len(self.steps) - 1)):\n\u001b[0;32m    355\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_final_estimator\u001b[0m \u001b[1;33m!=\u001b[0m \u001b[1;34m'passthrough'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 356\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_final_estimator\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mXt\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    357\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    358\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, sample_weight)\u001b[0m\n\u001b[0;32m   1530\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1531\u001b[0m         X, y = check_X_y(X, y, accept_sparse='csr', dtype=_dtype, order=\"C\",\n\u001b[1;32m-> 1532\u001b[1;33m                          accept_large_sparse=solver != 'liblinear')\n\u001b[0m\u001b[0;32m   1533\u001b[0m         \u001b[0mcheck_classification_targets\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1534\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclasses_\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0munique\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py\u001b[0m in \u001b[0;36mcheck_X_y\u001b[1;34m(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, warn_on_dtype, estimator)\u001b[0m\n\u001b[0;32m    717\u001b[0m                     \u001b[0mensure_min_features\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mensure_min_features\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    718\u001b[0m                     \u001b[0mwarn_on_dtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mwarn_on_dtype\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 719\u001b[1;33m                     estimator=estimator)\n\u001b[0m\u001b[0;32m    720\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mmulti_output\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    721\u001b[0m         y = check_array(y, 'csr', force_all_finite=True, ensure_2d=False,\n",
      "\u001b[1;32m~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py\u001b[0m in \u001b[0;36mcheck_array\u001b[1;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)\u001b[0m\n\u001b[0;32m    494\u001b[0m             \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    495\u001b[0m                 \u001b[0mwarnings\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msimplefilter\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'error'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mComplexWarning\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 496\u001b[1;33m                 \u001b[0marray\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0morder\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0morder\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    497\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mComplexWarning\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    498\u001b[0m                 raise ValueError(\"Complex data not supported\\n\"\n",
      "\u001b[1;32m~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\numpy\\core\\numeric.py\u001b[0m in \u001b[0;36masarray\u001b[1;34m(a, dtype, order)\u001b[0m\n\u001b[0;32m    536\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    537\u001b[0m     \"\"\"\n\u001b[1;32m--> 538\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0morder\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0morder\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    539\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    540\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: setting an array element with a sequence."
     ]
    }
   ],
   "source": [
    "dico_pipelined_models[2]['log'].fit(X_train, np.ravel(y_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('preproc',\n",
       "                 Pipeline(memory=None,\n",
       "                          steps=[('age_bin', AgeBinAdder()),\n",
       "                                 ('gender_age_bin', GenderXAgeBinAdder()),\n",
       "                                 ('gender_marriage', GenderXMarriageAdder()),\n",
       "                                 ('next_bill', NextBillAdder())],\n",
       "                          verbose=False)),\n",
       "                ('classifier',\n",
       "                 LogisticRegression(C=0.1, class_weight='balanced', dual=False,\n",
       "                                    fit_intercept=True, intercept_scaling=1,\n",
       "                                    l1_ratio=None, max_iter=100,\n",
       "                                    multi_class='warn', n_jobs=-1, penalty='l2',\n",
       "                                    random_state=42, solver='liblinear',\n",
       "                                    tol=0.0001, verbose=0, warm_start=False))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 186,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dico_pipelined_models[2]['log']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>limit_bal</th>\n",
       "      <th>age</th>\n",
       "      <th>pay_1</th>\n",
       "      <th>pay_2</th>\n",
       "      <th>pay_3</th>\n",
       "      <th>pay_4</th>\n",
       "      <th>pay_5</th>\n",
       "      <th>pay_6</th>\n",
       "      <th>bill_amt1</th>\n",
       "      <th>bill_amt2</th>\n",
       "      <th>bill_amt3</th>\n",
       "      <th>bill_amt4</th>\n",
       "      <th>bill_amt5</th>\n",
       "      <th>bill_amt6</th>\n",
       "      <th>pay_amt1</th>\n",
       "      <th>pay_amt2</th>\n",
       "      <th>pay_amt3</th>\n",
       "      <th>pay_amt4</th>\n",
       "      <th>pay_amt5</th>\n",
       "      <th>pay_amt6</th>\n",
       "      <th>pred_bill_amt0</th>\n",
       "      <th>gender_2</th>\n",
       "      <th>gender_1</th>\n",
       "      <th>education_3</th>\n",
       "      <th>education_2</th>\n",
       "      <th>education_1</th>\n",
       "      <th>education_4</th>\n",
       "      <th>marriage_1</th>\n",
       "      <th>marriage_2</th>\n",
       "      <th>marriage_3</th>\n",
       "      <th>age_bin_20s</th>\n",
       "      <th>age_bin_30s</th>\n",
       "      <th>age_bin_40s</th>\n",
       "      <th>age_bin_50s</th>\n",
       "      <th>age_bin_60+</th>\n",
       "      <th>gen_ageBin_(1, '20s')</th>\n",
       "      <th>gen_ageBin_(1, '30s')</th>\n",
       "      <th>gen_ageBin_(1, '40s')</th>\n",
       "      <th>gen_ageBin_(1, '50s')</th>\n",
       "      <th>gen_ageBin_(1, '60+')</th>\n",
       "      <th>gen_ageBin_(2, '20s')</th>\n",
       "      <th>gen_ageBin_(2, '30s')</th>\n",
       "      <th>gen_ageBin_(2, '40s')</th>\n",
       "      <th>gen_ageBin_(2, '50s')</th>\n",
       "      <th>gen_ageBin_(2, '60+')</th>\n",
       "      <th>gen_mar_(1, 1)</th>\n",
       "      <th>gen_mar_(1, 2)</th>\n",
       "      <th>gen_mar_(1, 3)</th>\n",
       "      <th>gen_mar_(2, 1)</th>\n",
       "      <th>gen_mar_(2, 2)</th>\n",
       "      <th>gen_mar_(2, 3)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10574</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19255.0</td>\n",
       "      <td>20650.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>19269.757474</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11330</th>\n",
       "      <td>300000.0</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>285038.0</td>\n",
       "      <td>291480.0</td>\n",
       "      <td>298378.0</td>\n",
       "      <td>131103.0</td>\n",
       "      <td>128742.0</td>\n",
       "      <td>124882.0</td>\n",
       "      <td>12507.0</td>\n",
       "      <td>15056.0</td>\n",
       "      <td>5027.0</td>\n",
       "      <td>5007.0</td>\n",
       "      <td>5063.0</td>\n",
       "      <td>3039.0</td>\n",
       "      <td>285123.402526</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7127</th>\n",
       "      <td>500000.0</td>\n",
       "      <td>40</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>33179.0</td>\n",
       "      <td>97882.0</td>\n",
       "      <td>48017.0</td>\n",
       "      <td>49403.0</td>\n",
       "      <td>40018.0</td>\n",
       "      <td>54109.0</td>\n",
       "      <td>99159.0</td>\n",
       "      <td>48276.0</td>\n",
       "      <td>49450.0</td>\n",
       "      <td>201.0</td>\n",
       "      <td>54379.0</td>\n",
       "      <td>9905.0</td>\n",
       "      <td>34020.039927</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25150</th>\n",
       "      <td>70000.0</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>70349.0</td>\n",
       "      <td>70068.0</td>\n",
       "      <td>71318.0</td>\n",
       "      <td>59665.0</td>\n",
       "      <td>29510.0</td>\n",
       "      <td>26738.0</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>3700.0</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>70345.498357</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6000</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13538.0</td>\n",
       "      <td>9199.0</td>\n",
       "      <td>6879.0</td>\n",
       "      <td>6753.0</td>\n",
       "      <td>5440.0</td>\n",
       "      <td>5440.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>1284.0</td>\n",
       "      <td>264.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13480.622867</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20951</th>\n",
       "      <td>340000.0</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11379.0</td>\n",
       "      <td>11086.0</td>\n",
       "      <td>3112.0</td>\n",
       "      <td>3351.0</td>\n",
       "      <td>1259.0</td>\n",
       "      <td>19271.0</td>\n",
       "      <td>11258.0</td>\n",
       "      <td>3112.0</td>\n",
       "      <td>3351.0</td>\n",
       "      <td>1259.0</td>\n",
       "      <td>19271.0</td>\n",
       "      <td>16992.0</td>\n",
       "      <td>11373.778043</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13894</th>\n",
       "      <td>300000.0</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>145138.0</td>\n",
       "      <td>148586.0</td>\n",
       "      <td>151878.0</td>\n",
       "      <td>123417.0</td>\n",
       "      <td>104922.0</td>\n",
       "      <td>107005.0</td>\n",
       "      <td>5785.0</td>\n",
       "      <td>6163.0</td>\n",
       "      <td>4728.0</td>\n",
       "      <td>3811.0</td>\n",
       "      <td>3830.0</td>\n",
       "      <td>3970.0</td>\n",
       "      <td>145183.779419</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15285</th>\n",
       "      <td>230000.0</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>970.0</td>\n",
       "      <td>1475.0</td>\n",
       "      <td>1636.0</td>\n",
       "      <td>1149.0</td>\n",
       "      <td>334.0</td>\n",
       "      <td>459.0</td>\n",
       "      <td>1483.0</td>\n",
       "      <td>1640.0</td>\n",
       "      <td>1152.0</td>\n",
       "      <td>335.0</td>\n",
       "      <td>460.0</td>\n",
       "      <td>553.0</td>\n",
       "      <td>976.657948</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6881</th>\n",
       "      <td>200000.0</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>26344.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>320.0</td>\n",
       "      <td>170.0</td>\n",
       "      <td>2865.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>329.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2865.0</td>\n",
       "      <td>801.0</td>\n",
       "      <td>25998.069408</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22145</th>\n",
       "      <td>60000.0</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23304.0</td>\n",
       "      <td>20517.0</td>\n",
       "      <td>20261.0</td>\n",
       "      <td>22832.0</td>\n",
       "      <td>24234.0</td>\n",
       "      <td>24928.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>23267.364751</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10270</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>49423.0</td>\n",
       "      <td>48332.0</td>\n",
       "      <td>46043.0</td>\n",
       "      <td>18759.0</td>\n",
       "      <td>19161.0</td>\n",
       "      <td>19516.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>655.0</td>\n",
       "      <td>688.0</td>\n",
       "      <td>666.0</td>\n",
       "      <td>649.0</td>\n",
       "      <td>49408.217276</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28740</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4145.0</td>\n",
       "      <td>3930.0</td>\n",
       "      <td>5859.0</td>\n",
       "      <td>10299.0</td>\n",
       "      <td>11825.0</td>\n",
       "      <td>13121.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>5008.0</td>\n",
       "      <td>3007.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>4142.519487</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19310</th>\n",
       "      <td>360000.0</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4087</th>\n",
       "      <td>150000.0</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>135587.0</td>\n",
       "      <td>138473.0</td>\n",
       "      <td>141297.0</td>\n",
       "      <td>144605.0</td>\n",
       "      <td>148420.0</td>\n",
       "      <td>150500.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>4500.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>135625.391520</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13760</th>\n",
       "      <td>480000.0</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7174.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7079.795960</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27714</th>\n",
       "      <td>130000.0</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>44930.0</td>\n",
       "      <td>40688.0</td>\n",
       "      <td>40880.0</td>\n",
       "      <td>37851.0</td>\n",
       "      <td>37816.0</td>\n",
       "      <td>39849.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>8002.0</td>\n",
       "      <td>44874.323218</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7584</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>83480.0</td>\n",
       "      <td>89264.0</td>\n",
       "      <td>89896.0</td>\n",
       "      <td>86706.0</td>\n",
       "      <td>87385.0</td>\n",
       "      <td>88351.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>83556.053289</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29650</th>\n",
       "      <td>260000.0</td>\n",
       "      <td>36</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>334214.0</td>\n",
       "      <td>310546.0</td>\n",
       "      <td>288566.0</td>\n",
       "      <td>291597.0</td>\n",
       "      <td>214353.0</td>\n",
       "      <td>216238.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11000.0</td>\n",
       "      <td>11000.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>8500.0</td>\n",
       "      <td>8300.0</td>\n",
       "      <td>333899.422605</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9472</th>\n",
       "      <td>440000.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>225416.0</td>\n",
       "      <td>220405.0</td>\n",
       "      <td>226731.0</td>\n",
       "      <td>231456.0</td>\n",
       "      <td>237707.0</td>\n",
       "      <td>243555.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>225351.300676</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7022</th>\n",
       "      <td>100000.0</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>59586.0</td>\n",
       "      <td>60842.0</td>\n",
       "      <td>61749.0</td>\n",
       "      <td>65130.0</td>\n",
       "      <td>63816.0</td>\n",
       "      <td>65011.0</td>\n",
       "      <td>2800.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2400.0</td>\n",
       "      <td>2400.0</td>\n",
       "      <td>59602.656941</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21323</th>\n",
       "      <td>250000.0</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>900.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>900.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.037329</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23409</th>\n",
       "      <td>200000.0</td>\n",
       "      <td>43</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>198595.0</td>\n",
       "      <td>194734.0</td>\n",
       "      <td>190447.0</td>\n",
       "      <td>187224.0</td>\n",
       "      <td>183641.0</td>\n",
       "      <td>178239.0</td>\n",
       "      <td>7003.0</td>\n",
       "      <td>7000.0</td>\n",
       "      <td>7000.0</td>\n",
       "      <td>6510.0</td>\n",
       "      <td>6550.0</td>\n",
       "      <td>6001.0</td>\n",
       "      <td>198543.553379</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3884</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>780.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>395.054847</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9456</th>\n",
       "      <td>310000.0</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2948.0</td>\n",
       "      <td>62593.0</td>\n",
       "      <td>152141.0</td>\n",
       "      <td>125325.0</td>\n",
       "      <td>109142.0</td>\n",
       "      <td>103020.0</td>\n",
       "      <td>62635.0</td>\n",
       "      <td>100042.0</td>\n",
       "      <td>4017.0</td>\n",
       "      <td>9184.0</td>\n",
       "      <td>4081.0</td>\n",
       "      <td>6042.0</td>\n",
       "      <td>3746.596855</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15820</th>\n",
       "      <td>30000.0</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23575.0</td>\n",
       "      <td>24584.0</td>\n",
       "      <td>25885.0</td>\n",
       "      <td>26851.0</td>\n",
       "      <td>26808.0</td>\n",
       "      <td>27599.0</td>\n",
       "      <td>1400.0</td>\n",
       "      <td>1700.0</td>\n",
       "      <td>1900.0</td>\n",
       "      <td>557.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>23588.476014</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13152</th>\n",
       "      <td>180000.0</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>43507.0</td>\n",
       "      <td>44282.0</td>\n",
       "      <td>23946.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>43513.615983</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10707</th>\n",
       "      <td>80000.0</td>\n",
       "      <td>34</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17294.0</td>\n",
       "      <td>22256.0</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>22055.0</td>\n",
       "      <td>22603.0</td>\n",
       "      <td>23133.0</td>\n",
       "      <td>5210.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>17359.055717</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1541</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23062</th>\n",
       "      <td>150000.0</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23364.0</td>\n",
       "      <td>22846.0</td>\n",
       "      <td>25779.0</td>\n",
       "      <td>22303.0</td>\n",
       "      <td>16981.0</td>\n",
       "      <td>16837.0</td>\n",
       "      <td>1450.0</td>\n",
       "      <td>5779.0</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>700.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>23357.695692</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11118</th>\n",
       "      <td>200000.0</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>1879.0</td>\n",
       "      <td>1707.0</td>\n",
       "      <td>617.0</td>\n",
       "      <td>620.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1879.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>617.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>184.514338</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3063</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>43685.0</td>\n",
       "      <td>44694.0</td>\n",
       "      <td>43673.0</td>\n",
       "      <td>44659.0</td>\n",
       "      <td>49512.0</td>\n",
       "      <td>48454.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>300.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>43698.075819</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21234</th>\n",
       "      <td>300000.0</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50281.0</td>\n",
       "      <td>51441.0</td>\n",
       "      <td>54107.0</td>\n",
       "      <td>55111.0</td>\n",
       "      <td>56688.0</td>\n",
       "      <td>58272.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>50296.694346</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2459</th>\n",
       "      <td>250000.0</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>37137.0</td>\n",
       "      <td>51803.0</td>\n",
       "      <td>4997.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>51811.0</td>\n",
       "      <td>9544.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>37321.501701</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24360</th>\n",
       "      <td>100000.0</td>\n",
       "      <td>34</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>-342.0</td>\n",
       "      <td>1158.0</td>\n",
       "      <td>2485.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-341.999955</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9048</th>\n",
       "      <td>140000.0</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>854.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>842.785859</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25080</th>\n",
       "      <td>30000.0</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14476.0</td>\n",
       "      <td>15297.0</td>\n",
       "      <td>8531.0</td>\n",
       "      <td>8020.0</td>\n",
       "      <td>-620.0</td>\n",
       "      <td>-620.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>1152.0</td>\n",
       "      <td>1081.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14485.612723</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6886</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>55</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>390.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>911.0</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>930.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>911.0</td>\n",
       "      <td>559.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>390.090215</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18264</th>\n",
       "      <td>180000.0</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>77408.0</td>\n",
       "      <td>74156.0</td>\n",
       "      <td>75659.0</td>\n",
       "      <td>72046.0</td>\n",
       "      <td>70355.0</td>\n",
       "      <td>71208.0</td>\n",
       "      <td>3293.0</td>\n",
       "      <td>3350.0</td>\n",
       "      <td>3111.0</td>\n",
       "      <td>2615.0</td>\n",
       "      <td>2689.0</td>\n",
       "      <td>2766.0</td>\n",
       "      <td>77365.547903</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17546</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>42</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>31422.0</td>\n",
       "      <td>32420.0</td>\n",
       "      <td>33462.0</td>\n",
       "      <td>36199.0</td>\n",
       "      <td>36803.0</td>\n",
       "      <td>37566.0</td>\n",
       "      <td>1521.0</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>3300.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>1600.0</td>\n",
       "      <td>31435.290939</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12326</th>\n",
       "      <td>240000.0</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>133230.0</td>\n",
       "      <td>136579.0</td>\n",
       "      <td>139281.0</td>\n",
       "      <td>132487.0</td>\n",
       "      <td>107437.0</td>\n",
       "      <td>109237.0</td>\n",
       "      <td>5500.0</td>\n",
       "      <td>5036.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>6200.0</td>\n",
       "      <td>5500.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>133274.426549</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2440</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>40</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>67925.0</td>\n",
       "      <td>69771.0</td>\n",
       "      <td>73233.0</td>\n",
       "      <td>72328.0</td>\n",
       "      <td>74234.0</td>\n",
       "      <td>75500.0</td>\n",
       "      <td>3509.0</td>\n",
       "      <td>5824.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2204.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>67949.835369</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8905</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>42883.0</td>\n",
       "      <td>47887.0</td>\n",
       "      <td>46836.0</td>\n",
       "      <td>45785.0</td>\n",
       "      <td>47003.0</td>\n",
       "      <td>48085.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>42948.525522</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5739</th>\n",
       "      <td>150000.0</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112391.0</td>\n",
       "      <td>98583.0</td>\n",
       "      <td>95810.0</td>\n",
       "      <td>87216.0</td>\n",
       "      <td>88201.0</td>\n",
       "      <td>87483.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>3290.0</td>\n",
       "      <td>3200.0</td>\n",
       "      <td>3215.0</td>\n",
       "      <td>112209.185246</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>360</th>\n",
       "      <td>500000.0</td>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>507726.0</td>\n",
       "      <td>509229.0</td>\n",
       "      <td>499936.0</td>\n",
       "      <td>505507.0</td>\n",
       "      <td>484612.0</td>\n",
       "      <td>419643.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>16500.0</td>\n",
       "      <td>16000.0</td>\n",
       "      <td>19000.0</td>\n",
       "      <td>507744.145924</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4816</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40019.0</td>\n",
       "      <td>20394.0</td>\n",
       "      <td>19597.0</td>\n",
       "      <td>17830.0</td>\n",
       "      <td>18397.0</td>\n",
       "      <td>19064.0</td>\n",
       "      <td>2006.0</td>\n",
       "      <td>1519.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>39761.156568</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15178</th>\n",
       "      <td>90000.0</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28276.0</td>\n",
       "      <td>29305.0</td>\n",
       "      <td>30336.0</td>\n",
       "      <td>28800.0</td>\n",
       "      <td>9852.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>1505.0</td>\n",
       "      <td>1536.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>9852.0</td>\n",
       "      <td>2160.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>28289.685854</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18321</th>\n",
       "      <td>480000.0</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3655.0</td>\n",
       "      <td>38022.0</td>\n",
       "      <td>5785.0</td>\n",
       "      <td>9233.0</td>\n",
       "      <td>4893.0</td>\n",
       "      <td>2168.0</td>\n",
       "      <td>38024.0</td>\n",
       "      <td>5785.0</td>\n",
       "      <td>9233.0</td>\n",
       "      <td>4893.0</td>\n",
       "      <td>2168.0</td>\n",
       "      <td>3625.0</td>\n",
       "      <td>4100.732845</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23609</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>43886.0</td>\n",
       "      <td>50411.0</td>\n",
       "      <td>9688.0</td>\n",
       "      <td>857.0</td>\n",
       "      <td>3810.0</td>\n",
       "      <td>4741.0</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>7026.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>43964.639987</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16230</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>49450.0</td>\n",
       "      <td>50067.0</td>\n",
       "      <td>69594.0</td>\n",
       "      <td>46181.0</td>\n",
       "      <td>16770.0</td>\n",
       "      <td>19890.0</td>\n",
       "      <td>1743.0</td>\n",
       "      <td>2236.0</td>\n",
       "      <td>2807.0</td>\n",
       "      <td>629.0</td>\n",
       "      <td>3810.0</td>\n",
       "      <td>719.0</td>\n",
       "      <td>49461.415202</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21927</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3849.0</td>\n",
       "      <td>6393.0</td>\n",
       "      <td>6969.0</td>\n",
       "      <td>9044.0</td>\n",
       "      <td>2325.0</td>\n",
       "      <td>3998.0</td>\n",
       "      <td>6458.0</td>\n",
       "      <td>6980.0</td>\n",
       "      <td>9044.0</td>\n",
       "      <td>2325.0</td>\n",
       "      <td>4997.0</td>\n",
       "      <td>9947.0</td>\n",
       "      <td>3882.509880</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27267</th>\n",
       "      <td>160000.0</td>\n",
       "      <td>42</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>14137.0</td>\n",
       "      <td>13613.0</td>\n",
       "      <td>14634.0</td>\n",
       "      <td>16532.0</td>\n",
       "      <td>15969.0</td>\n",
       "      <td>17701.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1247.0</td>\n",
       "      <td>2145.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14130.299526</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11593</th>\n",
       "      <td>170000.0</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-1020.0</td>\n",
       "      <td>25607.0</td>\n",
       "      <td>23922.0</td>\n",
       "      <td>24399.0</td>\n",
       "      <td>24919.0</td>\n",
       "      <td>26884.0</td>\n",
       "      <td>27107.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>863.0</td>\n",
       "      <td>903.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>6186.0</td>\n",
       "      <td>-670.641976</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15732</th>\n",
       "      <td>50000.0</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27369.0</td>\n",
       "      <td>6425.0</td>\n",
       "      <td>4580.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1050.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27093.649272</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25950</th>\n",
       "      <td>410000.0</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>110345.0</td>\n",
       "      <td>116128.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>388.0</td>\n",
       "      <td>110345.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>4200.0</td>\n",
       "      <td>388.003271</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5863</th>\n",
       "      <td>320000.0</td>\n",
       "      <td>44</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>36831.0</td>\n",
       "      <td>28785.0</td>\n",
       "      <td>13104.0</td>\n",
       "      <td>2621.0</td>\n",
       "      <td>9075.0</td>\n",
       "      <td>6564.0</td>\n",
       "      <td>28791.0</td>\n",
       "      <td>13227.0</td>\n",
       "      <td>2621.0</td>\n",
       "      <td>9075.0</td>\n",
       "      <td>6564.0</td>\n",
       "      <td>12767.0</td>\n",
       "      <td>36722.618013</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22847</th>\n",
       "      <td>310000.0</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>67940.0</td>\n",
       "      <td>45746.0</td>\n",
       "      <td>46979.0</td>\n",
       "      <td>42045.0</td>\n",
       "      <td>40334.0</td>\n",
       "      <td>49542.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>13000.0</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>67648.765025</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17496</th>\n",
       "      <td>80000.0</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>52062.0</td>\n",
       "      <td>49884.0</td>\n",
       "      <td>22348.0</td>\n",
       "      <td>22063.0</td>\n",
       "      <td>8723.0</td>\n",
       "      <td>7254.0</td>\n",
       "      <td>2158.0</td>\n",
       "      <td>1708.0</td>\n",
       "      <td>1411.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>220.0</td>\n",
       "      <td>52028.650887</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2145</th>\n",
       "      <td>170000.0</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>171098.0</td>\n",
       "      <td>166176.0</td>\n",
       "      <td>169759.0</td>\n",
       "      <td>11760.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4902.0</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>5695.0</td>\n",
       "      <td>11760.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4902.0</td>\n",
       "      <td>6000.0</td>\n",
       "      <td>171033.627401</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15305</th>\n",
       "      <td>100000.0</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>83013.0</td>\n",
       "      <td>89589.0</td>\n",
       "      <td>92887.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8000.0</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>83099.709874</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25509</th>\n",
       "      <td>60000.0</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>56136.0</td>\n",
       "      <td>57280.0</td>\n",
       "      <td>56969.0</td>\n",
       "      <td>58695.0</td>\n",
       "      <td>57453.0</td>\n",
       "      <td>61153.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>2685.0</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>4800.0</td>\n",
       "      <td>3400.0</td>\n",
       "      <td>56150.972468</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>18000 rows × 51 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       limit_bal  age  pay_1  pay_2  pay_3  pay_4  pay_5  pay_6  bill_amt1  \\\n",
       "10574    20000.0   51      0      0      0      0      0      0    19255.0   \n",
       "11330   300000.0   31      0      0      0      0      0      0   285038.0   \n",
       "7127    500000.0   40      0      0      0      0      0      0    33179.0   \n",
       "25150    70000.0   46      0      0      0      0      0      0    70349.0   \n",
       "6000     50000.0   33      0      0      0      0      0      0    13538.0   \n",
       "20951   340000.0   41      0      0      0      0      0      0    11379.0   \n",
       "13894   300000.0   39      0      0      0      0      0      0   145138.0   \n",
       "15285   230000.0   39      0      0      0      0      0      0      970.0   \n",
       "6881    200000.0   38      0      0      0      0      1      0    26344.0   \n",
       "22145    60000.0   29      0      0      0      0      0      0    23304.0   \n",
       "10270    50000.0   46      1      1      0      0      0      0    49423.0   \n",
       "28740   160000.0   28      0      1      0      0      0      0     4145.0   \n",
       "19310   360000.0   33      0      0      0      0      0      0        0.0   \n",
       "4087    150000.0   34      0      0      0      0      0      0   135587.0   \n",
       "13760   480000.0   49      0      0      0      0      0      0     7174.0   \n",
       "27714   130000.0   23      0      0      0      0      0      0    44930.0   \n",
       "7584     90000.0   28      1      1      1      1      0      0    83480.0   \n",
       "29650   260000.0   36      1      0      0      0      0      0   334214.0   \n",
       "9472    440000.0   25      1      1      0      0      0      0   225416.0   \n",
       "7022    100000.0   44      1      1      1      1      1      1    59586.0   \n",
       "21323   250000.0   50      1      0      0      0      0      0        0.0   \n",
       "23409   200000.0   43      0      0      0      0      0      0   198595.0   \n",
       "3884     20000.0   24      0      0      1      0      0      0      390.0   \n",
       "9456    310000.0   35      0      0      0      0      0      0     2948.0   \n",
       "15820    30000.0   27      1      0      0      0      0      0    23575.0   \n",
       "13152   180000.0   39      0      0      0      0      0      0    43507.0   \n",
       "10707    80000.0   34      1      1      1      0      0      0    17294.0   \n",
       "1541     50000.0   22      1      0      0      0      0      0        0.0   \n",
       "23062   150000.0   32      0      0      0      0      0      0    23364.0   \n",
       "11118   200000.0   39      0      0      1      0      0      0      162.0   \n",
       "...          ...  ...    ...    ...    ...    ...    ...    ...        ...   \n",
       "3063     50000.0   25      1      1      1      0      0      1    43685.0   \n",
       "21234   300000.0   34      0      0      0      0      0      0    50281.0   \n",
       "2459    250000.0   41      0      0      0      0      0      0    37137.0   \n",
       "24360   100000.0   34      1      0      0      0      0      0     -342.0   \n",
       "9048    140000.0   30      0      0      0      0      0      0      854.0   \n",
       "25080    30000.0   52      0      0      0      0      0      0    14476.0   \n",
       "6886     50000.0   55      0      0      0      0      0      1      390.0   \n",
       "18264   180000.0   52      0      0      0      0      0      0    77408.0   \n",
       "17546    50000.0   42      1      0      0      0      1      1    31422.0   \n",
       "12326   240000.0   28      0      0      0      0      0      0   133230.0   \n",
       "2440    160000.0   40      0      0      0      0      0      0    67925.0   \n",
       "8905     50000.0   25      1      1      1      1      1      1    42883.0   \n",
       "5739    150000.0   34      0      0      0      0      0      0   112391.0   \n",
       "360     500000.0   23      1      1      1      0      0      0   507726.0   \n",
       "4816     50000.0   31      0      0      0      0      0      0    40019.0   \n",
       "15178    90000.0   25      0      0      0      0      0      0    28276.0   \n",
       "18321   480000.0   32      0      0      0      0      0      0     3655.0   \n",
       "23609    50000.0   37      0      0      0      0      0      0    43886.0   \n",
       "16230    50000.0   33      0      0      0      0      0      0    49450.0   \n",
       "21927   160000.0   28      0      0      0      0      0      0     3849.0   \n",
       "27267   160000.0   42      1      1      0      0      1      1    14137.0   \n",
       "11593   170000.0   27      1      0      0      0      0      0    -1020.0   \n",
       "15732    50000.0   30      1      1      0      0      0      0    27369.0   \n",
       "25950   410000.0   32      0      0      0      0      0      0      388.0   \n",
       "5863    320000.0   44      0      0      0      0      0      0    36831.0   \n",
       "22847   310000.0   36      0      0      0      0      0      0    67940.0   \n",
       "17496    80000.0   28      1      0      0      0      0      0    52062.0   \n",
       "2145    170000.0   28      0      0      0      0      0      0   171098.0   \n",
       "15305   100000.0   33      0      0      0      0      0      0    83013.0   \n",
       "25509    60000.0   25      0      0      0      0      0      0    56136.0   \n",
       "\n",
       "       bill_amt2  bill_amt3  bill_amt4  bill_amt5  bill_amt6  pay_amt1  \\\n",
       "10574    20650.0        0.0        0.0        0.0        0.0    2000.0   \n",
       "11330   291480.0   298378.0   131103.0   128742.0   124882.0   12507.0   \n",
       "7127     97882.0    48017.0    49403.0    40018.0    54109.0   99159.0   \n",
       "25150    70068.0    71318.0    59665.0    29510.0    26738.0    2600.0   \n",
       "6000      9199.0     6879.0     6753.0     5440.0     5440.0    2000.0   \n",
       "20951    11086.0     3112.0     3351.0     1259.0    19271.0   11258.0   \n",
       "13894   148586.0   151878.0   123417.0   104922.0   107005.0    5785.0   \n",
       "15285     1475.0     1636.0     1149.0      334.0      459.0    1483.0   \n",
       "6881         0.0        0.0      320.0      170.0     2865.0       0.0   \n",
       "22145    20517.0    20261.0    22832.0    24234.0    24928.0    3000.0   \n",
       "10270    48332.0    46043.0    18759.0    19161.0    19516.0      10.0   \n",
       "28740     3930.0     5859.0    10299.0    11825.0    13121.0       0.0   \n",
       "19310        0.0        0.0        0.0        0.0        0.0       0.0   \n",
       "4087    138473.0   141297.0   144605.0   148420.0   150500.0    4000.0   \n",
       "13760        0.0        0.0        0.0        0.0        0.0       0.0   \n",
       "27714    40688.0    40880.0    37851.0    37816.0    39849.0    6000.0   \n",
       "7584     89264.0    89896.0    86706.0    87385.0    88351.0    8000.0   \n",
       "29650   310546.0   288566.0   291597.0   214353.0   216238.0   12000.0   \n",
       "9472    220405.0   226731.0   231456.0   237707.0   243555.0       0.0   \n",
       "7022     60842.0    61749.0    65130.0    63816.0    65011.0    2800.0   \n",
       "21323        0.0      191.0     2160.0        0.0      900.0       0.0   \n",
       "23409   194734.0   190447.0   187224.0   183641.0   178239.0    7003.0   \n",
       "3884       780.0      390.0      780.0      780.0        0.0     780.0   \n",
       "9456     62593.0   152141.0   125325.0   109142.0   103020.0   62635.0   \n",
       "15820    24584.0    25885.0    26851.0    26808.0    27599.0    1400.0   \n",
       "13152    44282.0    23946.0        0.0        0.0        0.0    2000.0   \n",
       "10707    22256.0    21660.0    22055.0    22603.0    23133.0    5210.0   \n",
       "1541         0.0        0.0        0.0        0.0        0.0       0.0   \n",
       "23062    22846.0    25779.0    22303.0    16981.0    16837.0    1450.0   \n",
       "11118     1879.0     1707.0      617.0      620.0        0.0    1879.0   \n",
       "...          ...        ...        ...        ...        ...       ...   \n",
       "3063     44694.0    43673.0    44659.0    49512.0    48454.0    2000.0   \n",
       "21234    51441.0    54107.0    55111.0    56688.0    58272.0    2000.0   \n",
       "2459     51803.0     4997.0        0.0        0.0        0.0   51811.0   \n",
       "24360     -342.0     -342.0     -342.0     1158.0     2485.0       0.0   \n",
       "9048         0.0        0.0        0.0        0.0        0.0       0.0   \n",
       "25080    15297.0     8531.0     8020.0     -620.0     -620.0    1129.0   \n",
       "6886       390.0      911.0     1080.0      930.0      390.0     390.0   \n",
       "18264    74156.0    75659.0    72046.0    70355.0    71208.0    3293.0   \n",
       "17546    32420.0    33462.0    36199.0    36803.0    37566.0    1521.0   \n",
       "12326   136579.0   139281.0   132487.0   107437.0   109237.0    5500.0   \n",
       "2440     69771.0    73233.0    72328.0    74234.0    75500.0    3509.0   \n",
       "8905     47887.0    46836.0    45785.0    47003.0    48085.0    6000.0   \n",
       "5739     98583.0    95810.0    87216.0    88201.0    87483.0    5000.0   \n",
       "360     509229.0   499936.0   505507.0   484612.0   419643.0   10000.0   \n",
       "4816     20394.0    19597.0    17830.0    18397.0    19064.0    2006.0   \n",
       "15178    29305.0    30336.0    28800.0     9852.0     2160.0    1505.0   \n",
       "18321    38022.0     5785.0     9233.0     4893.0     2168.0   38024.0   \n",
       "23609    50411.0     9688.0      857.0     3810.0     4741.0    2500.0   \n",
       "16230    50067.0    69594.0    46181.0    16770.0    19890.0    1743.0   \n",
       "21927     6393.0     6969.0     9044.0     2325.0     3998.0    6458.0   \n",
       "27267    13613.0    14634.0    16532.0    15969.0    17701.0       0.0   \n",
       "11593    25607.0    23922.0    24399.0    24919.0    26884.0   27107.0   \n",
       "15732     6425.0     4580.0        0.0        0.0        0.0       0.0   \n",
       "25950      388.0      388.0      388.0   110345.0   116128.0     388.0   \n",
       "5863     28785.0    13104.0     2621.0     9075.0     6564.0   28791.0   \n",
       "22847    45746.0    46979.0    42045.0    40334.0    49542.0    2000.0   \n",
       "17496    49884.0    22348.0    22063.0     8723.0     7254.0    2158.0   \n",
       "2145    166176.0   169759.0    11760.0        0.0     4902.0   14000.0   \n",
       "15305    89589.0    92887.0        0.0        0.0        0.0    8000.0   \n",
       "25509    57280.0    56969.0    58695.0    57453.0    61153.0    2100.0   \n",
       "\n",
       "       pay_amt2  pay_amt3  pay_amt4  pay_amt5  pay_amt6  pred_bill_amt0  \\\n",
       "10574       0.0       0.0       0.0       0.0       0.0    19269.757474   \n",
       "11330   15056.0    5027.0    5007.0    5063.0    3039.0   285123.402526   \n",
       "7127    48276.0   49450.0     201.0   54379.0    9905.0    34020.039927   \n",
       "25150    3700.0    2600.0    1200.0    1200.0    1000.0    70345.498357   \n",
       "6000     1284.0     264.0     109.0     390.0       0.0    13480.622867   \n",
       "20951    3112.0    3351.0    1259.0   19271.0   16992.0    11373.778043   \n",
       "13894    6163.0    4728.0    3811.0    3830.0    3970.0   145183.779419   \n",
       "15285    1640.0    1152.0     335.0     460.0     553.0      976.657948   \n",
       "6881        0.0     329.0       0.0    2865.0     801.0    25998.069408   \n",
       "22145    3000.0    3000.0    2000.0    2000.0    2000.0    23267.364751   \n",
       "10270    2000.0     655.0     688.0     666.0     649.0    49408.217276   \n",
       "28740    2000.0    5008.0    3007.0    2000.0    2000.0     4142.519487   \n",
       "19310       0.0       0.0       0.0       0.0       0.0        0.000000   \n",
       "4087     4000.0    4500.0    5000.0    3500.0       0.0   135625.391520   \n",
       "13760       0.0       0.0       0.0       0.0       0.0     7079.795960   \n",
       "27714    6000.0    4000.0   10000.0    5000.0    8002.0    44874.323218   \n",
       "7584     3000.0       0.0    3000.0    5000.0    5000.0    83556.053289   \n",
       "29650   11000.0   11000.0    8000.0    8500.0    8300.0   333899.422605   \n",
       "9472    10000.0   10000.0   10000.0   10000.0   10000.0   225351.300676   \n",
       "7022     2500.0    5000.0       0.0    2400.0    2400.0    59602.656941   \n",
       "21323     191.0    2160.0       0.0     900.0    2000.0        0.037329   \n",
       "23409    7000.0    7000.0    6510.0    6550.0    6001.0   198543.553379   \n",
       "3884        0.0     780.0       0.0       0.0       0.0      395.054847   \n",
       "9456   100042.0    4017.0    9184.0    4081.0    6042.0     3746.596855   \n",
       "15820    1700.0    1900.0     557.0    1000.0    2000.0    23588.476014   \n",
       "13152    2000.0       0.0       0.0       0.0       0.0    43513.615983   \n",
       "10707       0.0    1000.0    1000.0    1000.0    1000.0    17359.055717   \n",
       "1541        0.0       0.0       0.0       0.0       0.0        0.000000   \n",
       "23062    5779.0    1300.0     700.0    1000.0    1000.0    23357.695692   \n",
       "11118       8.0     617.0       3.0       0.0       0.0      184.514338   \n",
       "...         ...       ...       ...       ...       ...             ...   \n",
       "3063        0.0    2000.0    6000.0     300.0    3000.0    43698.075819   \n",
       "21234    3500.0    3000.0    2500.0    2500.0    2500.0    50296.694346   \n",
       "2459     9544.0       0.0       0.0       0.0       0.0    37321.501701   \n",
       "24360       0.0       0.0    1500.0    1500.0       0.0     -341.999955   \n",
       "9048        0.0       0.0       0.0       0.0       0.0      842.785859   \n",
       "25080    1152.0    1081.0       0.0       0.0       0.0    14485.612723   \n",
       "6886      911.0     559.0     390.0       0.0     390.0      390.090215   \n",
       "18264    3350.0    3111.0    2615.0    2689.0    2766.0    77365.547903   \n",
       "17546    1563.0    3300.0    1500.0    1500.0    1600.0    31435.290939   \n",
       "12326    5036.0    4000.0    6200.0    5500.0    5000.0   133274.426549   \n",
       "2440     5824.0       0.0    2204.0    2000.0       0.0    67949.835369   \n",
       "8905        0.0       0.0    2100.0    2000.0       0.0    42948.525522   \n",
       "5739     4000.0    4000.0    3290.0    3200.0    3215.0   112209.185246   \n",
       "360         0.0   14000.0   16500.0   16000.0   19000.0   507744.145924   \n",
       "4816     1519.0    1000.0    1000.0    1000.0    1000.0    39761.156568   \n",
       "15178    1536.0    1000.0    9852.0    2160.0     400.0    28289.685854   \n",
       "18321    5785.0    9233.0    4893.0    2168.0    3625.0     4100.732845   \n",
       "23609    7026.0    1000.0    3000.0    1000.0    1000.0    43964.639987   \n",
       "16230    2236.0    2807.0     629.0    3810.0     719.0    49461.415202   \n",
       "21927    6980.0    9044.0    2325.0    4997.0    9947.0     3882.509880   \n",
       "27267    1247.0    2145.0       0.0    2000.0       0.0    14130.299526   \n",
       "11593    1500.0     863.0     903.0    3000.0    6186.0     -670.641976   \n",
       "15732    1050.0       0.0       0.0       0.0       0.0    27093.649272   \n",
       "25950     388.0     388.0  110345.0    8000.0    4200.0      388.003271   \n",
       "5863    13227.0    2621.0    9075.0    6564.0   12767.0    36722.618013   \n",
       "22847    2000.0    2000.0    2000.0   13000.0    2000.0    67648.765025   \n",
       "17496    1708.0    1411.0       4.0      21.0     220.0    52028.650887   \n",
       "2145     5695.0   11760.0       0.0    4902.0    6000.0   171033.627401   \n",
       "15305    5000.0       0.0       0.0       0.0       0.0    83099.709874   \n",
       "25509    2100.0    2685.0    2300.0    4800.0    3400.0    56150.972468   \n",
       "\n",
       "       gender_2  gender_1  education_3  education_2  education_1  education_4  \\\n",
       "10574         1         0            1            0            0            0   \n",
       "11330         0         1            0            1            0            0   \n",
       "7127          0         1            0            1            0            0   \n",
       "25150         1         0            0            1            0            0   \n",
       "6000          0         1            0            1            0            0   \n",
       "20951         0         1            0            0            1            0   \n",
       "13894         1         0            0            1            0            0   \n",
       "15285         1         0            0            0            1            0   \n",
       "6881          1         0            0            0            1            0   \n",
       "22145         1         0            0            0            1            0   \n",
       "10270         0         1            0            1            0            0   \n",
       "28740         1         0            0            0            1            0   \n",
       "19310         1         0            0            0            1            0   \n",
       "4087          0         1            0            1            0            0   \n",
       "13760         0         1            0            0            0            1   \n",
       "27714         1         0            0            1            0            0   \n",
       "7584          1         0            0            1            0            0   \n",
       "29650         0         1            0            1            0            0   \n",
       "9472          1         0            0            1            0            0   \n",
       "7022          0         1            0            1            0            0   \n",
       "21323         0         1            0            0            1            0   \n",
       "23409         1         0            0            0            0            1   \n",
       "3884          0         1            0            1            0            0   \n",
       "9456          1         0            0            0            1            0   \n",
       "15820         1         0            0            0            1            0   \n",
       "13152         1         0            0            0            1            0   \n",
       "10707         1         0            0            1            0            0   \n",
       "1541          0         1            1            0            0            0   \n",
       "23062         1         0            0            0            1            0   \n",
       "11118         0         1            0            1            0            0   \n",
       "...         ...       ...          ...          ...          ...          ...   \n",
       "3063          0         1            1            0            0            0   \n",
       "21234         0         1            0            1            0            0   \n",
       "2459          1         0            0            1            0            0   \n",
       "24360         0         1            0            1            0            0   \n",
       "9048          1         0            0            1            0            0   \n",
       "25080         0         1            0            0            1            0   \n",
       "6886          1         0            1            0            0            0   \n",
       "18264         0         1            1            0            0            0   \n",
       "17546         1         0            1            0            0            0   \n",
       "12326         1         0            0            0            1            0   \n",
       "2440          1         0            0            0            1            0   \n",
       "8905          0         1            0            1            0            0   \n",
       "5739          1         0            1            0            0            0   \n",
       "360           1         0            0            1            0            0   \n",
       "4816          1         0            0            1            0            0   \n",
       "15178         1         0            0            1            0            0   \n",
       "18321         1         0            0            0            1            0   \n",
       "23609         1         0            0            0            1            0   \n",
       "16230         1         0            0            1            0            0   \n",
       "21927         1         0            0            0            1            0   \n",
       "27267         0         1            0            0            1            0   \n",
       "11593         1         0            0            1            0            0   \n",
       "15732         1         0            0            1            0            0   \n",
       "25950         1         0            1            0            0            0   \n",
       "5863          0         1            0            1            0            0   \n",
       "22847         1         0            0            0            1            0   \n",
       "17496         1         0            0            1            0            0   \n",
       "2145          0         1            0            0            0            1   \n",
       "15305         1         0            0            1            0            0   \n",
       "25509         1         0            0            0            1            0   \n",
       "\n",
       "       marriage_1  marriage_2  marriage_3  age_bin_20s  age_bin_30s  \\\n",
       "10574           1           0           0            0            0   \n",
       "11330           0           1           0            0            1   \n",
       "7127            0           1           0            0            0   \n",
       "25150           1           0           0            0            0   \n",
       "6000            1           0           0            0            1   \n",
       "20951           1           0           0            0            0   \n",
       "13894           1           0           0            0            1   \n",
       "15285           1           0           0            0            1   \n",
       "6881            1           0           0            0            1   \n",
       "22145           0           1           0            1            0   \n",
       "10270           0           1           0            0            0   \n",
       "28740           0           1           0            1            0   \n",
       "19310           0           1           0            0            1   \n",
       "4087            0           1           0            0            1   \n",
       "13760           0           1           0            0            0   \n",
       "27714           0           1           0            1            0   \n",
       "7584            0           1           0            1            0   \n",
       "29650           0           1           0            0            1   \n",
       "9472            0           1           0            1            0   \n",
       "7022            1           0           0            0            0   \n",
       "21323           1           0           0            0            0   \n",
       "23409           1           0           0            0            0   \n",
       "3884            0           1           0            1            0   \n",
       "9456            1           0           0            0            1   \n",
       "15820           0           1           0            1            0   \n",
       "13152           1           0           0            0            1   \n",
       "10707           1           0           0            0            1   \n",
       "1541            0           1           0            1            0   \n",
       "23062           1           0           0            0            1   \n",
       "11118           1           0           0            0            1   \n",
       "...           ...         ...         ...          ...          ...   \n",
       "3063            0           1           0            1            0   \n",
       "21234           0           1           0            0            1   \n",
       "2459            1           0           0            0            0   \n",
       "24360           0           1           0            0            1   \n",
       "9048            0           1           0            1            0   \n",
       "25080           1           0           0            0            0   \n",
       "6886            0           0           1            0            0   \n",
       "18264           0           1           0            0            0   \n",
       "17546           1           0           0            0            0   \n",
       "12326           0           1           0            1            0   \n",
       "2440            0           1           0            0            0   \n",
       "8905            0           1           0            1            0   \n",
       "5739            1           0           0            0            1   \n",
       "360             0           1           0            1            0   \n",
       "4816            0           1           0            0            1   \n",
       "15178           0           1           0            1            0   \n",
       "18321           0           1           0            0            1   \n",
       "23609           0           1           0            0            1   \n",
       "16230           1           0           0            0            1   \n",
       "21927           0           1           0            1            0   \n",
       "27267           1           0           0            0            0   \n",
       "11593           0           1           0            1            0   \n",
       "15732           1           0           0            1            0   \n",
       "25950           1           0           0            0            1   \n",
       "5863            0           1           0            0            0   \n",
       "22847           0           1           0            0            1   \n",
       "17496           1           0           0            1            0   \n",
       "2145            1           0           0            1            0   \n",
       "15305           1           0           0            0            1   \n",
       "25509           0           1           0            1            0   \n",
       "\n",
       "       age_bin_40s  age_bin_50s  age_bin_60+  gen_ageBin_(1, '20s')  \\\n",
       "10574            0            1            0                      0   \n",
       "11330            0            0            0                      0   \n",
       "7127             1            0            0                      0   \n",
       "25150            1            0            0                      0   \n",
       "6000             0            0            0                      0   \n",
       "20951            1            0            0                      0   \n",
       "13894            0            0            0                      0   \n",
       "15285            0            0            0                      0   \n",
       "6881             0            0            0                      0   \n",
       "22145            0            0            0                      0   \n",
       "10270            1            0            0                      0   \n",
       "28740            0            0            0                      0   \n",
       "19310            0            0            0                      0   \n",
       "4087             0            0            0                      0   \n",
       "13760            0            1            0                      0   \n",
       "27714            0            0            0                      0   \n",
       "7584             0            0            0                      0   \n",
       "29650            0            0            0                      0   \n",
       "9472             0            0            0                      0   \n",
       "7022             1            0            0                      0   \n",
       "21323            0            1            0                      0   \n",
       "23409            1            0            0                      0   \n",
       "3884             0            0            0                      1   \n",
       "9456             0            0            0                      0   \n",
       "15820            0            0            0                      0   \n",
       "13152            0            0            0                      0   \n",
       "10707            0            0            0                      0   \n",
       "1541             0            0            0                      1   \n",
       "23062            0            0            0                      0   \n",
       "11118            0            0            0                      0   \n",
       "...            ...          ...          ...                    ...   \n",
       "3063             0            0            0                      1   \n",
       "21234            0            0            0                      0   \n",
       "2459             1            0            0                      0   \n",
       "24360            0            0            0                      0   \n",
       "9048             0            0            0                      0   \n",
       "25080            0            1            0                      0   \n",
       "6886             0            1            0                      0   \n",
       "18264            0            1            0                      0   \n",
       "17546            1            0            0                      0   \n",
       "12326            0            0            0                      0   \n",
       "2440             1            0            0                      0   \n",
       "8905             0            0            0                      1   \n",
       "5739             0            0            0                      0   \n",
       "360              0            0            0                      0   \n",
       "4816             0            0            0                      0   \n",
       "15178            0            0            0                      0   \n",
       "18321            0            0            0                      0   \n",
       "23609            0            0            0                      0   \n",
       "16230            0            0            0                      0   \n",
       "21927            0            0            0                      0   \n",
       "27267            1            0            0                      0   \n",
       "11593            0            0            0                      0   \n",
       "15732            0            0            0                      0   \n",
       "25950            0            0            0                      0   \n",
       "5863             1            0            0                      0   \n",
       "22847            0            0            0                      0   \n",
       "17496            0            0            0                      0   \n",
       "2145             0            0            0                      1   \n",
       "15305            0            0            0                      0   \n",
       "25509            0            0            0                      0   \n",
       "\n",
       "       gen_ageBin_(1, '30s')  gen_ageBin_(1, '40s')  gen_ageBin_(1, '50s')  \\\n",
       "10574                      0                      0                      0   \n",
       "11330                      1                      0                      0   \n",
       "7127                       0                      1                      0   \n",
       "25150                      0                      0                      0   \n",
       "6000                       1                      0                      0   \n",
       "20951                      0                      1                      0   \n",
       "13894                      0                      0                      0   \n",
       "15285                      0                      0                      0   \n",
       "6881                       0                      0                      0   \n",
       "22145                      0                      0                      0   \n",
       "10270                      0                      1                      0   \n",
       "28740                      0                      0                      0   \n",
       "19310                      0                      0                      0   \n",
       "4087                       1                      0                      0   \n",
       "13760                      0                      0                      1   \n",
       "27714                      0                      0                      0   \n",
       "7584                       0                      0                      0   \n",
       "29650                      1                      0                      0   \n",
       "9472                       0                      0                      0   \n",
       "7022                       0                      1                      0   \n",
       "21323                      0                      0                      1   \n",
       "23409                      0                      0                      0   \n",
       "3884                       0                      0                      0   \n",
       "9456                       0                      0                      0   \n",
       "15820                      0                      0                      0   \n",
       "13152                      0                      0                      0   \n",
       "10707                      0                      0                      0   \n",
       "1541                       0                      0                      0   \n",
       "23062                      0                      0                      0   \n",
       "11118                      1                      0                      0   \n",
       "...                      ...                    ...                    ...   \n",
       "3063                       0                      0                      0   \n",
       "21234                      1                      0                      0   \n",
       "2459                       0                      0                      0   \n",
       "24360                      1                      0                      0   \n",
       "9048                       0                      0                      0   \n",
       "25080                      0                      0                      1   \n",
       "6886                       0                      0                      0   \n",
       "18264                      0                      0                      1   \n",
       "17546                      0                      0                      0   \n",
       "12326                      0                      0                      0   \n",
       "2440                       0                      0                      0   \n",
       "8905                       0                      0                      0   \n",
       "5739                       0                      0                      0   \n",
       "360                        0                      0                      0   \n",
       "4816                       0                      0                      0   \n",
       "15178                      0                      0                      0   \n",
       "18321                      0                      0                      0   \n",
       "23609                      0                      0                      0   \n",
       "16230                      0                      0                      0   \n",
       "21927                      0                      0                      0   \n",
       "27267                      0                      1                      0   \n",
       "11593                      0                      0                      0   \n",
       "15732                      0                      0                      0   \n",
       "25950                      0                      0                      0   \n",
       "5863                       0                      1                      0   \n",
       "22847                      0                      0                      0   \n",
       "17496                      0                      0                      0   \n",
       "2145                       0                      0                      0   \n",
       "15305                      0                      0                      0   \n",
       "25509                      0                      0                      0   \n",
       "\n",
       "       gen_ageBin_(1, '60+')  gen_ageBin_(2, '20s')  gen_ageBin_(2, '30s')  \\\n",
       "10574                      0                      0                      0   \n",
       "11330                      0                      0                      0   \n",
       "7127                       0                      0                      0   \n",
       "25150                      0                      0                      0   \n",
       "6000                       0                      0                      0   \n",
       "20951                      0                      0                      0   \n",
       "13894                      0                      0                      1   \n",
       "15285                      0                      0                      1   \n",
       "6881                       0                      0                      1   \n",
       "22145                      0                      1                      0   \n",
       "10270                      0                      0                      0   \n",
       "28740                      0                      1                      0   \n",
       "19310                      0                      0                      1   \n",
       "4087                       0                      0                      0   \n",
       "13760                      0                      0                      0   \n",
       "27714                      0                      1                      0   \n",
       "7584                       0                      1                      0   \n",
       "29650                      0                      0                      0   \n",
       "9472                       0                      1                      0   \n",
       "7022                       0                      0                      0   \n",
       "21323                      0                      0                      0   \n",
       "23409                      0                      0                      0   \n",
       "3884                       0                      0                      0   \n",
       "9456                       0                      0                      1   \n",
       "15820                      0                      1                      0   \n",
       "13152                      0                      0                      1   \n",
       "10707                      0                      0                      1   \n",
       "1541                       0                      0                      0   \n",
       "23062                      0                      0                      1   \n",
       "11118                      0                      0                      0   \n",
       "...                      ...                    ...                    ...   \n",
       "3063                       0                      0                      0   \n",
       "21234                      0                      0                      0   \n",
       "2459                       0                      0                      0   \n",
       "24360                      0                      0                      0   \n",
       "9048                       0                      1                      0   \n",
       "25080                      0                      0                      0   \n",
       "6886                       0                      0                      0   \n",
       "18264                      0                      0                      0   \n",
       "17546                      0                      0                      0   \n",
       "12326                      0                      1                      0   \n",
       "2440                       0                      0                      0   \n",
       "8905                       0                      0                      0   \n",
       "5739                       0                      0                      1   \n",
       "360                        0                      1                      0   \n",
       "4816                       0                      0                      1   \n",
       "15178                      0                      1                      0   \n",
       "18321                      0                      0                      1   \n",
       "23609                      0                      0                      1   \n",
       "16230                      0                      0                      1   \n",
       "21927                      0                      1                      0   \n",
       "27267                      0                      0                      0   \n",
       "11593                      0                      1                      0   \n",
       "15732                      0                      1                      0   \n",
       "25950                      0                      0                      1   \n",
       "5863                       0                      0                      0   \n",
       "22847                      0                      0                      1   \n",
       "17496                      0                      1                      0   \n",
       "2145                       0                      0                      0   \n",
       "15305                      0                      0                      1   \n",
       "25509                      0                      1                      0   \n",
       "\n",
       "       gen_ageBin_(2, '40s')  gen_ageBin_(2, '50s')  gen_ageBin_(2, '60+')  \\\n",
       "10574                      0                      1                      0   \n",
       "11330                      0                      0                      0   \n",
       "7127                       0                      0                      0   \n",
       "25150                      1                      0                      0   \n",
       "6000                       0                      0                      0   \n",
       "20951                      0                      0                      0   \n",
       "13894                      0                      0                      0   \n",
       "15285                      0                      0                      0   \n",
       "6881                       0                      0                      0   \n",
       "22145                      0                      0                      0   \n",
       "10270                      0                      0                      0   \n",
       "28740                      0                      0                      0   \n",
       "19310                      0                      0                      0   \n",
       "4087                       0                      0                      0   \n",
       "13760                      0                      0                      0   \n",
       "27714                      0                      0                      0   \n",
       "7584                       0                      0                      0   \n",
       "29650                      0                      0                      0   \n",
       "9472                       0                      0                      0   \n",
       "7022                       0                      0                      0   \n",
       "21323                      0                      0                      0   \n",
       "23409                      1                      0                      0   \n",
       "3884                       0                      0                      0   \n",
       "9456                       0                      0                      0   \n",
       "15820                      0                      0                      0   \n",
       "13152                      0                      0                      0   \n",
       "10707                      0                      0                      0   \n",
       "1541                       0                      0                      0   \n",
       "23062                      0                      0                      0   \n",
       "11118                      0                      0                      0   \n",
       "...                      ...                    ...                    ...   \n",
       "3063                       0                      0                      0   \n",
       "21234                      0                      0                      0   \n",
       "2459                       1                      0                      0   \n",
       "24360                      0                      0                      0   \n",
       "9048                       0                      0                      0   \n",
       "25080                      0                      0                      0   \n",
       "6886                       0                      1                      0   \n",
       "18264                      0                      0                      0   \n",
       "17546                      1                      0                      0   \n",
       "12326                      0                      0                      0   \n",
       "2440                       1                      0                      0   \n",
       "8905                       0                      0                      0   \n",
       "5739                       0                      0                      0   \n",
       "360                        0                      0                      0   \n",
       "4816                       0                      0                      0   \n",
       "15178                      0                      0                      0   \n",
       "18321                      0                      0                      0   \n",
       "23609                      0                      0                      0   \n",
       "16230                      0                      0                      0   \n",
       "21927                      0                      0                      0   \n",
       "27267                      0                      0                      0   \n",
       "11593                      0                      0                      0   \n",
       "15732                      0                      0                      0   \n",
       "25950                      0                      0                      0   \n",
       "5863                       0                      0                      0   \n",
       "22847                      0                      0                      0   \n",
       "17496                      0                      0                      0   \n",
       "2145                       0                      0                      0   \n",
       "15305                      0                      0                      0   \n",
       "25509                      0                      0                      0   \n",
       "\n",
       "       gen_mar_(1, 1)  gen_mar_(1, 2)  gen_mar_(1, 3)  gen_mar_(2, 1)  \\\n",
       "10574               0               0               0               1   \n",
       "11330               0               1               0               0   \n",
       "7127                0               1               0               0   \n",
       "25150               0               0               0               1   \n",
       "6000                1               0               0               0   \n",
       "20951               1               0               0               0   \n",
       "13894               0               0               0               1   \n",
       "15285               0               0               0               1   \n",
       "6881                0               0               0               1   \n",
       "22145               0               0               0               0   \n",
       "10270               0               1               0               0   \n",
       "28740               0               0               0               0   \n",
       "19310               0               0               0               0   \n",
       "4087                0               1               0               0   \n",
       "13760               0               1               0               0   \n",
       "27714               0               0               0               0   \n",
       "7584                0               0               0               0   \n",
       "29650               0               1               0               0   \n",
       "9472                0               0               0               0   \n",
       "7022                1               0               0               0   \n",
       "21323               1               0               0               0   \n",
       "23409               0               0               0               1   \n",
       "3884                0               1               0               0   \n",
       "9456                0               0               0               1   \n",
       "15820               0               0               0               0   \n",
       "13152               0               0               0               1   \n",
       "10707               0               0               0               1   \n",
       "1541                0               1               0               0   \n",
       "23062               0               0               0               1   \n",
       "11118               1               0               0               0   \n",
       "...               ...             ...             ...             ...   \n",
       "3063                0               1               0               0   \n",
       "21234               0               1               0               0   \n",
       "2459                0               0               0               1   \n",
       "24360               0               1               0               0   \n",
       "9048                0               0               0               0   \n",
       "25080               1               0               0               0   \n",
       "6886                0               0               0               0   \n",
       "18264               0               1               0               0   \n",
       "17546               0               0               0               1   \n",
       "12326               0               0               0               0   \n",
       "2440                0               0               0               0   \n",
       "8905                0               1               0               0   \n",
       "5739                0               0               0               1   \n",
       "360                 0               0               0               0   \n",
       "4816                0               0               0               0   \n",
       "15178               0               0               0               0   \n",
       "18321               0               0               0               0   \n",
       "23609               0               0               0               0   \n",
       "16230               0               0               0               1   \n",
       "21927               0               0               0               0   \n",
       "27267               1               0               0               0   \n",
       "11593               0               0               0               0   \n",
       "15732               0               0               0               1   \n",
       "25950               0               0               0               1   \n",
       "5863                0               1               0               0   \n",
       "22847               0               0               0               0   \n",
       "17496               0               0               0               1   \n",
       "2145                1               0               0               0   \n",
       "15305               0               0               0               1   \n",
       "25509               0               0               0               0   \n",
       "\n",
       "       gen_mar_(2, 2)  gen_mar_(2, 3)  \n",
       "10574               0               0  \n",
       "11330               0               0  \n",
       "7127                0               0  \n",
       "25150               0               0  \n",
       "6000                0               0  \n",
       "20951               0               0  \n",
       "13894               0               0  \n",
       "15285               0               0  \n",
       "6881                0               0  \n",
       "22145               1               0  \n",
       "10270               0               0  \n",
       "28740               1               0  \n",
       "19310               1               0  \n",
       "4087                0               0  \n",
       "13760               0               0  \n",
       "27714               1               0  \n",
       "7584                1               0  \n",
       "29650               0               0  \n",
       "9472                1               0  \n",
       "7022                0               0  \n",
       "21323               0               0  \n",
       "23409               0               0  \n",
       "3884                0               0  \n",
       "9456                0               0  \n",
       "15820               1               0  \n",
       "13152               0               0  \n",
       "10707               0               0  \n",
       "1541                0               0  \n",
       "23062               0               0  \n",
       "11118               0               0  \n",
       "...               ...             ...  \n",
       "3063                0               0  \n",
       "21234               0               0  \n",
       "2459                0               0  \n",
       "24360               0               0  \n",
       "9048                1               0  \n",
       "25080               0               0  \n",
       "6886                0               1  \n",
       "18264               0               0  \n",
       "17546               0               0  \n",
       "12326               1               0  \n",
       "2440                1               0  \n",
       "8905                0               0  \n",
       "5739                0               0  \n",
       "360                 1               0  \n",
       "4816                1               0  \n",
       "15178               1               0  \n",
       "18321               1               0  \n",
       "23609               1               0  \n",
       "16230               0               0  \n",
       "21927               1               0  \n",
       "27267               0               0  \n",
       "11593               1               0  \n",
       "15732               0               0  \n",
       "25950               0               0  \n",
       "5863                0               0  \n",
       "22847               1               0  \n",
       "17496               0               0  \n",
       "2145                0               0  \n",
       "15305               0               0  \n",
       "25509               1               0  \n",
       "\n",
       "[18000 rows x 51 columns]"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dico_pipelined_models_num[2]['log'].steps[0][1].fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc-autonumbering": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}