(Advanced) Amazon SageMaker AutoPilot (Hyperparameter tuning).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (Advanced) Amazon SageMaker AutoPilot (Hyperparameter tuning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data load / set-up procedue is identical with the basic SageMaker AutoPilot Model Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please refer to the other notebook (Amazon SageMaker AutoPilot (model building to deploy)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.64.0\n"
     ]
    }
   ],
   "source": [
    "import sagemaker\n",
    "import smdebug_rulesconfig as rule_configs\n",
    "import boto3\n",
    "import os\n",
    "import sys\n",
    "import numpy as np \n",
    "import pandas as pd\n",
    "\n",
    "print(sagemaker.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sagemaker session : <sagemaker.session.Session object at 0x000002289785B908>\n",
      "S3 bucket : hyunjoon-sagemaker-us\n",
      "Prefix : sagemaker/autopilot\n",
      "Region selected : us-west-2\n",
      "IAM role : arn:aws:iam::570447867175:role/SageMakerNotebookRole\n"
     ]
    }
   ],
   "source": [
    "sess   = sagemaker.Session()\n",
    "bucket = sess.default_bucket()                     \n",
    "prefix = 'sagemaker/autopilot'\n",
    "region = boto3.Session().region_name\n",
    "role = 'arn:aws:iam::570447867175:role/SageMakerNotebookRole' # pass your IAM role name\n",
    "\n",
    "print('Sagemaker session :', sess)\n",
    "print('S3 bucket :', bucket)\n",
    "print('Prefix :', prefix)\n",
    "print('Region selected :', region)\n",
    "print('IAM role :', role)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the direct marketing dataset from UCI's ML Repository.\n",
    "The data is related with direct marketing campaigns of a Portuguese banking institution. The marketing campaigns were based on phone calls. Often, more than one contact to the same client was required, in order to access if the product (bank term deposit) would be ('yes') or not ('no') subscribed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Download Dataset Manually here](https://archive.ics.uci.edu/ml/datasets/bank+marketing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or try:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Wget Download\n",
    "!wget -N https://archive.ics.uci.edu/ml/machine-learning-databases/00222/bank-additional.zip \\\n",
    "!unzip -o bank-additional.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>job</th>\n",
       "      <th>marital</th>\n",
       "      <th>education</th>\n",
       "      <th>default</th>\n",
       "      <th>housing</th>\n",
       "      <th>loan</th>\n",
       "      <th>contact</th>\n",
       "      <th>month</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>duration</th>\n",
       "      <th>campaign</th>\n",
       "      <th>pdays</th>\n",
       "      <th>previous</th>\n",
       "      <th>poutcome</th>\n",
       "      <th>emp.var.rate</th>\n",
       "      <th>cons.price.idx</th>\n",
       "      <th>cons.conf.idx</th>\n",
       "      <th>euribor3m</th>\n",
       "      <th>nr.employed</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>56</td>\n",
       "      <td>housemaid</td>\n",
       "      <td>married</td>\n",
       "      <td>basic.4y</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>telephone</td>\n",
       "      <td>may</td>\n",
       "      <td>mon</td>\n",
       "      <td>261</td>\n",
       "      <td>1</td>\n",
       "      <td>999</td>\n",
       "      <td>0</td>\n",
       "      <td>nonexistent</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>57</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>high.school</td>\n",
       "      <td>unknown</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>telephone</td>\n",
       "      <td>may</td>\n",
       "      <td>mon</td>\n",
       "      <td>149</td>\n",
       "      <td>1</td>\n",
       "      <td>999</td>\n",
       "      <td>0</td>\n",
       "      <td>nonexistent</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>37</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>high.school</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>telephone</td>\n",
       "      <td>may</td>\n",
       "      <td>mon</td>\n",
       "      <td>226</td>\n",
       "      <td>1</td>\n",
       "      <td>999</td>\n",
       "      <td>0</td>\n",
       "      <td>nonexistent</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>40</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>basic.6y</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>telephone</td>\n",
       "      <td>may</td>\n",
       "      <td>mon</td>\n",
       "      <td>151</td>\n",
       "      <td>1</td>\n",
       "      <td>999</td>\n",
       "      <td>0</td>\n",
       "      <td>nonexistent</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>56</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>high.school</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>telephone</td>\n",
       "      <td>may</td>\n",
       "      <td>mon</td>\n",
       "      <td>307</td>\n",
       "      <td>1</td>\n",
       "      <td>999</td>\n",
       "      <td>0</td>\n",
       "      <td>nonexistent</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age        job  marital    education  default housing loan    contact  \\\n",
       "0   56  housemaid  married     basic.4y       no      no   no  telephone   \n",
       "1   57   services  married  high.school  unknown      no   no  telephone   \n",
       "2   37   services  married  high.school       no     yes   no  telephone   \n",
       "3   40     admin.  married     basic.6y       no      no   no  telephone   \n",
       "4   56   services  married  high.school       no      no  yes  telephone   \n",
       "\n",
       "  month day_of_week  duration  campaign  pdays  previous     poutcome  \\\n",
       "0   may         mon       261         1    999         0  nonexistent   \n",
       "1   may         mon       149         1    999         0  nonexistent   \n",
       "2   may         mon       226         1    999         0  nonexistent   \n",
       "3   may         mon       151         1    999         0  nonexistent   \n",
       "4   may         mon       307         1    999         0  nonexistent   \n",
       "\n",
       "   emp.var.rate  cons.price.idx  cons.conf.idx  euribor3m  nr.employed   y  \n",
       "0           1.1          93.994          -36.4      4.857       5191.0  no  \n",
       "1           1.1          93.994          -36.4      4.857       5191.0  no  \n",
       "2           1.1          93.994          -36.4      4.857       5191.0  no  \n",
       "3           1.1          93.994          -36.4      4.857       5191.0  no  \n",
       "4           1.1          93.994          -36.4      4.857       5191.0  no  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.set_option('display.max_columns', 500)     # Make sure we can see all of the columns\n",
    "pd.set_option('display.max_rows', 50)         # Keep the output on one page\n",
    "\n",
    "# Read data\n",
    "path = 'C:\\\\Users\\\\bokhy\\\\Desktop\\\\Python\\\\github\\\\amazon-sagemaker\\\\data'\n",
    "\n",
    "data = pd.read_csv(os.path.join(path, 'bank-additional-full.csv'), sep=';')\n",
    "data[:5] # Show the first 5 lines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. EDA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- We see a class imbalance problem in the response variable (y), which might highly effect the prediction accuracy in training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Positive samples: 4640\n",
      "Negative samples: 36548\n",
      "Ratio: 7.88\n"
     ]
    }
   ],
   "source": [
    "one_class = data[data['y']=='yes']\n",
    "one_class_count = one_class.shape[0]\n",
    "print(\"Positive samples: %d\" % one_class_count)\n",
    "\n",
    "zero_class = data[data['y']=='no']\n",
    "zero_class_count = zero_class.shape[0]\n",
    "print(\"Negative samples: %d\" % zero_class_count)\n",
    "\n",
    "zero_to_one_ratio = zero_class_count/one_class_count\n",
    "print(\"Ratio: %.2f\" % zero_to_one_ratio)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. PreProcess the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Many rows have the value of \"999\" for 'pdays' column. It is very likely that it indicates no contact was made before. So, lets' create a new column called \"no_previous_contact\", then grant it value of \"1\" when pdays is 999 and \"0\" otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 999]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[np.min(data['pdays']), np.max(data['pdays'])] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['no_previous_contact'] = np.where(data['pdays'] == 999, 1, 0)\n",
    "data = data.drop(['pdays'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Generate a new column to show whether the customer is working based on \"job\" column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "admin.           10422\n",
       "blue-collar       9254\n",
       "technician        6743\n",
       "services          3969\n",
       "management        2924\n",
       "retired           1720\n",
       "entrepreneur      1456\n",
       "self-employed     1421\n",
       "housemaid         1060\n",
       "unemployed        1014\n",
       "student            875\n",
       "unknown            330\n",
       "Name: job, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['job'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['not_working'] = np.where(np.in1d(data['job'], ['student', 'retired', 'unemployed']), 1, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    37579\n",
       "1     3609\n",
       "Name: not_working, dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['not_working'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Convert categorical columns to numeric columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>duration</th>\n",
       "      <th>campaign</th>\n",
       "      <th>previous</th>\n",
       "      <th>emp.var.rate</th>\n",
       "      <th>cons.price.idx</th>\n",
       "      <th>cons.conf.idx</th>\n",
       "      <th>euribor3m</th>\n",
       "      <th>nr.employed</th>\n",
       "      <th>no_previous_contact</th>\n",
       "      <th>not_working</th>\n",
       "      <th>job_admin.</th>\n",
       "      <th>job_blue-collar</th>\n",
       "      <th>job_entrepreneur</th>\n",
       "      <th>job_housemaid</th>\n",
       "      <th>job_management</th>\n",
       "      <th>job_retired</th>\n",
       "      <th>job_self-employed</th>\n",
       "      <th>job_services</th>\n",
       "      <th>job_student</th>\n",
       "      <th>job_technician</th>\n",
       "      <th>job_unemployed</th>\n",
       "      <th>job_unknown</th>\n",
       "      <th>marital_divorced</th>\n",
       "      <th>marital_married</th>\n",
       "      <th>marital_single</th>\n",
       "      <th>marital_unknown</th>\n",
       "      <th>education_basic.4y</th>\n",
       "      <th>education_basic.6y</th>\n",
       "      <th>education_basic.9y</th>\n",
       "      <th>education_high.school</th>\n",
       "      <th>education_illiterate</th>\n",
       "      <th>education_professional.course</th>\n",
       "      <th>education_university.degree</th>\n",
       "      <th>education_unknown</th>\n",
       "      <th>default_no</th>\n",
       "      <th>default_unknown</th>\n",
       "      <th>default_yes</th>\n",
       "      <th>housing_no</th>\n",
       "      <th>housing_unknown</th>\n",
       "      <th>housing_yes</th>\n",
       "      <th>loan_no</th>\n",
       "      <th>loan_unknown</th>\n",
       "      <th>loan_yes</th>\n",
       "      <th>contact_cellular</th>\n",
       "      <th>contact_telephone</th>\n",
       "      <th>month_apr</th>\n",
       "      <th>month_aug</th>\n",
       "      <th>month_dec</th>\n",
       "      <th>month_jul</th>\n",
       "      <th>month_jun</th>\n",
       "      <th>month_mar</th>\n",
       "      <th>month_may</th>\n",
       "      <th>month_nov</th>\n",
       "      <th>month_oct</th>\n",
       "      <th>month_sep</th>\n",
       "      <th>day_of_week_fri</th>\n",
       "      <th>day_of_week_mon</th>\n",
       "      <th>day_of_week_thu</th>\n",
       "      <th>day_of_week_tue</th>\n",
       "      <th>day_of_week_wed</th>\n",
       "      <th>poutcome_failure</th>\n",
       "      <th>poutcome_nonexistent</th>\n",
       "      <th>poutcome_success</th>\n",
       "      <th>y_no</th>\n",
       "      <th>y_yes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>56</td>\n",
       "      <td>261</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>57</td>\n",
       "      <td>149</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>37</td>\n",
       "      <td>226</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>40</td>\n",
       "      <td>151</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>56</td>\n",
       "      <td>307</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>45</td>\n",
       "      <td>198</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>59</td>\n",
       "      <td>139</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>41</td>\n",
       "      <td>217</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>24</td>\n",
       "      <td>380</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>25</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.994</td>\n",
       "      <td>-36.4</td>\n",
       "      <td>4.857</td>\n",
       "      <td>5191.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  duration  campaign  previous  emp.var.rate  cons.price.idx  \\\n",
       "0   56       261         1         0           1.1          93.994   \n",
       "1   57       149         1         0           1.1          93.994   \n",
       "2   37       226         1         0           1.1          93.994   \n",
       "3   40       151         1         0           1.1          93.994   \n",
       "4   56       307         1         0           1.1          93.994   \n",
       "5   45       198         1         0           1.1          93.994   \n",
       "6   59       139         1         0           1.1          93.994   \n",
       "7   41       217         1         0           1.1          93.994   \n",
       "8   24       380         1         0           1.1          93.994   \n",
       "9   25        50         1         0           1.1          93.994   \n",
       "\n",
       "   cons.conf.idx  euribor3m  nr.employed  no_previous_contact  not_working  \\\n",
       "0          -36.4      4.857       5191.0                    1            0   \n",
       "1          -36.4      4.857       5191.0                    1            0   \n",
       "2          -36.4      4.857       5191.0                    1            0   \n",
       "3          -36.4      4.857       5191.0                    1            0   \n",
       "4          -36.4      4.857       5191.0                    1            0   \n",
       "5          -36.4      4.857       5191.0                    1            0   \n",
       "6          -36.4      4.857       5191.0                    1            0   \n",
       "7          -36.4      4.857       5191.0                    1            0   \n",
       "8          -36.4      4.857       5191.0                    1            0   \n",
       "9          -36.4      4.857       5191.0                    1            0   \n",
       "\n",
       "   job_admin.  job_blue-collar  job_entrepreneur  job_housemaid  \\\n",
       "0           0                0                 0              1   \n",
       "1           0                0                 0              0   \n",
       "2           0                0                 0              0   \n",
       "3           1                0                 0              0   \n",
       "4           0                0                 0              0   \n",
       "5           0                0                 0              0   \n",
       "6           1                0                 0              0   \n",
       "7           0                1                 0              0   \n",
       "8           0                0                 0              0   \n",
       "9           0                0                 0              0   \n",
       "\n",
       "   job_management  job_retired  job_self-employed  job_services  job_student  \\\n",
       "0               0            0                  0             0            0   \n",
       "1               0            0                  0             1            0   \n",
       "2               0            0                  0             1            0   \n",
       "3               0            0                  0             0            0   \n",
       "4               0            0                  0             1            0   \n",
       "5               0            0                  0             1            0   \n",
       "6               0            0                  0             0            0   \n",
       "7               0            0                  0             0            0   \n",
       "8               0            0                  0             0            0   \n",
       "9               0            0                  0             1            0   \n",
       "\n",
       "   job_technician  job_unemployed  job_unknown  marital_divorced  \\\n",
       "0               0               0            0                 0   \n",
       "1               0               0            0                 0   \n",
       "2               0               0            0                 0   \n",
       "3               0               0            0                 0   \n",
       "4               0               0            0                 0   \n",
       "5               0               0            0                 0   \n",
       "6               0               0            0                 0   \n",
       "7               0               0            0                 0   \n",
       "8               1               0            0                 0   \n",
       "9               0               0            0                 0   \n",
       "\n",
       "   marital_married  marital_single  marital_unknown  education_basic.4y  \\\n",
       "0                1               0                0                   1   \n",
       "1                1               0                0                   0   \n",
       "2                1               0                0                   0   \n",
       "3                1               0                0                   0   \n",
       "4                1               0                0                   0   \n",
       "5                1               0                0                   0   \n",
       "6                1               0                0                   0   \n",
       "7                1               0                0                   0   \n",
       "8                0               1                0                   0   \n",
       "9                0               1                0                   0   \n",
       "\n",
       "   education_basic.6y  education_basic.9y  education_high.school  \\\n",
       "0                   0                   0                      0   \n",
       "1                   0                   0                      1   \n",
       "2                   0                   0                      1   \n",
       "3                   1                   0                      0   \n",
       "4                   0                   0                      1   \n",
       "5                   0                   1                      0   \n",
       "6                   0                   0                      0   \n",
       "7                   0                   0                      0   \n",
       "8                   0                   0                      0   \n",
       "9                   0                   0                      1   \n",
       "\n",
       "   education_illiterate  education_professional.course  \\\n",
       "0                     0                              0   \n",
       "1                     0                              0   \n",
       "2                     0                              0   \n",
       "3                     0                              0   \n",
       "4                     0                              0   \n",
       "5                     0                              0   \n",
       "6                     0                              1   \n",
       "7                     0                              0   \n",
       "8                     0                              1   \n",
       "9                     0                              0   \n",
       "\n",
       "   education_university.degree  education_unknown  default_no  \\\n",
       "0                            0                  0           1   \n",
       "1                            0                  0           0   \n",
       "2                            0                  0           1   \n",
       "3                            0                  0           1   \n",
       "4                            0                  0           1   \n",
       "5                            0                  0           0   \n",
       "6                            0                  0           1   \n",
       "7                            0                  1           0   \n",
       "8                            0                  0           1   \n",
       "9                            0                  0           1   \n",
       "\n",
       "   default_unknown  default_yes  housing_no  housing_unknown  housing_yes  \\\n",
       "0                0            0           1                0            0   \n",
       "1                1            0           1                0            0   \n",
       "2                0            0           0                0            1   \n",
       "3                0            0           1                0            0   \n",
       "4                0            0           1                0            0   \n",
       "5                1            0           1                0            0   \n",
       "6                0            0           1                0            0   \n",
       "7                1            0           1                0            0   \n",
       "8                0            0           0                0            1   \n",
       "9                0            0           0                0            1   \n",
       "\n",
       "   loan_no  loan_unknown  loan_yes  contact_cellular  contact_telephone  \\\n",
       "0        1             0         0                 0                  1   \n",
       "1        1             0         0                 0                  1   \n",
       "2        1             0         0                 0                  1   \n",
       "3        1             0         0                 0                  1   \n",
       "4        0             0         1                 0                  1   \n",
       "5        1             0         0                 0                  1   \n",
       "6        1             0         0                 0                  1   \n",
       "7        1             0         0                 0                  1   \n",
       "8        1             0         0                 0                  1   \n",
       "9        1             0         0                 0                  1   \n",
       "\n",
       "   month_apr  month_aug  month_dec  month_jul  month_jun  month_mar  \\\n",
       "0          0          0          0          0          0          0   \n",
       "1          0          0          0          0          0          0   \n",
       "2          0          0          0          0          0          0   \n",
       "3          0          0          0          0          0          0   \n",
       "4          0          0          0          0          0          0   \n",
       "5          0          0          0          0          0          0   \n",
       "6          0          0          0          0          0          0   \n",
       "7          0          0          0          0          0          0   \n",
       "8          0          0          0          0          0          0   \n",
       "9          0          0          0          0          0          0   \n",
       "\n",
       "   month_may  month_nov  month_oct  month_sep  day_of_week_fri  \\\n",
       "0          1          0          0          0                0   \n",
       "1          1          0          0          0                0   \n",
       "2          1          0          0          0                0   \n",
       "3          1          0          0          0                0   \n",
       "4          1          0          0          0                0   \n",
       "5          1          0          0          0                0   \n",
       "6          1          0          0          0                0   \n",
       "7          1          0          0          0                0   \n",
       "8          1          0          0          0                0   \n",
       "9          1          0          0          0                0   \n",
       "\n",
       "   day_of_week_mon  day_of_week_thu  day_of_week_tue  day_of_week_wed  \\\n",
       "0                1                0                0                0   \n",
       "1                1                0                0                0   \n",
       "2                1                0                0                0   \n",
       "3                1                0                0                0   \n",
       "4                1                0                0                0   \n",
       "5                1                0                0                0   \n",
       "6                1                0                0                0   \n",
       "7                1                0                0                0   \n",
       "8                1                0                0                0   \n",
       "9                1                0                0                0   \n",
       "\n",
       "   poutcome_failure  poutcome_nonexistent  poutcome_success  y_no  y_yes  \n",
       "0                 0                     1                 0     1      0  \n",
       "1                 0                     1                 0     1      0  \n",
       "2                 0                     1                 0     1      0  \n",
       "3                 0                     1                 0     1      0  \n",
       "4                 0                     1                 0     1      0  \n",
       "5                 0                     1                 0     1      0  \n",
       "6                 0                     1                 0     1      0  \n",
       "7                 0                     1                 0     1      0  \n",
       "8                 0                     1                 0     1      0  \n",
       "9                 0                     1                 0     1      0  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_data = pd.get_dummies(data)  # Convert categorical variables to sets of indicators\n",
    "model_data[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Split the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data, validation_data, test_data = np.split(model_data.sample(frac=1, random_state=123), \n",
    "                                                  [int(0.7 * len(model_data)), int(0.9*len(model_data))])  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop the two columns for 'yes' and 'no' and add 'yes' back as first column of the dataframe\n",
    "pd.concat([train_data['y_yes'], train_data.drop(['y_no', 'y_yes'], axis=1)], axis=1).to_csv(os.path.join(path, 'train.csv'), index=False, header=False)\n",
    "pd.concat([validation_data['y_yes'], validation_data.drop(['y_no', 'y_yes'], axis=1)], axis=1).to_csv(os.path.join(path, 'validation.csv'), index=False, header=False)\n",
    "pd.concat([test_data['y_yes'], test_data.drop(['y_no', 'y_yes'], axis=1)], axis=1).to_csv(os.path.join(path, 'test.csv'), index=False, header=False)\n",
    "\n",
    "# Dropping the target value, as we will use this CSV file for batch transform\n",
    "test_data.drop(['y_no', 'y_yes'], axis=1).to_csv('test.csv', index=False, header=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Upload data to S3 bucket"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First upload data to the specifed S3 bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'train/train.csv')).upload_file(os.path.join(path, 'train.csv'))\n",
    "boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'validation/validation.csv')).upload_file(os.path.join(path, 'validation.csv'))\n",
    "boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'test/test.csv')).upload_file(os.path.join(path, 'test.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, SageMaker needs to know where the training and validation sets are located, so let's define that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s3://sagemaker-us-west-2-570447867175/sagemaker/autopilot/input/train.csv\n"
     ]
    }
   ],
   "source": [
    "s3_input_train = sess.upload_data(path=\"./data/train.csv\", key_prefix=prefix + \"/input\")\n",
    "s3_input_validation = sess.upload_data(path=\"./data/validation.csv\", key_prefix=prefix + \"/input\")\n",
    "print(s3_input_train)\n",
    "\n",
    "s3_data = {'train': s3_input_train, 'validation': s3_input_validation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:sagemaker:'s3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2.\n",
      "WARNING:sagemaker:'s3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2.\n"
     ]
    }
   ],
   "source": [
    "s3_input_train = sagemaker.s3_input(s3_data='s3://{}/{}/train'.format(bucket, prefix), content_type='csv')\n",
    "s3_input_validation = sagemaker.s3_input(s3_data='s3://{}/{}/validation/'.format(bucket, prefix), content_type='csv')\n",
    "s3_data = {'train': s3_input_train, 'validation': s3_input_validation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Launching an Automatic Model Tuning job (Select Algorithm / Hyperparameter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should be able to check live-time in your AWS console. Please nagivate to:\n",
    "### AWS Console --> Training --> Training Jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:sagemaker.amazon.amazon_estimator:'get_image_uri' method will be deprecated in favor of 'ImageURIProvider' class in SageMaker Python SDK v2.\n",
      "WARNING:root:Parameter image_name will be renamed to image_uri in SageMaker Python SDK v2.\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "\n",
    "from sagemaker.estimator import Estimator\n",
    "# https://sagemaker.readthedocs.io/en/stable/estimators.html\n",
    "\n",
    "# Specify Docker Contatiner\n",
    "container = get_image_uri(region, 'xgboost', repo_version='1.0-1')\n",
    "\n",
    "xgb = Estimator(\n",
    "    \n",
    "    container,                                               # The contatiner that contatins algorithm (XGBoost)\n",
    "    role=role,                                               # Your IAM role for Sagemaker\n",
    "    sagemaker_session=sess,                                  # session\n",
    "                                    \n",
    "    input_mode='File',                                       # Copy the dataset and then train\n",
    "    output_path='s3://{}/{}/output'.format(bucket, prefix),  # Save the model here\n",
    "                                    \n",
    "    train_instance_count=1,                                  # Instance requirements\n",
    "    train_instance_type='ml.m4.xlarge',                      # Instance type\n",
    "                                    \n",
    "    train_use_spot_instances=True,                           # Use a spot instance\n",
    "    train_max_run=300,                                       # Max training time\n",
    "    train_max_wait=600,                                      # Max training time + spot waiting time\n",
    ")\n",
    "\n",
    "xgb.set_hyperparameters(objective='binary:logistic', \n",
    "                        num_round=100,\n",
    "                        early_stopping_rounds=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use SageMaker tuning to automate the searching process effectively. Specifically, we specify a range, or a list of possible values in the case of categorical hyperparameters, for each of the hyperparameter that we plan to tune. SageMaker hyperparameter tuning will automatically launch multiple training jobs with different hyperparameter settings, evaluate results of those training jobs based on a predefined \"objective metric\", and select the hyperparameter settings for future attempts based on previous results. \\\n",
    "\n",
    "For each hyperparameter tuning job, we will give it a budget (max number of training jobs) and it will complete once that many training jobs have been executed.\\\n",
    "\n",
    "We will tune 4 hyperparameters in this XGboost problem \\\n",
    "\n",
    "All available set of hyperparameter are [here](https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost-tuning.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tuner import IntegerParameter, ContinuousParameter\n",
    "\n",
    "# We give a range of hyperparameters\n",
    "hyperparameter_ranges = {\n",
    "    'eta': ContinuousParameter(0, 1),\n",
    "    'min_child_weight': ContinuousParameter(1, 10),\n",
    "    'alpha': ContinuousParameter(0, 2),\n",
    "    'max_depth': IntegerParameter(1, 10)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we'll specify the objective metric that we'd like to tune for and its definition. Several metrics are available. /\n",
    "\n",
    "All available set of metric are [here](https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost-tuning.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we'll create a HyperparameterTuner object, to which we pass:\n",
    "- The XGBoost estimator we created above\n",
    "- Our hyperparameter ranges\n",
    "- Objective metric name and definition\n",
    "- Tuning resource configurations (Number of training jobs to run in total and how many training jobs can be run in parallel)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "objective_metric_name = 'validation:auc'\n",
    "objective_type = 'Maximize'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tuner import HyperparameterTuner\n",
    "\n",
    "tuner = HyperparameterTuner(\n",
    "    xgb,\n",
    "    objective_metric_name,\n",
    "    hyperparameter_ranges,\n",
    "    objective_type=objective_type,\n",
    "    max_jobs=40,\n",
    "    max_parallel_jobs=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Start Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:sagemaker:'s3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2.\n",
      "WARNING:sagemaker:'s3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2.\n"
     ]
    }
   ],
   "source": [
    "tuner.fit(s3_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sagemaker.tuner.HyperparameterTuner at 0x22897aa1ec8>"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tuner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Inspecting jobs with Amazon SageMaker Experiments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While model tuning is running, we can explore its progress with SageMaker Experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.analytics import HyperparameterTuningJobAnalytics\n",
    "\n",
    "exp = HyperparameterTuningJobAnalytics(\n",
    "    sagemaker_session=sess, \n",
    "    hyperparameter_tuning_job_name=tuner.latest_tuning_job.name\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sagemaker.HyperparameterTuningJobAnalytics for sagemaker-xgboost-200618-1639>"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>alpha</th>\n",
       "      <th>eta</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>min_child_weight</th>\n",
       "      <th>TrainingJobName</th>\n",
       "      <th>TrainingJobStatus</th>\n",
       "      <th>FinalObjectiveValue</th>\n",
       "      <th>TrainingStartTime</th>\n",
       "      <th>TrainingEndTime</th>\n",
       "      <th>TrainingElapsedTimeSeconds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.591646</td>\n",
       "      <td>0.888403</td>\n",
       "      <td>7.0</td>\n",
       "      <td>2.673249</td>\n",
       "      <td>sagemaker-xgboost-200618-1639-008-7bcb13bd</td>\n",
       "      <td>Stopped</td>\n",
       "      <td>None</td>\n",
       "      <td>2020-06-18 16:46:31-07:00</td>\n",
       "      <td>2020-06-18 16:47:16-07:00</td>\n",
       "      <td>45.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      alpha       eta  max_depth  min_child_weight  \\\n",
       "0  0.591646  0.888403        7.0          2.673249   \n",
       "\n",
       "                              TrainingJobName TrainingJobStatus  \\\n",
       "0  sagemaker-xgboost-200618-1639-008-7bcb13bd           Stopped   \n",
       "\n",
       "  FinalObjectiveValue         TrainingStartTime           TrainingEndTime  \\\n",
       "0                None 2020-06-18 16:46:31-07:00 2020-06-18 16:47:16-07:00   \n",
       "\n",
       "   TrainingElapsedTimeSeconds  \n",
       "0                        45.0  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = exp.dataframe()\n",
    "df.sort_values('FinalObjectiveValue', ascending=0)[:1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>alpha</th>\n",
       "      <th>eta</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>min_child_weight</th>\n",
       "      <th>TrainingJobName</th>\n",
       "      <th>TrainingJobStatus</th>\n",
       "      <th>FinalObjectiveValue</th>\n",
       "      <th>TrainingStartTime</th>\n",
       "      <th>TrainingEndTime</th>\n",
       "      <th>TrainingElapsedTimeSeconds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.591646</td>\n",
       "      <td>0.888403</td>\n",
       "      <td>7.0</td>\n",
       "      <td>2.673249</td>\n",
       "      <td>sagemaker-xgboost-200618-1639-008-7bcb13bd</td>\n",
       "      <td>Stopped</td>\n",
       "      <td>None</td>\n",
       "      <td>2020-06-18 16:46:31-07:00</td>\n",
       "      <td>2020-06-18 16:47:16-07:00</td>\n",
       "      <td>45.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.710656</td>\n",
       "      <td>0.729853</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.941976</td>\n",
       "      <td>sagemaker-xgboost-200618-1639-007-8048b959</td>\n",
       "      <td>Failed</td>\n",
       "      <td>None</td>\n",
       "      <td>2020-06-18 16:45:48-07:00</td>\n",
       "      <td>2020-06-18 16:47:00-07:00</td>\n",
       "      <td>72.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.097842</td>\n",
       "      <td>0.287681</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.017434</td>\n",
       "      <td>sagemaker-xgboost-200618-1639-006-cc6efd69</td>\n",
       "      <td>Failed</td>\n",
       "      <td>None</td>\n",
       "      <td>2020-06-18 16:45:30-07:00</td>\n",
       "      <td>2020-06-18 16:47:13-07:00</td>\n",
       "      <td>103.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.559950</td>\n",
       "      <td>0.464547</td>\n",
       "      <td>6.0</td>\n",
       "      <td>8.896817</td>\n",
       "      <td>sagemaker-xgboost-200618-1639-005-0b158347</td>\n",
       "      <td>Failed</td>\n",
       "      <td>None</td>\n",
       "      <td>2020-06-18 16:45:54-07:00</td>\n",
       "      <td>2020-06-18 16:47:07-07:00</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.885231</td>\n",
       "      <td>0.356845</td>\n",
       "      <td>8.0</td>\n",
       "      <td>3.577142</td>\n",
       "      <td>sagemaker-xgboost-200618-1639-004-f809c522</td>\n",
       "      <td>Failed</td>\n",
       "      <td>None</td>\n",
       "      <td>2020-06-18 16:41:59-07:00</td>\n",
       "      <td>2020-06-18 16:43:22-07:00</td>\n",
       "      <td>83.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      alpha       eta  max_depth  min_child_weight  \\\n",
       "0  0.591646  0.888403        7.0          2.673249   \n",
       "1  1.710656  0.729853        2.0          2.941976   \n",
       "2  1.097842  0.287681        1.0          2.017434   \n",
       "3  0.559950  0.464547        6.0          8.896817   \n",
       "4  1.885231  0.356845        8.0          3.577142   \n",
       "\n",
       "                              TrainingJobName TrainingJobStatus  \\\n",
       "0  sagemaker-xgboost-200618-1639-008-7bcb13bd           Stopped   \n",
       "1  sagemaker-xgboost-200618-1639-007-8048b959            Failed   \n",
       "2  sagemaker-xgboost-200618-1639-006-cc6efd69            Failed   \n",
       "3  sagemaker-xgboost-200618-1639-005-0b158347            Failed   \n",
       "4  sagemaker-xgboost-200618-1639-004-f809c522            Failed   \n",
       "\n",
       "  FinalObjectiveValue         TrainingStartTime           TrainingEndTime  \\\n",
       "0                None 2020-06-18 16:46:31-07:00 2020-06-18 16:47:16-07:00   \n",
       "1                None 2020-06-18 16:45:48-07:00 2020-06-18 16:47:00-07:00   \n",
       "2                None 2020-06-18 16:45:30-07:00 2020-06-18 16:47:13-07:00   \n",
       "3                None 2020-06-18 16:45:54-07:00 2020-06-18 16:47:07-07:00   \n",
       "4                None 2020-06-18 16:41:59-07:00 2020-06-18 16:43:22-07:00   \n",
       "\n",
       "   TrainingElapsedTimeSeconds  \n",
       "0                        45.0  \n",
       "1                        72.0  \n",
       "2                       103.0  \n",
       "3                        73.0  \n",
       "4                        83.0  "
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 8. Model Deploy "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import strftime, gmtime\n",
    "timestamp = strftime('%d-%H-%M-%S', gmtime())\n",
    "\n",
    "xgb_endpoint = tuner.deploy(\n",
    "    endpoint_name = 'xgboost-{}'.format(timestamp),\n",
    "    initial_instance_count = 1,                    # Infrastructure requirements\n",
    "    instance_type = 'ml.m4.xlarge'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 9. Predicting with the test set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predicting with our model\n",
    "- First we'll need to determine how we pass data into and receive data from our endpoint. Our data is currently stored as NumPy arrays in memory of our notebook instance./\n",
    "- To send it in an HTTP POST request, we'll serialize it as a CSV string and then decode the resulting CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sm = boto3.Session().client(service_name='runtime.sagemaker') \n",
    "\n",
    "test_samples = [line.rstrip('\\n') for line in open('test.csv')]\n",
    "test_samples = test_samples[:100] # We'll predict the first 100 samples\n",
    "\n",
    "for sample in test_samples:\n",
    "    sample = bytes(sample, 'utf-8')\n",
    "    response = sm.invoke_endpoint(EndpointName=xgb_endpoint.endpoint, \n",
    "                                  ContentType='text/csv', \n",
    "                                  Body=sample)\n",
    "    print(response['Body'].read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 10. Close the SageMaker Instance "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make sure we don't get charged after the training/inference is over, we have to delete the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker.Session().delete_endpoint(xgb_endpoint.endpoint) "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}