ML0101EN_Clas_Logistic_Reg_churn.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Grundmane/MachineLearningCoursera/blob/main/ML0101EN_Clas_Logistic_Reg_churn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "GS1McMWBhO-n"
      },
      "source": [
        "<p style=\"text-align:center\">\n",
        "    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork1047-2023-01-01\">\n",
        "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n",
        "    </a>\n",
        "</p>\n",
        "\n",
        "\n",
        "# Logistic Regression with Python\n",
        "\n",
        "\n",
        "Estimated time needed: **25** minutes\n",
        "    \n",
        "\n",
        "## Objectives\n",
        "\n",
        "After completing this lab you will be able to:\n",
        "\n",
        "* Use scikit Logistic Regression to classify\n",
        "* Understand confusion matrix\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bDlVk5MehO-q"
      },
      "source": [
        "In this notebook, you will learn Logistic Regression, and then, you'll create a model for a telecommunication company, to predict when its customers will leave for a competitor, so that they can take some action to retain the customers.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3bJXL3S4hO-q"
      },
      "source": [
        "<h1>Table of contents</h1>\n",
        "\n",
        "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
        "    <ol>\n",
        "        <li><a href=\"#about_dataset\">About the dataset</a></li>\n",
        "        <li><a href=\"#preprocessing\">Data pre-processing and selection</a></li>\n",
        "        <li><a href=\"#modeling\">Modeling (Logistic Regression with Scikit-learn)</a></li>\n",
        "        <li><a href=\"#evaluation\">Evaluation</a></li>\n",
        "        <li><a href=\"#practice\">Practice</a></li>\n",
        "    </ol>\n",
        "</div>\n",
        "<br>\n",
        "<hr>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "zG2NuuFxhO-r"
      },
      "source": [
        "<a id=\"ref1\"></a>\n",
        "## What is the difference between Linear and Logistic Regression?\n",
        "\n",
        "While Linear Regression is suited for estimating continuous values (e.g. estimating house price), it is not the best tool for predicting the class of an observed data point. In order to estimate the class of a data point, we need some sort of guidance on what would be the <b>most probable class</b> for that data point. For this, we use <b>Logistic Regression</b>.\n",
        "\n",
        "<div class=\"alert alert-success alertsuccess\" style=\"margin-top: 20px\">\n",
        "<font size = 3><strong>Recall linear regression:</strong></font>\n",
        "<br>\n",
        "<br>\n",
        "    As you know, <b>Linear regression</b> finds a function that relates a continuous dependent variable, <b>y</b>, to some predictors (independent variables $x_1$, $x_2$, etc.). For example, simple linear regression assumes a function of the form:\n",
        "<br><br>\n",
        "$$\n",
        "y = \\theta_0 + \\theta_1  x_1 + \\theta_2  x_2 + \\cdots\n",
        "$$\n",
        "<br>\n",
        "and finds the values of parameters $\\theta_0, \\theta_1, \\theta_2$, etc, where the term $\\theta_0$ is the \"intercept\". It can be generally shown as:\n",
        "<br><br>\n",
        "$$\n",
        "ℎ_\\theta(𝑥) = \\theta^TX\n",
        "$$\n",
        "<p></p>\n",
        "\n",
        "</div>\n",
        "\n",
        "Logistic Regression is a variation of Linear Regression, used when the observed dependent variable, <b>y</b>, is categorical. It produces a formula that predicts the probability of the class label as a function of the independent variables.\n",
        "\n",
        "Logistic regression fits a special s-shaped curve by taking the linear regression function and transforming the numeric estimate into a probability with the following function, which is called the sigmoid function 𝜎:\n",
        "\n",
        "$$\n",
        "ℎ_\\theta(𝑥) = \\sigma({\\theta^TX}) =  \\frac {e^{(\\theta_0 + \\theta_1  x_1 + \\theta_2  x_2 +...)}}{1 + e^{(\\theta_0 + \\theta_1  x_1 + \\theta_2  x_2 +\\cdots)}}\n",
        "$$\n",
        "Or:\n",
        "$$\n",
        "ProbabilityOfaClass_1 =  P(Y=1|X) = \\sigma({\\theta^TX}) = \\frac{e^{\\theta^TX}}{1+e^{\\theta^TX}}\n",
        "$$\n",
        "\n",
        "In this equation, ${\\theta^TX}$ is the regression result (the sum of the variables weighted by the coefficients), `exp` is the exponential function and $\\sigma(\\theta^TX)$ is the sigmoid or [logistic function](http://en.wikipedia.org/wiki/Logistic_function?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork1047-2023-01-01), also called logistic curve. It is a common \"S\" shape (sigmoid curve).\n",
        "\n",
        "So, briefly, Logistic Regression passes the input through the logistic/sigmoid but then treats the result as a probability:\n",
        "\n",
        "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/images/mod_ID_24_final.png\" width=\"400\" align=\"center\">\n",
        "\n",
        "\n",
        "The objective of the __Logistic Regression__ algorithm, is to find the best parameters θ, for $ℎ_\\theta(𝑥)$ = $\\sigma({\\theta^TX})$, in such a way that the model best predicts the class of each case.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HwMRsdsnhO-r"
      },
      "source": [
        "### Customer churn with Logistic Regression\n",
        "A telecommunications company is concerned about the number of customers leaving their land-line business for cable competitors. They need to understand who is leaving. Imagine that you are an analyst at this company and you have to find out who is leaving and why.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sPnyOEdFhO-s",
        "outputId": "15cfa764-b9b5-4dd7-c68e-e6d69b53c68f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting scikit-learn==0.23.1\n",
            "  Downloading scikit-learn-0.23.1.tar.gz (7.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.2/7.2 MB\u001b[0m \u001b[31m17.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  \u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n",
            "  \n",
            "  \u001b[31m×\u001b[0m \u001b[32mpip subprocess to install build dependencies\u001b[0m did not run successfully.\n",
            "  \u001b[31m│\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n",
            "  \u001b[31m╰─>\u001b[0m See above for output.\n",
            "  \n",
            "  \u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n",
            "  Installing build dependencies ... \u001b[?25l\u001b[?25herror\n",
            "\u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n",
            "\n",
            "\u001b[31m×\u001b[0m \u001b[32mpip subprocess to install build dependencies\u001b[0m did not run successfully.\n",
            "\u001b[31m│\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n",
            "\u001b[31m╰─>\u001b[0m See above for output.\n",
            "\n",
            "\u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n"
          ]
        }
      ],
      "source": [
        "!pip install scikit-learn==0.23.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "U2aH2qnlhO-s"
      },
      "source": [
        "Let's first import required libraries:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "PMm1-HR1hO-t"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import pylab as pl\n",
        "import numpy as np\n",
        "import scipy.optimize as opt\n",
        "from sklearn import preprocessing\n",
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "0fN2W-r8hO-t"
      },
      "source": [
        "<h2 id=\"about_dataset\">About the dataset</h2>\n",
        "We will use a telecommunications dataset for predicting customer churn. This is a historical customer dataset where each row represents one customer. The data is relatively easy to understand, and you may uncover insights you can use immediately. Typically it is less expensive to keep customers than acquire new ones, so the focus of this analysis is to predict the customers who will stay with the company.\n",
        "\n",
        "\n",
        "This data set provides information to help you predict what behavior will help you to retain customers. You can analyze all relevant customer data and develop focused customer retention programs.\n",
        "\n",
        "\n",
        "\n",
        "The dataset includes information about:\n",
        "\n",
        "- Customers who left within the last month – the column is called Churn\n",
        "- Services that each customer has signed up for – phone, multiple lines, internet, online security, online backup, device protection, tech support, and streaming TV and movies\n",
        "- Customer account information – how long they had been a customer, contract, payment method, paperless billing, monthly charges, and total charges\n",
        "- Demographic info about customers – gender, age range, and if they have partners and dependents\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "fvt0dOaYhO-t"
      },
      "source": [
        "###  Load the Telco Churn data\n",
        "Telco Churn is a hypothetical data file that concerns a telecommunications company's efforts to reduce turnover in its customer base. Each case corresponds to a separate customer and it records various demographic and service usage information. Before you can work with the data, you must use the URL to get the ChurnData.csv.\n",
        "\n",
        "To download the data, we will use `!wget` to download it from IBM Object Storage.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9YV-yh7zhO-u",
        "outputId": "fea60bb1-f924-4a86-e6f2-e74120a2989e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-08-01 09:16:37--  https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/data/ChurnData.csv\n",
            "Resolving cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud (cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud)... 198.23.119.245\n",
            "Connecting to cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud (cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud)|198.23.119.245|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 35943 (35K) [text/csv]\n",
            "Saving to: ‘ChurnData.csv’\n",
            "\n",
            "ChurnData.csv       100%[===================>]  35.10K  --.-KB/s    in 0.02s   \n",
            "\n",
            "2023-08-01 09:16:37 (2.16 MB/s) - ‘ChurnData.csv’ saved [35943/35943]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "#Click here and press Shift+Enter\n",
        "!wget -O ChurnData.csv https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/data/ChurnData.csv"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6OXm9RKShO-u"
      },
      "source": [
        "__Did you know?__ When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "oYe0aUv1hO-u"
      },
      "source": [
        "## Load Data From CSV File  \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 236
        },
        "id": "pykRiIrkhO-u",
        "outputId": "f51805f8-3363-4f95-9d9a-ace095efa572"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   tenure   age  address  income   ed  employ  equip  callcard  wireless  \\\n",
              "0    11.0  33.0      7.0   136.0  5.0     5.0    0.0       1.0       1.0   \n",
              "1    33.0  33.0     12.0    33.0  2.0     0.0    0.0       0.0       0.0   \n",
              "2    23.0  30.0      9.0    30.0  1.0     2.0    0.0       0.0       0.0   \n",
              "3    38.0  35.0      5.0    76.0  2.0    10.0    1.0       1.0       1.0   \n",
              "4     7.0  35.0     14.0    80.0  2.0    15.0    0.0       1.0       0.0   \n",
              "\n",
              "   longmon  ...  pager  internet  callwait  confer  ebill  loglong  logtoll  \\\n",
              "0     4.40  ...    1.0       0.0       1.0     1.0    0.0    1.482    3.033   \n",
              "1     9.45  ...    0.0       0.0       0.0     0.0    0.0    2.246    3.240   \n",
              "2     6.30  ...    0.0       0.0       0.0     1.0    0.0    1.841    3.240   \n",
              "3     6.05  ...    1.0       1.0       1.0     1.0    1.0    1.800    3.807   \n",
              "4     7.10  ...    0.0       0.0       1.0     1.0    0.0    1.960    3.091   \n",
              "\n",
              "   lninc  custcat  churn  \n",
              "0  4.913      4.0    1.0  \n",
              "1  3.497      1.0    1.0  \n",
              "2  3.401      3.0    0.0  \n",
              "3  4.331      4.0    0.0  \n",
              "4  4.382      3.0    0.0  \n",
              "\n",
              "[5 rows x 28 columns]"
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-2aa9039c-c4f0-4313-b2a1-52e835382ef6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tenure</th>\n",
              "      <th>age</th>\n",
              "      <th>address</th>\n",
              "      <th>income</th>\n",
              "      <th>ed</th>\n",
              "      <th>employ</th>\n",
              "      <th>equip</th>\n",
              "      <th>callcard</th>\n",
              "      <th>wireless</th>\n",
              "      <th>longmon</th>\n",
              "      <th>...</th>\n",
              "      <th>pager</th>\n",
              "      <th>internet</th>\n",
              "      <th>callwait</th>\n",
              "      <th>confer</th>\n",
              "      <th>ebill</th>\n",
              "      <th>loglong</th>\n",
              "      <th>logtoll</th>\n",
              "      <th>lninc</th>\n",
              "      <th>custcat</th>\n",
              "      <th>churn</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>11.0</td>\n",
              "      <td>33.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>136.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>4.40</td>\n",
              "      <td>...</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.482</td>\n",
              "      <td>3.033</td>\n",
              "      <td>4.913</td>\n",
              "      <td>4.0</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>33.0</td>\n",
              "      <td>33.0</td>\n",
              "      <td>12.0</td>\n",
              "      <td>33.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>9.45</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.246</td>\n",
              "      <td>3.240</td>\n",
              "      <td>3.497</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>23.0</td>\n",
              "      <td>30.0</td>\n",
              "      <td>9.0</td>\n",
              "      <td>30.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>6.30</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.841</td>\n",
              "      <td>3.240</td>\n",
              "      <td>3.401</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>38.0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>76.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>10.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>6.05</td>\n",
              "      <td>...</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.800</td>\n",
              "      <td>3.807</td>\n",
              "      <td>4.331</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7.0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>14.0</td>\n",
              "      <td>80.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>15.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.10</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.960</td>\n",
              "      <td>3.091</td>\n",
              "      <td>4.382</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 28 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2aa9039c-c4f0-4313-b2a1-52e835382ef6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-939ed897-7184-4f20-9f05-af9cccdb70b9\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-939ed897-7184-4f20-9f05-af9cccdb70b9')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-939ed897-7184-4f20-9f05-af9cccdb70b9 button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-2aa9039c-c4f0-4313-b2a1-52e835382ef6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-2aa9039c-c4f0-4313-b2a1-52e835382ef6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "churn_df = pd.read_csv(\"ChurnData.csv\")\n",
        "churn_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M8id2-Q5hO-u"
      },
      "source": [
        "<h2 id=\"preprocessing\">Data pre-processing and selection</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TQhocU09hO-u"
      },
      "source": [
        "Let's select some features for the modeling. Also, we change the target data type to be an integer, as it is a requirement by the skitlearn algorithm:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "8XUjJSy3hO-u",
        "outputId": "92fc7965-11a0-4466-ef69-7722b6a18906"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   tenure   age  address  income   ed  employ  equip  callcard  wireless  \\\n",
              "0    11.0  33.0      7.0   136.0  5.0     5.0    0.0       1.0       1.0   \n",
              "1    33.0  33.0     12.0    33.0  2.0     0.0    0.0       0.0       0.0   \n",
              "2    23.0  30.0      9.0    30.0  1.0     2.0    0.0       0.0       0.0   \n",
              "3    38.0  35.0      5.0    76.0  2.0    10.0    1.0       1.0       1.0   \n",
              "4     7.0  35.0     14.0    80.0  2.0    15.0    0.0       1.0       0.0   \n",
              "\n",
              "   churn  \n",
              "0      1  \n",
              "1      1  \n",
              "2      0  \n",
              "3      0  \n",
              "4      0  "
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-53b07468-71cf-4d8e-892d-43fea2e497b4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tenure</th>\n",
              "      <th>age</th>\n",
              "      <th>address</th>\n",
              "      <th>income</th>\n",
              "      <th>ed</th>\n",
              "      <th>employ</th>\n",
              "      <th>equip</th>\n",
              "      <th>callcard</th>\n",
              "      <th>wireless</th>\n",
              "      <th>churn</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>11.0</td>\n",
              "      <td>33.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>136.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>33.0</td>\n",
              "      <td>33.0</td>\n",
              "      <td>12.0</td>\n",
              "      <td>33.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>23.0</td>\n",
              "      <td>30.0</td>\n",
              "      <td>9.0</td>\n",
              "      <td>30.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>38.0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>76.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>10.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7.0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>14.0</td>\n",
              "      <td>80.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>15.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-53b07468-71cf-4d8e-892d-43fea2e497b4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-7d7cb955-7bb2-406d-9dbc-41321534c486\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7d7cb955-7bb2-406d-9dbc-41321534c486')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-7d7cb955-7bb2-406d-9dbc-41321534c486 button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-53b07468-71cf-4d8e-892d-43fea2e497b4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-53b07468-71cf-4d8e-892d-43fea2e497b4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "churn_df = churn_df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip',   'callcard', 'wireless','churn']]\n",
        "churn_df['churn'] = churn_df['churn'].astype('int')\n",
        "churn_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": true,
        "new_sheet": true,
        "run_control": {
          "read_only": false
        },
        "id": "t-0A0RcPhO-u"
      },
      "source": [
        "## Practice\n",
        "How many rows and columns are in this dataset in total? What are the names of columns?\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gx-3qCeHhO-u",
        "outputId": "11eb1f65-793f-4ae3-9d36-94ea43ab2f92"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(200, 10)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "# write your code here\n",
        "churn_df.shape\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OUyQlXcvhO-v"
      },
      "source": [
        "<details><summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "churn_df.shape\n",
        "\n",
        "```\n",
        "\n",
        "</details>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k49xza5MhO-v"
      },
      "source": [
        "Let's define X, and y for our dataset:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jafs2KW_hO-v",
        "outputId": "1d5aa652-f3cb-4e56-979d-b95b75eaf519"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 11.,  33.,   7., 136.,   5.,   5.,   0.],\n",
              "       [ 33.,  33.,  12.,  33.,   2.,   0.,   0.],\n",
              "       [ 23.,  30.,   9.,  30.,   1.,   2.,   0.],\n",
              "       [ 38.,  35.,   5.,  76.,   2.,  10.,   1.],\n",
              "       [  7.,  35.,  14.,  80.,   2.,  15.,   0.]])"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "X = np.asarray(churn_df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip']])\n",
        "X[0:5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mg04paiohO-v",
        "outputId": "a2c0d486-7153-4f62-b236-524c607a8519"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 1, 0, 0, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "y = np.asarray(churn_df['churn'])\n",
        "y [0:5]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "siTSrc0vhO-v"
      },
      "source": [
        "Also, we normalize the dataset:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HrWaY7KzhO-v",
        "outputId": "6c47ffc9-f781-40d7-ae65-d868d0fb7117"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-1.13518441, -0.62595491, -0.4588971 ,  0.4751423 ,  1.6961288 ,\n",
              "        -0.58477841, -0.85972695],\n",
              "       [-0.11604313, -0.62595491,  0.03454064, -0.32886061, -0.6433592 ,\n",
              "        -1.14437497, -0.85972695],\n",
              "       [-0.57928917, -0.85594447, -0.261522  , -0.35227817, -1.42318853,\n",
              "        -0.92053635, -0.85972695],\n",
              "       [ 0.11557989, -0.47262854, -0.65627219,  0.00679109, -0.6433592 ,\n",
              "        -0.02518185,  1.16316   ],\n",
              "       [-1.32048283, -0.47262854,  0.23191574,  0.03801451, -0.6433592 ,\n",
              "         0.53441472, -0.85972695]])"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "from sklearn import preprocessing\n",
        "X = preprocessing.StandardScaler().fit(X).transform(X)\n",
        "X[0:5]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w-HJoPychO-v"
      },
      "source": [
        "## Train/Test dataset\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D9CtoV7XhO-v"
      },
      "source": [
        "We split our dataset into train and test set:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "H9FAe7DThO-v",
        "outputId": "af7aff4b-2d33-4ba7-89a6-d47ebefac67f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Train set: (160, 7) (160,)\n",
            "Test set: (40, 7) (40,)\n"
          ]
        }
      ],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)\n",
        "print ('Train set:', X_train.shape,  y_train.shape)\n",
        "print ('Test set:', X_test.shape,  y_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ileN1xLShO-w"
      },
      "source": [
        "<h2 id=\"modeling\">Modeling (Logistic Regression with Scikit-learn)</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h4ZDDsGrhO-w"
      },
      "source": [
        "Let's build our model using __LogisticRegression__ from the Scikit-learn package. This function implements logistic regression and can use different numerical optimizers to find parameters, including ‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’ solvers. You can find extensive information about the pros and cons of these optimizers if you search it in the internet.\n",
        "\n",
        "The version of Logistic Regression in Scikit-learn, support regularization. Regularization is a technique used to solve the overfitting problem of machine learning models.\n",
        "__C__ parameter indicates __inverse of regularization strength__ which must be a positive float. Smaller values specify stronger regularization.\n",
        "Now let's fit our model with train set:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        },
        "id": "FqYEdSemhO-w",
        "outputId": "20360932-366f-4d2f-8fe5-07d2462fe3e2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=0.01, solver='liblinear')"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression(C=0.01, solver=&#x27;liblinear&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(C=0.01, solver=&#x27;liblinear&#x27;)</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import confusion_matrix\n",
        "LR = LogisticRegression(C=0.01, solver='liblinear').fit(X_train,y_train)\n",
        "LR"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DacyIvUHhO-w"
      },
      "source": [
        "Now we can predict using our test set:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vCKXYKDJhO-w",
        "outputId": "c0ee7026-6753-404e-d7cb-0166bbc8ddbb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0,\n",
              "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "yhat = LR.predict(X_test)\n",
        "yhat"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7EulkMRMhO-w"
      },
      "source": [
        "__predict_proba__  returns estimates for all classes, ordered by the label of classes. So, the first column is the probability of class 0, P(Y=0|X), and second column is probability of class 1, P(Y=1|X):\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d1x7aI48hO-w",
        "outputId": "3cd75517-8ee6-43f1-be02-f6cb2dc379a0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.54132919, 0.45867081],\n",
              "       [0.60593357, 0.39406643],\n",
              "       [0.56277713, 0.43722287],\n",
              "       [0.63432489, 0.36567511],\n",
              "       [0.56431839, 0.43568161],\n",
              "       [0.55386646, 0.44613354],\n",
              "       [0.52237207, 0.47762793],\n",
              "       [0.60514349, 0.39485651],\n",
              "       [0.41069572, 0.58930428],\n",
              "       [0.6333873 , 0.3666127 ],\n",
              "       [0.58068791, 0.41931209],\n",
              "       [0.62768628, 0.37231372],\n",
              "       [0.47559883, 0.52440117],\n",
              "       [0.4267593 , 0.5732407 ],\n",
              "       [0.66172417, 0.33827583],\n",
              "       [0.55092315, 0.44907685],\n",
              "       [0.51749946, 0.48250054],\n",
              "       [0.485743  , 0.514257  ],\n",
              "       [0.49011451, 0.50988549],\n",
              "       [0.52423349, 0.47576651],\n",
              "       [0.61619519, 0.38380481],\n",
              "       [0.52696302, 0.47303698],\n",
              "       [0.63957168, 0.36042832],\n",
              "       [0.52205164, 0.47794836],\n",
              "       [0.50572852, 0.49427148],\n",
              "       [0.70706202, 0.29293798],\n",
              "       [0.55266286, 0.44733714],\n",
              "       [0.52271594, 0.47728406],\n",
              "       [0.51638863, 0.48361137],\n",
              "       [0.71331391, 0.28668609],\n",
              "       [0.67862111, 0.32137889],\n",
              "       [0.50896403, 0.49103597],\n",
              "       [0.42348082, 0.57651918],\n",
              "       [0.71495838, 0.28504162],\n",
              "       [0.59711064, 0.40288936],\n",
              "       [0.63808839, 0.36191161],\n",
              "       [0.39957895, 0.60042105],\n",
              "       [0.52127638, 0.47872362],\n",
              "       [0.65975464, 0.34024536],\n",
              "       [0.5114172 , 0.4885828 ]])"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "yhat_prob = LR.predict_proba(X_test)\n",
        "yhat_prob"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yIFLzVxbhO-w"
      },
      "source": [
        "<h2 id=\"evaluation\">Evaluation</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wDNL3QcPhO-w"
      },
      "source": [
        "### jaccard index\n",
        "Let's try the jaccard index for accuracy evaluation. we can define jaccard as the size of the intersection divided by the size of the union of the two label sets. If the entire set of predicted labels for a sample strictly matches with the true set of labels, then the subset accuracy is 1.0; otherwise it is 0.0.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ORTzLfaFhO-w",
        "outputId": "3dba4384-fa36-4e35-e777-d8b9d3175852"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7058823529411765"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "from sklearn.metrics import jaccard_score\n",
        "jaccard_score(y_test, yhat,pos_label=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mDNsYlA9hO-w"
      },
      "source": [
        "### confusion matrix\n",
        "Another way of looking at the accuracy of the classifier is to look at __confusion matrix__.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hPQJPHFQhO-0",
        "outputId": "9de5eaf2-d6cd-40a0-8f59-83f207b9a2a1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[ 6  9]\n",
            " [ 1 24]]\n"
          ]
        }
      ],
      "source": [
        "from sklearn.metrics import classification_report, confusion_matrix\n",
        "import itertools\n",
        "def plot_confusion_matrix(cm, classes,\n",
        "                          normalize=False,\n",
        "                          title='Confusion matrix',\n",
        "                          cmap=plt.cm.Blues):\n",
        "    \"\"\"\n",
        "    This function prints and plots the confusion matrix.\n",
        "    Normalization can be applied by setting `normalize=True`.\n",
        "    \"\"\"\n",
        "    if normalize:\n",
        "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
        "        print(\"Normalized confusion matrix\")\n",
        "    else:\n",
        "        print('Confusion matrix, without normalization')\n",
        "\n",
        "    print(cm)\n",
        "\n",
        "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
        "    plt.title(title)\n",
        "    plt.colorbar()\n",
        "    tick_marks = np.arange(len(classes))\n",
        "    plt.xticks(tick_marks, classes, rotation=45)\n",
        "    plt.yticks(tick_marks, classes)\n",
        "\n",
        "    fmt = '.2f' if normalize else 'd'\n",
        "    thresh = cm.max() / 2.\n",
        "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
        "        plt.text(j, i, format(cm[i, j], fmt),\n",
        "                 horizontalalignment=\"center\",\n",
        "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
        "\n",
        "    plt.tight_layout()\n",
        "    plt.ylabel('True label')\n",
        "    plt.xlabel('Predicted label')\n",
        "print(confusion_matrix(y_test, yhat, labels=[1,0]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 558
        },
        "id": "9bmWpvQmhO-0",
        "outputId": "0efff4f8-9787-4bad-94be-393b91452928"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Confusion matrix, without normalization\n",
            "[[ 6  9]\n",
            " [ 1 24]]\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiQAAAHpCAYAAACybSeHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABArElEQVR4nO3dd3wUdf7H8fdsIIWQBEILkVBDCQKBgxMREDiQYgOReiABAXuhCQI/qkIUVIoiFpQIioIiKHjiQeiC3oFixRyBICBVEEJNQjK/PzCrayi7ZMPsbF5PH/N4sDOz3/ksN7d+/Hy+31nDNE1TAAAAFnJYHQAAAAAJCQAAsBwJCQAAsBwJCQAAsBwJCQAAsBwJCQAAsBwJCQAAsBwJCQAAsBwJCQAAsBwJCeBnduzYobZt2yoiIkKGYWjp0qVeHX/37t0yDENJSUleHdcfVK5cWX379rU6DMCWSEiAArBz507df//9qlq1qoKDgxUeHq6mTZtqxowZOnv2bIFeOyEhQd99950mTZqk+fPnq1GjRgV6PX/0448/avz48dq9e7fVoQCFhsFv2QDe9cknn6hr164KCgpSnz59VKdOHWVmZmrjxo1avHix+vbtq9dee61Arn327FkVK1ZMo0eP1tNPP10g1zBNUxkZGSpatKgCAgIK5BpW++CDD9S1a1etWbNGLVu2dPt9GRkZcjgcKlq0aMEFB/ipIlYHAPiTtLQ09ejRQ5UqVdLq1atVvnx557GHH35Yqamp+uSTTwrs+keOHJEklShRosCuYRiGgoODC2x8uzFNU+fOnVNISIiCgoKsDgewLVo2gBdNmTJFp06d0htvvOGSjOSKjY3V448/7nx9/vx5PfXUU6pWrZqCgoJUuXJljRo1ShkZGS7vq1y5sm6//XZt3LhRN9xwg4KDg1W1alXNmzfPec748eNVqVIlSdITTzwhwzBUuXJlSVLfvn2df/6z8ePHyzAMl30rV65Us2bNVKJECRUvXlw1a9bUqFGjnMcvNYdk9erVat68uUJDQ1WiRAl17NhR27dvv+j1UlNT1bdvX5UoUUIRERHq16+fzpw5c+m/2N+1bNlSderU0bfffqsWLVqoWLFiio2N1QcffCBJWrdunRo3bqyQkBDVrFlTq1atcnn/zz//rIceekg1a9ZUSEiISpUqpa5du7q0ZpKSktS1a1dJUqtWrWQYhgzD0Nq1ayX98b/FZ599pkaNGikkJESvvvqq81juHBLTNNWqVSuVKVNGhw8fdo6fmZmpunXrqlq1ajp9+vQVPzNQWJCQAF60bNkyVa1aVTfddJNb5w8YMEBjx47V3/72N02bNk0tWrRQYmKievTokefc1NRUdenSRbfccouef/55lSxZUn379tUPP/wgSercubOmTZsmSerZs6fmz5+v6dOnexT/Dz/8oNtvv10ZGRmaOHGinn/+ed155536/PPPL/u+VatWqV27djp8+LDGjx+vIUOGaNOmTWratOlF52F069ZNJ0+eVGJiorp166akpCRNmDDBrRh/++033X777WrcuLGmTJmioKAg9ejRQwsXLlSPHj1066236plnntHp06fVpUsXnTx50vne//73v9q0aZN69OihmTNn6oEHHlBycrJatmzpTIhuvvlmPfbYY5KkUaNGaf78+Zo/f77i4uKc46SkpKhnz5665ZZbNGPGDNWvXz9PnIZh6M0339S5c+f0wAMPOPePGzdOP/zwg+bOnavQ0FC3PjNQKJgAvOLEiROmJLNjx45unb9t2zZTkjlgwACX/cOGDTMlmatXr3buq1SpkinJXL9+vXPf4cOHzaCgIHPo0KHOfWlpaaYkc+rUqS5jJiQkmJUqVcoTw7hx48w/fw1MmzbNlGQeOXLkknHnXmPu3LnOffXr1zfLli1rHj161Lnvm2++MR0Oh9mnT58817v33ntdxrzrrrvMUqVKXfKauVq0aGFKMhcsWODc99NPP5mSTIfDYX7xxRfO/Z999lmeOM+cOZNnzM2bN5uSzHnz5jn3vf/++6Ykc82aNXnOz/3fYsWKFRc9lpCQ4LLv1VdfNSWZb7/9tvnFF1+YAQEB5qBBg674WYHChgoJ4CXp6emSpLCwMLfO/9e//iVJGjJkiMv+oUOHSlKeuSa1a9dW8+bNna/LlCmjmjVrateuXVcd81/lzj356KOPlJOT49Z7Dhw4oG3btqlv376KjIx07q9Xr55uueUW5+f8sz9XDCSpefPmOnr0qPPv8HKKFy/uUkGqWbOmSpQoobi4ODVu3Ni5P/fPf/77CQkJcf45KytLR48eVWxsrEqUKKGvvvrKjU97QZUqVdSuXTu3zr3vvvvUrl07Pfroo7rnnntUrVo1TZ482e1rAYUFCQngJeHh4ZLk0iK4nJ9//lkOh0OxsbEu+6OiolSiRAn9/PPPLvsrVqyYZ4ySJUvqt99+u8qI8+revbuaNm2qAQMGqFy5curRo4cWLVp02eQkN86aNWvmORYXF6dff/01z1yJv36WkiVLSpJbn6VChQp55r1EREQoJiYmz76/jnn27FmNHTtWMTExCgoKUunSpVWmTBkdP35cJ06cuOK1c1WpUsXtcyXpjTfe0JkzZ7Rjxw4lJSW5JEYALiAhAbwkPDxc0dHR+v777z1631//5Xopl1pia7qxcv9S18jOznZ5HRISovXr12vVqlW655579O2336p79+665ZZb8pybH/n5LJd6rztjPvroo5o0aZK6deumRYsW6d///rdWrlypUqVKuV0RkuRxQrF27VrnROXvvvvOo/cChQUJCeBFt99+u3bu3KnNmzdf8dxKlSopJydHO3bscNl/6NAhHT9+3LlixhtKliyp48eP59n/1yqMJDkcDrVu3VovvPCCfvzxR02aNEmrV6/WmjVrLjp2bpwpKSl5jv30008qXbq0z0ze/OCDD5SQkKDnn3/eOUG4WbNmef5u3E0S3XHgwAE9+uijatu2rW6//XYNGzbson/vQGFHQgJ40fDhwxUaGqoBAwbo0KFDeY7v3LlTM2bMkCTdeuutkpRnJcwLL7wgSbrtttu8Fle1atV04sQJffvtt859Bw4c0JIlS1zOO3bsWJ735q4g+etS5Fzly5dX/fr19dZbb7n8i/3777/Xv//9b+fn9AUBAQF5qjAvvvhinupPbgJ1sSTOUwMHDlROTo7eeOMNvfbaaypSpIj69+/vVjUIKEx4MBrgRdWqVdOCBQvUvXt3xcXFuTypddOmTXr//fedz6mIj49XQkKCXnvtNR0/flwtWrTQf/7zH7311lvq1KmTWrVq5bW4evTooREjRuiuu+7SY489pjNnzmj27NmqUaOGy2TOiRMnav369brttttUqVIlHT58WC+//LIqVKigZs2aXXL8qVOnqkOHDmrSpIn69++vs2fP6sUXX1RERITGjx/vtc+RX7fffrvmz5+viIgI1a5dW5s3b9aqVatUqlQpl/Pq16+vgIAAPfvsszpx4oSCgoL0j3/8Q2XLlvXoenPnztUnn3yipKQkVahQQdKFBKh3796aPXu2HnroIa99NsDuSEgAL7vzzjv17bffaurUqfroo480e/ZsBQUFqV69enr++ec1cOBA57lz5sxR1apVlZSUpCVLligqKkojR47UuHHjvBpTqVKltGTJEg0ZMkTDhw9XlSpVlJiYqB07drgkJHfeead2796tN998U7/++qtKly6tFi1aaMKECc5JohfTpk0brVixQuPGjdPYsWNVtGhRtWjRQs8++6zHE0AL0owZMxQQEKB33nlH586dU9OmTZ3PUPmzqKgovfLKK0pMTFT//v2VnZ2tNWvWeJSQ7Nu3T4MHD9Ydd9yhhIQE5/5evXpp8eLFGj58uDp06OBTfz+AlfgtGwAAYDnmkAAAAMuRkAAAAMuRkAAAAMuRkAAAAMuRkAAAAMuRkAAAAMvxHBIfkZOTo/379yssLMyrj60GABQM0zR18uRJRUdHy+G4Nv99f+7cOWVmZnplrMDAQAUHB3tlLG8gIfER+/fvz/NrpQAA37d3717nk3gL0rlz5xQSVko6f8Yr40VFRSktLc1nkhISEh8RFhYmSfpk048KLR5mcTSAd6X+dtLqEACvO3v6lB699Qbn93dBy8zMlM6fUdD1/aSAwPwNlp2pgz/MVWZmJgkJXOW2aUKLh6l4WLjF0QDeVSzL6giAgnPN2+wBgTLymZD44iPaSUgAALATQ1J+kyAfnKpIQgIAgJ0YjgtbfsfwMSQkAADYiWF4oULieyUS30uRAABAoUOFBAAAO6FlAwAALEfLBgAAoGBQIQEAwFa80LLxwXoECQkAAHZCywYAAKBgUCEBAMBOWGUDAAAs56ctGxISAADsxE8rJL4XEQAAKHSokAAAYCe0bAAAgOVo2QAAABQMKiQAANiJYXihQkLLBgAA5IfDuLDldwwfQ8sGAABYjgoJAAB24qeTWklIAACwE5b9AgAAy/lphcT3IgIAAIUOFRIAAOyElg0AALAcLRsAAICCQYUEAAA7oWUDAAAsR8sGAACgYFAhAQDATmjZAAAA63mhZeODDRISEgAA7MRPKyS+lyIBAIBChwoJAAB2YhheWGXjexUSEhIAAOyEZb8AAAAFgwoJAAB24qeTWklIAACwE1o2AAAABYMKCQAAdkLLBgAAWM5PWzYkJAAA2ImfVkh8L0UCAACFDhUSAABsxDAMGX5YISEhAQDARvw1IaFlAwAALEeFBAAAOzF+3/I7ho8hIQEAwEZo2QAAABQQKiQAANiIv1ZISEgAALAREhIAAGA5f01ImEMCAAAsR4UEAAA7YdkvAACwGi0bAABQ6CQmJurvf/+7wsLCVLZsWXXq1EkpKSku55w7d04PP/ywSpUqpeLFi+vuu+/WoUOHPLoOCQkAADZiGH9USa5+c/9669at08MPP6wvvvhCK1euVFZWltq2bavTp087zxk8eLCWLVum999/X+vWrdP+/fvVuXNnjz4XLRsAAGzEkBdaNh5MIlmxYoXL66SkJJUtW1Zbt27VzTffrBMnTuiNN97QggUL9I9//EOSNHfuXMXFxemLL77QjTfe6NZ1qJAAAFBIpaenu2wZGRlXfM+JEyckSZGRkZKkrVu3KisrS23atHGeU6tWLVWsWFGbN292OxYSEgAAbCT/7Zo/KiwxMTGKiIhwbomJiZe9dk5OjgYNGqSmTZuqTp06kqSDBw8qMDBQJUqUcDm3XLlyOnjwoNufi5YNAAB24sVlv3v37lV4eLhzd1BQ0GXf9vDDD+v777/Xxo0b8xlAXiQkAADYiReW/Zq/vz88PNwlIbmcRx55RMuXL9f69etVoUIF5/6oqChlZmbq+PHjLlWSQ4cOKSoqyu2YaNkAAIBLMk1TjzzyiJYsWaLVq1erSpUqLscbNmyookWLKjk52bkvJSVFe/bsUZMmTdy+DhUSAABsxBsPRvPk/Q8//LAWLFigjz76SGFhYc55IREREQoJCVFERIT69++vIUOGKDIyUuHh4Xr00UfVpEkTt1fYSCQkAADYyrVOSGbPni1Jatmypcv+uXPnqm/fvpKkadOmyeFw6O6771ZGRobatWunl19+2aOYSEgAAMAlmaZ5xXOCg4M1a9YszZo166qvQ0ICAICd8ON6AADAate6ZXOtsMoGAABYjgoJAAA24q8VEhISAABshIQEAABYzl8TEuaQAAAAy1EhAQDATlj2CwAArEbLBgAAoIBQIQEAwEb8tUJCQgIAgI34a0JCywaFyuGD+zVm0EC1blBZTWuVU/f2TfTjt19ZHRaQb2dPn9K858brsdtuVMJNsRrXr5N2/rDN6rAAt1EhQaGRfuI39e/STo2aNNeMuYtVslQp7U3bqfCIElaHBuTb6089ob07/6cHn5qukmXKaeO/lmjyg//U1A+SFVm2vNXhwZtYZQPY21uvTFe58tdp3NSXnfuui6lsXUCAl2SeO6v/rP5UQ59/Q3F/u1GS1OX+Ifpq/Sqt+mC+uj003OII4U20bACbW7/qU8XVa6ARD/XRLY2q6Z+3NdOSd5OsDgvIt+zsbOVkZ6toUJDL/sCgYKVs+69FUaGg5CYk+d18jU8mJLt375ZhGNq2bZvVocCP/LJntxa//YYqVqmmF9/6UF169ddzE0Zo+eIFVocG5EtIaHFVr9dQS+bM0G9HDionO1sb//Whdny3Vcd/PWx1eIBbfDIhsZv169frjjvuUHR0tAzD0NKlS60OCReRY+aoVp14PfzEONW6Pl6d/9lPnXokaPE7b1odGpBvD02cLtM09XD7v6tPk2pa8d6buqldRxkGX/P+xpAXKiQ+OImkUN2pmZmZBTLu6dOnFR8fr1mzZhXI+PCO0mWiVCW2psu+KrE1dHD/PosiArynXExljX39A725MUUvfvKlnp63XNnns1T2uopWhwYvo2VTAHJycjRlyhTFxsYqKChIFStW1KRJk5zHd+3apVatWqlYsWKKj4/X5s2bncfGjx+v+vXru4w3ffp0Va5c2fm6b9++6tSpkyZNmqTo6GjVrFnT2Q768MMPLzm2pzp06KCnn35ad91111WPgYIX36ixft6V6rLv57SdKn9djEURAd4XHFJMJcuU06n04/p283o1bNnW6pAAt1iakIwcOVLPPPOMxowZox9//FELFixQuXLlnMdHjx6tYcOGadu2bapRo4Z69uyp8+fPe3SN5ORkpaSkaOXKlVq+fLlbY+/Zs0fFixe/7DZ58uR8ffaMjAylp6e7bChY/7z3IX237b96c9Zz2rt7p1Z89L6WvJukrvcMtDo0IN++2bRW32xao8O/7NF3X6zXpPu7K7pyNbW4o5vVocHbDC9tPsayZb8nT57UjBkz9NJLLykhIUGSVK1aNTVr1ky7d++WJA0bNky33XabJGnChAm6/vrrlZqaqlq1arl9ndDQUM2ZM0eBgYGS5NbY0dHRV5xQGxkZ6cGnzSsxMVETJkzI1xjwzPXxDfXcK+/opakTNGfmFEXHVNLQMYnq0IkvbNjf2VMn9d5Lz+jY4YMqHl5Cf2/dQd0fGq4iRYtaHRq8zF+X/VqWkGzfvl0ZGRlq3br1Jc+pV6+e88/ly194sM/hw4c9Skjq1q3rTEbcHbtIkSKKjY11+xpXY+TIkRoyZIjzdXp6umJiaB0UtOat26t56/ZWhwF43Y1t79CNbe+wOgzgqlnWsgkJCbniOUX/lNnnZnM5OTmSJIfDIdM0Xc7PysrKM0ZoaKjHY1+Llk1QUJDCw8NdNgAArsRfJ7VaViGpXr26QkJClJycrAEDBnj8/jJlyujgwYMyTdP5F+ut55Zci5YNAABXwzAubPkdw9dYlpAEBwdrxIgRGj58uAIDA9W0aVMdOXJEP/zww2XbOLlatmypI0eOaMqUKerSpYtWrFihTz/91CuVBk9bNqdOnVJq6h+rN9LS0rRt2zZFRkaqYkWW3AEAvOdCQpLfOSReCsaLLF1lM2bMGA0dOlRjx45VXFycunfvrsOH3XuqYFxcnF5++WXNmjVL8fHx+s9//qNhw4YVcMQXt2XLFjVo0EANGjSQJA0ZMkQNGjTQ2LFjLYkHAAC7Mcy/TsSAJdLT0xUREaG13+5V8TDmk8C//O8Yy9rhf86cOqkBLWrrxIkT12QeYO6/J6o+9oECgi4+P9Jd2RmntWtml2sWuzv4tV8AAGzEX5f9FqpHxwMAAN9EhQQAABthlQ0AALCcw2HI4chfRmHm8/0FgZYNAACwHBUSAABshJYNAACwnL+usiEhAQDARvy1QsIcEgAAYDkqJAAA2AgtGwAAYDl/TUho2QAAAMtRIQEAwEb8dVIrCQkAADZiyAstG/leRkLLBgAAWI4KCQAANkLLBgAAWM5fV9mQkAAAYCP+WiFhDgkAALAcFRIAAGyElg0AALAcLRsAAIACQoUEAAAboWUDAACs54WWjQ8+qJWWDQAAsB4VEgAAbISWDQAAsJy/rrIhIQEAwEb8tULCHBIAAGA5KiQAANgILRsAAGA5WjYAAAAFhAoJAAA24q8VEhISAABsxF/nkNCyAQAAlqNCAgCAjdCyAQAAlvPXlg0JCQAANuKvFRLmkAAAAMtRIQEAwEYMeaFl45VIvIuEBAAAG3EYhhz5zEjy+/6CQMsGAABc1vr163XHHXcoOjpahmFo6dKlLsf79u3rnNuSu7Vv396ja5CQAABgI7mrbPK7eeL06dOKj4/XrFmzLnlO+/btdeDAAef27rvvenQNWjYAANiIFatsOnTooA4dOlz2nKCgIEVFRV11TFRIAAAopNLT0122jIyMqx5r7dq1Klu2rGrWrKkHH3xQR48e9ej9JCQAANiIw/DOJkkxMTGKiIhwbomJiVcVU/v27TVv3jwlJyfr2Wef1bp169ShQwdlZ2e7PQYtGwAA7MTwwoPNfn/73r17FR4e7twdFBR0VcP16NHD+ee6deuqXr16qlatmtauXavWrVu7NQYVEgAAbMSbk1rDw8NdtqtNSP6qatWqKl26tFJTU91+DwkJAADwqn379uno0aMqX7682++hZQMAgI0Yv/+T3zE8cerUKZdqR1pamrZt26bIyEhFRkZqwoQJuvvuuxUVFaWdO3dq+PDhio2NVbt27dy+BgkJAAA28udJqfkZwxNbtmxRq1atnK+HDBkiSUpISNDs2bP17bff6q233tLx48cVHR2ttm3b6qmnnvKoBURCAgAALqtly5YyTfOSxz/77LN8X4OEBAAAG7HiwWjXglsJyccff+z2gHfeeedVBwMAAC7vah79frExfI1bCUmnTp3cGswwDI8eggIAACC5mZDk5OQUdBwAAMANDsOQI58ljvy+vyDkaw7JuXPnFBwc7K1YAADAFfhry8bjB6NlZ2frqaee0nXXXafixYtr165dkqQxY8bojTfe8HqAAADgD7mTWvO7+RqPE5JJkyYpKSlJU6ZMUWBgoHN/nTp1NGfOHK8GBwAACgePE5J58+bptddeU69evRQQEODcHx8fr59++smrwQEAAFfe/C0bX+LxHJJffvlFsbGxefbn5OQoKyvLK0EBAICL89dJrR5XSGrXrq0NGzbk2f/BBx+oQYMGXgkKAAAULh5XSMaOHauEhAT98ssvysnJ0YcffqiUlBTNmzdPy5cvL4gYAQDA74zft/yO4Ws8rpB07NhRy5Yt06pVqxQaGqqxY8dq+/btWrZsmW655ZaCiBEAAPzOX1fZXNVzSJo3b66VK1d6OxYAAFBIXfWD0bZs2aLt27dLujCvpGHDhl4LCgAAXJzDuLDldwxf43FCsm/fPvXs2VOff/65SpQoIUk6fvy4brrpJr333nuqUKGCt2MEAAC/89df+/V4DsmAAQOUlZWl7du369ixYzp27Ji2b9+unJwcDRgwoCBiBAAAf+JvzyCRrqJCsm7dOm3atEk1a9Z07qtZs6ZefPFFNW/e3KvBAQCAwsHjhCQmJuaiD0DLzs5WdHS0V4ICAAAXR8vmd1OnTtWjjz6qLVu2OPdt2bJFjz/+uJ577jmvBgcAAFzlTmrN7+Zr3KqQlCxZ0iWbOn36tBo3bqwiRS68/fz58ypSpIjuvfdederUqUACBQAA/suthGT69OkFHAYAAHCHv7Zs3EpIEhISCjoOAADgBn99dPxVPxhNks6dO6fMzEyXfeHh4fkKCAAAFD4eJySnT5/WiBEjtGjRIh09ejTP8ezsbK8EBgAA8nIYhhz5bLnk9/0FweNVNsOHD9fq1as1e/ZsBQUFac6cOZowYYKio6M1b968gogRAAD8Lr8PRfPVh6N5XCFZtmyZ5s2bp5YtW6pfv35q3ry5YmNjValSJb3zzjvq1atXQcQJAADkv5NaPa6QHDt2TFWrVpV0Yb7IsWPHJEnNmjXT+vXrvRsdAAAoFDxOSKpWraq0tDRJUq1atbRo0SJJFyonuT+2BwAACoa/tmw8Tkj69eunb775RpL05JNPatasWQoODtbgwYP1xBNPeD1AAADwh9xJrfndfI3Hc0gGDx7s/HObNm30008/aevWrYqNjVW9evW8GhwAACgc8vUcEkmqVKmSKlWq5I1YAADAFXij5eKDBRL3EpKZM2e6PeBjjz121cEAAIDL89dVNm4lJNOmTXNrMMMwSEjyKe66cJ52C7/T7K5RVocAeJ2ZnXnlk+A2txKS3FU1AADAWg5dxYqUi4zha/I9hwQAAFw7hbplAwAAfINhSA4/nNTqi1UbAABQyFAhAQDARhxeqJDk9/0FgYQEAAAb8dc5JFfVstmwYYN69+6tJk2a6JdffpEkzZ8/Xxs3bvRqcAAAoHDwOCFZvHix2rVrp5CQEH399dfKyMiQJJ04cUKTJ0/2eoAAAOAPuS2b/G6+xuOE5Omnn9Yrr7yi119/XUWLFnXub9q0qb766iuvBgcAAFzxa7+/S0lJ0c0335xnf0REhI4fP+6NmAAAQCHjcUISFRWl1NTUPPs3btyoqlWreiUoAABwcQ7D8MrmazxOSAYOHKjHH39cX375pQzD0P79+/XOO+9o2LBhevDBBwsiRgAA8DuHlzZf4/Gy3yeffFI5OTlq3bq1zpw5o5tvvllBQUEaNmyYHn300YKIEQAA/M4bc0B8sEDieUJiGIZGjx6tJ554QqmpqTp16pRq166t4sWLF0R8AACgELjqB6MFBgaqdu3a3owFAABcgUP5nwPikO+VSDxOSFq1anXZJ7ytXr06XwEBAIBLo2Xzu/r167u8zsrK0rZt2/T9998rISHBW3EBAIBCxOOEZNq0aRfdP378eJ06dSrfAQEAgEvz1x/X89rKn969e+vNN9/01nAAAOAiDCP/zyLxxZaN1xKSzZs3Kzg42FvDAQCAQsTjlk3nzp1dXpumqQMHDmjLli0aM2aM1wIDAAB5Man1dxERES6vHQ6HatasqYkTJ6pt27ZeCwwAAOTlr3NIPEpIsrOz1a9fP9WtW1clS5YsqJgAAMAlGL//k98xfI1Hc0gCAgLUtm1bftUXAAB4lceTWuvUqaNdu3YVRCwAAOAKcls2+d18jccJydNPP61hw4Zp+fLlOnDggNLT0102AABQcPw1IXF7DsnEiRM1dOhQ3XrrrZKkO++80+UR8qZpyjAMZWdnez9KAADg19xOSCZMmKAHHnhAa9asKch4AADAZRiGcdnflHN3DF/jdkJimqYkqUWLFgUWDAAAuDx/Xfbr0RwSX8yoAACA/Xn0HJIaNWpcMSk5duxYvgICAACXxpNadWEeyV+f1AoAAK6d3B/Iy+8YvsajhKRHjx4qW7ZsQcUCAACuoNDPIWH+CAAAKCger7IBAAAW8sIcEh/8KRv3KyQ5OTm0awAAsJhDhlc2T6xfv1533HGHoqOjZRiGli5d6nLcNE2NHTtW5cuXV0hIiNq0aaMdO3Z4+LkAAAAu4/Tp04qPj9esWbMuenzKlCmaOXOmXnnlFX355ZcKDQ1Vu3btdO7cObev4dGkVgAAYC0rlv126NBBHTp0uOgx0zQ1ffp0/d///Z86duwoSZo3b57KlSunpUuXqkePHm5dgwoJAAA24s0f1/vrD+RmZGR4HE9aWpoOHjyoNm3aOPdFRESocePG2rx5s/ufy+MrAwAAvxATE6OIiAjnlpiY6PEYBw8elCSVK1fOZX+5cuWcx9xBywYAABvx5oPR9u7dq/DwcOf+oKCgfI2bH1RIAACwkdw5JPndJCk8PNxlu5qEJCoqSpJ06NAhl/2HDh1yHnMHCQkAADbikOGsklz15sUHkVSpUkVRUVFKTk527ktPT9eXX36pJk2auD0OLRsAAHBZp06dUmpqqvN1Wlqatm3bpsjISFWsWFGDBg3S008/rerVq6tKlSoaM2aMoqOj1alTJ7evQUICAICNWLHsd8uWLWrVqpXz9ZAhQyRJCQkJSkpK0vDhw3X69Gndd999On78uJo1a6YVK1YoODjY7WuQkAAAYCMO5X++hafvb9my5WV/QsYwDE2cOFETJ068ZjEBAAB4HRUSAABsxDAMGfns2eT3/QWBhAQAABsxlP8f6/W9dISWDQAA8AFUSAAAsBFvPqnVl5CQAABgM76XTuQfCQkAADZixXNIrgXmkAAAAMtRIQEAwEZY9gsAACxnxZNarwVfjAkAABQyVEgAALARWjYAAMByPKkVAACggFAhAQDARmjZAAAAy/nrKhsSEgAAbMRfKyS+mCQBAIBChgoJAAA24q+rbEhIAACwEX5cDwAAoIBQIQEAwEYcMuTIZ9Mlv+8vCCQkAADYCC0bAACAAkKFBAAAGzF+/ye/Y/gaEhIAAGzEX1s2JCQAANiI4YVJrb5YIWEOCQAAsBwVEgAAbISWDQAAsJy/JiS0bAAAgOWokAAAYCMs+wUAAJZzGBe2/I7ha2jZAAAAy1EhAQDARmjZAAAAy7HKBvADGzes192d7lCVitEKKWro44+WWh0S4LFh97bVxref0OGNz+nn5EQtemGgqlcqe8nzl770oM5+/ZLuaFnvGkaJgmLojyrJ1f/je0hIUKicPn1adevFa/rMWVaHAly15n+L1SsL16tFn+d0+4MvqUiRAC2f/YiKBQfmOffRXq1kmhYECXiIlg0KlXbtO6hd+w5WhwHkS8dHXnZ5fd+4t7V39TNqUDtGn3+107m/Xo3r9Pg9/1DTXlO0e1XitQ4TBcRfV9mQkACAzYUXD5Yk/XbijHNfSHBRJSX21aBnFunQ0ZNWhYYC4K+TWn2yZbN7924ZhqFt27ZZHQoA+DTDMDR1WBdt+nqnftx5wLl/ytC79cU3aVq+9jsLowPc55MJiR3NmjVLlStXVnBwsBo3bqz//Oc/VocEoBCYPrKbro8trz5PznXuu61FXbW8oYaemPqBhZGhoOSussnv5msKVUKSmZlZIOMuXLhQQ4YM0bhx4/TVV18pPj5e7dq10+HDhwvkegAgSdNGdNWtzeuo3cCZ+uXwcef+ln+voaoVSuvg+qk6+d8ZOvnfGZKkd58boM9ef9yiaOEthpc2X2NpQpKTk6MpU6YoNjZWQUFBqlixoiZNmuQ8vmvXLrVq1UrFihVTfHy8Nm/e7Dw2fvx41a9f32W86dOnq3Llys7Xffv2VadOnTRp0iRFR0erZs2aznbQhx9+eMmxPfXCCy9o4MCB6tevn2rXrq1XXnlFxYoV05tvvnnVYwLA5Uwb0VV3/iNe7e+fqZ/3H3U59tzcf+vv3RLVuMczzk2Shj+/WPeNe9uKcIErsnRS68iRI/X6669r2rRpatasmQ4cOKCffvrJeXz06NF67rnnVL16dY0ePVo9e/ZUamqqihRxP+zk5GSFh4dr5cqVLvsvN/aePXtUu3bty447atQojRo1SpmZmdq6datGjhzpPOZwONSmTZvLJjkZGRnKyMhwvk5PT3f7M+HqnTp1SjtTU52vd6el6Ztt21QyMlIVK1a0MDLAfdNHdlP3Do3UdfBrOnX6nMqVCpMknTh1TucysnTo6MmLTmTde+C3PMkL7MchQ4589lwcPlgjsSwhOXnypGbMmKGXXnpJCQkJkqRq1aqpWbNm2r17tyRp2LBhuu222yRJEyZM0PXXX6/U1FTVqlXL7euEhoZqzpw5Cgy8sD7fnbGjo6OvOKE2MjJSkvTrr78qOztb5cqVczlerlw5l+TqrxITEzVhwgS3Pwe846utW9SuTSvn6xFPDJEk9b4nQa+/mWRRVIBn7u92syRp5ZxBLvsHjp2vt5d9aUFEuJa80XLxvXTEwoRk+/btysjIUOvWrS95Tr16fzxVsHz58pKkw4cPe5SQ1K1b15mMuDt2kSJFFBsb6/Y1rsbIkSM1ZMgQ5+v09HTFxMQU6DUh3dyipc5m8ZQo2FtIg0euyXvgo/w0I7FsDklISMgVzylatKjzz8bv5amcnBxJF9oi5l8eP5iVlZVnjNDQUI/H3rNnj4oXL37ZbfLkyZKk0qVLKyAgQIcOHXIZ/9ChQ4qKirrkZwsKClJ4eLjLBgBAYWVZhaR69eoKCQlRcnKyBgwY4PH7y5Qpo4MHD8o0TWdC4a3nlnjSsgkMDFTDhg2VnJysTp06SbqQ2CQnJ+uRR/gvEgCAd/nrg9EsS0iCg4M1YsQIDR8+XIGBgWratKmOHDmiH3744bJtnFwtW7bUkSNHNGXKFHXp0kUrVqzQp59+6pVKg6ctmyFDhighIUGNGjXSDTfcoOnTp+v06dPq169fvmMBAMCFN54j4nv5iLWrbMaMGaMiRYpo7Nix2r9/v8qXL68HHnjArffGxcXp5Zdf1uTJk/XUU0/p7rvv1rBhw/Taa68VcNR5de/eXUeOHNHYsWN18OBB1a9fXytWrMgz0RUAAFycYf51IgYskZ6eroiICB06eoL5JPA7Jf9O+xL+x8zOVMZ3r+vEiWvzvZ3774nV2/aoeFj+rnfqZLr+Ub/iNYvdHfy4HgAAdsIqGwAAgIJBhQQAABthlQ0AALCcN36t1xd/7ZeEBAAAG/HTKSTMIQEAANajQgIAgJ34aYmEhAQAABvx10mttGwAAIDlqJAAAGAjrLIBAACW89MpJLRsAACA9aiQAABgJ35aIiEhAQDARvx1lQ0JCQAANuKvk1qZQwIAACxHhQQAABvx0ykkVEgAALAVw0ubm8aPHy/DMFy2WrVqee3j5KJCAgAALuv666/XqlWrnK+LFPF++kBCAgCAjVixyqZIkSKKiorK1zWvhJYNAAA2krvKJr+bJKWnp7tsGRkZF73mjh07FB0drapVq6pXr17as2eP1z8XCQkAAIVUTEyMIiIinFtiYmKecxo3bqykpCStWLFCs2fPVlpampo3b66TJ096NRZaNgAA2Ig3V9ns3btX4eHhzv1BQUF5zu3QoYPzz/Xq1VPjxo1VqVIlLVq0SP37989nJH8gIQEAwE68mJGEh4e7JCTuKFGihGrUqKHU1NR8BuGKlg0AADZieOmfq3Xq1Cnt3LlT5cuX9+KnIiEBAACXMWzYMK1bt067d+/Wpk2bdNdddykgIEA9e/b06nVo2QAAYCPX+rds9u3bp549e+ro0aMqU6aMmjVrpi+++EJlypTJXxB/QUICAICNXOtHx7/33nv5vJp7aNkAAADLUSEBAMBO/PTX9UhIAACwESseHX8t0LIBAACWo0ICAICdeGGVjQ8WSEhIAACwEz+dQkJCAgCArfhpRsIcEgAAYDkqJAAA2Ii/rrIhIQEAwEau9aPjrxVaNgAAwHJUSAAAsBE/ndNKQgIAgK34aUZCywYAAFiOCgkAADbCKhsAAGA5Q15YZeOVSLyLhAQAABvx0ykkzCEBAADWo0ICAICN+OuD0UhIAACwFf9s2tCyAQAAlqNCAgCAjdCyAQAAlvPPhg0tGwAA4AOokAAAYCO0bAAAgOV4dDwAALCen04iYQ4JAACwHBUSAABsxE8LJCQkAADYib9OaqVlAwAALEeFBAAAG2GVDQAAsJ6fTiKhZQMAACxHhQQAABvx0wIJCQkAAHbir6tsSEgAALCV/E9q9cUaCXNIAACA5aiQAABgI/7asqFCAgAALEdCAgAALEfLBgAAG/HXlg0JCQAANuKvj46nZQMAACxHhQQAABuhZQMAACzHo+MBAID1/DQjYQ4JAACwHBUSAABsxF9X2ZCQAABgI/46qZWWDQAAsBwVEgAAbMRP57SSkAAAYCt+mpHQsgEAAJajQgIAgI2wygYFyjRNSdLJ9HSLIwG8z8zOtDoEwOty7+vc7+9r5eTJ9Hyvkjl50vf+XUNC4iNOnjwpSYqtEmNxJAAAT5w8eVIREREFfp3AwEBFRUWpupf+PREVFaXAwECvjOUNhnmtUztcVE5Ojvbv36+wsDAZvrhA3M+kp6crJiZGe/fuVXh4uNXhAF7BfX1tmaapkydPKjo6Wg7HtZmSee7cOWVmeqfiGBgYqODgYK+M5Q1USHyEw+FQhQoVrA6j0AkPD+eLG36H+/rauRaVkT8LDg72qSTCm1hlAwAALEdCAgAALEdCgkIpKChI48aNU1BQkNWhAF7DfQ07Y1IrAACwHBUSAABgORISAABgORISAABgORISAABgORISAABgORISAABgORISACjk/vr0B54GASvwWzaAG3JycuRwOHT+/HkVKcL/beA/cu/tY8eO6eTJk8rJyVGVKlWcx03T5Ac/cU1QIQGuIPcLOyUlRePHj1daWprVIQFekXtvf/fdd2rSpIluu+02Va9eXX369NHSpUslSYZhUDHBNcF/6gFX4HA4tHPnTrVo0UKHDx/WwYMHNWHCBF133XVWhwbki8Ph0IEDB9ShQwd1795d99xzj3bt2qXXXntNTz/9tH755Rc9/PDDVEhwTZCQAFdw9uxZTZs2TW3btlXHjh31z3/+U1lZWZo8eTJJCWzvxx9/VGRkpP7v//5PJUuWVP369VWjRg29+uqrevnllxUcHKz+/ftbHSYKARIS4Aqys7P197//XYGBgbr77ru1Zs0atWrVSpJISmB7gYGB2rdvn1JSUnTjjTdKkurUqaPHHntMmZmZevvtt9WoUSPFx8dbHCn8HXNIgCsoXry4OnXqpJ49e0qSbrrpJiUnJ+u9997TyJEj9csvv0i60I//8ccfrQwV8Fh0dLSio6P12WefKSMjw7m/evXquu+++5SSkqItW7ZYGCEKCxIS4DJyJ/NFREQ4X+fk5KhZs2ZKTk7WwoULNWrUKKWlpWnw4MEaPHiwTp48aWXIgFty7+1q1appwIABmjhxohYuXOgygbVhw4Zq1KiRVq9ebVWYKERo2QCXcLElvoZhyDAMZWdnq1mzZlq9erXatWun1atX69ChQ/ryyy8VFhZmUcSAe3Lv7dxVNoMGDdLBgwc1cOBAnTlzRt26dVNkZKSkC4lLxYoVLY4YhYFhsp4LyOPPX9gPPvig+vTpo6ZNm7qck/tl3qlTJ23YsEHr1q1TnTp1LIoYcM9f7+3evXurefPmkqTx48fr2Wef1Z133qmyZcsqKytL7777rjZv3qzatWtbHDn8HRUS4C/+/IXdqFEjhYeHq3Hjxhc9d+TIkfr444/19ddfk4zA513s3m7SpInz+Pjx41W3bl2tXbtW27ZtU+XKlbVhwwaSEVwTVEiAP/nzF/YNN9ygyMhIffLJJypatKimTJmiBg0a6JZbbnGeu2zZMsXGxqpu3boWRw5c3pXu7Xr16ql9+/aS/phfkpWVpcDAQCvDRiFCQgL87q9f2CVKlNCnn36qokWLqm/fvkpOTtbnn39OPx224+m9zePiYQVW2QC68KyR3C/sxo0bu3xh33vvvfr888/12WefkYzAdq7m3iYZgRVISABJAQEBysnJUfXq1RUZGenyhb1u3Tp9/PHH9NFhS9zbsAsSEuB3O3bs0E033aRly5apaNGi6t+/v9atW6fly5crLi7O6vCAq8a9DTtgDgkKpSv1yLt27apvvvlGH330EV/YsBXubdgVy35R6OR+Ya9Zs0aff/65Tp8+raFDh6p06dKSpD179ujIkSP68MMP+cKGrXBvw86okKBQWr58ubp27aobb7xRqampkqS3335bzZs3l8PhUEZGhoKCgiyOEvAc9zbsijkkKHROnz6tFStW6OWXX1ZycrL+97//6YYbblDXrl21du1amabJFzZsiXsbdkZCgkLl66+/VlxcnL777jtVr15dDodDISEhWrx4sZo3b67u3bs7v7gBO+Heht2RkMDv5X4Br1+/XpGRkbr++uu1YcMG/fbbb5Iu/CaNJC1evFj/+Mc/1Lp1a23YsMGyeAF3cW/Dn5CQwO/lTvJr2bKlvv76ay1YsEAtWrTQoEGDtH37djkcDucX+8KFC3XPPfcoKirK4qiBK+Pehj9hUiv83q5duzR//nxFRERo0KBBkqTjx4/r9ttv1+HDh53LH3lcNuyGexv+hAoJ/NoPP/yg/v37KykpSRUqVJB04Xc9SpQooeXLl6ts2bLq0qWLvv/+e76wYSvc2/A3JCTwO38u+gUGBiomJkbHjh3T+vXrJUlFihRRdna2SpQooU8++USGYahv377KzMy0KmTALdzb8Ge0bOCXNm/erKioKFWpUkU///yzEhMTtWHDBg0cONBZ2s7OzlZAQIBOnDih3377TZUrV7Y0ZsAd3NvwVzypFX7n2LFjGj16tPbt26eVK1eqUqVKGj58uEzT1MKFC2UYhh5//HEFBAQoOztbERERioiIsDps4Iq4t+HPaNnA70RGRurJJ59U1apV1alTJ6Wlpalq1aoaPny46tWrpw8++EDPPPOMpAu/hArYBfc2/BkJCWwvt+t4/vx55762bdvqiSeeUKlSpdS5c2elpaWpWrVqGjFihCpWrKjVq1c7n9UA+CrubRQmzCGBX/jyyy81fvx4vfPOO4qMjHTuT05O1rhx45SRkaElS5aoQoUK2r17t4KDg3keA2yBexuFBRUS+IXU1FQdOHBA/fr1c/mvw9atW6tHjx7aunWrWrVqpd27d6ty5cp8YcM2uLdRWJCQwHYuVtTr3r27RowYoSNHjqh37946evSo81jt2rXVqlUr3Xjjjc5HaQO+iHsbhRktG9hO7lMnv/32W/36668KCgpS06ZNlZOTo0WLFunFF19UWFiY3n77bZUuXVrjxo3TkSNH9OyzzyosLMzq8IFL4t5GoWYCNjB58mRz1KhRZnZ2tmmapvnhhx+aoaGhZo0aNUzDMMzhw4ebmZmZZnZ2tvn++++bTZo0MYsUKWI2bdrULFasmPndd99Z/AmAi+PeBi7gOSSwheLFi2v06NEKDQ3VgAEDNHnyZL300ktq3ry5tm7dqt69e+v48eOaMWOGunTpoqZNm2rhwoWSpLlz56p69eoWfwLg4ri3gQtISODzTNPUo48+qpCQEN1///1KT09XvXr1dPfddyssLEzVqlVTeHi4OnbsKEmaOnWqypcv73xqJeCruLeBPzCHBD4t9/Y0DEOmaerdd99Vnz59VLp0aW3dulXXXXedcnJy5HA49Nlnn+nuu+/WXXfdpZkzZ6pkyZIWRw9cGvc24IpVNvB5hmFo1apVGjp0qOLj47VgwQIdOXJEr7/+uvML2zRNtWvXTu+++67+/e9/KyMjw+qwgSvi3gb+QMsGPs0wDH344Yfq3bu3Ro0apbNnz6pbt246ceKEHnjgARUtWlQjR450fnHfcccd2rVrl0JDQ60OHbgs7m3AFQkJfNr//vc/DRs2TM8//7wefPBB5/6BAwfKNE09+OCDcjgcGjFihByOCwU/vrBhB9zbgCsSEvi0PXv2qGjRorr11lud+3JL2ffdd59CQ0N1zz33qGjRoho2bJiFkQKe4d4GXJGQwKedOnVKZ8+edb7OycmRYRiSpLVr16phw4ZauHCh6tSpY1WIwFXh3gZcMakVPi0+Pl6//vqrXnvtNUmSw+Fwfml/9NFHWrBggTp37qy4uDgrwwQ8xr0NuKJCAp9WpUoVvfTSS3rggQeUlZWlPn36KCAgQElJSUpKStLmzZsVEBBgdZiAx7i3AVc8hwQ+LycnR4sXL9b999+v0NBQBQcHKyAgQO+++64aNGhgdXjAVePeBv5AQgLb2L9/v37++WcZhqEqVaqoXLlyVocEeAX3NkBCAgAAfACTWgEAgOVISAAAgOVISAAAgOVISAAAgOVISAAAgOVISAAAgOVISAAAgOVISAAAgOVISACob9++6tSpk/N1y5YtNWjQoGsex9q1a2UYho4fP37JcwzD0NKlS90ec/z48apfv36+4tq9e7cMw9C2bdvyNQ6ASyMhAXxU3759ZRiGDMNQYGCgYmNjNXHiRJ0/f77Ar/3hhx/qqaeecutcd5IIALgSfu0X8GHt27fX3LlzlZGRoX/96196+OGHVbRoUY0cOTLPuZmZmQoMDPTKdSMjI70yDgC4iwoJ4MOCgoIUFRWlSpUq6cEHH1SbNm308ccfS/qjzTJp0iRFR0erZs2akqS9e/eqW7duKlGihCIjI9WxY0ft3r3bOWZ2draGDBmiEiVKqFSpUho+fLj++pNWf23ZZGRkaMSIEYqJiVFQUJBiY2P1xhtvaPfu3WrVqpUkqWTJkjIMQ3379pV04ZdsExMTVaVKFYWEhCg+Pl4ffPCBy3X+9a9/qUaNGgoJCVGrVq1c4nTXiBEjVKNGDRUrVkxVq1bVmDFjlJWVlee8V199VTExMSpWrJi6deumEydOuByfM2eO4uLiFBwcrFq1aunll1/2OBYAV4+EBLCRkJAQZWZmOl8nJycrJSVFK1eu1PLly5WVlaV27dopLCxMGzZs0Oeff67ixYurffv2zvc9//zzSkpK0ptvvqmNGzfq2LFjWrJkyWWv26dPH7377ruaOXOmtm/frldffVXFixdXTEyMFi9eLElKSUnRgQMHNGPGDElSYmKi5s2bp1deeUU//PCDBg8erN69e2vdunWSLiROnTt31h133KFt27ZpwIABevLJJz3+OwkLC1NSUpJ+/PFHzZgxQ6+//rqmTZvmck5qaqoWLVqkZcuWacWKFfr666/10EMPOY+/8847Gjt2rCZNmqTt27dr8uTJGjNmjN566y2P4wFwlUwAPikhIcHs2LGjaZqmmZOTY65cudIMCgoyhw0b5jxerlw5MyMjw/me+fPnmzVr1jRzcnKc+zIyMsyQkBDzs88+M03TNMuXL29OmTLFeTwrK8usUKGC81qmaZotWrQwH3/8cdM0TTMlJcWUZK5cufKica5Zs8aUZP7222/OfefOnTOLFStmbtq0yeXc/v37mz179jRN0zRHjhxp1q5d2+X4iBEj8oz1V5LMJUuWXPL41KlTzYYNGzpfjxs3zgwICDD37dvn3Pfpp5+aDofDPHDggGmaplmtWjVzwYIFLuM89dRTZpMmTUzTNM20tDRTkvn1119f8roA8oc5JIAPW758uYoXL66srCzl5OTon//8p8aPH+88XrduXZd5I998841SU1MVFhbmMs65c+e0c+dOnThxQgcOHFDjxo2dx4oUKaJGjRrladvk2rZtmwICAtSiRQu3405NTdWZM2d0yy23uOzPzMxUgwYNJEnbt293iUOSmjRp4vY1ci1cuFAzZ87Uzp07derUKZ0/f17h4eEu51SsWFHXXXedy3VycnKUkpKisLAw7dy5U/3799fAgQOd55w/f14REREexwPg6pCQAD6sVatWmj17tgIDAxUdHa0iRVz/LxsaGury+tSpU2rYsKHeeeedPGOVKVPmqmIICQnx+D2nTp2SJH3yyScuiYB0YV6Mt2zevFm9evXShAkT1K5dO0VEROi9997T888/73Gsr7/+ep4EKSAgwGuxArg8EhLAh4WGhio2Ntbt8//2t79p4cKFKlu2bJ4qQa7y5cvryy+/1M033yzpQiVg69at+tvf/nbR8+vWraucnBytW7dObdq0yXM8t0KTnZ3t3Fe7dm0FBQVpz549l6ysxMXFOSfo5vriiy+u/CH/ZNOmTapUqZJGjx7t3Pfzzz/nOW/Pnj3av3+/oqOjnddxOByqWbOmypUrp+joaO3atUu9evXy6PoAvIdJrYAf6dWrl0qXLq2OHTtqw4YNSktL09q1a/XYY49p3759kqTHH39czzzzjJYuXaqffvpJDz300GWfIVK5cmUlJCTo3nvv1dKlS51jLlq0SJJUqVIlGYah5cuX68iRIzp16pTCwsI0bNgwDR48WG+99ZZ27typr776Si+++KJzougDDzygHTt26IknnlBKSooWLFigpKQkjz5v9erVtWfPHr333nvauXOnZs6cedEJusHBwUpISNA333yjDRs26LHHHlO3bt0UFRUlSZowYYISExM1c+ZM/e9//9N3332nuXPn6oUXXvAoHgBXj4QE8CPFihXT+vXrVbFiRXXu3FlxcXHq37+/zp0756yYDB06VPfcc48SEhLUpEkThYWF6a677rrsuLNnz1aXLl300EMPqVatWho4cKBOnz4tSbruuus0YcIEPfnkkypXrpweeeQRSdJTTz2lMWPGKDExUXFxcWrfvr0++eQTValSRdKFeR2LFy/W0qVLFR8fr1deeUWTJ0/26PPeeeedGjx4sB555BHVr19fmzZt0pgxY/KcFxsbq86dO+vWW29V27ZtVa9ePZdlvQMGDNCcOXM0d+5c1a1bVy1atFBSUpIzVgAFzzAvNZMNAADgGqFCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALEdCAgAALPf/LUev8fM1Hn8AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "# Compute confusion matrix\n",
        "cnf_matrix = confusion_matrix(y_test, yhat, labels=[1,0])\n",
        "np.set_printoptions(precision=2)\n",
        "\n",
        "\n",
        "# Plot non-normalized confusion matrix\n",
        "plt.figure()\n",
        "plot_confusion_matrix(cnf_matrix, classes=['churn=1','churn=0'],normalize= False,  title='Confusion matrix')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AP1WjWDmhO-0"
      },
      "source": [
        "Let's look at first row. The first row is for customers whose actual churn value in the test set is 1.\n",
        "As you can calculate, out of 40 customers, the churn value of 15 of them is 1.\n",
        "Out of these 15 cases, the classifier correctly predicted 6 of them as 1, and 9 of them as 0.\n",
        "\n",
        "This means, for 6 customers, the actual churn value was 1 in test set and classifier also correctly predicted those as 1. However, while the actual label of 9 customers was 1, the classifier predicted those as 0, which is not very good. We can consider it as the error of the model for first row.\n",
        "\n",
        "What about the customers with churn value 0? Lets look at the second row.\n",
        "It looks like  there were 25 customers whom their churn value were 0.\n",
        "\n",
        "\n",
        "The classifier correctly predicted 24 of them as 0, and one of them wrongly as 1. So, it has done a good job in predicting the customers with churn value 0. A good thing about the confusion matrix is that it shows the model’s ability to correctly predict or separate the classes.  In a specific case of the binary classifier, such as this example,  we can interpret these numbers as the count of true positives, false positives, true negatives, and false negatives.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u402pnZahO-1",
        "outputId": "02bb8ea6-11d1-47da-9135-04b087059bb7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.73      0.96      0.83        25\n",
            "           1       0.86      0.40      0.55        15\n",
            "\n",
            "    accuracy                           0.75        40\n",
            "   macro avg       0.79      0.68      0.69        40\n",
            "weighted avg       0.78      0.75      0.72        40\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print (classification_report(y_test, yhat))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BEgYrmb7hO-1"
      },
      "source": [
        "Based on the count of each section, we can calculate precision and recall of each label:\n",
        "\n",
        "\n",
        "- __Precision__ is a measure of the accuracy provided that a class label has been predicted. It is defined by: precision = TP / (TP + FP)\n",
        "\n",
        "- __Recall__ is the true positive rate. It is defined as: Recall =  TP / (TP + FN)\n",
        "\n",
        "    \n",
        "So, we can calculate the precision and recall of each class.\n",
        "\n",
        "__F1 score:__\n",
        "Now we are in the position to calculate the F1 scores for each label based on the precision and recall of that label.\n",
        "\n",
        "The F1 score is the harmonic average of the precision and recall, where an F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0. It is a good way to show that a classifer has a good value for both recall and precision.\n",
        "\n",
        "\n",
        "Finally, we can tell the average accuracy for this classifier is the average of the F1-score for both labels, which is 0.72 in our case.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3VADurkEhO-1"
      },
      "source": [
        "### log loss\n",
        "Now, let's try __log loss__ for evaluation. In logistic regression, the output can be the probability of customer churn is yes (or equals to 1). This probability is a value between 0 and 1.\n",
        "Log loss( Logarithmic loss) measures the performance of a classifier where the predicted output is a probability value between 0 and 1.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BwYDYuHyhO-1",
        "outputId": "1a0e9969-c36b-4eb1-dbe4-14e6c7269504"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6017092478101185"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "from sklearn.metrics import log_loss\n",
        "log_loss(y_test, yhat_prob)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CrfIkOhMhO-1"
      },
      "source": [
        "<h2 id=\"practice\">Practice</h2>\n",
        "Try to build Logistic Regression model again for the same dataset, but this time, use different __solver__ and __regularization__ values? What is new __logLoss__ value?\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a_sGBQNGhO-1",
        "outputId": "e04a9471-7d6b-4526-b361-2353b051316b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "LogLoss: : 0.61\n"
          ]
        }
      ],
      "source": [
        "# write your code here\n",
        "\n",
        "LR2 = LogisticRegression(C=0.01, solver='sag').fit(X_train,y_train)\n",
        "yhat_prob2 = LR2.predict_proba(X_test)\n",
        "print (\"LogLoss: : %.2f\" % log_loss(y_test, yhat_prob2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k3l47kIjhO-1"
      },
      "source": [
        "<details><summary>Click here for the solution</summary>\n",
        "\n",
        "```python\n",
        "LR2 = LogisticRegression(C=0.01, solver='sag').fit(X_train,y_train)\n",
        "yhat_prob2 = LR2.predict_proba(X_test)\n",
        "print (\"LogLoss: : %.2f\" % log_loss(y_test, yhat_prob2))\n",
        "\n",
        "```\n",
        "\n",
        "</details>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "button": false,
        "new_sheet": false,
        "run_control": {
          "read_only": false
        },
        "id": "MImLqIRYhO-1"
      },
      "source": [
        "<h2>Want to learn more?</h2>\n",
        "\n",
        "IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"https://www.ibm.com/analytics/spss-statistics-software?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork1047-2023-01-01\">SPSS Modeler</a>\n",
        "\n",
        "Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://www.ibm.com/cloud/watson-studio?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork1047-2023-01-01\">Watson Studio</a>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Ro1JkjuhO-2"
      },
      "source": [
        "### Thank you for completing this lab!\n",
        "\n",
        "\n",
        "## Author\n",
        "\n",
        "Saeed Aghabozorgi\n",
        "\n",
        "\n",
        "### Other Contributors\n",
        "\n",
        "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork1047-2023-01-01\">Joseph Santarcangelo</a>\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "## Change Log\n",
        "\n",
        "\n",
        "|  Date (YYYY-MM-DD) |  Version | Changed By  |  Change Description |\n",
        "|---|---|---|---|\n",
        "| 2021-01-21  | 2.2  | Lakshmi  |  Updated sklearn library|\n",
        "| 2020-11-03  | 2.1  | Lakshmi  |  Updated URL of csv |\n",
        "| 2020-08-27  | 2.0  | Lavanya  |  Moved lab to course repo in GitLab |\n",
        "|   |   |   |   |\n",
        "|   |   |   |   |\n",
        "\n",
        "\n",
        "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python",
      "language": "python",
      "name": "conda-env-python-py"
    },
    "language_info": {
      "name": ""
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}