0c-intro_scikit-learn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting started with scikit-learn\n",
    "\n",
    "Author : Alexandre Gramfort\n",
    "\n",
    "This just aims to have you started with scikit-learn. It assumes you\n",
    "understand machine learning concepts such as training/fitting and\n",
    "predicting, using cross-validation etc.\n",
    "\n",
    "Here we do classification using logistic regression\n",
    "or a linear SVM on a demo dataset.\n",
    "\n",
    "We encourage you to read:\n",
    "\n",
    "https://scikit-learn.org/stable/tutorial/index.html\n",
    "\n",
    "to learn more.\n",
    "\n",
    "`\n",
    "Reference:\n",
    "Scikit-learn: Machine Learning in Python,\n",
    "Pedregosa et al., JMLR 12, pp. 2825-2830, 2011.\n",
    "`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 4)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150,)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD5CAYAAAA3Os7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAeOUlEQVR4nO3df2wc533n8feXSzYJk1Q/aqJ1LIm8Qw4F7NSpLcJ1LkHPDdVDErsKDklTHRS0CmqwJZM2uSbwJRWg2gIE+3BFY7SubDBxD07ENs65aU+2k7vGTnyQ0EYFZTtyHBmFm4oW4+TMxI58ttqcSH7vj11K5HKXM8t99tlnZj8vYKDdZ2Znvvtw+Gg4853vmLsjIiLF19ftAEREJAwN6CIiJaEBXUSkJDSgi4iUhAZ0EZGS0IAuIlIS/XkXNLMKMAN8191vqpu3D/ivwHdrTXe5+2fXW99ll13mIyMjLQUrItLrTp48+QN3H2o0L/eADnwUOA38ZJP597v7R/KubGRkhJmZmRY2LyIiZjbbbF6uUy5mtg24EVj3qFtERLon7zn0O4FbgKV1lnmfmZ0yswfMbHujBcxs3MxmzGxmfn6+1VhFRGQdmQO6md0EvODuJ9dZ7EFgxN2vBh4B7mu0kLtPufuou48ODTU8BSQiIhuU5wj97cBuMzsDfAF4p5kdWbmAu//Q3X9ce/sZYGfQKEVEJFPmgO7un3L3be4+AuwBvubuH1y5jJldvuLtbqoXT0VEJKJWslxWMbODwIy7HwV+18x2AwvAi8C+MOGJiEheLd1Y5O6PLeegu/uB2mC+fBR/lbu/1d1/yd2f6USwUlLT0zAyAn191X+np7sdkUghbfgIXSSI6WkYH4fz56vvZ2er7wH27u1eXCIFpFv/pbv27780mC87f77aLiIt0YAu3fXcc621i0hTGtClu3bsaK1dRJrSgC7ddegQDA6ubhscrLaLSEs0oEt37d0LU1MwPAxm1X+npnRBVGQDlOUi3bd3rwZwkQB0hC4iUhIa0EVESkIDuohISWhAFxEpCQ3oIiIloQFdRKQkNKCLiJSEBnQRkZLQgC4iUhIa0KV9ekCFSBJ067+0Rw+oEEmGjtClPXpAhUgyNKBLe/SACpFkaECX9ugBFSLJ0IAu7dEDKkSSoQFd2qMHVIgkQ1ku0j49oEIkCTpCLzvliIv0DB2hl5lyxEV6io7Qy0w54iI9RQN6mSlHXKSnaEAvM+WIi/QUDehlphxxkZ6iAb3MlCMu0lOU5VJ2yhEX6Rm5j9DNrGJmT5jZQw3mvcbM7jezZ83shJmNhAxSRPn0ItlaOeXyUeB0k3m/Cbzk7m8GPg38l3YDE7loOZ9+dhbcL+XTa1AXWSXXgG5m24Abgc82WeS9wH211w8AY2Zm7YcngvLpRXLKe4R+J3ALsNRk/hXAWQB3XwDOAT9Vv5CZjZvZjJnNzM/PbyBc6UnKpxfJJXNAN7ObgBfc/eR6izVo8zUN7lPuPuruo0NDQy2EKT1N+fQiueQ5Qn87sNvMzgBfAN5pZkfqlpkDtgOYWT+wCXgxYJzSy5RPL5JL5oDu7p9y923uPgLsAb7m7h+sW+wo8Bu11++vLbPmCF1kQ5RPL5LLhvPQzewgMOPuR4F7gc+b2bNUj8z3BIpPpEr59CKZWrpT1N0fc/ebaq8P1AZz3P1f3P1X3f3N7n6du3+nE8FKF0xOQn9/9ci4v7/6XkSSpDtFpbnJSbj77kvvFxcvvT98uDsxiUhTquUizU1NtdYuIl2lAV2aW1xsrV1EukoDujRXqbTWLiJdpQFdmlt+/mjedhHpKl0UleaWL3xOTVVPs1Qq1cFcF0RFkqQBXdZ3+LAGcJGC0CmXItu1q5ofvjzt2tXtiDZGtc4lYSF2z2i7uLt3Zdq5c6dLG8bG3KvVwVdPY2Pdjqw1R464Dw6u/g6Dg9V2kS4LsXuG3sWp3qHfcFw171LJldHRUZ+ZmenKtkthvXLzRSqjMzJSfWBFveFhOHMmdjQiq4TYPUPv4mZ20t1HG83TKRfpLtU6l4SF2D1j7uIa0KW7VOtcEhZi94y5i2tAL6qxsdbaU6Va55KwELtnzF1cA3pRPfLI2sF7bKzaXiSqdS4JC7F7xtzFdVFURKRAdFG0rGIkyCpHXKQwdKdoUU1PV2/DP3+++n529lKNlbx/y2WtI8Q2RCQanXIpqhgJssoRF0mOTrmUUYwEWeWIixSKBvSiipEgqxxxkULRgF5UMRJklSMuUiga0IsqRoKscsRFCkUXRUVECkQXRVsVI/c6zzaUAy4lpt27A5rV1e30lGw99Bj1ufNsQ3XCpcS0e28cqofeghi513m2oRxwKTHt3hu33ikXDej1+voaPyDCDJaW4m0jRhwiXaLde+N0Dr0VMXKv82xDOeBSYtq9O0MDer0Yudd5tqEccCkx7d4d0uzkeqenZC+KulevzAwPu5tV/+3ElZo824gRh0iXaPfeGHRRVESkHNo6h25mrzWzvzezb5rZ02Z2W4Nl9pnZvJk9WZtuDhF4z5uchP7+6pWi/v7q+1bmQzo59SLSec0O3ZcnwIA31F4PACeA6+uW2QfclbUuL8oplxRMTKxO0l2eJibyzXdPJ6deRIIh1CkXMxsEjgMT7n5iRfs+YNTdP5J3XTrlkqG/HxYX17ZXKrCwkD0f0smpF5Fg2k5bNLOKmT0JvAB8deVgvsL7zOyUmT1gZtubrGfczGbMbGZ+fj73F+hJjQbrle1Z8yFOPXPVTBdJRq4B3d0X3f3ngW3AdWb2lrpFHgRG3P1q4BHgvibrmXL3UXcfHRoaaifu8qtU1m/Pmg/p5NSLSBQt5aG7+4+Ax4B31bX/0N1/XHv7GWBnkOh62fKzO5u1Z82HdHLqRSSOZifXlydgCNhce/064BhwU90yl694/R+Ab2StVxdFc5iYcK9UqhcaK5XVFzzzzHdPJ6deRIKgnYuiZnY11VMoFapH9F9094NmdrC24qNmdjuwG1gAXqR60fSZ9dari6IiIq1r66Kou59y92vc/Wp3f4u7H6y1H3D3o7XXn3L3q9z9re7+S1mDefJC5FXnyRFvdx0xaqqH+B4JmH5qmpE7R+i7rY+RO0eYfqr1n6lK2Evymh26d3pK9pRLiLzqPDni7a4jRk31EN8jAUdOHfHBQ4POrVycBg8N+pFT+X+mKmEvqUC3/rcgRF51nhzxdtcRo6Z6iO+RgJE7R5g9t7YfhjcNc+ZjZ/KtY0Ql7CUNqofeihCFms2az8vb31nriFFTPcT3SEDfbX04a+M1jKU/yPczVQl7SYXqobciRF51nhzxdtcRo6Z6iO+RgB2bGn/fZu0Nl1UJeykADej1QuRV58kRb3cdMWqqh/geCTg0dojBgdX9MDgwyKGx/D9TlbCXQmh2cr3TU7IXRd3D5FXnyRFvdx0xaqqH+B4JOHLqiA9/etjtVvPhTw+3dEH04jpUwl4SgC6KioiUg86hF1VWUrOSnpMzecdx+rfOYbZE/9Y5Ju84Hj+Gctw6IBvR7NC901PSp1xSkJXUrKTn5EzcfswZeGV12v7AKz5x+7F4MZTj1gFZBzrlUkBZSc1Kek5O/9Y5Fl/atqa9smWOhRfXtnckhnLcOiDr0CmXIsqqM6465MlZfOlNLbV3JIYcZfKlvDSgpyorqVlJz8mpbHm+pfaOxFCOWwdkgzSgpyorqVlJz8kZv+UMDLy6unHg1Wp7rBjKceuAbFSzk+udnnRRNIespGYlPSdn4vZjXtly1mHRK1vORr0gejGGctw6IE2gi6IiIuXQexdF283PzvP5GMm+yjPPLUS98xiy8tRj/MhD1HWPVV9eWtTs0L3TU8dOubSbn53n8zGSfZVnnluIeucxZOWpx/iRh6jrHqu+vDRGT51yaTc/O8/nYyT7Ks88txD1zmPIylOP8SMPUdc9Vn15aay36qG3W5Q6z+dj1AlXce3cQtQ7j8FsicZnOZdw74vyIw9R1z1WfXlprLfOobebn53n8zGSfZVnnluIeucxZOWpx/iRh6jrHqu+vLSufAN6u/nZeT4fI9lXeea5hah3HkNWnnqMH3mIuu6x6svLBjQ7ud7pqaN56O3mZ+f5fIxkX+WZ5xai3nkMWXnqMX7kIeq6x6ovL2vRUxdFRURKrLfOoYcQIkE2ax0qWt2TYuRe7/rEn2GbZzFbwjbPsusTf9b6On7tGayygJljlQV2/doz4QOV8Jodund6SvbW/xAJslnrUNHqnhQj93rs4/c2zHUf+/i9+dfxgdMOS3W755KPfeB0uEBlw9AplxaESJDNWoeKVvekGLnXtnkWzg2vnbFpFv9Rg/ZG66gswFL/2hl9C/hig3aJSqdcWhGiznjWOlS0uidFKWF/bntr7Y0sNUm/bdYuydCAXi9EgmzWOlS0uidFyb3edLa19kb6mhxYNGuXZGhArxciQTZrHSpa3ZNi5F6P3fxow1z3sZsfzb+O9z8La+4E9Vq7JK3ZyfVOT8leFHUPkyCbtQ4Vre5JMXKvxz5+r7PpjMOis+lMSxdEL67jA6edvgvVi6N9F3RBNCHooqiISDm0dVHUzF5rZn9vZt80s6fN7LYGy7zGzO43s2fN7ISZjbQfdhMhijnHkJVnXpDvEaLu9eTDk/Qf7MduM/oP9jP58Oq+CLKNAHXGs9YRw65d1V1medq1a+0yWf2VQs31PNtJYPcuyq9hfs0O3ZcnwIA31F4PACeA6+uWmQTuqb3eA9yftd4NnXIJUcw5hqw884J8jxB1rycemlj1+eVp4qGJcNsIUGc8ax0xjI013m3Gxi4tk9VfKdRcz7OdBHbvovwarkGoUy5mNggcBybc/cSK9v8F3Oruf2dm/cD3gSFfZ+UbOuUSophzDFl55gX5HiHqXvcf7GfR1/ZFxSosHFgIs40Adcaz1hFDnqrMWf2VQs11yFFTPVIc6ynIr+EabddDN7MKcBJ4M/Cn7v6f6+Z/C3iXu8/V3v8j8Avu/oO65caBcYAdO3bsnG3UU+sJUcw5hqzfzIJ8jxB1r+225n3hf+BhthGgznjWOmLIM6Bn9VcKNdchR0317u/eRfk1bLDtNm8scvdFd/95YBtwnZm9pX4bjT7WYD1T7j7q7qNDQ0N5Nr1aiGLOMWTlmRfke4Soe12xxn2x3B5kGwHqjGetIxVZ/ZVCzfU820lg9y7Kr2FLWjr0cPcfAY8B76qbNQdsB6idctkEvBggvtVCFHOOISvPvCDfI0Td6/GdjftiuT3INgLUGc9aRwxjY9ntWf2VQs31PNtJYPcuyq9ha5qdXF+egCFgc+3164BjwE11y3yY1RdFv5i13g3noYco5hxDVp55Qb5HiLrXEw9NeOW2inMrXrmtcvGCaNBtBKgznrWOGOovjK68ILosq79SqLmeZzsJ7N5F+TVchXYuiprZ1cB9QIXqEf0X3f2gmR2srfiomb0W+DxwDdUj8z3u/p311qs8dBGR1q13Dj2zdJq7n6I6UNe3H1jx+l+AX20nSBERaU85a7kU6k6A8su6ESbEjUXtxhAqzsybaQJ81xj9lQL9GreufLf+T09XLz6eP3+pbXAQpqZg797w25N1TT81zfiD45y/cOnnMTgwyNSvTLH35/Zmzo8RQ6g4s3a9EN81Rn+lQL/GzbWdh94JHRvQU7wToIdl3QgT4saidmMIFWfmzTQBvmuM/kqBfo2b660HXER5ioDk9dy5xv2+3J41P0YMeZbJtY6MXS/Ed43RXynQr/HGlG9AL9qdACWXdSNMiBuL2o0hzzK51pF1M02A7xqjv1KgX+ONKd+AXrg7Acot60aYEDcWtRtDqDgzb6YJ8F1j9FcK9Gu8Qc0S1Ds9dfQBF6ndCdDjsm6ECXFjUbsxhIoz82aaAN81Rn+lQL/GjaEHXIiIlENvXRSV5GTlE2c9ACPPOkLIfBBHgIchZD73JJGc/KKIsV8UKh++2aF7p6eknykqwWQ9ICDrARh51hFC5oM4AjwMIfO5JwEe9pElxjZiibFflPoBFyHplEtvyMonznoARp51hJD5II4cMWR+16znniSSk18UMfaLFPPhdcpFuiYrn7jRIFrfHiMnOSuOPDFkftfGm7jYnkpOflHE2C+Klg+vAV06KiufOOsBGHnWEULmgzgCPAwh87knieTkF0WM/aJo+fAa0KWjsvKJsx6AkWcdIWQ+iCPAwxAyn3uSSE5+UcTYLwqXD9/s5HqnJ10U7R1Z+cRZD8DIs44QMh/EEeBhCJnPPUkkJ78oYuwXqeXDo4uiIiLloIuiPSyFnOMQMVx142NY3wJmjvUtcNWNj0WPIdd2suqhFymnWQpHR+gllkLt7BAxXHXjY3z7y/8OsBWtzpXv+d88/fANUWLII7Meump8SwC9VQ9dLkoh5zhEDNa3AN7gaYm2gC9lPkUxWj9k1kPPmC+Sh0659KgUco6DxOBN8v2atXcihjzbyaqHXrCcZikeDegllkLOcZAYrMkdOc3aOxFDnu1k1UMvWE6zFI8G9BJLIec4RAxXvvs4UH9q0GvtcWLItZ2seuhFy2mW4mmWz9jpSXnocaSQcxwihivf83XHLjgsOXbBr3zP16PHkGs7WfXQE8tpluJBeegiIuWgi6LSUe3meOf5fJQ64coRlwaKtF9k53yJrKM+x3v23CzjD1aLk+TJ8c7z+Xa3ket71OWIz85eqrGiHPHeVbT9QqdcpC3t5njn+XyUOuEjyhGXtVLcL3TKRTqm3RzvPJ+PUidcOeLSQNH2Cw3o0pZ2c7zzfD5KnXDliEsDRdsvNKBLW9rN8c7z+Sh1wpUjLg0Ubb/QgC5t2ftze5n6lSmGNw1jGMObhlsqepXn8+1uI1cce6tFsoaHwaz6r4pmSdH2C10UFREpkLYuiprZdjP7upmdNrOnzeyjDZa5wczOmdmTtelAiMDLLERuawq1zvPEkTm/QHm+WSbvOE7/1jnMlujfOsfkHfnKE4RUpv6UFjW7hXR5Ai4Hrq29fiPwD8CVdcvcADyUta6VUy/f+n/kiPvgYPUxZMvT4GBrt4EfOXXEBw8NOrdycRo8NBj91v6sODLnB+iLVEzcfswZeGXVd2HgFZ+4/Vi0GMrUn9IYIW/9N7P/Adzl7l9d0XYD8Al3vynvenr5lEuI3NYUap3niSNz/kh6eb4b1b91jsWXtq1pr2yZY+HFte2dUKb+lMaC5aGb2QhwDXCiwey3mdk3zewrZnZVk8+Pm9mMmc3Mz8+3sulSCZHbmkKt8zxxZM4vWJ7vehZfelNL7Z1Qpv6U1uUe0M3sDcBfAh9z95frZj8ODLv7W4E/Af660TrcfcrdR919dGhoaKMxF16I3NYUap3niSNzfsHyfNdT2fJ8S+2dUKb+lNblGtDNbIDqYD7t7l+qn+/uL7v7K7XXXwYGzOyyoJGWSIjc1hRqneeJI3N+wfJ81zN+yxkYeHV148Cr1fZIytSfsgHNTq4vT1SfzPs54M51lvkZLqVAXgc8t/y+2dTLF0Xdw9TFTqHWeZ44MueXqEb4xO3HvLLlrMOiV7acjXpBdFmZ+lPWop2Lomb2DuAY8BSwVGv+fWBH7T+Ee8zsI8AEsAD8M/B77v636623ly+Kiohs1HoXRTPL57r7capH6estcxdw18bC603TT02z/9H9PHfuOXZs2sGhsUNB73yMafLhSaZOTrHoi1SswvjOcQ7feLjbYYn0HNVD74IY9b1jmXx4krtn7r74ftEXL77XoC4Sl2q5dMH+R/dfHMyXnb9wnv2P7u9SRBs3dXKqpXYR6RwN6F2QSg55CIu+2FK7iHSOBvQuSCWHPISKVVpqF5HO0YDeBankkIcwvnO8pXYR6RwN6F0Qo753LIdvPMzE6MTFI/KKVZgYndAFUZEuUD10EZEC0UOi6xWkYHQq9c6zFCXOGNQX0k29l4c+PQ3j43C+ljY4O1t9D0k9V6oouepFiTMG9YV0W++dcilIwehU6p1nKUqcMagvJAadclmpIAWji5KrXpQ4Y1BfSLf13oBekILRRclVL0qcMagvpNt6b0AvSMHoouSqFyXOGNQX0m29N6Dv3QtTU9Vz5mbVf6emkrogCsXJVS9KnDGoL6Tbeu+iqIhIgemiqPS8yTuO0791DrMl+rfOMXnH8ZbXoRxzSV3v5aFLz5m84zh3H7gGLrwegMWXtnH3gS3AcQ5/8h251qEccykCnXKR0uvfOsfiS9vWtFe2zLHw4tr2RpRjLqnQKRfpaYsvvaml9kaUYy5FoAFdSq+y5fmW2htRjrkUgQZ0Kb3xW87AwKurGwderbbnpBxzKQIN6FJ6hz/5DiYOPkFlyxywRGXLHBMHn8h9QRSUYy7FoIuiIiIFoouiIiI9QAO6iEhJaEAXESkJDegiIiWhAV1EpCQ0oIuIlIQGdBGRktCALiJSEpkDupltN7Ovm9lpM3vazD7aYBkzsz82s2fN7JSZXduZcHuL6m+LSCvy1ENfAD7u7o+b2RuBk2b2VXf/9opl3g38m9r0C8DdtX9lg1R/W0RalXmE7u7fc/fHa6//L3AauKJusfcCn/OqbwCbzezy4NH2kP2P7r84mC87f+E8+x/d36WIRCR1LZ1DN7MR4BrgRN2sK4CzK97PsXbQx8zGzWzGzGbm5+dbi7THqP62iLQq94BuZm8A/hL4mLu/XD+7wUfWVP1y9yl3H3X30aGhodYi7TGqvy0irco1oJvZANXBfNrdv9RgkTlg+4r324D8Tw+QNVR/W0RalSfLxYB7gdPu/kdNFjsK/Hot2+V64Jy7fy9gnD1H9bdFpFWZ9dDN7B3AMeApYKnW/PvADgB3v6c26N8FvAs4D3zI3dctdq566CIirVuvHnpm2qK7H6fxOfKVyzjw4Y2FJyIiIehOURGRktCALiJSEhrQRURKQgO6iEhJaEAXESkJDegiIiWhAV1EpCQybyzq2IbN5oHZrmz8ksuAH3Q5hjwUZzhFiBEUZ2hlinPY3RsWw+ragJ4CM5tpdsdVShRnOEWIERRnaL0Sp065iIiUhAZ0EZGS6PUBfarbAeSkOMMpQoygOEPriTh7+hy6iEiZ9PoRuohIaWhAFxEpiZ4Y0M2sYmZPmNlDDebtM7N5M3uyNt3cjRhrsZwxs6dqcax5+kftiVB/bGbPmtkpM7s2wRhvMLNzK/rzQOwYa3FsNrMHzOwZMzttZm+rm9/1vswZZ9f708x+dsX2nzSzl83sY3XLdL0/c8bZ9f6sxfGfzOxpM/uWmf2Fmb22bv5rzOz+Wn+eMLORXCt299JPwO8Bfw481GDePuCubsdYi+UMcNk6898DfIXqA0euB04kGOMNjfq5C3HeB9xce/0TwObU+jJnnEn054p4KsD3qd7cklx/5oiz6/0JXAH8E/C62vsvAvvqlpkE7qm93gPcn2fdpT9CN7NtwI3AZ7sdSwDvBT7nVd8ANpvZ5d0OKjVm9pPAL1J9Fi7u/v/c/Ud1i3W9L3PGmZox4B/dvf4u7673Z51mcaaiH3idmfUDg8DzdfPfS/U/e4AHgLHaoz7XVfoBHbgTuIVLz0Nt5H21PxMfMLPtkeJqxIG/MbOTZjbeYP4VwNkV7+dqbTFlxQjwNjP7ppl9xcyuihlczb8G5oH/VjvV9lkze33dMin0ZZ44ofv9udIe4C8atKfQnys1ixO63J/u/l3gD4HngO8B59z9b+oWu9if7r4AnAN+KmvdpR7Qzewm4AV3P7nOYg8CI+5+NfAIl/5X7Ia3u/u1wLuBD5vZL9bNb/Q/dOy806wYH6f6Z+5bgT8B/jpyfFA9+rkWuNvdrwFeBT5Zt0wKfZknzhT6EwAz+wlgN/DfG81u0NaVnOiMOLven2a2heoR+L8C3gS83sw+WL9Yg49m9mepB3Tg7cBuMzsDfAF4p5kdWbmAu//Q3X9ce/sZYGfcEFfF8nzt3xeAvwKuq1tkDlj5F8Q21v6p1lFZMbr7y+7+Su31l4EBM7ssZoxU+2nO3U/U3j9AdeCsX6arfUmOOBPpz2XvBh539//TYF4K/bmsaZyJ9Ocu4J/cfd7dLwBfAv5t3TIX+7N2WmYT8GLWiks9oLv7p9x9m7uPUP0T7Gvuvup/wrrzfLuB0xFDXBnH683sjcuvgX8PfKtusaPAr9cyCq6n+qfa91KK0cx+Zvlcn5ldR3Uf+2GsGAHc/fvAWTP72VrTGPDtusW62pd540yhP1f4jzQ/jdH1/lyhaZyJ9OdzwPVmNliLZYy1485R4Ddqr99PdezKPELvDxpmQZjZQWDG3Y8Cv2tmu4EFqv8D7utSWD8N/FVtX+sH/tzd/6eZ/TaAu98DfJlqNsGzwHngQwnG+H5gwswWgH8G9uTZETvgd4Dp2p/f3wE+lFhf5o0zif40s0Hgl4HfWtGWXH/miLPr/enuJ8zsAaqnfxaAJ4CpunHpXuDzZvYs1XFpT55169Z/EZGSKPUpFxGRXqIBXUSkJDSgi4iUhAZ0EZGS0IAuIlISGtBFREpCA7qISEn8f7UlenviJ18tAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[y == 0, 0], X[y == 0, 1], color='r');\n",
    "plt.scatter(X[y == 1, 0], X[y == 1, 1], color='g');\n",
    "plt.scatter(X[y == 2, 0], X[y == 2, 1], color='b');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Let's do some machine learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = LogisticRegression(C=1., solver='liblinear', multi_class='auto')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='auto', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='liblinear', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 4)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.coef_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = clf.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       2, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.96"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(y == y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X[::2]\n",
    "y_train = y[::2]\n",
    "X_test = X[1::2]\n",
    "y_test = y[1::2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((75, 4), (75, 4))"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = clf.fit(X_train, y_train).predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9466666666666667"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(y_pred == y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.96078431 0.92156863 0.95833333]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score, StratifiedKFold\n",
    "clf = LogisticRegression(C=1., solver='liblinear', multi_class='auto')\n",
    "cv = StratifiedKFold(n_splits=3)\n",
    "scores = cross_val_score(clf, X, y, cv=cv)\n",
    "print(scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CV Accuracy : 0.9468954248366014 (std : 0.017936681699428268)\n"
     ]
    }
   ],
   "source": [
    "print(\"CV Accuracy : %s (std : %s)\" % (np.mean(scores), np.std(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# if you want to do it with a support vector machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "clf = SVC(C=1., kernel='linear')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9733333333333334"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred = clf.fit(X_train, y_train).predict(X_test)\n",
    "np.mean(y_pred == y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.         0.96078431 0.97916667]\n"
     ]
    }
   ],
   "source": [
    "scores = cross_val_score(clf, X, y, cv=cv)\n",
    "print(scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CV Accuracy : 0.9799836601307189 (std : 0.01602015649601925)\n"
     ]
    }
   ],
   "source": [
    "print(\"CV Accuracy : %s (std : %s)\" % (np.mean(scores), np.std(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# To learn more\n",
    "\n",
    "- https://scikit-learn.org/stable/tutorial/index.html\n",
    "- https://jakevdp.github.io/PythonDataScienceHandbook/05.00-machine-learning.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}