notebooks/Python SVM Classifier Example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python SVM Classifier Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(C) 2017-2019 by [Damir Cavar](http://damir.cavar.me/)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Download:** This and various other Jupyter notebooks are available from my [GitHub repo](https://github.com/dcavar/python-tutorial-for-ipython)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Version:** 1.1, September 2019"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**License:** [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/) ([CA BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a tutorial related to the discussion of an SVN classifier in the textbook [Machine Learning: The Art and Science of Algorithms that Make Sense of Data](https://www.cs.bris.ac.uk/~flach/mlbook/) by [Peter Flach](https://www.cs.bris.ac.uk/~flach/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial was developed as part of my course material for the course Machine Learning for Computational Linguistics in the [Computational Linguistics Program](http://cl.indiana.edu/) of the [Department of Linguistics](http://www.indiana.edu/~lingdept/) at [Indiana University](https://www.indiana.edu/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVN Example using "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The basic idea and storyline for this example was taken from or inspired by [the tutorial Simple Support Vector Machine (SVM) example with character recognition](https://pythonprogramming.net/support-vector-machine-svm-example-tutorial-scikit-learn-python/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial requires [Scikit-learn](https://pypi.python.org/pypi/scikit-learn/) and [Matplotlib](http://matplotlib.org). These modules come with the default [Anaconda Python](https://www.continuum.io/downloads) installation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start the tutorial and run the example, we will import *pyplot* from *matplotlib*, and *datasets* and *svm* from *scikit-learn*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn import datasets\n",
    "from sklearn import svm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the digits data set into memory, refering to it with the variable *digits*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "digits = datasets.load_digits()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can output the data set in *digits*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0.   0.   5. ...,   0.   0.   0.]\n",
      " [  0.   0.   0. ...,  10.   0.   0.]\n",
      " [  0.   0.   0. ...,  16.   9.   0.]\n",
      " ..., \n",
      " [  0.   0.   1. ...,   6.   0.   0.]\n",
      " [  0.   0.   2. ...,  12.   0.   0.]\n",
      " [  0.   0.  10. ...,  12.   1.   0.]]\n"
     ]
    }
   ],
   "source": [
    "print(digits.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data contains the actual **features**. You will find a brief description of the [digits dataset](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html) on the [Scikit-learn website](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html). It contains datapoints with 8x8 images of the digits 0 to 9."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can print out the image of the digits. In this case we are printing the digit 0:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1109b2d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAFdCAYAAABGoXXzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAEX9JREFUeJzt3X+s3XV9x/Hnq1jlhyI34nBOmRaUaTAgt6DOAZtUMERR\nkllF4kIIM6AzxG2JEDFkizF1RiSgGLM5QVCX9g8jOhCobHMtsoYidQzQ6AVhDqotHRCqA+xnf5zT\neL32Ws/t993vPYfnI7l/3EP5nlf643m//Z7T701rDUlS95b0PUCSJpWBlaQiBlaSihhYSSpiYCWp\niIGVpCIGVpKKGFhJKmJgJamIgZWkImMT2CTvS3Jvkp8luTXJsX1vAkhyfJJrk/w4yY4kp/W9CSDJ\nhUk2JHk0yeYkX0ny8r53ASQ5N8mmJI8MP25J8qa+d82V5ILhr+kli2DLxcMtsz/u6nvXTklemOTq\nJFuSbB/++h6zCHbdu4uftx1JLt8bzz8WgU3yDuATwMXAq4FNwA1JDu512MABwB3Ae4HFdGOH44HL\ngdcAK4ClwI1J9ut11cADwAeBY4Bp4Gbgq0le0euqWYZfwN/D4PfaYnEncAjwguHHH/U7ZyDJQcB6\n4P+AU4BXAH8FbOtz19Byfvnz9QLgjQz+nK7eG0+ecbjZS5Jbgf9orZ0//DwM/pBe1lr7u17HzZJk\nB/C21tq1fW+Za/jF6CfACa21dX3vmSvJVuCvW2ufXwRbng1sBM4DPgx8p7X2lz1vuhh4a2ut97PC\nuZKsAl7XWjux7y27k+RS4NTW2l7529yiP4NNspTBWc43dz7WBl8V1gKv62vXGDqIwVfuh/seMluS\nJUneCewPfLvvPUOfBr7WWru57yFzvGx4KeqHSa5J8uK+Bw29Bbgtyerh5ajbk5zT96i5hi05E/jc\n3nrORR9Y4GBgH2DznMc3Mzjl124Mz/gvBda11hbFdbskRyZ5jMFfK68ATm+t3dPzLIaxPxq4sO8t\nc9wKnMXgr+DnAi8FvpXkgD5HDS1jcLb/PeBk4DPAZUne3euqX3c68Fzgqr31hM/YW0+kXl0BvBJ4\nfd9DZrkHOIrBb/g/Bb6Q5IQ+I5vkRQy+EK1orT3Z145daa3dMOvTO5NsAH4ErAT6vqyyBNjQWvvw\n8PNNSY5k8IXg6v5m/Zqzgetbaw/trScchzPYLcAvGFzcn+0QYK/9RI2rJJ8CTgX+uLX2YN97dmqt\nPdVam2mtfae19iEGLyad3/OsaeD5wO1JnkzyJHAicH6SJ4Z/E1gUWmuPAN8HDu97C/AgcPecx+4G\nDu1hyy4lOZTBi71/vzefd9EHdngmsRE4aedjw9/oJwG39LVrHAzj+lbgT1pr9/e9ZzeWAM/qecNa\n4FUMLhEcNfy4DbgGOKotoleEhy/EHc4gbn1bDxwx57EjGJxhLxZnM7iseN3efNJxuURwCXBlko3A\nBuADDF4UubLPUQDDa2CHAzvPbpYlOQp4uLX2QI+7rgDOAE4DHk+y828Aj7TWft7XLoAkHwWuB+4H\nnsPghYcTGVy/601r7XHgV65RJ3kc2Npam3uGtlcl+TjwNQbR+j3gb4AngS/3uWvok8D6JBcyePvT\na4BzgD/vddXQ8ITsLODK1tqOvfrkrbWx+GDwPtP7gJ8xeLV5ed+bhrtOBHYwuIwx++Mfe961q02/\nAP5sEfyc/QMwM/y1fAi4EXhD37vm2XozcMki2PFl4L+HP2f3A18CXtr3rln7TgW+C2wH/gs4u+9N\ns7a9cfh7//C9/dxj8T5YSRpHi/4arCSNKwMrSUUMrCQVMbCSVMTASlIRAytJRUr/oUGS5zG4OcV9\nQK9vbpekjuwLvAS4obW29Tf9wOp/yXUK8MXi55CkPpzJ4B98zKs6sPcVH39irVixorNj3XHHHRx9\n9NGdHe/9739/Z8f6yEc+wkUXXdTJsTZs2NDJcXZavXo1K1eu7ORYl112WSfH2Wn79u3sv//+nRzr\nscce6+Q4T0P37e4HVAfWywILNDU11dmxli5d2unxjjzyyM6OdeCBB3Z2vIce6vbmavvttx+HHtrN\nDaGe8Yxu/6gtWbKk82NqZLvtmy9ySVIRAytJRQysJBUxsE8DXV1HrPDmN7+57wnzOvbYY/ueMK9n\nPvOZfU/Qb8HAPg0s5sCedtppfU+Y13HHHdf3hHkZ2PFgYCWpiIGVpCIGVpKKGFhJKmJgJamIgZWk\nIgsKbJL3Jbk3yc+S3Jpk8b5hUJJ6MnJgk7wD+ARwMfBqYBNwQ5KDO94mSWNtIWewHwA+21r7Qmvt\nHuBcYDtwdqfLJGnMjRTYJEuBaeCbOx9rrTVgLfC6bqdJ0ngb9Qz2YGAfYPOcxzcDL+hkkSRNCN9F\nIElFRg3sFuAXwCFzHj8E6PZ28pI05kYKbGvtSWAjcNLOx5Jk+Pkt3U6TpPG2kG/qcwlwZZKNwAYG\n7yrYH7iyw12SNPZGDmxrbfXwPa9/y+DSwB3AKa21n3Y9TpLG2YK+LWVr7Qrgio63SNJE8V0EklTE\nwEpSEQMrSUUMrCQVMbCSVMTASlIRAytJRQysJBUxsJJUxMBKUhEDK0lFDKwkFTGwklRkQXfTUr1V\nq1b1PWFey5Yt63vCLk1NTfU9YV4PP/xw3xPmtXLlyr4nzGvNmjV9T9gjnsFKUhEDK0lFDKwkFTGw\nklTEwEpSEQMrSUUMrCQVMbCSVMTASlIRAytJRQysJBUxsJJUxMBKUhEDK0lFRg5skuOTXJvkx0l2\nJDmtYpgkjbuFnMEeANwBvBdo3c6RpMkx8g23W2vfAL4BkCSdL5KkCeE1WEkqYmAlqYiBlaQiBlaS\nihhYSSoy8rsIkhwAHA7sfAfBsiRHAQ+31h7ocpwkjbORAwssB/6FwXtgG/CJ4eNXAWd3tEuSxt5C\n3gf7b3hpQZJ2y1BKUhEDK0lFDKwkFTGwklTEwEpSEQMrSUUMrCQVMbCSVMTASlIRAytJRQysJBUx\nsJJUxMBKUpGF3K5wYkxPT/c9YV7Lli3re8K8DjvssL4n7NLMzEzfE+Z100039T1hXov5z8GaNWv6\nnrBHPIOVpCIGVpKKGFhJKmJgJamIgZWkIgZWkooYWEkqYmAlqYiBlaQiBlaSihhYSSpiYCWpiIGV\npCIjBTbJhUk2JHk0yeYkX0ny8qpxkjTORj2DPR64HHgNsAJYCtyYZL+uh0nSuBvpfrCttVNnf57k\nLOAnwDSwrrtZkjT+9vQa7EFAAx7uYIskTZQFBzZJgEuBda21u7qbJEmTYU++ZcwVwCuB13e0RZIm\nyoICm+RTwKnA8a21B7udJEmTYeTADuP6VuDE1tr93U+SpMkwUmCTXAGcAZwGPJ7kkOF/eqS19vOu\nx0nSOBv1Ra5zgQOBfwX+Z9bHym5nSdL4G/V9sP7TWkn6LRlMSSpiYCWpiIGVpCIGVpKKGFhJKmJg\nJamIgZWkIgZWkooYWEkqYmAlqYiBlaQiBlaSihhYSSpiYCWpyJ58T66xNzU11feEeW3cuLHvCfOa\nmZnpe8LYWcy/nqrjGawkFTGwklTEwEpSEQMrSUUMrCQVMbCSVMTASlIRAytJRQysJBUxsJJUxMBK\nUhEDK0lFDKwkFRkpsEnOTbIpySPDj1uSvKlqnCSNs1HPYB8APggcA0wDNwNfTfKKrodJ0rgb6X6w\nrbV/nvPQRUnOA14L3N3ZKkmaAAu+4XaSJcBKYH/g250tkqQJMXJgkxzJIKj7Ao8Bp7fW7ul6mCSN\nu4W8i+Ae4CjgOOAzwBeS/EGnqyRpAox8BttaewrY+U2ZvpPkOOB84Lwuh0nSuOvifbBLgGd1cBxJ\nmigjncEm+ShwPXA/8BzgTOBE4OTup0nSeBv1EsHvAFcBvws8AnwXOLm1dnPXwyRp3I36PthzqoZI\n0qTxXgSSVMTASlIRAytJRQysJBUxsJJUxMBKUhEDK0lFDKwkFTGwklTEwEpSEQMrSUUMrCQVMbCS\nVMTASlKRBX9X2UkwNTXV94R5rV27tu8J6tBi/r22bdu2vidMLM9gJamIgZWkIgZWkooYWEkqYmAl\nqYiBlaQiBlaSihhYSSpiYCWpiIGVpCIGVpKKGFhJKmJgJanIHgU2yQVJdiS5pKtBkjQpFhzYJMcC\n7wE2dTdHkibHggKb5NnANcA5wP92ukiSJsRCz2A/DXyttXZzl2MkaZKM/B0NkrwTOBpY3v0cSZoc\nIwU2yYuAS4EVrbUnayZJ0mQY9Qx2Gng+cHuSDB/bBzghyV8Az2qttS4HStK4GjWwa4FXzXnsSuBu\nYJVxlaRfGimwrbXHgbtmP5bkcWBra+3uLodJ0rjr4l9yedYqSbsw8rsI5mqtvaGLIZI0abwXgSQV\nMbCSVMTASlIRAytJRQysJBUxsJJUxMBKUhEDK0lFDKwkFTGwklTEwEpSEQMrSUUMrCQV2eO7aY2z\nbdu29T1hXtPT031PGDtTU1N9T5jXYv71XLNmTd8TJpZnsJJUxMBKUhEDK0lFDKwkFTGwklTEwEpS\nEQMrSUUMrCQVMbCSVMTASlIRAytJRQysJBUxsJJUxMBKUpGRApvk4iQ75nzcVTVOksbZQu4Heydw\nEpDh5091N0eSJsdCAvtUa+2nnS+RpAmzkGuwL0vy4yQ/THJNkhd3vkqSJsCogb0VOAs4BTgXeCnw\nrSQHdLxLksbeSJcIWms3zPr0ziQbgB8BK4HPdzlMksbdHr1Nq7X2CPB94PBu5kjS5NijwCZ5NoO4\nPtjNHEmaHKO+D/bjSU5I8vtJ/hD4CvAk8OWSdZI0xkZ9m9aLgC8BzwN+CqwDXtta29r1MEkad6O+\nyHVG1RBJmjTei0CSihhYSSpiYCWpiIGVpCIGVpKKGFhJKmJgJamIgZWkIgZWkooYWEkqYmAlqYiB\nlaQiBlaSiizku8pOjJmZmb4nzGt6errvCfN6+9vf3veEXVqsuxa7j33sY31PmFiewUpSEQMrSUUM\nrCQVMbCSVMTASlIRAytJRQysJBUxsJJUxMBKUhEDK0lFDKwkFTGwklTEwEpSkZEDm+SFSa5OsiXJ\n9iSbkhxTMU6SxtlItytMchCwHvgmcAqwBXgZsK37aZI03ka9H+wFwP2ttXNmPfajDvdI0sQY9RLB\nW4DbkqxOsjnJ7UnO2e3/JUlPQ6MGdhlwHvA94GTgM8BlSd7d9TBJGnejXiJYAmxorX14+PmmJEcC\n5wJXd7pMksbcqGewDwJ3z3nsbuDQbuZI0uQYNbDrgSPmPHYEvtAlSb9m1MB+EnhtkguTHJbkXcA5\nwKe6nyZJ422kwLbWbgNOB84A/hP4EHB+a+2fCrZJ0lgb9UUuWmvXAdcVbJGkieK9CCSpiIGVpCIG\nVpKKGFhJKmJgJamIgZWkIgZWkooYWEkqYmAlqYiBlaQiBlaSihhYSSpiYCWpiIGVpCIj365wkszM\nzPQ9YV4XXHBB3xPmtWrVqr4n7NLGjRv7njCv5cuX9z1BPfAMVpKKGFhJKmJgJamIgZWkIgZWkooY\nWEkqYmAlqYiBlaQiBlaSihhYSSpiYCWpiIGVpCIGVpKKjBTYJPcm2bGLj8urBkrSuBr1doXLgX1m\nff4q4EZgdWeLJGlCjBTY1trW2Z8neQvww9bav3e6SpImwIKvwSZZCpwJfK67OZI0OfbkRa7TgecC\nV3W0RZImyp4E9mzg+tbaQ12NkaRJsqDvyZXkUGAF8LZu50jS5FjoGezZwGbgug63SNJEGTmwSQKc\nBVzZWtvR+SJJmhALOYNdAbwY+HzHWyRpoox8Dba1dhO/+o8NJEm74L0IJKmIgZWkIgZWkooYWEkq\nYmAlqYiBlaQiBlaSihhYSSpiYJ8GfvCDH/Q9YV7XXntt3xPmtW7dur4naMwZ2KeBxRzYr3/9631P\nmNf69ev7nqAxZ2AlqYiBlaQiBlaSiizoOxqMYN/i40+sLVu2dHasJ554otPj3XnnnZ0d69FHH+3s\neDMzM50cZ6ft27d3fkxNlN32La21smdP8i7gi2VPIEn9ObO19qXf9AOqA/s84BTgPuDnZU8kSXvP\nvsBLgBtaa1t/0w8sDawkPZ35IpckFTGwklTEwEpSEQMrSUUMrCQVMbCSVMTASlKR/wfmW6zHvgHd\n0QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1109b2fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.gray()\n",
    "plt.matshow(digits.images[0])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *target* vector contains the actual labels of the datapoints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2 ..., 8 9 8]\n"
     ]
    }
   ],
   "source": [
    "print(digits.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use a default classifier from the Scikit-learn module, the [C-Support Vector Classifier](http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html). The penalty parameter *C* is set to 1.0 in the default. In this example *C* is set to 100. The kernel coeefficient is optional and in this example it is set to 0.001. The meaning and effect of *C* and *gamma* is explained on [the Scikit-learn pages](http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "classifier = svm.SVC(gamma=0.001, C=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can train the classifier now on all datapoints but the last 10. We leave the last 10 datapoints out for testing. The *X* variable contains the coordinates or features, and the *y* variable the targets or labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X,y = digits.data[:-10], digits.target[:-10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train the classifier on this data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=100, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape=None, degree=3, gamma=0.001, kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now test the classifier on one of the test datapoints that we left out from the training corpus. Note that newer Scikit-learn modules deprecate the passing of one-dimensional arrays as data, which *digits.data[-5]*. Since *digits.data[-5]* contains a single sample, we need to reshape it using *.reshape(1,-1)*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9]\n"
     ]
    }
   ],
   "source": [
    "print(classifier.predict(digits.data[-5].reshape(1,-1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *reshape* method converts vectors. For example, imagine we have an array of 10 digits arranged as a 1-dimensional columnar array as in the following example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0],\n",
       "       [1],\n",
       "       [2],\n",
       "       [3],\n",
       "       [4],\n",
       "       [5],\n",
       "       [6],\n",
       "       [7],\n",
       "       [8],\n",
       "       [9]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy\n",
    "\n",
    "numpy.array([[0],[1],[2],[3],[4],[5],[6],[7],[8],[9]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This array can be converted to an 1-dimensional row array using the *reshape* function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0 1 2 3 4 5 6 7 8 9]]\n"
     ]
    }
   ],
   "source": [
    "t = numpy.array([[0],[1],[2],[3],[4],[5],[6],[7],[8],[9]])\n",
    "\n",
    "print(t.reshape(1,-1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, an 1-dimensional row array can be reshaped to a 1-dimensional columnar array in the following way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t as a      row-array: [[0 1 2 3 4 5 6 7 8 9]]\n",
      "t as a columnar array: [[0]\n",
      " [1]\n",
      " [2]\n",
      " [3]\n",
      " [4]\n",
      " [5]\n",
      " [6]\n",
      " [7]\n",
      " [8]\n",
      " [9]]\n"
     ]
    }
   ],
   "source": [
    "t = t.reshape(1,-1)\n",
    "print(\"t as a      row-array:\", t)\n",
    "print(\"t as a columnar array:\", t.reshape((-1,1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example the digits data contains a single sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  0.   0.   4.  10.  13.   6.   0.   0.   0.   1.  16.  14.  12.  16.   3.\n",
      "   0.   0.   4.  16.   6.   3.  16.   4.   0.   0.   0.  12.  16.  16.  16.\n",
      "   5.   0.   0.   0.   0.   4.   4.  16.   8.   0.   0.   0.   0.   0.   0.\n",
      "  15.   5.   0.   0.   0.   5.   7.   7.  16.   4.   0.   0.   0.   2.  14.\n",
      "  15.   9.   0.   0.]\n"
     ]
    }
   ],
   "source": [
    "print(digits.data[-5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *classifier.predict()* function in the [Scikit-learn module](http://scikit-learn.org/) requires the vector of this one sample to be reshaped to actually an array that contains the entire sample as an element, that is an array with an array that contains the sample data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0.   0.   4.  10.  13.   6.   0.   0.   0.   1.  16.  14.  12.  16.\n",
      "    3.   0.   0.   4.  16.   6.   3.  16.   4.   0.   0.   0.  12.  16.\n",
      "   16.  16.   5.   0.   0.   0.   0.   4.   4.  16.   8.   0.   0.   0.\n",
      "    0.   0.   0.  15.   5.   0.   0.   0.   5.   7.   7.  16.   4.   0.\n",
      "    0.   0.   2.  14.  15.   9.   0.   0.]]\n"
     ]
    }
   ],
   "source": [
    "print(digits.data[-5].reshape(1,-1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Returning to our classifier result, let us look at the 5th datapoint in our test data, that is the fifth element from the back (-5) of *digits*, given that we left out the last ten datapoints for testing. We see that the classifier guessed that 5th sample represents a 9:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9]\n"
     ]
    }
   ],
   "source": [
    "print(classifier.predict(digits.data[-5].reshape(1,-1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can print the image and see whether the classifier was right:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAFdCAYAAABGoXXzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAE8RJREFUeJzt3X+MZWV9x/H3F7oRF+ysAYul/lgoQtZgWGagYC3s8kMw\nJCI0Br2gDRBqQJqs0zZRYg22fxhTI0NAME1r5Zfchn9USEEoUmpB6QrXXUtZ1kRBqMAi4A5kEVjY\np3/cu+nMMHd37p3znXPv3fcruX/cZ+95ziezcz/zzLlnzolSCpKk6u1VdwBJGlUWrCQlsWAlKYkF\nK0lJLFhJSmLBSlISC1aSkvxO5uQRsT9wGvAY8HLmviRpiewDrATuKKU8t6sXphYs7XL9VvI+JKkO\n5wI37eoF2QX7GMCNN97IqlWrKplwcnKSqampSuaqWpXZHnjggUrmAbj66qu55JJLKpvv1ltvrWyu\nH//4xxxzzDGVzLV58+ZK5tnpySef5KCDDqpkrnPOOaeSeXZqNps0Go1K5jrjjDMqmQf2jPfnpk2b\n+MQnPgGdftuV7IJ9GWDVqlWMj49XMuHY2Fhlc1WtymwvvPBCJfMA7Lfffhx22GGVzbf//vtXNtey\nZcsqm2/58uWVzLPT3nvvXdmcK1eurGSenZYvX17ZnFW+n/aU92fHbg97+iGXJCWxYCUpiQUrSUmG\nrmCrOrCfYVCznXTSSXVH6Orggw+uO0JXK1asqDtCV8cee2zdEeY1qO8BqCebBVuhQc128skn1x2h\nq0MOOaTuCF299a1vrTtCV8cdd1zdEeY1qO8BsGAlaaRYsJKUxIKVpCQWrCQlsWAlKYkFK0lJ+irY\niLgkIh6NiN9GxP0RUc3VOiRphPRcsBHxMeCrwGXAUcBG4I6IOKDibJI01PpZwU4C/1BKub6U8ghw\nEfAScEGlySRpyPVUsBGxDJgAvr9zrJRSgLuA91cbTZKGW68r2AOAvYEtc8a3AG+vJJEkjQjPIpCk\nJL3e0eBZ4HXgwDnjBwJPd9tocnKSsbGxWWONRmOgLwwhSc1mk2azOWtsenp6wdv3VLCllO0R8SBw\nMnALQERE5/mV3babmpoa2NtISFI38y0EW60WExMTC9q+n3tyXQ5c2yna9bTPKlgOXNvHXJI0snou\n2FLKzZ1zXv+O9qGBDcBppZRfVx1OkoZZX3eVLaVcA1xTcRZJGimeRSBJSSxYSUpiwUpSEgtWkpJY\nsJKUxIKVpCQWrCQlsWAlKYkFK0lJLFhJSmLBSlISC1aSkliwkpSkr6tpjYqtW7fWHaGrE088se4I\nXb373e+uO8K8Vq5cWXeErs4///y6I3S1evXquiN0NcjZFsIVrCQlsWAlKYkFK0lJLFhJSmLBSlIS\nC1aSkliwkpTEgpWkJBasJCWxYCUpiQUrSUksWElKYsFKUhILVpKS9FywEXF8RNwSEb+KiB0RcUZG\nMEkadv2sYPcFNgCfBkq1cSRpdPR8we1SyveA7wFERFSeSJJGhMdgJSmJBStJSSxYSUqyJDc9nJyc\nZGxsbNZYo9Gg0Wgsxe4lqS/NZpNmszlrbHp6esHbL0nBTk1NMT4+vhS7kqTKzLcQbLVaTExMLGj7\nngs2IvYFDgV2nkFwSEQcCTxfSnmi1/kkaVT1s4I9Gvh32ufAFuCrnfHrgAsqyiVJQ6+f82D/Az8c\nk6TdsiglKYkFK0lJLFhJSmLBSlISC1aSkliwkpTEgpWkJBasJCWxYCUpiQUrSUksWElKYsFKUhIL\nVpKSLMkFtwfVhg0b6o4wlL74xS/WHWFeq1evrjtCV0cddVTdEbraunVr3RFGlitYSUpiwUpSEgtW\nkpJYsJKUxIKVpCQWrCQlsWAlKYkFK0lJLFhJSmLBSlISC1aSkliwkpTEgpWkJBasJCXpqWAj4tKI\nWB8RL0TEloj4dkQclhVOkoZZryvY44GrgGOBU4BlwJ0R8eaqg0nSsOvpgtullNNnPo+I84BngAng\n3upiSdLwW+wx2BVAAZ6vIIskjZS+CzYiArgCuLeU8nB1kSRpNCzmnlzXAO8FPrC7F05OTjI2NjZr\nrNFo0Gg0FrF7ScrVbDZpNpuzxqanpxe8fV8FGxFfA04Hji+lPLW7109NTTE+Pt7PriSpNvMtBFut\nFhMTEwvavueC7ZTrR4A1pZTHe91ekvYUPRVsRFwDNIAzgG0RcWDnn6ZLKS9XHU6ShlmvH3JdBPwu\ncA/w5IzH2dXGkqTh1+t5sP5prSQtkIUpSUksWElKYsFKUhILVpKSWLCSlMSClaQkFqwkJbFgJSmJ\nBStJSSxYSUpiwUpSEgtWkpJYsJKUZDG3jBl6a9eurTtCV2vWrKk7Qlfnn39+3RFUoa1bt9YdYWS5\ngpWkJBasJCWxYCUpiQUrSUksWElKYsFKUhILVpKSWLCSlMSClaQkFqwkJbFgJSmJBStJSSxYSUpi\nwUpSkp4KNiIuioiNETHdefwwIj6UFU6ShlmvK9gngM8C48AEcDfw3YhYVXUwSRp2PV1wu5Tyr3OG\n/iYiLgaOAzZVlkqSRkDfdzSIiL2As4HlwI8qSyRJI6Lngo2II2gX6j7Ai8BZpZRHqg4mScOunxXs\nI8CRwBjwUeD6iDhhVyU7OTnJ2NjYrLFGo0Gj0ehj95K0NJrNJs1mc9bY9PT0grfvuWBLKa8Bv+g8\n/UlE/BGwDri42zZTU1OMj4/3uitJqtV8C8FWq8XExMSCtq/iPNi9gDdVMI8kjZSeVrAR8SXgduBx\n4C3AucAa4NTqo0nScOv1EMHvAdcBvw9MAz8FTi2l3F11MEkadr2eB3thVhBJGjVei0CSkliwkpTE\ngpWkJBasJCWxYCUpiQUrSUksWElKYsFKUhILVpKSWLCSlMSClaQkFqwkJbFgJSlJ3zc9VK577rmn\n7ghdDXK2QXXiiSfWHaGrDRs21B2hqzPPPLPuCIviClaSkliwkpTEgpWkJBasJCWxYCUpiQUrSUks\nWElKYsFKUhILVpKSWLCSlMSClaQkFqwkJbFgJSnJogo2Ij4XETsi4vKqAknSqOi7YCPiGOBTwMbq\n4kjS6OirYCNiP+BG4EJga6WJJGlE9LuCvRq4tZRyd5VhJGmU9HxHg4j4OLAaOLr6OJI0Onoq2Ih4\nB3AFcEopZXtOJEkaDb2uYCeAtwGtiIjO2N7ACRHxF8CbSill7kaTk5OMjY3NGms0GjQajT4iS9LS\naDabNJvNWWPT09ML3r7Xgr0LeN+csWuBTcCX5ytXgKmpKcbHx3vclSTVa76FYKvVYmJiYkHb91Sw\npZRtwMMzxyJiG/BcKWVTL3NJ0qir4i+55l21StKeruezCOYqpZxURRBJGjVei0CSkliwkpTEgpWk\nJBasJCWxYCUpiQUrSUksWElKYsFKUhILVpKSWLCSlMSClaQkFqwkJbFgJSnJoq+mpT3P2rVr644w\ndObe0WOQrF69uu4II8sVrCQlsWAlKYkFK0lJLFhJSmLBSlISC1aSkliwkpTEgpWkJBasJCWxYCUp\niQUrSUksWElKYsFKUhILVpKS9FSwEXFZROyY83g4K5wkDbN+rgf7EHAyEJ3nr1UXR5JGRz8F+1op\n5deVJ5GkEdPPMdj3RMSvIuLnEXFjRLyz8lSSNAJ6Ldj7gfOA04CLgIOBH0TEvhXnkqSh19MhglLK\nHTOePhQR64FfAmcD36wymCQNu0Xd9LCUMh0RPwMO3dXrJicn33DTt0ajQaPRWMzuJSlVs9mk2WzO\nGpuenl7w9osq2IjYj3a5Xr+r101NTTE+Pr6YXUnSkptvIdhqtZiYmFjQ9r2eB/uViDghIt4dEX8M\nfBvYDjR3s6kk7XF6XcG+A7gJ2B/4NXAvcFwp5bmqg0nSsOv1Qy4PmkrSAnktAklKYsFKUhILVpKS\nWLCSlMSClaQkFqwkJbFgJSmJBStJSSxYSUpiwUpSEgtWkpJYsJKUxIKVpCSLuuC28nznO9+pO0JX\njz32WN0Rhk4vV8FfaitWrKg7wshyBStJSSxYSUpiwUpSEgtWkpJYsJKUxIKVpCQWrCQlsWAlKYkF\nK0lJLFhJSmLBSlISC1aSkliwkpTEgpWkJD0XbEQcFBE3RMSzEfFSRGyMiPGMcJI0zHq6HmxErADu\nA74PnAY8C7wH+E310SRpuPV6we3PAY+XUi6cMfbLCvNI0sjo9RDBh4EHIuLmiNgSEa2IuHC3W0nS\nHqjXgj0EuBjYDJwKfB24MiI+WXUwSRp2vR4i2AtYX0r5Quf5xog4ArgIuKHbRpOTk4yNjc0aazQa\nNBqNHncvSUun2WzSbDZnjfVyf7VeC/YpYNOcsU3An+5qo6mpKcbHPdFA0nCZbyHYarWYmJhY0Pa9\nHiK4Dzh8ztjh+EGXJL1BrwU7BRwXEZdGxB9GxDnAhcDXqo8mScOtp4ItpTwAnAU0gP8GPg+sK6X8\nS0I2SRpqvR6DpZRyG3BbQhZJGilei0CSkliwkpTEgpWkJBasJCWxYCUpiQUrSUksWElKYsFKUhIL\nVpKSWLCSlMSClaQkFqwkJbFgJSlJz1fT0tJYvXp13RG6+sxnPlN3hHlt3bq17ghdrVu3ru4IXa1d\nu7buCCPLFawkJbFgJSmJBStJSSxYSUpiwUpSEgtWkpJYsJKUxIKVpCQWrCQlsWAlKYkFK0lJLFhJ\nSmLBSlISC1aSkvRUsBHxaETsmOdxVVZASRpWvV4P9mhg7xnP3wfcCdxcWSJJGhE9FWwp5bmZzyPi\nw8DPSyn/WWkqSRoBfR+DjYhlwLnAN6qLI0mjYzEfcp0FjAHXVZRFkkbKYu7JdQFweynl6d29cHJy\nkrGxsVljjUaDRqOxiN1LUq5ms0mz2Zw1Nj09veDt+yrYiHgXcApw5kJePzU1xfj4eD+7kqTazLcQ\nbLVaTExMLGj7fg8RXABsAW7rc3tJGnk9F2xEBHAecG0pZUfliSRpRPSzgj0FeCfwzYqzSNJI6fkY\nbCnl35j9xwaSpHl4LQJJSmLBSlISC1aSkliwkpTEgpWkJBasJCWxYCUpydAV7NwLLwySQc12yy23\n1B2hq23bttUdoatXX3217ghdbd68ue4I8xrU9wDUk82CrdCgZrNg+7N9+/a6I3RlwfbOgpWkEWLB\nSlISC1aSkizmjgYLsQ/Apk2bKptwenqaVqtV2XxVqjLbk08+Wck8AC+++CIPPfRQZfO98sorlc21\nY8eOyuZ7/fXXK5lnp1JKZXM+88wzlcyz06uvvlrZnFW+n/aE9+eMPttnd6+NUsqid9h18ohzgG+l\n7UCS6nNuKeWmXb0gu2D3B04DHgNeTtuRJC2dfYCVwB2llOd29cLUgpWkPZkfcklSEgtWkpJYsJKU\nxIKVpCQWrCQlGZqCjYhLIuLRiPhtRNwfEcfUnQkgIo6PiFsi4lcRsSMizqg7E0BEXBoR6yPihYjY\nEhHfjojD6s4FEBEXRcTGiJjuPH4YER+qO9dcEfG5zv/p5QOQ5bJOlpmPh+vOtVNEHBQRN0TEsxHx\nUuf/d3wAcj06z9dtR0RctRT7H4qCjYiPAV8FLgOOAjYCd0TEAbUGa9sX2AB8Ghikc96OB64CjgVO\nAZYBd0bEm2tN1fYE8FlgHJgA7ga+GxGrak01Q+cH+Kdof68NioeAA4G3dx5/Um+ctohYAdwHvEL7\nvPdVwF8Bv6kzV8fR/P/X6+3AB2m/T29eip0PxXmwEXE/8F+llHWd50H7TXplKeXvaw03Q0TsAM4s\npQzc9QE7P4yeAU4opdxbd565IuI54K9LKd8cgCz7AQ8CFwNfAH5SSvnLmjNdBnyklFL7qnCuiPgy\n8P5Sypq6s+xORFwBnF5KWZLf5gZ+BRsRy2ivcr6/c6y0fyrcBby/rlxDaAXtn9zP1x1kpojYKyI+\nDiwHflR3no6rgVtLKXfXHWSO93QORf08Im6MiHfWHajjw8ADEXFz53BUKyIurDvUXJ0uORf4xlLt\nc+ALFjgA2BvYMmd8C+0lv3ajs+K/Ari3lDIQx+0i4oiIeJH2r5XXAGeVUh6pORadsl8NXFp3ljnu\nB86j/Sv4RcDBwA8iYt86Q3UcQnu1vxk4Ffg6cGVEfLLWVG90FjAGXLdUO8y+mpYGwzXAe4EP1B1k\nhkeAI2l/w38UuD4iTqizZCPiHbR/EJ1SShmo2xmUUu6Y8fShiFgP/BI4G6j7sMpewPpSyhc6zzdG\nxBG0fxDcUF+sN7gAuL2U8vRS7XAYVrDPAq/TPrg/04HAkn2hhlVEfA04HVhbSnmq7jw7lVJeK6X8\nopTyk1LK52l/mLSu5lgTwNuAVkRsj4jtwBpgXUS82vlNYCCUUqaBnwGH1p0FeAqYe03STcC7asgy\nr4h4F+0Pe/9xKfc78AXbWUk8CJy8c6zzjX4y8MO6cg2DTrl+BDixlPJ43Xl2Yy/gTTVnuAt4H+1D\nBEd2Hg8ANwJHlgH6RLjzQdyhtMutbvcBh88ZO5z2CntQXED7sOJtS7nTYTlEcDlwbUQ8CKwHJml/\nKHJtnaEAOsfADgV2rm4OiYgjgedLKU/UmOsaoAGcAWyLiJ2/AUyXUmq9dGREfAm4HXgceAvtDx7W\n0D5+V5tSyjZg1jHqiNgGPFdKqe6q8X2IiK8At9IurT8A/hbYDgzCXQangPsi4lLapz8dC1wI/Hmt\nqTo6C7LzgGtLKTuWdOellKF40D7P9DHgt7Q/bT667kydXGuAHbQPY8x8/HPNuebL9DrwZwPwNfsn\n4Bed/8ungTuBk+rO1SXr3cDlA5CjCfxv52v2OHATcHDduWbkOx34KfAS8D/ABXVnmpHtg53v/UOX\net9DcR6sJA2jgT8GK0nDyoKVpCQWrCQlsWAlKYkFK0lJLFhJSmLBSlISC1aSkliwkpTEgpWkJBas\nJCX5P5kok3cOUiQcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11093d940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(digits.images[-5], cmap=plt.cm.gray_r, interpolation='nearest')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(C) 2017-2019 by [Damir Cavar](http://cavar.me/damir/) - [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/) ([CA BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)); portions taken from the referenced sources."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "latex_metadata": {
   "affiliation": "Indiana University, Department of Linguistics, Bloomington, IN, USA",
   "author": "Damir Cavar",
   "title": "Python SVM Classifier Example"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": false,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}