training_7.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python for Data Analysis III\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Agenda:**\n",
    "\n",
    "    * CProfile \n",
    "    * Cython\n",
    "    * sklearn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Writing programs is fun, but making them fast can be a pain. Python programs are no exception to that, but the basic profiling toolchain is actually not that complicated to use. Here, I would like to show you how you can quickly profile and analyze your Python code to find what part of the code you should optimize."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can do profiling manually"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting line_profiler\n",
      "Collecting IPython>=0.13 (from line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/b1/7f/91d50f28af3e3a24342561983a7857e399ce24093876e6970b986a0b6677/ipython-6.4.0-py3-none-any.whl\n",
      "Collecting setuptools>=18.5 (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/7f/e1/820d941153923aac1d49d7fc37e17b6e73bfbd2904959fffbad77900cf92/setuptools-39.2.0-py2.py3-none-any.whl\n",
      "Collecting pygments (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl\n",
      "Collecting pickleshare (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/9f/17/daa142fc9be6b76f26f24eeeb9a138940671490b91cb5587393f297c8317/pickleshare-0.7.4-py2.py3-none-any.whl\n",
      "Collecting pexpect; sys_platform != \"win32\" (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/89/e6/b5a1de8b0cc4e07ca1b305a4fcc3f9806025c1b651ea302646341222f88b/pexpect-4.6.0-py2.py3-none-any.whl\n",
      "Collecting backcall (from IPython>=0.13->line_profiler)\n",
      "Collecting jedi>=0.10 (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/e7/42/074192a165622e645ed4aeade63e76e56b3496a044569b3c6cae3a918352/jedi-0.12.0-py2.py3-none-any.whl\n",
      "Collecting simplegeneric>0.8 (from IPython>=0.13->line_profiler)\n",
      "Collecting decorator (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/bc/bb/a24838832ba35baf52f32ab1a49b906b5f82fb7c76b2f6a7e35e140bac30/decorator-4.3.0-py2.py3-none-any.whl\n",
      "Collecting prompt-toolkit<2.0.0,>=1.0.15 (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/04/d1/c6616dd03701e7e2073f06d5c3b41b012256e42b72561f16a7bd86dd7b43/prompt_toolkit-1.0.15-py3-none-any.whl\n",
      "Collecting traitlets>=4.2 (from IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/93/d6/abcb22de61d78e2fc3959c964628a5771e47e7cc60d53e9342e21ed6cc9a/traitlets-4.3.2-py2.py3-none-any.whl\n",
      "Collecting ptyprocess>=0.5 (from pexpect; sys_platform != \"win32\"->IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/ff/4e/fa4a73ccfefe2b37d7b6898329e7dbcd1ac846ba3a3b26b294a78a3eb997/ptyprocess-0.5.2-py2.py3-none-any.whl\n",
      "Collecting parso>=0.2.0 (from jedi>=0.10->IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/cd/3e/5908f9577dbd1e5df53e64349bfd11e46b726c1e4d8cd676bbe8aa4de316/parso-0.2.1-py2.py3-none-any.whl\n",
      "Collecting six>=1.9.0 (from prompt-toolkit<2.0.0,>=1.0.15->IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl\n",
      "Collecting wcwidth (from prompt-toolkit<2.0.0,>=1.0.15->IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/7e/9f/526a6947247599b084ee5232e4f9190a38f398d7300d866af3ab571a5bfe/wcwidth-0.1.7-py2.py3-none-any.whl\n",
      "Collecting ipython-genutils (from traitlets>=4.2->IPython>=0.13->line_profiler)\n",
      "  Using cached https://files.pythonhosted.org/packages/fa/bc/9bd3b5c2b4774d5f33b2d544f1460be9df7df2fe42f352135381c347c69a/ipython_genutils-0.2.0-py2.py3-none-any.whl\n",
      "Installing collected packages: setuptools, pygments, pickleshare, ptyprocess, pexpect, backcall, parso, jedi, simplegeneric, decorator, six, wcwidth, prompt-toolkit, ipython-genutils, traitlets, IPython, line-profiler\n",
      "Successfully installed IPython-6.2.1 backcall-0.1.0 decorator-4.2.1 ipython-genutils-0.2.0 jedi-0.11.1 line-profiler-2.1.2 parso-0.1.1 pexpect-4.6.0 pickleshare-0.7.4 prompt-toolkit-1.0.15 ptyprocess-0.5.2 pygments-2.2.0 setuptools-39.2.0 simplegeneric-0.8.1 six-1.11.0 traitlets-4.3.2 wcwidth-0.1.7\n",
      "\u001b[33mYou are using pip version 8.1.1, however version 10.0.1 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip3 install line_profiler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting speedup.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile speedup.py\n",
    "\n",
    "import random\n",
    "\n",
    "class Matrix(list):\n",
    "    @classmethod\n",
    "    def zeros(cls, shape):\n",
    "        n_rows, n_cols = shape\n",
    "        return cls([[0] * n_cols for i in range(n_rows)])\n",
    "\n",
    "    @classmethod\n",
    "    def random(cls, shape):\n",
    "        M, (n_rows, n_cols) = cls(), shape \n",
    "        for i in range (n_rows):\n",
    "            M.append([random.randint(-255, 255) for j in range (n_cols)])\n",
    "        return M\n",
    "\n",
    "    @property\n",
    "    def shape(self):\n",
    "        return ((0, 0) if not self else (len(self), len(self[0])))\n",
    "    \n",
    "    \n",
    "def dot_product(X, Y):\n",
    "    n_xrows, n_xcols = X.shape\n",
    "    n_yrows, n_ycols = Y.shape\n",
    "    Z = Matrix.zeros((n_xrows, n_ycols))\n",
    "    for i in range(n_xrows):\n",
    "        for j in range(n_xcols):\n",
    "            for k in range(n_ycols):\n",
    "                Z[i][k] += X[i][j] * Y[j][k]\n",
    "    return Z\n",
    "\n",
    "def bench(shape=(64, 64), n_iter=16):\n",
    "    X = Matrix.random(shape)\n",
    "    Y = Matrix.random(shape)\n",
    "    for iter in range(n_iter):\n",
    "        dot_product(X, Y)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    bench()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.86 µs ± 113 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "a1 = np.random.rand(3,2)\n",
    "a2 = np.random.rand(2,3)\n",
    "a1.dot(a2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cProfile module allows you to profile Python code up to a function or method call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "         41377 function calls in 2.356 seconds\n",
      "\n",
      "   Ordered by: internal time\n",
      "\n",
      "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
      "       16    2.330    0.146    2.331    0.146 <string>:22(dot_product)\n",
      "     8192    0.008    0.000    0.017    0.000 random.py:170(randrange)\n",
      "     8192    0.007    0.000    0.009    0.000 random.py:220(_randbelow)\n",
      "     8192    0.004    0.000    0.020    0.000 random.py:214(randint)\n",
      "      128    0.003    0.000    0.023    0.000 <string>:14(<listcomp>)\n",
      "     8201    0.002    0.000    0.002    0.000 {method 'getrandbits' of '_random.Random' objects}\n",
      "        1    0.001    0.001    2.356    2.356 {built-in method builtins.exec}\n",
      "        1    0.001    0.001    2.355    2.355 <string>:32(bench)\n",
      "     8192    0.001    0.000    0.001    0.000 {method 'bit_length' of 'int' objects}\n",
      "       16    0.000    0.000    0.000    0.000 <string>:8(<listcomp>)\n",
      "        2    0.000    0.000    0.023    0.012 <string>:10(random)\n",
      "       16    0.000    0.000    0.000    0.000 <string>:5(zeros)\n",
      "        1    0.000    0.000    2.355    2.355 <string>:2(<module>)\n",
      "       32    0.000    0.000    0.000    0.000 <string>:17(shape)\n",
      "        1    0.000    0.000    0.000    0.000 {built-in method builtins.__build_class__}\n",
      "      128    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}\n",
      "       64    0.000    0.000    0.000    0.000 {built-in method builtins.len}\n",
      "        1    0.000    0.000    0.000    0.000 <string>:4(Matrix)\n",
      "        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import cProfile\n",
    "\n",
    "source = open(\"speedup.py\").read()\n",
    "cProfile.run(source, sort=\"tottime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext line_profiler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from speedup import dot_product, bench\n",
    "%lprun -f dot_product bench"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Scikit-Learn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Scikit-Learn` is a library, in which implemented a large number of machine learning algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can separate learning problems in a few large categories:\n",
    "\n",
    "1. supervised learning, in which the data comes with additional attributes that we want to predict.This problem can be either:\n",
    "\n",
    "    - classification: samples belong to two or more classes and we want to learn from already labeled data how to predict the class of unlabeled data.\n",
    "    - regression: if the desired output consists of one or more continuous variables, then the task is called regression.\n",
    "\n",
    "2. unsupervised learning, in which the training data consists of a set of input vectors x without any corresponding target values. The goal in such problems may be to discover groups of similar examples within the data, where it is called clustering, or to determine the distribution of data within the input space, known as density estimation, or to project the data from a high-dimensional space down to two or three dimensions for the purpose of visualization.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, a learning problem considers a set of `n` samples of data and then tries to predict properties of unknown data. If each sample is more than a single number and, for instance, a multi-dimensional entry (aka multivariate data), it is said to have several attributes or features.\n",
    "\n",
    "This idea of first learn known samples and then predict new samples is implemented in scikit-learn with two basic functions: `fit` and `predict`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('Churn-Modelling.csv')\n",
    "df.dropna(inplace=True)\n",
    "df = df[['CreditScore', 'Age', 'Balance', 'EstimatedSalary', 'Exited']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CreditScore</th>\n",
       "      <th>Age</th>\n",
       "      <th>Balance</th>\n",
       "      <th>EstimatedSalary</th>\n",
       "      <th>Exited</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>619</td>\n",
       "      <td>42</td>\n",
       "      <td>0.00</td>\n",
       "      <td>101348.88</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>608</td>\n",
       "      <td>41</td>\n",
       "      <td>83807.86</td>\n",
       "      <td>112542.58</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>699</td>\n",
       "      <td>39</td>\n",
       "      <td>0.00</td>\n",
       "      <td>93826.63</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>850</td>\n",
       "      <td>43</td>\n",
       "      <td>125510.82</td>\n",
       "      <td>79084.10</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>822</td>\n",
       "      <td>50</td>\n",
       "      <td>0.00</td>\n",
       "      <td>10062.80</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   CreditScore  Age    Balance  EstimatedSalary  Exited\n",
       "0          619   42       0.00        101348.88       1\n",
       "1          608   41   83807.86        112542.58       0\n",
       "3          699   39       0.00         93826.63       0\n",
       "4          850   43  125510.82         79084.10       0\n",
       "6          822   50       0.00         10062.80       0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Everyone is familiar with the fact that most gradient methods (strongly or strangely to scale data). Therefore, before running algorithms, either normalization or so-called standardization is usually done. Normalization involves replacing the nominal characteristics so that each of them lies in the range from 0 to 1. Standardization implies the same preprocessing of data, after which each attribute has an average of 0 and a variance of 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "# normalize the data attributes\n",
    "normalized_df = preprocessing.normalize(df)\n",
    "# standardize the data attributes\n",
    "standardized_df = preprocessing.scale(df) # Standardization isn't required for logistic regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training set and testing set\n",
    "\n",
    "Machine learning is about learning some properties of a data set and applying them to new data. This is why a common practice in machine learning to evaluate an algorithm is to split the data at hand into two sets, one that we call the training set on which we learn data properties and one that we call the testing set on which we test these properties"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "train,test = train_test_split(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7483, 2495)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train), len(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CreditScore</th>\n",
       "      <th>Age</th>\n",
       "      <th>Balance</th>\n",
       "      <th>EstimatedSalary</th>\n",
       "      <th>Exited</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7965</th>\n",
       "      <td>625</td>\n",
       "      <td>51</td>\n",
       "      <td>124620.01</td>\n",
       "      <td>92243.94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1018</th>\n",
       "      <td>850</td>\n",
       "      <td>45</td>\n",
       "      <td>103909.86</td>\n",
       "      <td>60083.11</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7182</th>\n",
       "      <td>692</td>\n",
       "      <td>49</td>\n",
       "      <td>110540.43</td>\n",
       "      <td>107472.99</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8234</th>\n",
       "      <td>766</td>\n",
       "      <td>47</td>\n",
       "      <td>129289.98</td>\n",
       "      <td>169935.46</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2281</th>\n",
       "      <td>848</td>\n",
       "      <td>40</td>\n",
       "      <td>148495.64</td>\n",
       "      <td>158853.98</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      CreditScore  Age    Balance  EstimatedSalary  Exited\n",
       "7965          625   51  124620.01         92243.94       1\n",
       "1018          850   45  103909.86         60083.11       1\n",
       "7182          692   49  110540.43        107472.99       0\n",
       "8234          766   47  129289.98        169935.46       1\n",
       "2281          848   40  148495.64        158853.98       0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
      "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
      "          penalty='l2', random_state=22, solver='liblinear', tol=0.0001,\n",
      "          verbose=0, warm_start=False)\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "from sklearn.linear_model import LogisticRegression # <-- our model\n",
    "\n",
    "model = LogisticRegression(random_state=22)\n",
    "model.fit(train[['CreditScore', 'Age', 'Balance', 'EstimatedSalary']], train['Exited'])\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make predictions\n",
    "expected = test['Exited']\n",
    "predicted = model.predict(test[['CreditScore', 'Age', 'Balance', 'EstimatedSalary']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2495, 2495)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(predicted), len(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2005    4]\n",
      " [ 486    0]]\n"
     ]
    }
   ],
   "source": [
    "# summarize the fit of the model\n",
    "print(metrics.confusion_matrix(expected, predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `confusion_matrix()` function will calculate a confusion matrix and return the result as an array.\n",
    "The result is telling us that we have 1925+27 correct predictions and 506+37 incorrect predictions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is no need to describe the library scikit-learn - it's just a bunch of algorithms for solving machine learning problems. It does not solve your problems magically - as in our example above - we do not just put the data in the library and get an amazing result. You need to do some work before using the library.\n",
    "\n",
    "If you already know the algorithm/model for your particular problem, you can just go to the scilit-learn documentation and find out how to use it.\n",
    "\n",
    "http://scikit-learn.org/stable/documentation.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}