1-03_cudf_basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to cuDF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will begin your accelerated data science training with an introduction to [cuDF](https://github.com/rapidsai/cudf), the RAPIDS API that enables you to create and manipulate GPU-accelerated dataframes. cuDF implements a very similar interface to Pandas so that Python data scientists can use it with very little ramp up. Throughout this notebook we will provide Pandas counterparts to the cuDF operations you perform to build your intuition about how much faster cuDF can be, even for seemingly simple operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By the time you complete this notebook you will be able to:\n",
    "\n",
    "- Read and write data to and from disk with cuDF\n",
    "- Perform basic data exploration and cleaning operations with cuDF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we import cuDF and CuPy for GPU-accelerated dataframes and math operations, plus the CPU libraries Pandas and NumPy on which they are based and which we will use for performance comparisons:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cudf\n",
    "import cupy as cp\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading and Writing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using [cuDF](https://github.com/rapidsai/cudf), the RAPIDS API providing a GPU-accelerated dataframe, we can read data from [a variety of formats](https://rapidsai.github.io/projects/cudf/en/0.10.0/api.html#module-cudf.io.csv), including csv, json, parquet, feather, orc, and Pandas dataframes, among others.\n",
    "\n",
    "For the first part of this workshop, we will be reading almost 60 million records (corresponding to the entire population of England and Wales) which were sythesized from official UK census data. Here we read this data from a local csv file directly into GPU memory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4.74 s, sys: 5.53 s, total: 10.3 s\n",
      "Wall time: 11.3 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(58479894, 6)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time gdf = cudf.read_csv('./data/pop_1-03.csv')\n",
    "gdf.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age         int64\n",
       "sex        object\n",
       "county     object\n",
       "lat       float64\n",
       "long      float64\n",
       "name       object\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here for comparison we read the same data into a Pandas dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 30.1 s, sys: 3.71 s, total: 33.8 s\n",
      "Wall time: 33.8 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time df = pd.read_csv('./data/pop_1-03.csv')\n",
    "gdf.shape == df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because of the sophisticated GPU memory management behind the scenes in cuDF, the first data load into a fresh RAPIDS memory environment is sometimes substantially slower than subsequent loads. The RAPIDS Memory Manager is preparing additional memory to accommodate the array of data science operations that you may be interested in using on the data, rather than allocating and deallocating the memory repeatedly throughout your workflow.\n",
    "\n",
    "We will be using `gdf` regularly in this workshop to represent a GPU dataframe, as well as `df` for a CPU dataframe when comparing performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing to File"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cuDF also provides methods for writing data to files. Here we create a new dataframe specifically containing residents of Blackpool county and then write it to `blackpool.csv`, before doing the same with Pandas for comparison."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cuDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.52 s, sys: 1.05 s, total: 2.58 s\n",
      "Wall time: 3.93 s\n",
      "139305 residents\n"
     ]
    }
   ],
   "source": [
    "%time blackpool_residents = gdf.loc[gdf['county'] == 'BLACKPOOL']\n",
    "print(f'{blackpool_residents.shape[0]} residents')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 32 ms, sys: 104 ms, total: 136 ms\n",
      "Wall time: 134 ms\n"
     ]
    }
   ],
   "source": [
    "%time blackpool_residents.to_csv('blackpool.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.43 s, sys: 116 ms, total: 2.54 s\n",
      "Wall time: 2.53 s\n"
     ]
    }
   ],
   "source": [
    "%time blackpool_residents_pd = df.loc[df['county'] == 'BLACKPOOL']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 752 ms, sys: 8 ms, total: 760 ms\n",
      "Wall time: 759 ms\n"
     ]
    }
   ],
   "source": [
    "%time blackpool_residents_pd.to_csv('blackpool_pd.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Initial Data Exploration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have some data loaded, let's do some initial exploration.\n",
    "\n",
    "Use the `head`, `dtypes`, and `columns` methods on `gdf`, as well as the `value_counts` on individual `gdf` columns, to orient yourself to the data. If you're interested, use the `%time` magic command to compare performance against the same operations on the Pandas `df`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can create additional interactive cells by clicking the `+` button above, or by switching to command mode with `Esc` and using the keyboard shortuts `a` (for new cell above) and `b` (for new cell below).\n",
    "\n",
    "If you fill up the GPU memory at any time, don't forget that you can restart the kernel and rerun the cells up to this point quite quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Begin your initial exploration here. Create more cells as needed.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Operations with cuDF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Except for being much more performant with large datasets, cuDF looks and feels a lot like Pandas. In this section we highlight a few very simple operations. When performing data operations on cuDF dataframes, column operations are typically much more performant than row-wise operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Converting Data Types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For machine learning later in this workshop, we will sometimes need to convert integer values into floats. Here we convert the `age` column from `int64` to `float32`, comparing performance with Pandas:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cuDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 4 ms, total: 4 ms\n",
      "Wall time: 3.61 ms\n"
     ]
    }
   ],
   "source": [
    "%time gdf['age'] = gdf['age'].astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 92 ms, sys: 112 ms, total: 204 ms\n",
      "Wall time: 202 ms\n"
     ]
    }
   ],
   "source": [
    "%time df['age'] = df['age'].astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Column-Wise Aggregations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, column-wise aggregations take advantage of the GPU's architecture and RAPIDS' memory format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cuDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 ms, sys: 0 ns, total: 4 ms\n",
      "Wall time: 1 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "40.12419336806595"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time gdf['age'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 184 ms, sys: 84 ms, total: 268 ms\n",
      "Wall time: 267 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "40.12419"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time df['age'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### String Operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although strings are not a datatype traditionally associated with GPUs, cuDF supports powerful accelerated string operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cuDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 28 ms, sys: 28 ms, total: 56 ms\n",
      "Wall time: 71.7 ms\n"
     ]
    }
   ],
   "source": [
    "%time gdf['name'] = gdf['name'].str.title()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.533644</td>\n",
       "      <td>-1.524401</td>\n",
       "      <td>Francis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.426256</td>\n",
       "      <td>-1.465314</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.555200</td>\n",
       "      <td>-1.496417</td>\n",
       "      <td>Teddy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.547906</td>\n",
       "      <td>-1.572341</td>\n",
       "      <td>Angus</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.477639</td>\n",
       "      <td>-1.605995</td>\n",
       "      <td>Charlie</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age sex      county        lat      long     name\n",
       "0  0.0   m  DARLINGTON  54.533644 -1.524401  Francis\n",
       "1  0.0   m  DARLINGTON  54.426256 -1.465314   Edward\n",
       "2  0.0   m  DARLINGTON  54.555200 -1.496417    Teddy\n",
       "3  0.0   m  DARLINGTON  54.547906 -1.572341    Angus\n",
       "4  0.0   m  DARLINGTON  54.477639 -1.605995  Charlie"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 23.4 s, sys: 2.17 s, total: 25.6 s\n",
      "Wall time: 25.6 s\n"
     ]
    }
   ],
   "source": [
    "%time df['name'] = df['name'].str.title()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.533644</td>\n",
       "      <td>-1.524401</td>\n",
       "      <td>Francis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.426256</td>\n",
       "      <td>-1.465314</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.555200</td>\n",
       "      <td>-1.496417</td>\n",
       "      <td>Teddy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.547906</td>\n",
       "      <td>-1.572341</td>\n",
       "      <td>Angus</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.477639</td>\n",
       "      <td>-1.605995</td>\n",
       "      <td>Charlie</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age sex      county        lat      long     name\n",
       "0  0.0   m  DARLINGTON  54.533644 -1.524401  Francis\n",
       "1  0.0   m  DARLINGTON  54.426256 -1.465314   Edward\n",
       "2  0.0   m  DARLINGTON  54.555200 -1.496417    Teddy\n",
       "3  0.0   m  DARLINGTON  54.547906 -1.572341    Angus\n",
       "4  0.0   m  DARLINGTON  54.477639 -1.605995  Charlie"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Subsetting with `loc` and `iloc`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cuDF also supports the core data subsetting tools `loc` (label-based locator) and `iloc` (integer-based locator)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Range Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our data's labels happen to be incrementing numbers, though as with Pandas, `loc` will include every value it is passed whereas `iloc` will give the half-open range (omitting the final value)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.519527</td>\n",
       "      <td>-1.557723</td>\n",
       "      <td>Samuel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.530248</td>\n",
       "      <td>-1.500405</td>\n",
       "      <td>Alden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.515970</td>\n",
       "      <td>-1.628573</td>\n",
       "      <td>Samuel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.543373</td>\n",
       "      <td>-1.664323</td>\n",
       "      <td>Muhammad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.554589</td>\n",
       "      <td>-1.507385</td>\n",
       "      <td>Isaac</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.487209</td>\n",
       "      <td>-1.541073</td>\n",
       "      <td>Jayden</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     age sex      county        lat      long      name\n",
       "100  0.0   m  DARLINGTON  54.519527 -1.557723    Samuel\n",
       "101  0.0   m  DARLINGTON  54.530248 -1.500405     Alden\n",
       "102  0.0   m  DARLINGTON  54.515970 -1.628573    Samuel\n",
       "103  0.0   m  DARLINGTON  54.543373 -1.664323  Muhammad\n",
       "104  0.0   m  DARLINGTON  54.554589 -1.507385     Isaac\n",
       "105  0.0   m  DARLINGTON  54.487209 -1.541073    Jayden"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.loc[100:105]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.519527</td>\n",
       "      <td>-1.557723</td>\n",
       "      <td>Samuel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.530248</td>\n",
       "      <td>-1.500405</td>\n",
       "      <td>Alden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.515970</td>\n",
       "      <td>-1.628573</td>\n",
       "      <td>Samuel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.543373</td>\n",
       "      <td>-1.664323</td>\n",
       "      <td>Muhammad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.554589</td>\n",
       "      <td>-1.507385</td>\n",
       "      <td>Isaac</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     age sex      county        lat      long      name\n",
       "100  0.0   m  DARLINGTON  54.519527 -1.557723    Samuel\n",
       "101  0.0   m  DARLINGTON  54.530248 -1.500405     Alden\n",
       "102  0.0   m  DARLINGTON  54.515970 -1.628573    Samuel\n",
       "103  0.0   m  DARLINGTON  54.543373 -1.664323  Muhammad\n",
       "104  0.0   m  DARLINGTON  54.554589 -1.507385     Isaac"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.iloc[100:105]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `loc` with Boolean Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use `loc` with boolean selections:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cuDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.58 s, sys: 824 ms, total: 2.41 s\n",
      "Wall time: 2.41 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.426256</td>\n",
       "      <td>-1.465314</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.501872</td>\n",
       "      <td>-1.667874</td>\n",
       "      <td>Eamonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.483065</td>\n",
       "      <td>-1.501312</td>\n",
       "      <td>Ethan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.640205</td>\n",
       "      <td>-1.558986</td>\n",
       "      <td>Elvin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.575450</td>\n",
       "      <td>-1.600592</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    age sex      county        lat      long    name\n",
       "1   0.0   m  DARLINGTON  54.426256 -1.465314  Edward\n",
       "6   0.0   m  DARLINGTON  54.501872 -1.667874  Eamonn\n",
       "34  0.0   m  DARLINGTON  54.483065 -1.501312   Ethan\n",
       "45  0.0   m  DARLINGTON  54.640205 -1.558986   Elvin\n",
       "49  0.0   m  DARLINGTON  54.575450 -1.600592  Edward"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# as of version 0.10, the startswith method returns a list, so we convert it back to a Series for efficiency\n",
    "# in a future version, that method and other string methods will return a Series when appropriate\n",
    "%time e_names = gdf.loc[cudf.Series(gdf['name'].str.startswith('E'))]\n",
    "e_names.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 20.5 s, sys: 632 ms, total: 21.1 s\n",
      "Wall time: 21.1 s\n"
     ]
    }
   ],
   "source": [
    "%time e_names_pd = df.loc[df['name'].str.startswith('E')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combining with NumPy Methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can combine cuDF methods with NumPy methods. Here we use `np.logical_and` for elementwise boolean selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cuDF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5.39 s, sys: 744 ms, total: 6.13 s\n",
      "Wall time: 6.12 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.426256</td>\n",
       "      <td>-1.465314</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.575450</td>\n",
       "      <td>-1.600592</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.488042</td>\n",
       "      <td>-1.640927</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.492810</td>\n",
       "      <td>-1.509049</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.577920</td>\n",
       "      <td>-1.436109</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     age sex      county        lat      long    name\n",
       "1    0.0   m  DARLINGTON  54.426256 -1.465314  Edward\n",
       "49   0.0   m  DARLINGTON  54.575450 -1.600592  Edward\n",
       "106  0.0   m  DARLINGTON  54.488042 -1.640927  Edward\n",
       "145  0.0   m  DARLINGTON  54.492810 -1.509049  Edward\n",
       "170  0.0   m  DARLINGTON  54.577920 -1.436109  Edward"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time ed_names = gdf.loc[np.logical_and(gdf['name'].str.startswith('E'), gdf['name'].str.endswith('d'))]\n",
    "ed_names.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For better performance, we can use CuPy instead of NumPy, thereby performing the elementwise boolean `logical_and` operation on GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3.02 s, sys: 600 ms, total: 3.62 s\n",
      "Wall time: 3.62 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>county</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.426256</td>\n",
       "      <td>-1.465314</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.575450</td>\n",
       "      <td>-1.600592</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.488042</td>\n",
       "      <td>-1.640927</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.492810</td>\n",
       "      <td>-1.509049</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>0.0</td>\n",
       "      <td>m</td>\n",
       "      <td>DARLINGTON</td>\n",
       "      <td>54.577920</td>\n",
       "      <td>-1.436109</td>\n",
       "      <td>Edward</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     age sex      county        lat      long    name\n",
       "1    0.0   m  DARLINGTON  54.426256 -1.465314  Edward\n",
       "49   0.0   m  DARLINGTON  54.575450 -1.600592  Edward\n",
       "106  0.0   m  DARLINGTON  54.488042 -1.640927  Edward\n",
       "145  0.0   m  DARLINGTON  54.492810 -1.509049  Edward\n",
       "170  0.0   m  DARLINGTON  54.577920 -1.436109  Edward"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time ed_names = gdf.loc[cudf.Series(cp.logical_and(cudf.Series(gdf['name'].str.startswith('E')), cudf.Series(gdf['name'].str.endswith('d'))))]\n",
    "ed_names.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 31.6 s, sys: 476 ms, total: 32 s\n",
      "Wall time: 32 s\n"
     ]
    }
   ],
   "source": [
    "%time ed_names_pd = df.loc[np.logical_and(df['name'].str.startswith('E'), df['name'].str.endswith('d'))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Basic Data Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this exercise we ask you to perform two simple data cleaning tasks using several of the techniques described above:\n",
    "\n",
    "1. Modifying the data type of a couple columns\n",
    "2. Transforming string data into our desired format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Modify `dtypes`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Examine the `dtypes` of `gdf` and convert any 64-bit data types to their 32-bit counterparts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf['lat'] = gdf['lat'].astype('float32')\n",
    "gdf['long'] = gdf['long'].astype('float32')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age       float32\n",
       "sex        object\n",
       "county     object\n",
       "lat       float32\n",
       "long      float32\n",
       "name       object\n",
       "dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load solutions/modify_dtypes\n",
    "gdf['lat'] = gdf['lat'].astype('float32')\n",
    "gdf['long'] = gdf['long'].astype('float32')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Title Case the Counties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it stands, all of the counties are UPPERCASE:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    DARLINGTON\n",
       "1    DARLINGTON\n",
       "2    DARLINGTON\n",
       "3    DARLINGTON\n",
       "4    DARLINGTON\n",
       "Name: county, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf['county'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert them to title case as we have already done with the `name` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 26.2 s, sys: 2.48 s, total: 28.7 s\n",
      "Wall time: 28.6 s\n"
     ]
    }
   ],
   "source": [
    "%time df['county'] = df['county'].str.title()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load solutions/title_case_counties\n",
    "gdf['county'] = gdf['county'].str.title()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Counties North of Sunderland"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This exercise will require to use the `loc` method, and several of the techniques described above. Identify the latitude of the northernmost resident of Sunderland county (the person with the maximum `lat` value), and then determine which counties have any residents north of this resident. Use the `unique` method of a cudf `Series` to deduplicate the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_data = cudf.Series(gdf.loc[gdf['lat']>gdf.loc[gdf['county'] == 'Sunderland']['lat'].max()]['county']).unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          County Durham\n",
       "1                Cumbria\n",
       "2              Gateshead\n",
       "3    Newcastle Upon Tyne\n",
       "4         North Tyneside\n",
       "5        North Yorkshire\n",
       "6         Northumberland\n",
       "7         South Tyneside\n",
       "Name: county, dtype: object"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load solutions/counties_north_of_sunderland\n",
    "sunderland_residents = gdf.loc[gdf['county'] == 'Sunderland']\n",
    "northmost_sunderland_lat = sunderland_residents['lat'].max()\n",
    "counties_with_pop_north_of = gdf.loc[gdf['lat'] > northmost_sunderland_lat]['county'].unique()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          County Durham\n",
       "1                Cumbria\n",
       "2              Gateshead\n",
       "3    Newcastle Upon Tyne\n",
       "4         North Tyneside\n",
       "5        North Yorkshire\n",
       "6         Northumberland\n",
       "7         South Tyneside\n",
       "Name: county, dtype: object"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counties_with_pop_north_of"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<div align=\"center\"><h2>Please Restart the Kernel</h2></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import IPython\n",
    "app = IPython.Application.instance()\n",
    "app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next section, you will do some actual data preparation for use in our machine learning models later. As part of your work, we will create custom functions with CuPy, which can be used as a GPU-accelerated drop-in replacement for NumPy with drastic performance benefits."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}