2.tour/10-Iterators.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Iterators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Often an important piece of data analysis is repeating a similar calculation, over and over, in an automated fashion.\n",
    "\n",
    "For example, you may have a table of a names that you'd like to split into first and last, or perhaps of dates that you'd like to convert to some standard format.\n",
    "\n",
    "One of Python's answers to this is the *iterator* syntax.\n",
    "We've seen this already with the ``range`` iterator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<!--BOOK_INFORMATION-->\n",
    "<img align=\"left\" style=\"padding-right:10px;\" src=\"fig/cover-small.jpg\">\n",
    "*This notebook contains an excerpt from the [Whirlwind Tour of Python](http://www.oreilly.com/programming/free/a-whirlwind-tour-of-python.csp) by Jake VanderPlas; the content is available [on GitHub](https://github.com/jakevdp/WhirlwindTourOfPython).*\n",
    "\n",
    "*The text and code are released under the [CC0](https://github.com/jakevdp/WhirlwindTourOfPython/blob/master/LICENSE) license; see also the companion project, the [Python Data Science Handbook](https://github.com/jakevdp/PythonDataScienceHandbook).*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<!--NAVIGATION-->\n",
    "< [Errors and Exceptions](09-Errors-and-Exceptions.ipynb) | [Contents](Index.ipynb) | [List Comprehensions](11-List-Comprehensions.ipynb) >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 1 2 3 4 5 6 7 8 9 "
     ]
    }
   ],
   "source": [
    "#range(10)\n",
    "for i in range(10):\n",
    "    print(i, end=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here we're going to dig a bit deeper.\n",
    "\n",
    "- In Python 3, ``range`` is not a list, but is something called an *iterator*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Iterating over lists\n",
    "Iterators are perhaps most easily understood in the concrete case of iterating through a list.\n",
    "\n",
    "Consider the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3 5 7 9 11 "
     ]
    }
   ],
   "source": [
    "for value in [2, 4, 6, 8, 10]:\n",
    "    # do some operation\n",
    "    print(value + 1, end=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\"``for x in y``\" syntax allows us to repeat some operation for each value in the list.\n",
    "- \"*for [each] value in [the] list*\"\n",
    "- the Python interpreter checks whether it has an *iterator* interface\n",
    "    - check with the built-in ``iter`` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:16:55.399330Z",
     "start_time": "2018-10-17T07:16:55.394351Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<list_iterator at 0x10c9adef0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iter([2, 4, 6, 8, 10])\n",
    "# listi = iter([2, 4, 6, 8, 10])\n",
    "# print(next(listi), next(listi), next(listi))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The iterator object provides the functionality required by the ``for`` loop.\n",
    "- The ``iter`` object is a container\n",
    "- it gives you access to the next object for as long as it's valid\n",
    "    - which can be seen with the built-in function ``next``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "I = iter([2, 4, 6, 8, 10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "print(next(I))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    }
   ],
   "source": [
    "print(next(I))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n"
     ]
    }
   ],
   "source": [
    "print(next(I))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This is incredibly useful\n",
    "- because it allows Python to treat things as lists that are *not actually lists*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## ``range()``\n",
    "#### The most common example of indirect iteration \n",
    "- ``range()`` function in Python 3 \n",
    "- ``xrange()`` in Python 2\n",
    "\n",
    "``range()`` returns not a list, but a special ``range()`` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:19:22.830161Z",
     "start_time": "2018-10-17T07:19:22.826330Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "range"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "range(10)\n",
    "#type(range(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "``range``, like a list, exposes an iterator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:19:50.991062Z",
     "start_time": "2018-10-17T07:19:50.986572Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<range_iterator at 0x10c982960>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iter(range(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So Python knows to treat it *as if* it's a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:20:16.472815Z",
     "start_time": "2018-10-17T07:20:16.467395Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 1 2 3 4 5 6 7 8 9 "
     ]
    }
   ],
   "source": [
    "for i in range(10):\n",
    "    print(i, end=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The benefit of the iterator indirection\n",
    "- in python 3, ``range()`` does **NOT** explicitly create *the full list!*\n",
    "- in Python 2, ``range`` creates a list\n",
    "    - so in python 2 running a for-loop for a huge list would overwhelm our system memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "
     ]
    }
   ],
   "source": [
    "N = 10 ** 12\n",
    "for i in range(N):\n",
    "    if i >= 10: break\n",
    "    print(i, end=', ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "If we actually create a list of one trillion values\n",
    "- it would occupy tens of terabytes of machine memory\n",
    "- it is a waste, because our calcualtion does not need so many values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In fact, iterators do NOT have to end at all!\n",
    "- Python's ``itertools`` library contains a ``count`` function that acts as an infinite range:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0, 1, 2, 3, 4, 5, 6, 7, 8, 9, "
     ]
    }
   ],
   "source": [
    "from itertools import count\n",
    "\n",
    "for i in count():\n",
    "    if i >= 10:\n",
    "        break \n",
    "    print(i, end=', ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Without a loop break here, it would go on happily counting until the process is manually interrupted or killed (using, for example, ``ctrl-C``)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Some Useful Iterators\n",
    "\n",
    "Here we'll cover some of the more useful iterators in the Python language:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### ``enumerate``\n",
    "\n",
    "Often you need to \n",
    "- iterate not only the values in an array, \n",
    "- but also keep track of the index.\n",
    "\n",
    "You might be tempted to do things this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:26:26.004415Z",
     "start_time": "2018-10-17T07:26:25.998151Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 2\n",
      "1 4\n",
      "2 6\n",
      "3 8\n",
      "4 10\n"
     ]
    }
   ],
   "source": [
    "L = [2, 4, 6, 8, 10]\n",
    "for k in range(len(L)):\n",
    "    print(k, L[k])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Although this does work, Python provides a cleaner syntax using the ``enumerate`` iterator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:26:29.651157Z",
     "start_time": "2018-10-17T07:26:29.646674Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 2\n",
      "1 4\n",
      "2 6\n",
      "3 8\n",
      "4 10\n"
     ]
    }
   ],
   "source": [
    "#enumerate(L)\n",
    "for k, i in enumerate(L):\n",
    "    print(k, i) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This is the more \"Pythonic\" way to enumerate the indices and values in a list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### ``zip``\n",
    "\n",
    "to iterate over multiple lists simultaneously.\n",
    "\n",
    "- it is better to use the ``zip`` iterator\n",
    "    - which zips together iterables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:28:56.880751Z",
     "start_time": "2018-10-17T07:28:56.873972Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 3\n",
      "4 6\n",
      "6 9\n",
      "8 12\n",
      "10 15\n"
     ]
    }
   ],
   "source": [
    "L = [2, 4, 6, 8, 10]\n",
    "R = [3, 6, 9, 12, 15]\n",
    "for li, ri in zip(L, R):\n",
    "    print(li, ri)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:29:23.788236Z",
     "start_time": "2018-10-17T07:29:23.783391Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(2, 3), (4, 6), (6, 9), (8, 12), (10, 15)]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(L, R))\n",
    "#type(zip(L, R))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Any number of iterables can be zipped together, \n",
    "- and if they are different lengths, the shortest will determine the length of the ``zip``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 3\n",
      "4 6\n",
      "6 9\n"
     ]
    }
   ],
   "source": [
    "L = [2, 4, 6]\n",
    "R = [3, 6, 9, 12, 15]\n",
    "for li, ri in zip(L, R):\n",
    "    print(li, ri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### ``map`` and ``filter``\n",
    "The ``map`` iterator \n",
    "- takes a function \n",
    "- and applies the function to the values in an iterator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 1 4 9 16 25 36 49 64 81 "
     ]
    }
   ],
   "source": [
    "# find the first 10 square numbers\n",
    "square = lambda x: x ** 2\n",
    "for val in map(square, range(10)):\n",
    "    print(val, end=' ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(square, range(10)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The ``filter`` iterator looks similar, except it only passes-through values for which the filter function evaluates to True:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 2 4 6 8 "
     ]
    }
   ],
   "source": [
    "# find values up to 10 for which x % 2 is zero\n",
    "is_even = lambda x: x % 2 == 0\n",
    "for val in filter(is_even, range(10)):\n",
    "    print(val, end=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The ``map`` and ``filter`` functions, along with the ``reduce`` function (which lives in Python's ``functools`` module) are fundamental components of the *functional programming* style\n",
    "\n",
    "see, for example, the [pytoolz](https://toolz.readthedocs.org/en/latest/) library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Iterators as function arguments\n",
    "\n",
    "We have learnt [``*args`` and ``**kwargs``: Flexible Arguments](#*args-and-**kwargs:-Flexible-Arguments). \n",
    "\n",
    "- ``*args`` and ``**kwargs`` can be used to pass sequences and dictionaries to functions.\n",
    "\n",
    "the ``*args`` syntax works not just with sequences, but with any iterator:\n",
    "\n",
    "# ``*args`` in a function turns Iterators into function's iteratable arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:36:55.412717Z",
     "start_time": "2018-10-17T07:36:55.408123Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 1 2 3 4 5 6 7 8 9\n"
     ]
    }
   ],
   "source": [
    "print(*range(10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:37:57.318327Z",
     "start_time": "2018-10-17T07:37:57.313867Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 4) (2, 5) (3, 6)\n"
     ]
    }
   ],
   "source": [
    "print(*zip([1, 2, 3], [4, 5, 6]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:39:24.401951Z",
     "start_time": "2018-10-17T07:39:24.397972Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "can't use starred expression here (<ipython-input-23-822f3c7b2e82>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-23-822f3c7b2e82>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    *range(10)\u001b[0m\n\u001b[0m              ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m can't use starred expression here\n"
     ]
    }
   ],
   "source": [
    "*range(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "range(0, 10)\n"
     ]
    }
   ],
   "source": [
    "print(range(10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n"
     ]
    }
   ],
   "source": [
    "print(list(range(10)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So, for example, we can get tricky and compress the ``map`` example from before into the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:40:24.030390Z",
     "start_time": "2018-10-17T07:40:24.024865Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 1 4 9 16 25 36 49 64 81\n"
     ]
    }
   ],
   "source": [
    "print(  *map( lambda x: x ** 2, range(10) )  )\n",
    "\n",
    "# square = lambda x: x**2\n",
    "# map_object = map(square, range(10))\n",
    "# print(*map_object)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:40:55.830571Z",
     "start_time": "2018-10-17T07:40:55.826120Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map( lambda x: x ** 2, range(10) ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "map_object = map( lambda x: x ** 2, range(10) ) \n",
    "# it is a map object\n",
    "list( map_object )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "why is there no ``unzip()`` function which does the opposite of ``zip()``?\n",
    "- the opposite of ``zip()`` is... ``zip()``! \n",
    "- The key is that ``zip()`` can zip-together any number of iterators or sequences. Observe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "L1 = (1, 2, 3, 4)\n",
    "L2 = ('a', 'b', 'c', 'd')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 'a') (2, 'b') (3, 'c') (4, 'd')\n"
     ]
    }
   ],
   "source": [
    "z = zip(L1, L2)\n",
    "print(*z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2, 3, 4) ('a', 'b', 'c', 'd')\n"
     ]
    }
   ],
   "source": [
    "new_L1, new_L2 = zip((1, 'a'), (2, 'b'), (3, 'c') ,(4, 'd'))\n",
    "print(new_L1, new_L2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:43:59.963119Z",
     "start_time": "2018-10-17T07:43:59.957751Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2, 3, 4) ('a', 'b', 'c', 'd')\n"
     ]
    }
   ],
   "source": [
    "L1 = (1, 2, 3, 4)\n",
    "L2 = ('a', 'b', 'c', 'd')\n",
    "#z = zip(L1, L2)\n",
    "new_L1, new_L2 = zip(*zip(L1, L2))\n",
    "print(new_L1, new_L2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Ponder this for a while. \n",
    "- If you understand why it works, you'll have come a long way in understanding Python iterators!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Specialized Iterators: ``itertools``\n",
    "\n",
    "\n",
    "The ``itertools`` module contains a whole host of useful iterators\n",
    "- We have briefly looked at the infinite ``range`` iterator, ``itertools.count``.\n",
    "- consider the ``itertools.permutations`` function, \n",
    "    - which iterates over all permutations of a sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0, 1, 2) (0, 2, 1) (1, 0, 2) (1, 2, 0) (2, 0, 1) (2, 1, 0)\n"
     ]
    }
   ],
   "source": [
    "from itertools import permutations\n",
    "p = permutations(range(3))\n",
    "print(*p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<itertools.permutations at 0x109485bf8>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Similarly, the ``itertools.combinations`` function iterates over all unique combinations of ``N`` values within a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0, 1) (0, 2) (0, 3) (1, 2) (1, 3) (2, 3)\n"
     ]
    }
   ],
   "source": [
    "from itertools import combinations\n",
    "c = combinations(range(4), 2)\n",
    "print(*c)\n",
    "#print(c)\n",
    "#list(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Somewhat related is the ``product`` iterator, which iterates over all sets of pairs between two or more iterables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-17T07:47:51.867043Z",
     "start_time": "2018-10-17T07:47:51.861703Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('a', 0) ('a', 1) ('a', 2) ('b', 0) ('b', 1) ('b', 2)\n"
     ]
    }
   ],
   "source": [
    "from itertools import product\n",
    "#p = product(['a', 'b'], range(3))\n",
    "p = product('ab', range(3))\n",
    "\n",
    "print(*p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Many more useful iterators exist in ``itertools``: the full list can be found, along with some examples, in Python's [online documentation](https://docs.python.org/3.5/library/itertools.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [Errors and Exceptions](09-Errors-and-Exceptions.ipynb) | [Contents](Index.ipynb) | [List Comprehensions](11-List-Comprehensions.ipynb) >"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}