api-scraping.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting Data from APIs\n",
    "What is an API?\n",
    "- Application Programming Interface\n",
    "- Structured way to expose specific functionality and data access to users\n",
    "- Web APIs usually follow the \"REST\" standard\n",
    "\n",
    "How to interact with a REST API:\n",
    "- Make a \"request\" to a specific URL (an \"endpoint\"), and get the data back in a \"response\"\n",
    "- Most relevant request method for us is GET (other methods: POST, PUT, DELETE)\n",
    "- Response is often JSON format\n",
    "- Web console is sometimes available (allows you to explore an API)\n",
    "\n",
    "For this example we will be using SWAPI - the Star Wars API. \n",
    "This API is very user-friendly and an easy tool to learn with.\n",
    "https://swapi.co/\n",
    "\n",
    "Check out [the documentation](https://swapi.co/documentation) to get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use requests library to interact with a URL\n",
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#based on the documentation, https://swapi.co/documentation, what is this requesting\n",
    "full_request_url = 'http://swapi.co/api/starships'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = requests.get(full_request_url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check the status: 200 means success, 4xx means error\n",
    "r.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\"count\":37,\"next\":\"https://swapi.co/api/starships/?page=2\",\"previous\":null,\"results\":[{\"name\":\"Executor\",\"model\":\"Executor-class star dreadnought\",\"manufacturer\":\"Kuat Drive Yards, Fondor Shipyards\",\"cost_in_credits\":\"1143350000\",\"length\":\"19000\",\"max_atmosphering_speed\":\"n/a\",\"crew\":\"279144\",\"passengers\":\"38000\",\"cargo_capacity\":\"250000000\",\"consumables\":\"6 years\",\"hyperdrive_rating\":\"2.0\",\"MGLT\":\"40\",\"starship_class\":\"Star dreadnought\",\"pilots\":[],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/3/\"],\"created\":\"2014-12-15T12:31:42.547000Z\",\"edited\":\"2017-04-19T10:56:06.685592Z\",\"url\":\"https://swapi.co/api/starships/15/\"},{\"name\":\"Sentinel-class landing craft\",\"model\":\"Sentinel-class landing craft\",\"manufacturer\":\"Sienar Fleet Systems, Cyngus Spaceworks\",\"cost_in_credits\":\"240000\",\"length\":\"38\",\"max_atmosphering_speed\":\"1000\",\"crew\":\"5\",\"passengers\":\"75\",\"cargo_capacity\":\"180000\",\"consumables\":\"1 month\",\"hyperdrive_rating\":\"1.0\",\"MGLT\":\"70\",\"starship_class\":\"landing craft\",\"pilots\":[],\"films\":[\"https://swapi.co/api/films/1/\"],\"created\":\"2014-12-10T15:48:00.586000Z\",\"edited\":\"2014-12-22T17:35:44.431407Z\",\"url\":\"https://swapi.co/api/starships/5/\"},{\"name\":\"Death Star\",\"model\":\"DS-1 Orbital Battle Station\",\"manufacturer\":\"Imperial Department of Military Research, Sienar Fleet Systems\",\"cost_in_credits\":\"1000000000000\",\"length\":\"120000\",\"max_atmosphering_speed\":\"n/a\",\"crew\":\"342953\",\"passengers\":\"843342\",\"cargo_capacity\":\"1000000000000\",\"consumables\":\"3 years\",\"hyperdrive_rating\":\"4.0\",\"MGLT\":\"10\",\"starship_class\":\"Deep Space Mobile Battlestation\",\"pilots\":[],\"films\":[\"https://swapi.co/api/films/1/\"],\"created\":\"2014-12-10T16:36:50.509000Z\",\"edited\":\"2014-12-22T17:35:44.452589Z\",\"url\":\"https://swapi.co/api/starships/9/\"},{\"name\":\"Millennium Falcon\",\"model\":\"YT-1300 light freighter\",\"manufacturer\":\"Corellian Engineering Corporation\",\"cost_in_credits\":\"100000\",\"length\":\"34.37\",\"max_atmosphering_speed\":\"1050\",\"crew\":\"4\",\"passengers\":\"6\",\"cargo_capacity\":\"100000\",\"consumables\":\"2 months\",\"hyperdrive_rating\":\"0.5\",\"MGLT\":\"75\",\"starship_class\":\"Light freighter\",\"pilots\":[\"https://swapi.co/api/people/13/\",\"https://swapi.co/api/people/14/\",\"https://swapi.co/api/people/25/\",\"https://swapi.co/api/people/31/\"],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/7/\",\"https://swapi.co/api/films/3/\",\"https://swapi.co/api/films/1/\"],\"created\":\"2014-12-10T16:59:45.094000Z\",\"edited\":\"2014-12-22T17:35:44.464156Z\",\"url\":\"https://swapi.co/api/starships/10/\"},{\"name\":\"Y-wing\",\"model\":\"BTL Y-wing\",\"manufacturer\":\"Koensayr Manufacturing\",\"cost_in_credits\":\"134999\",\"length\":\"14\",\"max_atmosphering_speed\":\"1000km\",\"crew\":\"2\",\"passengers\":\"0\",\"cargo_capacity\":\"110\",\"consumables\":\"1 week\",\"hyperdrive_rating\":\"1.0\",\"MGLT\":\"80\",\"starship_class\":\"assault starfighter\",\"pilots\":[],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/3/\",\"https://swapi.co/api/films/1/\"],\"created\":\"2014-12-12T11:00:39.817000Z\",\"edited\":\"2014-12-22T17:35:44.479706Z\",\"url\":\"https://swapi.co/api/starships/11/\"},{\"name\":\"X-wing\",\"model\":\"T-65 X-wing\",\"manufacturer\":\"Incom Corporation\",\"cost_in_credits\":\"149999\",\"length\":\"12.5\",\"max_atmosphering_speed\":\"1050\",\"crew\":\"1\",\"passengers\":\"0\",\"cargo_capacity\":\"110\",\"consumables\":\"1 week\",\"hyperdrive_rating\":\"1.0\",\"MGLT\":\"100\",\"starship_class\":\"Starfighter\",\"pilots\":[\"https://swapi.co/api/people/1/\",\"https://swapi.co/api/people/9/\",\"https://swapi.co/api/people/18/\",\"https://swapi.co/api/people/19/\"],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/3/\",\"https://swapi.co/api/films/1/\"],\"created\":\"2014-12-12T11:19:05.340000Z\",\"edited\":\"2014-12-22T17:35:44.491233Z\",\"url\":\"https://swapi.co/api/starships/12/\"},{\"name\":\"TIE Advanced x1\",\"model\":\"Twin Ion Engine Advanced x1\",\"manufacturer\":\"Sienar Fleet Systems\",\"cost_in_credits\":\"unknown\",\"length\":\"9.2\",\"max_atmosphering_speed\":\"1200\",\"crew\":\"1\",\"passengers\":\"0\",\"cargo_capacity\":\"150\",\"consumables\":\"5 days\",\"hyperdrive_rating\":\"1.0\",\"MGLT\":\"105\",\"starship_class\":\"Starfighter\",\"pilots\":[\"https://swapi.co/api/people/4/\"],\"films\":[\"https://swapi.co/api/films/1/\"],\"created\":\"2014-12-12T11:21:32.991000Z\",\"edited\":\"2014-12-22T17:35:44.549047Z\",\"url\":\"https://swapi.co/api/starships/13/\"},{\"name\":\"Slave 1\",\"model\":\"Firespray-31-class patrol and attack\",\"manufacturer\":\"Kuat Systems Engineering\",\"cost_in_credits\":\"unknown\",\"length\":\"21.5\",\"max_atmosphering_speed\":\"1000\",\"crew\":\"1\",\"passengers\":\"6\",\"cargo_capacity\":\"70000\",\"consumables\":\"1 month\",\"hyperdrive_rating\":\"3.0\",\"MGLT\":\"70\",\"starship_class\":\"Patrol craft\",\"pilots\":[\"https://swapi.co/api/people/22/\"],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/5/\"],\"created\":\"2014-12-15T13:00:56.332000Z\",\"edited\":\"2014-12-22T17:35:44.716273Z\",\"url\":\"https://swapi.co/api/starships/21/\"},{\"name\":\"Imperial shuttle\",\"model\":\"Lambda-class T-4a shuttle\",\"manufacturer\":\"Sienar Fleet Systems\",\"cost_in_credits\":\"240000\",\"length\":\"20\",\"max_atmosphering_speed\":\"850\",\"crew\":\"6\",\"passengers\":\"20\",\"cargo_capacity\":\"80000\",\"consumables\":\"2 months\",\"hyperdrive_rating\":\"1.0\",\"MGLT\":\"50\",\"starship_class\":\"Armed government transport\",\"pilots\":[\"https://swapi.co/api/people/1/\",\"https://swapi.co/api/people/13/\",\"https://swapi.co/api/people/14/\"],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/3/\"],\"created\":\"2014-12-15T13:04:47.235000Z\",\"edited\":\"2014-12-22T17:35:44.795405Z\",\"url\":\"https://swapi.co/api/starships/22/\"},{\"name\":\"EF76 Nebulon-B escort frigate\",\"model\":\"EF76 Nebulon-B escort frigate\",\"manufacturer\":\"Kuat Drive Yards\",\"cost_in_credits\":\"8500000\",\"length\":\"300\",\"max_atmosphering_speed\":\"800\",\"crew\":\"854\",\"passengers\":\"75\",\"cargo_capacity\":\"6000000\",\"consumables\":\"2 years\",\"hyperdrive_rating\":\"2.0\",\"MGLT\":\"40\",\"starship_class\":\"Escort ship\",\"pilots\":[],\"films\":[\"https://swapi.co/api/films/2/\",\"https://swapi.co/api/films/3/\"],\"created\":\"2014-12-15T13:06:30.813000Z\",\"edited\":\"2014-12-22T17:35:44.848329Z\",\"url\":\"https://swapi.co/api/starships/23/\"}]}'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view the raw response text\n",
    "r.text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'count': 37,\n",
       " 'next': 'https://swapi.co/api/starships/?page=2',\n",
       " 'previous': None,\n",
       " 'results': [{'MGLT': '40',\n",
       "   'cargo_capacity': '250000000',\n",
       "   'consumables': '6 years',\n",
       "   'cost_in_credits': '1143350000',\n",
       "   'created': '2014-12-15T12:31:42.547000Z',\n",
       "   'crew': '279144',\n",
       "   'edited': '2017-04-19T10:56:06.685592Z',\n",
       "   'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/3/'],\n",
       "   'hyperdrive_rating': '2.0',\n",
       "   'length': '19000',\n",
       "   'manufacturer': 'Kuat Drive Yards, Fondor Shipyards',\n",
       "   'max_atmosphering_speed': 'n/a',\n",
       "   'model': 'Executor-class star dreadnought',\n",
       "   'name': 'Executor',\n",
       "   'passengers': '38000',\n",
       "   'pilots': [],\n",
       "   'starship_class': 'Star dreadnought',\n",
       "   'url': 'https://swapi.co/api/starships/15/'},\n",
       "  {'MGLT': '70',\n",
       "   'cargo_capacity': '180000',\n",
       "   'consumables': '1 month',\n",
       "   'cost_in_credits': '240000',\n",
       "   'created': '2014-12-10T15:48:00.586000Z',\n",
       "   'crew': '5',\n",
       "   'edited': '2014-12-22T17:35:44.431407Z',\n",
       "   'films': ['https://swapi.co/api/films/1/'],\n",
       "   'hyperdrive_rating': '1.0',\n",
       "   'length': '38',\n",
       "   'manufacturer': 'Sienar Fleet Systems, Cyngus Spaceworks',\n",
       "   'max_atmosphering_speed': '1000',\n",
       "   'model': 'Sentinel-class landing craft',\n",
       "   'name': 'Sentinel-class landing craft',\n",
       "   'passengers': '75',\n",
       "   'pilots': [],\n",
       "   'starship_class': 'landing craft',\n",
       "   'url': 'https://swapi.co/api/starships/5/'},\n",
       "  {'MGLT': '10',\n",
       "   'cargo_capacity': '1000000000000',\n",
       "   'consumables': '3 years',\n",
       "   'cost_in_credits': '1000000000000',\n",
       "   'created': '2014-12-10T16:36:50.509000Z',\n",
       "   'crew': '342953',\n",
       "   'edited': '2014-12-22T17:35:44.452589Z',\n",
       "   'films': ['https://swapi.co/api/films/1/'],\n",
       "   'hyperdrive_rating': '4.0',\n",
       "   'length': '120000',\n",
       "   'manufacturer': 'Imperial Department of Military Research, Sienar Fleet Systems',\n",
       "   'max_atmosphering_speed': 'n/a',\n",
       "   'model': 'DS-1 Orbital Battle Station',\n",
       "   'name': 'Death Star',\n",
       "   'passengers': '843342',\n",
       "   'pilots': [],\n",
       "   'starship_class': 'Deep Space Mobile Battlestation',\n",
       "   'url': 'https://swapi.co/api/starships/9/'},\n",
       "  {'MGLT': '75',\n",
       "   'cargo_capacity': '100000',\n",
       "   'consumables': '2 months',\n",
       "   'cost_in_credits': '100000',\n",
       "   'created': '2014-12-10T16:59:45.094000Z',\n",
       "   'crew': '4',\n",
       "   'edited': '2014-12-22T17:35:44.464156Z',\n",
       "   'films': ['https://swapi.co/api/films/2/',\n",
       "    'https://swapi.co/api/films/7/',\n",
       "    'https://swapi.co/api/films/3/',\n",
       "    'https://swapi.co/api/films/1/'],\n",
       "   'hyperdrive_rating': '0.5',\n",
       "   'length': '34.37',\n",
       "   'manufacturer': 'Corellian Engineering Corporation',\n",
       "   'max_atmosphering_speed': '1050',\n",
       "   'model': 'YT-1300 light freighter',\n",
       "   'name': 'Millennium Falcon',\n",
       "   'passengers': '6',\n",
       "   'pilots': ['https://swapi.co/api/people/13/',\n",
       "    'https://swapi.co/api/people/14/',\n",
       "    'https://swapi.co/api/people/25/',\n",
       "    'https://swapi.co/api/people/31/'],\n",
       "   'starship_class': 'Light freighter',\n",
       "   'url': 'https://swapi.co/api/starships/10/'},\n",
       "  {'MGLT': '80',\n",
       "   'cargo_capacity': '110',\n",
       "   'consumables': '1 week',\n",
       "   'cost_in_credits': '134999',\n",
       "   'created': '2014-12-12T11:00:39.817000Z',\n",
       "   'crew': '2',\n",
       "   'edited': '2014-12-22T17:35:44.479706Z',\n",
       "   'films': ['https://swapi.co/api/films/2/',\n",
       "    'https://swapi.co/api/films/3/',\n",
       "    'https://swapi.co/api/films/1/'],\n",
       "   'hyperdrive_rating': '1.0',\n",
       "   'length': '14',\n",
       "   'manufacturer': 'Koensayr Manufacturing',\n",
       "   'max_atmosphering_speed': '1000km',\n",
       "   'model': 'BTL Y-wing',\n",
       "   'name': 'Y-wing',\n",
       "   'passengers': '0',\n",
       "   'pilots': [],\n",
       "   'starship_class': 'assault starfighter',\n",
       "   'url': 'https://swapi.co/api/starships/11/'},\n",
       "  {'MGLT': '100',\n",
       "   'cargo_capacity': '110',\n",
       "   'consumables': '1 week',\n",
       "   'cost_in_credits': '149999',\n",
       "   'created': '2014-12-12T11:19:05.340000Z',\n",
       "   'crew': '1',\n",
       "   'edited': '2014-12-22T17:35:44.491233Z',\n",
       "   'films': ['https://swapi.co/api/films/2/',\n",
       "    'https://swapi.co/api/films/3/',\n",
       "    'https://swapi.co/api/films/1/'],\n",
       "   'hyperdrive_rating': '1.0',\n",
       "   'length': '12.5',\n",
       "   'manufacturer': 'Incom Corporation',\n",
       "   'max_atmosphering_speed': '1050',\n",
       "   'model': 'T-65 X-wing',\n",
       "   'name': 'X-wing',\n",
       "   'passengers': '0',\n",
       "   'pilots': ['https://swapi.co/api/people/1/',\n",
       "    'https://swapi.co/api/people/9/',\n",
       "    'https://swapi.co/api/people/18/',\n",
       "    'https://swapi.co/api/people/19/'],\n",
       "   'starship_class': 'Starfighter',\n",
       "   'url': 'https://swapi.co/api/starships/12/'},\n",
       "  {'MGLT': '105',\n",
       "   'cargo_capacity': '150',\n",
       "   'consumables': '5 days',\n",
       "   'cost_in_credits': 'unknown',\n",
       "   'created': '2014-12-12T11:21:32.991000Z',\n",
       "   'crew': '1',\n",
       "   'edited': '2014-12-22T17:35:44.549047Z',\n",
       "   'films': ['https://swapi.co/api/films/1/'],\n",
       "   'hyperdrive_rating': '1.0',\n",
       "   'length': '9.2',\n",
       "   'manufacturer': 'Sienar Fleet Systems',\n",
       "   'max_atmosphering_speed': '1200',\n",
       "   'model': 'Twin Ion Engine Advanced x1',\n",
       "   'name': 'TIE Advanced x1',\n",
       "   'passengers': '0',\n",
       "   'pilots': ['https://swapi.co/api/people/4/'],\n",
       "   'starship_class': 'Starfighter',\n",
       "   'url': 'https://swapi.co/api/starships/13/'},\n",
       "  {'MGLT': '70',\n",
       "   'cargo_capacity': '70000',\n",
       "   'consumables': '1 month',\n",
       "   'cost_in_credits': 'unknown',\n",
       "   'created': '2014-12-15T13:00:56.332000Z',\n",
       "   'crew': '1',\n",
       "   'edited': '2014-12-22T17:35:44.716273Z',\n",
       "   'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/5/'],\n",
       "   'hyperdrive_rating': '3.0',\n",
       "   'length': '21.5',\n",
       "   'manufacturer': 'Kuat Systems Engineering',\n",
       "   'max_atmosphering_speed': '1000',\n",
       "   'model': 'Firespray-31-class patrol and attack',\n",
       "   'name': 'Slave 1',\n",
       "   'passengers': '6',\n",
       "   'pilots': ['https://swapi.co/api/people/22/'],\n",
       "   'starship_class': 'Patrol craft',\n",
       "   'url': 'https://swapi.co/api/starships/21/'},\n",
       "  {'MGLT': '50',\n",
       "   'cargo_capacity': '80000',\n",
       "   'consumables': '2 months',\n",
       "   'cost_in_credits': '240000',\n",
       "   'created': '2014-12-15T13:04:47.235000Z',\n",
       "   'crew': '6',\n",
       "   'edited': '2014-12-22T17:35:44.795405Z',\n",
       "   'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/3/'],\n",
       "   'hyperdrive_rating': '1.0',\n",
       "   'length': '20',\n",
       "   'manufacturer': 'Sienar Fleet Systems',\n",
       "   'max_atmosphering_speed': '850',\n",
       "   'model': 'Lambda-class T-4a shuttle',\n",
       "   'name': 'Imperial shuttle',\n",
       "   'passengers': '20',\n",
       "   'pilots': ['https://swapi.co/api/people/1/',\n",
       "    'https://swapi.co/api/people/13/',\n",
       "    'https://swapi.co/api/people/14/'],\n",
       "   'starship_class': 'Armed government transport',\n",
       "   'url': 'https://swapi.co/api/starships/22/'},\n",
       "  {'MGLT': '40',\n",
       "   'cargo_capacity': '6000000',\n",
       "   'consumables': '2 years',\n",
       "   'cost_in_credits': '8500000',\n",
       "   'created': '2014-12-15T13:06:30.813000Z',\n",
       "   'crew': '854',\n",
       "   'edited': '2014-12-22T17:35:44.848329Z',\n",
       "   'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/3/'],\n",
       "   'hyperdrive_rating': '2.0',\n",
       "   'length': '300',\n",
       "   'manufacturer': 'Kuat Drive Yards',\n",
       "   'max_atmosphering_speed': '800',\n",
       "   'model': 'EF76 Nebulon-B escort frigate',\n",
       "   'name': 'EF76 Nebulon-B escort frigate',\n",
       "   'passengers': '75',\n",
       "   'pilots': [],\n",
       "   'starship_class': 'Escort ship',\n",
       "   'url': 'https://swapi.co/api/starships/23/'}]}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# decode the JSON response body into a dictionary\n",
    "r.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'Response' object is not subscriptable",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-27fb85ab037f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m#Based on the output, how would I select just the reuslts?\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mr\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'results'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'Response' object is not subscriptable"
     ]
    }
   ],
   "source": [
    "#Based on the output, how would I select just the reuslts?\n",
    "\n",
    "results = r['results']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Based on the output, how would I select just the reuslts?\n",
    "\n",
    "results = r.json()['results']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can even turn this dictionary into a dataframe\n",
    "import pandas as pd\n",
    "df = pd.DataFrame(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "full_request_url = 'http://swapi.co/api/people'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can use functions or for loops to make multiple requests in a row\n",
    "# for example, how would I write a for loop to select the first five people's ids\n",
    "# check out the documentation!\n",
    "\n",
    "people = []\n",
    "\n",
    "for i in range():\n",
    "    html= ''.format()\n",
    "    x = requests.get(html)\n",
    "    people.append(x.json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people = []\n",
    "\n",
    "for i in range(5):\n",
    "    html= full_request_url.format()\n",
    "    x = requests.get(html)\n",
    "    people.append(x.json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people = []\n",
    "\n",
    "for i in range(5):\n",
    "    html= 'http://swapi.co/api/people/' + str(i) + '/'\n",
    "    x = requests.get(html)\n",
    "    people.append(x.json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people = []\n",
    "\n",
    "for i in range(1,6):\n",
    "    html= 'http://swapi.co/api/people/' + str(i) + '/'\n",
    "    x = requests.get(html)\n",
    "    people.append(x.json())\n",
    "    time.sleep(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "people[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(people)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can also turn this into a dataframe\n",
    "df_people = pd.DataFrame(people)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_people.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Now save the new dataframe as a CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_people.to_csv('starwars_people.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}