Write a python program to perform web scrapping using beutiful soup.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m3XiDUJo3nT_"
   },
   "source": [
    "# LAB 8 : Write a python program to perform web scrapping using beutiful soup\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jfob8AW03nUB"
   },
   "source": [
    "## Installing Libraries if not available.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KR_yaRSk3nUB",
    "outputId": "51c05977-1b74-482c-ebf3-e82f5dbec1f3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash: mamba: command not found\n",
      "Collecting lxml==4.6.4\n",
      "  Downloading lxml-4.6.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.3 MB)\n",
      "\u001b[K     |████████████████████████████████| 6.3 MB 35.3 MB/s \n",
      "\u001b[?25hInstalling collected packages: lxml\n",
      "  Attempting uninstall: lxml\n",
      "    Found existing installation: lxml 4.2.6\n",
      "    Uninstalling lxml-4.2.6:\n",
      "      Successfully uninstalled lxml-4.2.6\n",
      "Successfully installed lxml-4.6.4\n",
      "/bin/bash: mamba: command not found\n"
     ]
    }
   ],
   "source": [
    "#!mamba install bs4==4.10.0 -y\n",
    "#!pip install lxml==4.6.4\n",
    "#!mamba install html5lib==1.1 -y\n",
    "#!pip install requests==2.26.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Tv1w5AZt3nUC"
   },
   "source": [
    "## Import the required modules and functions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wsFUL2ko3nUC"
   },
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup # this module helps in web scrapping.\n",
    "import requests  # this module helps us to download a web page"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tdoiATRY3nUD"
   },
   "source": [
    "<h2 id=\"BSO\">Beautiful Soup Objects</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GEG-IN5k3nUD"
   },
   "source": [
    "Beautiful Soup is a Python library for pulling data out of HTML and XML files, we will focus on HTML files. This is accomplished by representing the HTML as a set of objects with methods used to parse the HTML.  We can navigate the HTML as a tree and/or filter out what we are looking for.\n",
    "\n",
    "Consider the following HTML:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 203
    },
    "id": "6SBmhSnl3nUD",
    "outputId": "2d7fdf6a-8422-4278-8c25-14568c0b549c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html>\n",
       "<html>\n",
       "<head>\n",
       "<title>Page Title</title>\n",
       "</head>\n",
       "<body>\n",
       "<h3><b id='boldest'>Lebron James</b></h3>\n",
       "<p> Salary: $ 92,000,000 </p>\n",
       "<h3> Stephen Curry</h3>\n",
       "<p> Salary: $85,000, 000 </p>\n",
       "<h3> Kevin Durant </h3>\n",
       "<p> Salary: $73,200, 000</p>\n",
       "</body>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<!DOCTYPE html>\n",
    "<html>\n",
    "<head>\n",
    "<title>Page Title</title>\n",
    "</head>\n",
    "<body>\n",
    "<h3><b id='boldest'>Lebron James</b></h3>\n",
    "<p> Salary: $ 92,000,000 </p>\n",
    "<h3> Stephen Curry</h3>\n",
    "<p> Salary: $85,000, 000 </p>\n",
    "<h3> Kevin Durant </h3>\n",
    "<p> Salary: $73,200, 000</p>\n",
    "</body>\n",
    "</html>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gi8vdAxO3nUF"
   },
   "source": [
    "We can store it as a string in the variable HTML:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zYnbKO2h3nUF"
   },
   "outputs": [],
   "source": [
    "html=\"<!DOCTYPE html><html><head><title>Page Title</title></head><body><h3><b id='boldest'>Lebron James</b></h3><p> Salary: $ 92,000,000 </p><h3> Stephen Curry</h3><p> Salary: $85,000, 000 </p><h3> Kevin Durant </h3><p> Salary: $73,200, 000</p></body></html>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GlIgil3G3nUF"
   },
   "source": [
    "To parse a document, pass it into the <code>BeautifulSoup</code> constructor, the <code>BeautifulSoup</code> object, which represents the document as a nested data structure:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UiLL8U8I3nUF"
   },
   "outputs": [],
   "source": [
    "soup = BeautifulSoup(html, \"html.parser\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "auNqlDRh3nUG"
   },
   "source": [
    "First, the document is converted to Unicode, (similar to ASCII),  and HTML entities are converted to Unicode characters. Beautiful Soup transforms a complex HTML document into a complex tree of Python objects. The <code>BeautifulSoup</code> object can create other types of objects. In this lab, we will cover <code>BeautifulSoup</code> and <code>Tag</code> objects that for the purposes of this lab are identical, and <code>NavigableString</code> objects.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BOZo_0fZ3nUG"
   },
   "source": [
    "We can use the method <code>prettify()</code> to display the HTML in the nested structure:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-ZOCIv4j3nUG",
    "outputId": "953715b4-eb29-4942-e387-9d0972899db7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<!DOCTYPE html>\n",
      "<html>\n",
      " <head>\n",
      "  <title>\n",
      "   Page Title\n",
      "  </title>\n",
      " </head>\n",
      " <body>\n",
      "  <h3>\n",
      "   <b id=\"boldest\">\n",
      "    Lebron James\n",
      "   </b>\n",
      "  </h3>\n",
      "  <p>\n",
      "   Salary: $ 92,000,000\n",
      "  </p>\n",
      "  <h3>\n",
      "   Stephen Curry\n",
      "  </h3>\n",
      "  <p>\n",
      "   Salary: $85,000, 000\n",
      "  </p>\n",
      "  <h3>\n",
      "   Kevin Durant\n",
      "  </h3>\n",
      "  <p>\n",
      "   Salary: $73,200, 000\n",
      "  </p>\n",
      " </body>\n",
      "</html>\n"
     ]
    }
   ],
   "source": [
    "print(soup.prettify())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dpnRXN9k3nUG"
   },
   "source": [
    "## Tags\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oFEiYrTk3nUH"
   },
   "source": [
    "Let's say we want the  title of the page and the name of the top paid player we can use the <code>Tag</code>. The <code>Tag</code> object corresponds to an HTML tag in the original document, for example, the tag title.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TTsRrMPg3nUH",
    "outputId": "f290f0fc-da02-43ad-f02c-1bb2b386621a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tag object: <title>Page Title</title>\n"
     ]
    }
   ],
   "source": [
    "tag_object=soup.title\n",
    "print(\"tag object:\",tag_object)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PXud3Ebc3nUH"
   },
   "source": [
    "we can see the tag type <code>bs4.element.Tag</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4Z0X264E3nUH",
    "outputId": "0161aabe-d62c-49a1-bd0a-97925d81d7aa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tag object type: <class 'bs4.element.Tag'>\n"
     ]
    }
   ],
   "source": [
    "print(\"tag object type:\",type(tag_object))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kqMumysm3nUI"
   },
   "source": [
    "If there is more than one <code>Tag</code>  with the same name, the first element with that <code>Tag</code> name is called, this corresponds to the most paid player:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "FTe2QrQ43nUI",
    "outputId": "5dba4486-76bd-446e-ac11-3a9d60ff96a9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<h3><b id=\"boldest\">Lebron James</b></h3>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_object=soup.h3\n",
    "tag_object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KqLW28wY3nUI"
   },
   "source": [
    "Enclosed in the bold attribute <code>b</code>, it helps to use the tree representation. We can navigate down the tree using the child attribute to get the name.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8L4mPD8c3nUI"
   },
   "source": [
    "### Children, Parents, and Siblings\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MhG3H9hr3nUJ"
   },
   "source": [
    "As stated above the <code>Tag</code> object is a tree of objects we can access the child of the tag or navigate down the branch as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PkL05iLx3nUJ",
    "outputId": "e0924482-ab19-4839-d2b2-435049146d50"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<b id=\"boldest\">Lebron James</b>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_child =tag_object.b\n",
    "tag_child"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HtC0FviK3nUJ"
   },
   "source": [
    "You can access the parent with the <code> parent</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bLhYlNNo3nUJ",
    "outputId": "0ab3958d-d8ea-4d20-be3a-aba0623bd358"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<h3><b id=\"boldest\">Lebron James</b></h3>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "parent_tag=tag_child.parent\n",
    "parent_tag"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2udFJ0Hc3nUK"
   },
   "source": [
    "this is identical to\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8ZHVACnD3nUK",
    "outputId": "3fae8507-2048-4c8b-d783-f05b124a8c67"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<h3><b id=\"boldest\">Lebron James</b></h3>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z9dqH_if3nUL"
   },
   "source": [
    "<code>tag_object</code> parent is the <code>body</code> element.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NPRERDLj3nUL",
    "outputId": "de6437ff-7df8-4156-edc2-7e07cffd7ff2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<body><h3><b id=\"boldest\">Lebron James</b></h3><p> Salary: $ 92,000,000 </p><h3> Stephen Curry</h3><p> Salary: $85,000, 000 </p><h3> Kevin Durant </h3><p> Salary: $73,200, 000</p></body>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_object.parent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bt6Sa1ui3nUM"
   },
   "source": [
    "<code>tag_object</code> sibling is the <code>paragraph</code> element\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9JXUS1gu3nUM",
    "outputId": "242554f6-84a5-479b-fe2a-9647746107fc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<p> Salary: $ 92,000,000 </p>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sibling_1=tag_object.next_sibling\n",
    "sibling_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uxWqARc13nUN"
   },
   "source": [
    "`sibling_2` is the `header` element which is also a sibling of both `sibling_1` and `tag_object`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "54CFH3nG3nUN",
    "outputId": "206465b8-cfba-4282-cad6-c9a591ab78fe"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<h3> Stephen Curry</h3>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sibling_2=sibling_1.next_sibling\n",
    "sibling_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qHmFnZ_H3nUN"
   },
   "source": [
    "<h3 id=\"first_question\">Exercise: <code>next_sibling</code></h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CEZRNO3x3nUO"
   },
   "source": [
    "Using the object <code>sibling\\_2</code> and the property <code>next_sibling</code> to find the salary of Stephen Curry:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zYKf_5Jz3nUO"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8wV6SlAy3nUO"
   },
   "source": [
    "<details><summary>Click here for the solution</summary>\n",
    "\n",
    "```\n",
    "sibling_2.next_sibling\n",
    "\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "x8pOjkvl3nUP"
   },
   "source": [
    "### HTML Attributes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XGzPD7jB3nUP"
   },
   "source": [
    "If the tag has attributes, the tag <code>id=\"boldest\"</code> has an attribute <code>id</code> whose value is <code>boldest</code>. You can access a tag’s attributes by treating the tag like a dictionary:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "IRkYRKS73nUP",
    "outputId": "4dfb9a14-dbcb-426f-b120-a57fb2efefc0"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'boldest'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_child['id']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kwNj5XxN3nUQ"
   },
   "source": [
    "You can access that dictionary directly as <code>attrs</code>:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JikU6wJV3nUQ",
    "outputId": "8a3aeb8d-6c1c-4886-cc18-0b3b4b3695a1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'boldest'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_child.attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QEW4HL1C3nUQ"
   },
   "source": [
    "You can also work with Multi-valued attribute check out <a href=\"https://www.crummy.com/software/BeautifulSoup/bs4/doc/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0220ENSkillsNetwork23455606-2021-01-01\">\\[1]</a> for more.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ir2Pl9Ga3nUQ"
   },
   "source": [
    "We can also obtain the content if the attribute of the <code>tag</code> using the Python <code>get()</code> method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "ZMYuZHjf3nUQ",
    "outputId": "35db7b9c-4ac3-4cf2-a1b6-3a0c03b67699"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'boldest'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_child.get('id')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OAoMZaul3nUR"
   },
   "source": [
    "### Navigable String\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tRYiltJZ3nUR"
   },
   "source": [
    "A string corresponds to a bit of text or content within a tag. Beautiful Soup uses the <code>NavigableString</code> class to contain this text. In our HTML we can obtain the name of the first player by extracting the sting of the <code>Tag</code> object <code>tag_child</code> as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "VoiAwlXG3nUR",
    "outputId": "b8366925-9520-4602-a8bd-1496865e6c69"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'Lebron James'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_string=tag_child.string\n",
    "tag_string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CN5DQh223nUR"
   },
   "source": [
    "we can verify the type is Navigable String\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wcXbAIhC3nUS",
    "outputId": "02db2e3d-6e6b-49d7-f3d6-9830abf2bd47"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bs4.element.NavigableString"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(tag_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AkilQC913nUS"
   },
   "source": [
    "A NavigableString is just like a Python string or Unicode string, to be more precise. The main difference is that it also supports some  <code>BeautifulSoup</code> features. We can covert it to sting object in Python:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "VBdjGxBO3nUS",
    "outputId": "05849a9c-c611-44d5-8cd6-82bf8d013fb3"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'Lebron James'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unicode_string = str(tag_string)\n",
    "unicode_string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IGZEpdiP3nUS"
   },
   "source": [
    "<h2 id=\"filter\">Filter</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "X3Bod_eo3nUS"
   },
   "source": [
    "Filters allow you to find complex patterns, the simplest filter is a string. In this section we will pass a string to a different filter method and Beautiful Soup will perform a match against that exact string.  Consider the following HTML of rocket launchs:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 104
    },
    "id": "Gkcl-OOW3nUS",
    "outputId": "c7fcbc52-3db9-46ea-ad38-96262374cfad"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "  <tr>\n",
       "    <td id='flight' >Flight No</td>\n",
       "    <td>Launch site</td> \n",
       "    <td>Payload mass</td>\n",
       "   </tr>\n",
       "  <tr> \n",
       "    <td>1</td>\n",
       "    <td><a href='https://en.wikipedia.org/wiki/Florida'>Florida</a></td>\n",
       "    <td>300 kg</td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>2</td>\n",
       "    <td><a href='https://en.wikipedia.org/wiki/Texas'>Texas</a></td>\n",
       "    <td>94 kg</td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>3</td>\n",
       "    <td><a href='https://en.wikipedia.org/wiki/Florida'>Florida<a> </td>\n",
       "    <td>80 kg</td>\n",
       "  </tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<table>\n",
    "  <tr>\n",
    "    <td id='flight' >Flight No</td>\n",
    "    <td>Launch site</td>\n",
    "    <td>Payload mass</td>\n",
    "   </tr>\n",
    "  <tr>\n",
    "    <td>1</td>\n",
    "    <td><a href='https://en.wikipedia.org/wiki/Florida'>Florida</a></td>\n",
    "    <td>300 kg</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>2</td>\n",
    "    <td><a href='https://en.wikipedia.org/wiki/Texas'>Texas</a></td>\n",
    "    <td>94 kg</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>3</td>\n",
    "    <td><a href='https://en.wikipedia.org/wiki/Florida'>Florida<a> </td>\n",
    "    <td>80 kg</td>\n",
    "  </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-JWSDf773nUT"
   },
   "source": [
    "We can store it as a string in the variable <code>table</code>:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PU_EjM763nUT"
   },
   "outputs": [],
   "source": [
    "table=\"<table><tr><td id='flight'>Flight No</td><td>Launch site</td> <td>Payload mass</td></tr><tr> <td>1</td><td><a href='https://en.wikipedia.org/wiki/Florida'>Florida<a></td><td>300 kg</td></tr><tr><td>2</td><td><a href='https://en.wikipedia.org/wiki/Texas'>Texas</a></td><td>94 kg</td></tr><tr><td>3</td><td><a href='https://en.wikipedia.org/wiki/Florida'>Florida<a> </td><td>80 kg</td></tr></table>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VPcsyPEP3nUT"
   },
   "outputs": [],
   "source": [
    "table_bs = BeautifulSoup(table, \"html.parser\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3fQFdUO93nUT"
   },
   "source": [
    "## find All\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y8x3lX1g3nUU"
   },
   "source": [
    "The <code>find_all()</code> method looks through a tag’s descendants and retrieves all descendants that match your filters.\n",
    "\n",
    "<p>\n",
    "The Method signature for <code>find_all(name, attrs, recursive, string, limit, **kwargs)<c/ode>\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IaQX8z3F3nUU"
   },
   "source": [
    "### Name\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uY6P-Cdf3nUV"
   },
   "source": [
    "When we set the <code>name</code> parameter to a tag name, the method will extract all the tags with that name and its children.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bhhRgK913nUV",
    "outputId": "38e32b80-2e22-4093-fbfe-10b78837378f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tr><td id=\"flight\">Flight No</td><td>Launch site</td> <td>Payload mass</td></tr>,\n",
       " <tr> <td>1</td><td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a></td><td>300 kg</td></tr>,\n",
       " <tr><td>2</td><td><a href=\"https://en.wikipedia.org/wiki/Texas\">Texas</a></td><td>94 kg</td></tr>,\n",
       " <tr><td>3</td><td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a></td><td>80 kg</td></tr>]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table_rows=table_bs.find_all('tr')\n",
    "table_rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_RIcx09y3nUV"
   },
   "source": [
    "The result is a Python Iterable just like a list, each element is a <code>tag</code> object:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u3JKdm2E3nUW",
    "outputId": "5cf755d6-3733-4b09-cd93-0b8354f97aab"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tr><td id=\"flight\">Flight No</td><td>Launch site</td> <td>Payload mass</td></tr>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first_row =table_rows[0]\n",
    "first_row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jH0OUIOD3nUW"
   },
   "source": [
    "The type is <code>tag</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "G94TsPmH3nUW",
    "outputId": "434230f6-a121-46fa-f158-f01559bcb818"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'bs4.element.Tag'>\n"
     ]
    }
   ],
   "source": [
    "print(type(first_row))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K7kyU0Ez3nUX"
   },
   "source": [
    "we can obtain the child\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7hnTG3Um3nUX",
    "outputId": "3dad034c-a472-4147-a315-8a7e94dee8a4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<td id=\"flight\">Flight No</td>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first_row.td"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OAbSVuMR3nUX"
   },
   "source": [
    "If we iterate through the list, each element corresponds to a row in the table:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dDB9oTM73nUY",
    "outputId": "752f83c8-1e4d-4aa8-cdba-3fd673cf5eb2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "row 0 is <tr><td id=\"flight\">Flight No</td><td>Launch site</td> <td>Payload mass</td></tr>\n",
      "row 1 is <tr> <td>1</td><td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a></td><td>300 kg</td></tr>\n",
      "row 2 is <tr><td>2</td><td><a href=\"https://en.wikipedia.org/wiki/Texas\">Texas</a></td><td>94 kg</td></tr>\n",
      "row 3 is <tr><td>3</td><td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a></td><td>80 kg</td></tr>\n"
     ]
    }
   ],
   "source": [
    "for i,row in enumerate(table_rows):\n",
    "    print(\"row\",i,\"is\",row)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q06l7GZI3nUY"
   },
   "source": [
    "As <code>row</code> is a <code>cell</code> object, we can apply the method <code>find_all</code> to it and extract table cells in the object <code>cells</code> using the tag <code>td</code>, this is all the children with the name <code>td</code>. The result is a list, each element corresponds to a cell and is a <code>Tag</code> object, we can iterate through this list as well. We can extract the content using the <code>string</code>  attribute.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gzUjY_Rg3nUY",
    "outputId": "540dc3c0-cc8d-4676-d7d5-597685bb12c6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "row 0\n",
      "colunm 0 cell <td id=\"flight\">Flight No</td>\n",
      "colunm 1 cell <td>Launch site</td>\n",
      "colunm 2 cell <td>Payload mass</td>\n",
      "row 1\n",
      "colunm 0 cell <td>1</td>\n",
      "colunm 1 cell <td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a></td>\n",
      "colunm 2 cell <td>300 kg</td>\n",
      "row 2\n",
      "colunm 0 cell <td>2</td>\n",
      "colunm 1 cell <td><a href=\"https://en.wikipedia.org/wiki/Texas\">Texas</a></td>\n",
      "colunm 2 cell <td>94 kg</td>\n",
      "row 3\n",
      "colunm 0 cell <td>3</td>\n",
      "colunm 1 cell <td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a></td>\n",
      "colunm 2 cell <td>80 kg</td>\n"
     ]
    }
   ],
   "source": [
    "for i,row in enumerate(table_rows):\n",
    "    print(\"row\",i)\n",
    "    cells=row.find_all('td')\n",
    "    for j,cell in enumerate(cells):\n",
    "        print('colunm',j,\"cell\",cell)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E4IHGaD13nUZ"
   },
   "source": [
    "If we use a list we can match against any item in that list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "aiG38_fc3nUZ",
    "outputId": "cb3055d8-2fff-4eca-8d69-6b40f240bfe3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tr><td id=\"flight\">Flight No</td><td>Launch site</td> <td>Payload mass</td></tr>,\n",
       " <td id=\"flight\">Flight No</td>,\n",
       " <td>Launch site</td>,\n",
       " <td>Payload mass</td>,\n",
       " <tr> <td>1</td><td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a></td><td>300 kg</td></tr>,\n",
       " <td>1</td>,\n",
       " <td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a></td>,\n",
       " <td>300 kg</td>,\n",
       " <tr><td>2</td><td><a href=\"https://en.wikipedia.org/wiki/Texas\">Texas</a></td><td>94 kg</td></tr>,\n",
       " <td>2</td>,\n",
       " <td><a href=\"https://en.wikipedia.org/wiki/Texas\">Texas</a></td>,\n",
       " <td>94 kg</td>,\n",
       " <tr><td>3</td><td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a></td><td>80 kg</td></tr>,\n",
       " <td>3</td>,\n",
       " <td><a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a></td>,\n",
       " <td>80 kg</td>]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list_input=table_bs .find_all(name=[\"tr\", \"td\"])\n",
    "list_input"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Pm35Eq4s3nUZ"
   },
   "source": [
    "## Attributes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0oxmjuPK3nUZ"
   },
   "source": [
    "If the argument is not recognized it will be turned into a filter on the tag’s attributes. For example the <code>id</code>  argument, Beautiful Soup will filter against each tag’s <code>id</code> attribute. For example, the first <code>td</code> elements have a value of <code>id</code> of <code>flight</code>, therefore we can filter based on that <code>id</code> value.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Zj4-XmiP3nUZ",
    "outputId": "ba790263-4b46-404d-9e17-123ee8a8a64c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<td id=\"flight\">Flight No</td>]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table_bs.find_all(id=\"flight\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-VtzusC03nUa"
   },
   "source": [
    "We can find all the elements that have links to the Florida Wikipedia page:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_BWDvR4H3nUa",
    "outputId": "3174bea6-18fb-4116-f7ea-d2e930350ccf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a>,\n",
       " <a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a>]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list_input=table_bs.find_all(href=\"https://en.wikipedia.org/wiki/Florida\")\n",
    "list_input"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ESM4G5Z33nUa"
   },
   "source": [
    "If we set the  <code>href</code> attribute to True, regardless of what the value is, the code finds all tags with <code>href</code> value:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "s1eOUKDK3nUa",
    "outputId": "2f1b5610-678c-4635-b2e2-44bd051c078d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a></a></a>,\n",
       " <a href=\"https://en.wikipedia.org/wiki/Texas\">Texas</a>,\n",
       " <a href=\"https://en.wikipedia.org/wiki/Florida\">Florida<a> </a></a>]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table_bs.find_all(href=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uIAW4mNb3nUb"
   },
   "source": [
    "There are other methods for dealing with attributes and other related methods; Check out the following <a href='https://www.crummy.com/software/BeautifulSoup/bs4/doc/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0220ENSkillsNetwork23455606-2021-01-01#css-selectors'>link</a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "whaNGd613nUb"
   },
   "source": [
    "<h3 id=\"exer_type\">Exercise: <code>find_all</code></h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kLoBSa4R3nUc"
   },
   "source": [
    "Using the logic above, find all the elements without <code>href</code> value\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PO7_4neb3nUc"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6tH5dnbX3nUc"
   },
   "source": [
    "<details><summary>Click here for the solution</summary>\n",
    "\n",
    "```\n",
    "table_bs.find_all(href=False)\n",
    "\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oQZwP3SV3nUc"
   },
   "source": [
    "Using the soup object <code>soup</code>, find the element with the <code>id</code> attribute content set to <code>\"boldest\"</code>.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "v11ZofH_3nUc"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QbyJIPmv3nUd"
   },
   "source": [
    "<details><summary>Click here for the solution</summary>\n",
    "\n",
    "```\n",
    "soup.find_all(id=\"boldest\")\n",
    "\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bGnBnnxd3nUd"
   },
   "source": [
    "### string\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bolBAgZT3nUd"
   },
   "source": [
    "With string you can search for strings instead of tags, where we find all the elments with Florida:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "T1TS-4Fd3nUd",
    "outputId": "36d4098d-13a8-4f54-90f5-1a578e8020a4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Florida', 'Florida']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table_bs.find_all(string=\"Florida\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RNz1o2Rs3nUd"
   },
   "source": [
    "## find\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V6hPwwlV3nUe"
   },
   "source": [
    "The <code>find_all()</code> method scans the entire document looking for results, it’s if you are looking for one element you can use the <code>find()</code> method to find the first element in the document. Consider the following two table:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 275
    },
    "id": "eUpfwfFj3nUe",
    "outputId": "7522e13f-b3ab-49a9-d9bb-b67daad5afb3"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h3>Rocket Launch </h3>\n",
       "\n",
       "<p>\n",
       "<table class='rocket'>\n",
       "  <tr>\n",
       "    <td>Flight No</td>\n",
       "    <td>Launch site</td> \n",
       "    <td>Payload mass</td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>1</td>\n",
       "    <td>Florida</td>\n",
       "    <td>300 kg</td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>2</td>\n",
       "    <td>Texas</td>\n",
       "    <td>94 kg</td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>3</td>\n",
       "    <td>Florida </td>\n",
       "    <td>80 kg</td>\n",
       "  </tr>\n",
       "</table>\n",
       "</p>\n",
       "<p>\n",
       "\n",
       "<h3>Pizza Party  </h3>\n",
       "  \n",
       "    \n",
       "<table class='pizza'>\n",
       "  <tr>\n",
       "    <td>Pizza Place</td>\n",
       "    <td>Orders</td> \n",
       "    <td>Slices </td>\n",
       "   </tr>\n",
       "  <tr>\n",
       "    <td>Domino's Pizza</td>\n",
       "    <td>10</td>\n",
       "    <td>100</td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>Little Caesars</td>\n",
       "    <td>12</td>\n",
       "    <td >144 </td>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <td>Papa John's </td>\n",
       "    <td>15 </td>\n",
       "    <td>165</td>\n",
       "  </tr>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<h3>Rocket Launch </h3>\n",
    "\n",
    "<p>\n",
    "<table class='rocket'>\n",
    "  <tr>\n",
    "    <td>Flight No</td>\n",
    "    <td>Launch site</td>\n",
    "    <td>Payload mass</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>1</td>\n",
    "    <td>Florida</td>\n",
    "    <td>300 kg</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>2</td>\n",
    "    <td>Texas</td>\n",
    "    <td>94 kg</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>3</td>\n",
    "    <td>Florida </td>\n",
    "    <td>80 kg</td>\n",
    "  </tr>\n",
    "</table>\n",
    "</p>\n",
    "<p>\n",
    "\n",
    "<h3>Pizza Party  </h3>\n",
    "\n",
    "\n",
    "<table class='pizza'>\n",
    "  <tr>\n",
    "    <td>Pizza Place</td>\n",
    "    <td>Orders</td>\n",
    "    <td>Slices </td>\n",
    "   </tr>\n",
    "  <tr>\n",
    "    <td>Domino's Pizza</td>\n",
    "    <td>10</td>\n",
    "    <td>100</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Little Caesars</td>\n",
    "    <td>12</td>\n",
    "    <td >144 </td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Papa John's </td>\n",
    "    <td>15 </td>\n",
    "    <td>165</td>\n",
    "  </tr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HxTERIVO3nUe"
   },
   "source": [
    "We store the HTML as a Python string and assign <code>two_tables</code>:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-95tqnKx3nUe"
   },
   "outputs": [],
   "source": [
    "two_tables=\"<h3>Rocket Launch </h3><p><table class='rocket'><tr><td>Flight No</td><td>Launch site</td> <td>Payload mass</td></tr><tr><td>1</td><td>Florida</td><td>300 kg</td></tr><tr><td>2</td><td>Texas</td><td>94 kg</td></tr><tr><td>3</td><td>Florida </td><td>80 kg</td></tr></table></p><p><h3>Pizza Party  </h3><table class='pizza'><tr><td>Pizza Place</td><td>Orders</td> <td>Slices </td></tr><tr><td>Domino's Pizza</td><td>10</td><td>100</td></tr><tr><td>Little Caesars</td><td>12</td><td >144 </td></tr><tr><td>Papa John's </td><td>15 </td><td>165</td></tr>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bqj7eq5x3nUf"
   },
   "source": [
    "We create a <code>BeautifulSoup</code> object  <code>two_tables_bs</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "igMjBZuU3nUf"
   },
   "outputs": [],
   "source": [
    "two_tables_bs= BeautifulSoup(two_tables, 'html.parser')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H9XXvL4z3nUf"
   },
   "source": [
    "We can find the first table using the tag name table\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RgLzZoXS3nUf",
    "outputId": "4789ec86-9411-4705-b938-8f1972ce6268"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<table class=\"rocket\"><tr><td>Flight No</td><td>Launch site</td> <td>Payload mass</td></tr><tr><td>1</td><td>Florida</td><td>300 kg</td></tr><tr><td>2</td><td>Texas</td><td>94 kg</td></tr><tr><td>3</td><td>Florida </td><td>80 kg</td></tr></table>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "two_tables_bs.find(\"table\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5glIExCp3nUg"
   },
   "source": [
    "We can filter on the class attribute to find the second table, but because class is a keyword in Python, we add an underscore.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JYYC1vP93nUg",
    "outputId": "24b03a76-ebbc-4cd5-c141-f4474e4a5d92"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<table class=\"pizza\"><tr><td>Pizza Place</td><td>Orders</td> <td>Slices </td></tr><tr><td>Domino's Pizza</td><td>10</td><td>100</td></tr><tr><td>Little Caesars</td><td>12</td><td>144 </td></tr><tr><td>Papa John's </td><td>15 </td><td>165</td></tr></table>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "two_tables_bs.find(\"table\",class_='pizza')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2q-C-R7w3nUg"
   },
   "source": [
    "<h2 id=\"DSCW\">Downloading And Scraping The Contents Of A Web Page</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0bbKriqr3nUh"
   },
   "source": [
    "We Download the contents of the web page:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GDqq37nS3nUh"
   },
   "outputs": [],
   "source": [
    "url = \"http://www.ibm.com\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4UEbtPIo3nUh"
   },
   "source": [
    "We use <code>get</code> to download the contents of the webpage in text format and store in a variable called <code>data</code>:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FiXYuFPQ3nUh"
   },
   "outputs": [],
   "source": [
    "data  = requests.get(url).text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wvbnqee03nUi"
   },
   "source": [
    "We create a <code>BeautifulSoup</code> object using the <code>BeautifulSoup</code> constructor\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CtyBSH9b3nUi"
   },
   "outputs": [],
   "source": [
    "soup = BeautifulSoup(data,\"html.parser\")  # create a soup object using the variable 'data'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TAOgm0mo3nUi"
   },
   "source": [
    "Scrape all links\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "GVowo4R53nUi",
    "outputId": "52ffcd60-37d4-4448-a442-3bf9d2e6810a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.ibm.com/cloud?lnk=hpUSbt1\n"
     ]
    }
   ],
   "source": [
    "for link in soup.find_all('a',href=True):  # in html anchor/link is represented by the tag <a>\n",
    "\n",
    "    print(link.get('href'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KS-foerK3nUj"
   },
   "source": [
    "## Scrape  all images  Tags\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ji7_kLFy3nUj",
    "outputId": "9964b541-8fcf-4672-da06-484e5ec5cad7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<img alt=\"Portraits of IBM consultants\" class=\"bx--image__img\" id=\"image--1290136985\" loading=\"lazy\" src=\"/content/dam/adobe-cms/default-images/home-consultants.component.crop-16by9-xl.ts=1700568733825.jpg/content/adobe-cms/us/en/homepage/_jcr_content/root/table_of_contents/simple_image\"/>\n",
      "/content/dam/adobe-cms/default-images/home-consultants.component.crop-16by9-xl.ts=1700568733825.jpg/content/adobe-cms/us/en/homepage/_jcr_content/root/table_of_contents/simple_image\n"
     ]
    }
   ],
   "source": [
    "for link in soup.find_all('img'):# in html image is represented by the tag <img>\n",
    "    print(link)\n",
    "    print(link.get('src'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mTvbVEmI3nUj"
   },
   "source": [
    "## Scrape data from HTML tables\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-SOPF-D93nUk"
   },
   "outputs": [],
   "source": [
    "#The below url contains an html table with data about colors and color codes.\n",
    "url = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-DA0321EN-SkillsNetwork/labs/datasets/HTMLColorCodes.html\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vBSs_v1j3nUk"
   },
   "source": [
    "Before proceeding to scrape a web site, you need to examine the contents, and the way data is organized on the website. Open the above url in your browser and check how many rows and columns are there in the color table.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RCyPB6NG3nUk"
   },
   "outputs": [],
   "source": [
    "# get the contents of the webpage in text format and store in a variable called data\n",
    "data  = requests.get(url).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UH4yneh53nUl"
   },
   "outputs": [],
   "source": [
    "soup = BeautifulSoup(data,\"html.parser\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9EBhWlpJ3nUl"
   },
   "outputs": [],
   "source": [
    "#find a html table in the web page\n",
    "table = soup.find('table') # in html table is represented by the tag <table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "82ZUUaPm3nUl",
    "outputId": "ec7a8c04-7d17-4a57-ec0f-f82a2c42204f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Color Name--->None\n",
      "lightsalmon--->#FFA07A\n",
      "salmon--->#FA8072\n",
      "darksalmon--->#E9967A\n",
      "lightcoral--->#F08080\n",
      "coral--->#FF7F50\n",
      "tomato--->#FF6347\n",
      "orangered--->#FF4500\n",
      "gold--->#FFD700\n",
      "orange--->#FFA500\n",
      "darkorange--->#FF8C00\n",
      "lightyellow--->#FFFFE0\n",
      "lemonchiffon--->#FFFACD\n",
      "papayawhip--->#FFEFD5\n",
      "moccasin--->#FFE4B5\n",
      "peachpuff--->#FFDAB9\n",
      "palegoldenrod--->#EEE8AA\n",
      "khaki--->#F0E68C\n",
      "darkkhaki--->#BDB76B\n",
      "yellow--->#FFFF00\n",
      "lawngreen--->#7CFC00\n",
      "chartreuse--->#7FFF00\n",
      "limegreen--->#32CD32\n",
      "lime--->#00FF00\n",
      "forestgreen--->#228B22\n",
      "green--->#008000\n",
      "powderblue--->#B0E0E6\n",
      "lightblue--->#ADD8E6\n",
      "lightskyblue--->#87CEFA\n",
      "skyblue--->#87CEEB\n",
      "deepskyblue--->#00BFFF\n",
      "lightsteelblue--->#B0C4DE\n",
      "dodgerblue--->#1E90FF\n"
     ]
    }
   ],
   "source": [
    "#Get all rows from the table\n",
    "for row in table.find_all('tr'): # in html table row is represented by the tag <tr>\n",
    "    # Get all columns in each row.\n",
    "    cols = row.find_all('td') # in html a column is represented by the tag <td>\n",
    "    color_name = cols[2].string # store the value in column 3 as color_name\n",
    "    color_code = cols[3].string # store the value in column 4 as color_code\n",
    "    print(\"{}--->{}\".format(color_name,color_code))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hWwE2vAY3nUm"
   },
   "source": [
    "## Scrape data from HTML tables into a DataFrame using BeautifulSoup and Pandas\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3OBo9CB93nUm"
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4lh2fywf3nUn"
   },
   "outputs": [],
   "source": [
    "#The below url contains html tables with data about world population.\n",
    "url = \"https://en.wikipedia.org/wiki/World_population\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gIc_CSbJ3nUn"
   },
   "source": [
    "Before proceeding to scrape a web site, you need to examine the contents, and the way data is organized on the website. Open the above url in your browser and check the tables on the webpage.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "QI8Rmy6b3nUn"
   },
   "outputs": [],
   "source": [
    "# get the contents of the webpage in text format and store in a variable called data\n",
    "data  = requests.get(url).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "A9CuQ4C_3nUo"
   },
   "outputs": [],
   "source": [
    "soup = BeautifulSoup(data,\"html.parser\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_WeA7ZWH3nUo"
   },
   "outputs": [],
   "source": [
    "#find all html tables in the web page\n",
    "tables = soup.find_all('table') # in html table is represented by the tag <table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Pc2Fo5zV3nUo",
    "outputId": "78932c7d-32fc-417c-fd82-20d8694a315e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can see how many tables were found by checking the length of the tables list\n",
    "len(tables)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l3YYyqUw3nUp"
   },
   "source": [
    "Assume that we are looking for the `10 most densly populated countries` table, we can look through the tables list and find the right one we are look for based on the data in each table or we can search for the table name if it is in the table but this option might not always work.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PAYMcLxW3nUp",
    "outputId": "ca80f864-0b9d-4a66-f046-e77cc4f94788"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n"
     ]
    }
   ],
   "source": [
    "for index,table in enumerate(tables):\n",
    "    if (\"10 most densely populated countries\" in str(table)):\n",
    "        table_index = index\n",
    "print(table_index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tZdvm1gg3nUp"
   },
   "source": [
    "See if you can locate the table name of the table, `10 most densly populated countries`, below.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "GWXBQ84o3nUp",
    "outputId": "c317e6b7-d54f-4932-fe91-d2cbb05cd05e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<table class=\"wikitable sortable\" style=\"text-align:right\">\n",
      " <caption>\n",
      "  10 most densely populated countries\n",
      "  <small>\n",
      "   (with population above 5 million)\n",
      "  </small>\n",
      "  <sup class=\"reference\" id=\"cite_ref-:10_106-0\">\n",
      "   <a href=\"#cite_note-:10-106\">\n",
      "    [102]\n",
      "   </a>\n",
      "  </sup>\n",
      " </caption>\n",
      " <tbody>\n",
      "  <tr>\n",
      "   <th scope=\"col\">\n",
      "    Rank\n",
      "   </th>\n",
      "   <th scope=\"col\">\n",
      "    Country\n",
      "   </th>\n",
      "   <th scope=\"col\">\n",
      "    Population\n",
      "   </th>\n",
      "   <th scope=\"col\">\n",
      "    Area\n",
      "    <br/>\n",
      "    <small>\n",
      "     (km\n",
      "     <sup>\n",
      "      2\n",
      "     </sup>\n",
      "     )\n",
      "    </small>\n",
      "   </th>\n",
      "   <th scope=\"col\">\n",
      "    Density\n",
      "    <br/>\n",
      "    <small>\n",
      "     (pop/km\n",
      "     <sup>\n",
      "      2\n",
      "     </sup>\n",
      "     )\n",
      "    </small>\n",
      "   </th>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    1\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/4/48/Flag_of_Singapore.svg/23px-Flag_of_Singapore.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/4/48/Flag_of_Singapore.svg/35px-Flag_of_Singapore.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/4/48/Flag_of_Singapore.svg/45px-Flag_of_Singapore.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Singapore\" title=\"Singapore\">\n",
      "     Singapore\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    5,921,231\n",
      "   </td>\n",
      "   <td>\n",
      "    719\n",
      "   </td>\n",
      "   <td>\n",
      "    8,235\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    2\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"1000\" decoding=\"async\" height=\"14\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/f/f9/Flag_of_Bangladesh.svg/23px-Flag_of_Bangladesh.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/f/f9/Flag_of_Bangladesh.svg/35px-Flag_of_Bangladesh.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/f/f9/Flag_of_Bangladesh.svg/46px-Flag_of_Bangladesh.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Bangladesh\" title=\"Bangladesh\">\n",
      "     Bangladesh\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    165,650,475\n",
      "   </td>\n",
      "   <td>\n",
      "    148,460\n",
      "   </td>\n",
      "   <td>\n",
      "    1,116\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    3\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <p>\n",
      "     <span class=\"flagicon\">\n",
      "      <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "       <span>\n",
      "        <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"1200\" decoding=\"async\" height=\"12\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/0/00/Flag_of_Palestine.svg/23px-Flag_of_Palestine.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/0/00/Flag_of_Palestine.svg/35px-Flag_of_Palestine.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/0/00/Flag_of_Palestine.svg/46px-Flag_of_Palestine.svg.png 2x\" width=\"23\"/>\n",
      "       </span>\n",
      "      </span>\n",
      "     </span>\n",
      "     <a href=\"/wiki/State_of_Palestine\" title=\"State of Palestine\">\n",
      "      Palestine\n",
      "     </a>\n",
      "     <sup class=\"reference\" id=\"cite_ref-107\">\n",
      "      <a href=\"#cite_note-107\">\n",
      "       [note 3]\n",
      "      </a>\n",
      "     </sup>\n",
      "     <sup class=\"reference\" id=\"cite_ref-108\">\n",
      "      <a href=\"#cite_note-108\">\n",
      "       [103]\n",
      "      </a>\n",
      "     </sup>\n",
      "    </p>\n",
      "   </td>\n",
      "   <td>\n",
      "    5,223,000\n",
      "   </td>\n",
      "   <td>\n",
      "    6,025\n",
      "   </td>\n",
      "   <td>\n",
      "    867\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    4\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/7/72/Flag_of_the_Republic_of_China.svg/23px-Flag_of_the_Republic_of_China.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/7/72/Flag_of_the_Republic_of_China.svg/35px-Flag_of_the_Republic_of_China.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/7/72/Flag_of_the_Republic_of_China.svg/45px-Flag_of_the_Republic_of_China.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Taiwan\" title=\"Taiwan\">\n",
      "     Taiwan\n",
      "    </a>\n",
      "    <sup class=\"reference\" id=\"cite_ref-109\">\n",
      "     <a href=\"#cite_note-109\">\n",
      "      [note 4]\n",
      "     </a>\n",
      "    </sup>\n",
      "   </td>\n",
      "   <td>\n",
      "    23,580,712\n",
      "   </td>\n",
      "   <td>\n",
      "    35,980\n",
      "   </td>\n",
      "   <td>\n",
      "    655\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    5\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/0/09/Flag_of_South_Korea.svg/23px-Flag_of_South_Korea.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/0/09/Flag_of_South_Korea.svg/35px-Flag_of_South_Korea.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/0/09/Flag_of_South_Korea.svg/45px-Flag_of_South_Korea.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/South_Korea\" title=\"South Korea\">\n",
      "     South Korea\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    51,844,834\n",
      "   </td>\n",
      "   <td>\n",
      "    99,720\n",
      "   </td>\n",
      "   <td>\n",
      "    520\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    6\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/5/59/Flag_of_Lebanon.svg/23px-Flag_of_Lebanon.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/5/59/Flag_of_Lebanon.svg/35px-Flag_of_Lebanon.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/5/59/Flag_of_Lebanon.svg/45px-Flag_of_Lebanon.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Lebanon\" title=\"Lebanon\">\n",
      "     Lebanon\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    5,296,814\n",
      "   </td>\n",
      "   <td>\n",
      "    10,400\n",
      "   </td>\n",
      "   <td>\n",
      "    509\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    7\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/1/17/Flag_of_Rwanda.svg/23px-Flag_of_Rwanda.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/1/17/Flag_of_Rwanda.svg/35px-Flag_of_Rwanda.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/1/17/Flag_of_Rwanda.svg/45px-Flag_of_Rwanda.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Rwanda\" title=\"Rwanda\">\n",
      "     Rwanda\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    13,173,730\n",
      "   </td>\n",
      "   <td>\n",
      "    26,338\n",
      "   </td>\n",
      "   <td>\n",
      "    500\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    8\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"1000\" decoding=\"async\" height=\"14\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/5/50/Flag_of_Burundi.svg/23px-Flag_of_Burundi.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/5/50/Flag_of_Burundi.svg/35px-Flag_of_Burundi.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/5/50/Flag_of_Burundi.svg/46px-Flag_of_Burundi.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Burundi\" title=\"Burundi\">\n",
      "     Burundi\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    12,696,478\n",
      "   </td>\n",
      "   <td>\n",
      "    27,830\n",
      "   </td>\n",
      "   <td>\n",
      "    456\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    9\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/en/thumb/4/41/Flag_of_India.svg/23px-Flag_of_India.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/en/thumb/4/41/Flag_of_India.svg/35px-Flag_of_India.svg.png 1.5x, //upload.wikimedia.org/wikipedia/en/thumb/4/41/Flag_of_India.svg/45px-Flag_of_India.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/India\" title=\"India\">\n",
      "     India\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    1,389,637,446\n",
      "   </td>\n",
      "   <td>\n",
      "    3,287,263\n",
      "   </td>\n",
      "   <td>\n",
      "    423\n",
      "   </td>\n",
      "  </tr>\n",
      "  <tr>\n",
      "   <td>\n",
      "    10\n",
      "   </td>\n",
      "   <td align=\"left\">\n",
      "    <span class=\"flagicon\">\n",
      "     <span class=\"mw-image-border\" typeof=\"mw:File\">\n",
      "      <span>\n",
      "       <img alt=\"\" class=\"mw-file-element\" data-file-height=\"600\" data-file-width=\"900\" decoding=\"async\" height=\"15\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/2/20/Flag_of_the_Netherlands.svg/23px-Flag_of_the_Netherlands.svg.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/2/20/Flag_of_the_Netherlands.svg/35px-Flag_of_the_Netherlands.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/2/20/Flag_of_the_Netherlands.svg/45px-Flag_of_the_Netherlands.svg.png 2x\" width=\"23\"/>\n",
      "      </span>\n",
      "     </span>\n",
      "    </span>\n",
      "    <a href=\"/wiki/Netherlands\" title=\"Netherlands\">\n",
      "     Netherlands\n",
      "    </a>\n",
      "   </td>\n",
      "   <td>\n",
      "    17,400,824\n",
      "   </td>\n",
      "   <td>\n",
      "    41,543\n",
      "   </td>\n",
      "   <td>\n",
      "    419\n",
      "   </td>\n",
      "  </tr>\n",
      " </tbody>\n",
      "</table>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(tables[table_index].prettify())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 710
    },
    "id": "ruetfsw23nUp",
    "outputId": "09dccefa-96bd-409b-b04b-45080373ef77"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
      "<ipython-input-58-b5a72626b929>:11: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "  population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-fa69acba-956c-4693-a4d2-b95f00193ab2\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Rank</th>\n",
       "      <th>Country</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area</th>\n",
       "      <th>Density</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Singapore</td>\n",
       "      <td>5,921,231</td>\n",
       "      <td>719</td>\n",
       "      <td>8,235</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Bangladesh</td>\n",
       "      <td>165,650,475</td>\n",
       "      <td>148,460</td>\n",
       "      <td>1,116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>\\n Palestine[note 3][103]\\n\\n</td>\n",
       "      <td>5,223,000</td>\n",
       "      <td>6,025</td>\n",
       "      <td>867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Taiwan[note 4]</td>\n",
       "      <td>23,580,712</td>\n",
       "      <td>35,980</td>\n",
       "      <td>655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>South Korea</td>\n",
       "      <td>51,844,834</td>\n",
       "      <td>99,720</td>\n",
       "      <td>520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Lebanon</td>\n",
       "      <td>5,296,814</td>\n",
       "      <td>10,400</td>\n",
       "      <td>509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Rwanda</td>\n",
       "      <td>13,173,730</td>\n",
       "      <td>26,338</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Burundi</td>\n",
       "      <td>12,696,478</td>\n",
       "      <td>27,830</td>\n",
       "      <td>456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>India</td>\n",
       "      <td>1,389,637,446</td>\n",
       "      <td>3,287,263</td>\n",
       "      <td>423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Netherlands</td>\n",
       "      <td>17,400,824</td>\n",
       "      <td>41,543</td>\n",
       "      <td>419</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fa69acba-956c-4693-a4d2-b95f00193ab2')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-fa69acba-956c-4693-a4d2-b95f00193ab2 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-fa69acba-956c-4693-a4d2-b95f00193ab2');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-02512ad8-80b5-45e5-aa69-d9ba518ab81a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-02512ad8-80b5-45e5-aa69-d9ba518ab81a')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-02512ad8-80b5-45e5-aa69-d9ba518ab81a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "  Rank                        Country     Population       Area Density\n",
       "0    1                      Singapore      5,921,231        719   8,235\n",
       "1    2                     Bangladesh    165,650,475    148,460   1,116\n",
       "2    3  \\n Palestine[note 3][103]\\n\\n      5,223,000      6,025     867\n",
       "3    4                 Taiwan[note 4]     23,580,712     35,980     655\n",
       "4    5                    South Korea     51,844,834     99,720     520\n",
       "5    6                        Lebanon      5,296,814     10,400     509\n",
       "6    7                         Rwanda     13,173,730     26,338     500\n",
       "7    8                        Burundi     12,696,478     27,830     456\n",
       "8    9                          India  1,389,637,446  3,287,263     423\n",
       "9   10                    Netherlands     17,400,824     41,543     419"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "population_data = pd.DataFrame(columns=[\"Rank\", \"Country\", \"Population\", \"Area\", \"Density\"])\n",
    "\n",
    "for row in tables[table_index].tbody.find_all(\"tr\"):\n",
    "    col = row.find_all(\"td\")\n",
    "    if (col != []):\n",
    "        rank = col[0].text\n",
    "        country = col[1].text\n",
    "        population = col[2].text.strip()\n",
    "        area = col[3].text.strip()\n",
    "        density = col[4].text.strip()\n",
    "        population_data = population_data.append({\"Rank\":rank, \"Country\":country, \"Population\":population, \"Area\":area, \"Density\":density}, ignore_index=True)\n",
    "\n",
    "population_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ELxREaVh3nUq"
   },
   "source": [
    "## Scrape data from HTML tables into a DataFrame using BeautifulSoup and read_html\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8EXZJGve3nUq"
   },
   "source": [
    "Using the same `url`, `data`, `soup`, and `tables` object as in the last section we can use the `read_html` function to create a DataFrame.\n",
    "\n",
    "Remember the table we need is located in `tables[table_index]`\n",
    "\n",
    "We can now use the `pandas` function `read_html` and give it the string version of the table as well as the `flavor` which is the parsing engine `bs4`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "z4-OxfpH3nUq",
    "outputId": "759e75d0-a739-4b65-a348-ab38911af5fe"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[                                                    #  \\\n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       "       Graphs are unavailable due to technical issues.   \n",
       " 0                                                 NaN   \n",
       " 1                                                   1   \n",
       " 2                                                   2   \n",
       " 3                                                   3   \n",
       " 4                                                   4   \n",
       " 5                                                   5   \n",
       " 6                                                   6   \n",
       " 7                                                   7   \n",
       " 8                                                   8   \n",
       " 9                                                   9   \n",
       " 10                                                 10   \n",
       " 11                                                NaN   \n",
       " 12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       " \n",
       "                               Most populous countries  \\\n",
       "                                    Unnamed: 1_level_1   \n",
       "                                    Unnamed: 1_level_2   \n",
       "                                    Unnamed: 1_level_3   \n",
       "                                    Unnamed: 1_level_4   \n",
       "                                    Unnamed: 1_level_5   \n",
       "                                    Unnamed: 1_level_6   \n",
       "                                    Unnamed: 1_level_7   \n",
       "                                    Unnamed: 1_level_8   \n",
       "                                    Unnamed: 1_level_9   \n",
       "                                   Unnamed: 1_level_10   \n",
       "                                   Unnamed: 1_level_11   \n",
       " 0     Graphs are unavailable due to technical issues.   \n",
       " 1                                            China[B]   \n",
       " 2                                               India   \n",
       " 3                                       United States   \n",
       " 4                                           Indonesia   \n",
       " 5                                            Pakistan   \n",
       " 6                                              Brazil   \n",
       " 7                                             Nigeria   \n",
       " 8                                          Bangladesh   \n",
       " 9                                              Russia   \n",
       " 10                                             Mexico   \n",
       " 11                                        World total   \n",
       " 12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       " \n",
       "                                                  2000  \\\n",
       "                                    Unnamed: 2_level_1   \n",
       "                                    Unnamed: 2_level_2   \n",
       "                                    Unnamed: 2_level_3   \n",
       "                                    Unnamed: 2_level_4   \n",
       "                                    Unnamed: 2_level_5   \n",
       "                                    Unnamed: 2_level_6   \n",
       "                                    Unnamed: 2_level_7   \n",
       "                                    Unnamed: 2_level_8   \n",
       "                                    Unnamed: 2_level_9   \n",
       "                                   Unnamed: 2_level_10   \n",
       "                                   Unnamed: 2_level_11   \n",
       " 0                                                 NaN   \n",
       " 1                                                1270   \n",
       " 2                                                1053   \n",
       " 3                                                 283   \n",
       " 4                                                 212   \n",
       " 5                                                 136   \n",
       " 6                                                 176   \n",
       " 7                                                 123   \n",
       " 8                                                 131   \n",
       " 9                                                 146   \n",
       " 10                                                103   \n",
       " 11                                               6127   \n",
       " 12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       " \n",
       "                                                  2015  \\\n",
       "                                    Unnamed: 3_level_1   \n",
       "                                    Unnamed: 3_level_2   \n",
       "                                    Unnamed: 3_level_3   \n",
       "                                    Unnamed: 3_level_4   \n",
       "                                    Unnamed: 3_level_5   \n",
       "                                    Unnamed: 3_level_6   \n",
       "                                    Unnamed: 3_level_7   \n",
       "                                    Unnamed: 3_level_8   \n",
       "                                    Unnamed: 3_level_9   \n",
       "                                   Unnamed: 3_level_10   \n",
       "                                   Unnamed: 3_level_11   \n",
       " 0                                                 NaN   \n",
       " 1                                                1376   \n",
       " 2                                                1311   \n",
       " 3                                                 322   \n",
       " 4                                                 258   \n",
       " 5                                                 208   \n",
       " 6                                                 206   \n",
       " 7                                                 182   \n",
       " 8                                                 161   \n",
       " 9                                                 146   \n",
       " 10                                                127   \n",
       " 11                                               7349   \n",
       " 12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       " \n",
       "                                               2030[A]  \\\n",
       "                                    Unnamed: 4_level_1   \n",
       "                                    Unnamed: 4_level_2   \n",
       "                                    Unnamed: 4_level_3   \n",
       "                                    Unnamed: 4_level_4   \n",
       "                                    Unnamed: 4_level_5   \n",
       "                                    Unnamed: 4_level_6   \n",
       "                                    Unnamed: 4_level_7   \n",
       "                                    Unnamed: 4_level_8   \n",
       "                                    Unnamed: 4_level_9   \n",
       "                                   Unnamed: 4_level_10   \n",
       "                                   Unnamed: 4_level_11   \n",
       " 0                                                 NaN   \n",
       " 1                                                1416   \n",
       " 2                                                1528   \n",
       " 3                                                 356   \n",
       " 4                                                 295   \n",
       " 5                                                 245   \n",
       " 6                                                 228   \n",
       " 7                                                 263   \n",
       " 8                                                 186   \n",
       " 9                                                 149   \n",
       " 10                                                148   \n",
       " 11                                               8501   \n",
       " 12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       " \n",
       "       Graphs are unavailable due to technical issues.  \n",
       "                                    Unnamed: 5_level_1  \n",
       "                                    Unnamed: 5_level_2  \n",
       "                                    Unnamed: 5_level_3  \n",
       "                                    Unnamed: 5_level_4  \n",
       "                                    Unnamed: 5_level_5  \n",
       "                                    Unnamed: 5_level_6  \n",
       "                                    Unnamed: 5_level_7  \n",
       "                                    Unnamed: 5_level_8  \n",
       "                                    Unnamed: 5_level_9  \n",
       "                                   Unnamed: 5_level_10  \n",
       "                                   Unnamed: 5_level_11  \n",
       " 0                                                 NaN  \n",
       " 1                                                 NaN  \n",
       " 2                                                 NaN  \n",
       " 3                                                 NaN  \n",
       " 4                                                 NaN  \n",
       " 5                                                 NaN  \n",
       " 6                                                 NaN  \n",
       " 7                                                 NaN  \n",
       " 8                                                 NaN  \n",
       " 9                                                 NaN  \n",
       " 10                                                NaN  \n",
       " 11                                                NaN  \n",
       " 12  Notes: .mw-parser-output .reflist{font-size:90...  ,\n",
       "     0                                                1\n",
       " 0 NaN  Graphs are unavailable due to technical issues.]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_html(str(tables[5]), flavor='bs4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ETgshe6c3nUq"
   },
   "source": [
    "The function `read_html` always returns a list of DataFrames so we must pick the one we want out of the list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "M9CSgiLT3nUr",
    "outputId": "4b0d0a87-2ed7-493b-d241-897c62ff936d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ee53f93a-49c2-4111-b2b7-8b92a9bd03aa\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>#</th>\n",
       "      <th>Most populous countries</th>\n",
       "      <th>2000</th>\n",
       "      <th>2015</th>\n",
       "      <th>2030[A]</th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_1</th>\n",
       "      <th>Unnamed: 2_level_1</th>\n",
       "      <th>Unnamed: 3_level_1</th>\n",
       "      <th>Unnamed: 4_level_1</th>\n",
       "      <th>Unnamed: 5_level_1</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_2</th>\n",
       "      <th>Unnamed: 2_level_2</th>\n",
       "      <th>Unnamed: 3_level_2</th>\n",
       "      <th>Unnamed: 4_level_2</th>\n",
       "      <th>Unnamed: 5_level_2</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_3</th>\n",
       "      <th>Unnamed: 2_level_3</th>\n",
       "      <th>Unnamed: 3_level_3</th>\n",
       "      <th>Unnamed: 4_level_3</th>\n",
       "      <th>Unnamed: 5_level_3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_4</th>\n",
       "      <th>Unnamed: 2_level_4</th>\n",
       "      <th>Unnamed: 3_level_4</th>\n",
       "      <th>Unnamed: 4_level_4</th>\n",
       "      <th>Unnamed: 5_level_4</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_5</th>\n",
       "      <th>Unnamed: 2_level_5</th>\n",
       "      <th>Unnamed: 3_level_5</th>\n",
       "      <th>Unnamed: 4_level_5</th>\n",
       "      <th>Unnamed: 5_level_5</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_6</th>\n",
       "      <th>Unnamed: 2_level_6</th>\n",
       "      <th>Unnamed: 3_level_6</th>\n",
       "      <th>Unnamed: 4_level_6</th>\n",
       "      <th>Unnamed: 5_level_6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_7</th>\n",
       "      <th>Unnamed: 2_level_7</th>\n",
       "      <th>Unnamed: 3_level_7</th>\n",
       "      <th>Unnamed: 4_level_7</th>\n",
       "      <th>Unnamed: 5_level_7</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_8</th>\n",
       "      <th>Unnamed: 2_level_8</th>\n",
       "      <th>Unnamed: 3_level_8</th>\n",
       "      <th>Unnamed: 4_level_8</th>\n",
       "      <th>Unnamed: 5_level_8</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_9</th>\n",
       "      <th>Unnamed: 2_level_9</th>\n",
       "      <th>Unnamed: 3_level_9</th>\n",
       "      <th>Unnamed: 4_level_9</th>\n",
       "      <th>Unnamed: 5_level_9</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_10</th>\n",
       "      <th>Unnamed: 2_level_10</th>\n",
       "      <th>Unnamed: 3_level_10</th>\n",
       "      <th>Unnamed: 4_level_10</th>\n",
       "      <th>Unnamed: 5_level_10</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_11</th>\n",
       "      <th>Unnamed: 2_level_11</th>\n",
       "      <th>Unnamed: 3_level_11</th>\n",
       "      <th>Unnamed: 4_level_11</th>\n",
       "      <th>Unnamed: 5_level_11</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Graphs are unavailable due to technical issues.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>China[B]</td>\n",
       "      <td>1270</td>\n",
       "      <td>1376</td>\n",
       "      <td>1416</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>India</td>\n",
       "      <td>1053</td>\n",
       "      <td>1311</td>\n",
       "      <td>1528</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>United States</td>\n",
       "      <td>283</td>\n",
       "      <td>322</td>\n",
       "      <td>356</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Indonesia</td>\n",
       "      <td>212</td>\n",
       "      <td>258</td>\n",
       "      <td>295</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>Pakistan</td>\n",
       "      <td>136</td>\n",
       "      <td>208</td>\n",
       "      <td>245</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>Brazil</td>\n",
       "      <td>176</td>\n",
       "      <td>206</td>\n",
       "      <td>228</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>Nigeria</td>\n",
       "      <td>123</td>\n",
       "      <td>182</td>\n",
       "      <td>263</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>Bangladesh</td>\n",
       "      <td>131</td>\n",
       "      <td>161</td>\n",
       "      <td>186</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>Russia</td>\n",
       "      <td>146</td>\n",
       "      <td>146</td>\n",
       "      <td>149</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>Mexico</td>\n",
       "      <td>103</td>\n",
       "      <td>127</td>\n",
       "      <td>148</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World total</td>\n",
       "      <td>6127</td>\n",
       "      <td>7349</td>\n",
       "      <td>8501</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ee53f93a-49c2-4111-b2b7-8b92a9bd03aa')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ee53f93a-49c2-4111-b2b7-8b92a9bd03aa button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ee53f93a-49c2-4111-b2b7-8b92a9bd03aa');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-04ebfb06-861f-4dbc-8e99-bd9f4feb5a5a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-04ebfb06-861f-4dbc-8e99-bd9f4feb5a5a')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-04ebfb06-861f-4dbc-8e99-bd9f4feb5a5a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                                    #  \\\n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "0                                                 NaN   \n",
       "1                                                   1   \n",
       "2                                                   2   \n",
       "3                                                   3   \n",
       "4                                                   4   \n",
       "5                                                   5   \n",
       "6                                                   6   \n",
       "7                                                   7   \n",
       "8                                                   8   \n",
       "9                                                   9   \n",
       "10                                                 10   \n",
       "11                                                NaN   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                              Most populous countries  \\\n",
       "                                   Unnamed: 1_level_1   \n",
       "                                   Unnamed: 1_level_2   \n",
       "                                   Unnamed: 1_level_3   \n",
       "                                   Unnamed: 1_level_4   \n",
       "                                   Unnamed: 1_level_5   \n",
       "                                   Unnamed: 1_level_6   \n",
       "                                   Unnamed: 1_level_7   \n",
       "                                   Unnamed: 1_level_8   \n",
       "                                   Unnamed: 1_level_9   \n",
       "                                  Unnamed: 1_level_10   \n",
       "                                  Unnamed: 1_level_11   \n",
       "0     Graphs are unavailable due to technical issues.   \n",
       "1                                            China[B]   \n",
       "2                                               India   \n",
       "3                                       United States   \n",
       "4                                           Indonesia   \n",
       "5                                            Pakistan   \n",
       "6                                              Brazil   \n",
       "7                                             Nigeria   \n",
       "8                                          Bangladesh   \n",
       "9                                              Russia   \n",
       "10                                             Mexico   \n",
       "11                                        World total   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                                                 2000  \\\n",
       "                                   Unnamed: 2_level_1   \n",
       "                                   Unnamed: 2_level_2   \n",
       "                                   Unnamed: 2_level_3   \n",
       "                                   Unnamed: 2_level_4   \n",
       "                                   Unnamed: 2_level_5   \n",
       "                                   Unnamed: 2_level_6   \n",
       "                                   Unnamed: 2_level_7   \n",
       "                                   Unnamed: 2_level_8   \n",
       "                                   Unnamed: 2_level_9   \n",
       "                                  Unnamed: 2_level_10   \n",
       "                                  Unnamed: 2_level_11   \n",
       "0                                                 NaN   \n",
       "1                                                1270   \n",
       "2                                                1053   \n",
       "3                                                 283   \n",
       "4                                                 212   \n",
       "5                                                 136   \n",
       "6                                                 176   \n",
       "7                                                 123   \n",
       "8                                                 131   \n",
       "9                                                 146   \n",
       "10                                                103   \n",
       "11                                               6127   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                                                 2015  \\\n",
       "                                   Unnamed: 3_level_1   \n",
       "                                   Unnamed: 3_level_2   \n",
       "                                   Unnamed: 3_level_3   \n",
       "                                   Unnamed: 3_level_4   \n",
       "                                   Unnamed: 3_level_5   \n",
       "                                   Unnamed: 3_level_6   \n",
       "                                   Unnamed: 3_level_7   \n",
       "                                   Unnamed: 3_level_8   \n",
       "                                   Unnamed: 3_level_9   \n",
       "                                  Unnamed: 3_level_10   \n",
       "                                  Unnamed: 3_level_11   \n",
       "0                                                 NaN   \n",
       "1                                                1376   \n",
       "2                                                1311   \n",
       "3                                                 322   \n",
       "4                                                 258   \n",
       "5                                                 208   \n",
       "6                                                 206   \n",
       "7                                                 182   \n",
       "8                                                 161   \n",
       "9                                                 146   \n",
       "10                                                127   \n",
       "11                                               7349   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                                              2030[A]  \\\n",
       "                                   Unnamed: 4_level_1   \n",
       "                                   Unnamed: 4_level_2   \n",
       "                                   Unnamed: 4_level_3   \n",
       "                                   Unnamed: 4_level_4   \n",
       "                                   Unnamed: 4_level_5   \n",
       "                                   Unnamed: 4_level_6   \n",
       "                                   Unnamed: 4_level_7   \n",
       "                                   Unnamed: 4_level_8   \n",
       "                                   Unnamed: 4_level_9   \n",
       "                                  Unnamed: 4_level_10   \n",
       "                                  Unnamed: 4_level_11   \n",
       "0                                                 NaN   \n",
       "1                                                1416   \n",
       "2                                                1528   \n",
       "3                                                 356   \n",
       "4                                                 295   \n",
       "5                                                 245   \n",
       "6                                                 228   \n",
       "7                                                 263   \n",
       "8                                                 186   \n",
       "9                                                 149   \n",
       "10                                                148   \n",
       "11                                               8501   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "      Graphs are unavailable due to technical issues.  \n",
       "                                   Unnamed: 5_level_1  \n",
       "                                   Unnamed: 5_level_2  \n",
       "                                   Unnamed: 5_level_3  \n",
       "                                   Unnamed: 5_level_4  \n",
       "                                   Unnamed: 5_level_5  \n",
       "                                   Unnamed: 5_level_6  \n",
       "                                   Unnamed: 5_level_7  \n",
       "                                   Unnamed: 5_level_8  \n",
       "                                   Unnamed: 5_level_9  \n",
       "                                  Unnamed: 5_level_10  \n",
       "                                  Unnamed: 5_level_11  \n",
       "0                                                 NaN  \n",
       "1                                                 NaN  \n",
       "2                                                 NaN  \n",
       "3                                                 NaN  \n",
       "4                                                 NaN  \n",
       "5                                                 NaN  \n",
       "6                                                 NaN  \n",
       "7                                                 NaN  \n",
       "8                                                 NaN  \n",
       "9                                                 NaN  \n",
       "10                                                NaN  \n",
       "11                                                NaN  \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...  "
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "population_data_read_html = pd.read_html(str(tables[5]), flavor='bs4')[0]\n",
    "\n",
    "population_data_read_html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "enAgQmDR3nUr"
   },
   "source": [
    "## Scrape data from HTML tables into a DataFrame using read_html\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Qicp8lNt3nUr"
   },
   "source": [
    "We can also use the `read_html` function to directly get DataFrames from a `url`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ihI0oGdH3nUs"
   },
   "outputs": [],
   "source": [
    "dataframe_list = pd.read_html(url, flavor='bs4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rXbcd4923nUt"
   },
   "source": [
    "We can see there are 25 DataFrames just like when we used `find_all` on the `soup` object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_2OT2HLm3nUt",
    "outputId": "56c6da94-bcb7-49b8-e5cb-2cb59884a918"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "26"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(dataframe_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4TUz0VrN3nUt"
   },
   "source": [
    "Finally we can pick the DataFrame we need out of the list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "3W6GCeOP3nUt",
    "outputId": "c797cbf7-9752-41c1-889b-eec6eec00ff9"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-a7a5579f-23a8-4822-86c5-8feab5e76135\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>#</th>\n",
       "      <th>Most populous countries</th>\n",
       "      <th>2000</th>\n",
       "      <th>2015</th>\n",
       "      <th>2030[A]</th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_1</th>\n",
       "      <th>Unnamed: 2_level_1</th>\n",
       "      <th>Unnamed: 3_level_1</th>\n",
       "      <th>Unnamed: 4_level_1</th>\n",
       "      <th>Unnamed: 5_level_1</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_2</th>\n",
       "      <th>Unnamed: 2_level_2</th>\n",
       "      <th>Unnamed: 3_level_2</th>\n",
       "      <th>Unnamed: 4_level_2</th>\n",
       "      <th>Unnamed: 5_level_2</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_3</th>\n",
       "      <th>Unnamed: 2_level_3</th>\n",
       "      <th>Unnamed: 3_level_3</th>\n",
       "      <th>Unnamed: 4_level_3</th>\n",
       "      <th>Unnamed: 5_level_3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_4</th>\n",
       "      <th>Unnamed: 2_level_4</th>\n",
       "      <th>Unnamed: 3_level_4</th>\n",
       "      <th>Unnamed: 4_level_4</th>\n",
       "      <th>Unnamed: 5_level_4</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_5</th>\n",
       "      <th>Unnamed: 2_level_5</th>\n",
       "      <th>Unnamed: 3_level_5</th>\n",
       "      <th>Unnamed: 4_level_5</th>\n",
       "      <th>Unnamed: 5_level_5</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_6</th>\n",
       "      <th>Unnamed: 2_level_6</th>\n",
       "      <th>Unnamed: 3_level_6</th>\n",
       "      <th>Unnamed: 4_level_6</th>\n",
       "      <th>Unnamed: 5_level_6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_7</th>\n",
       "      <th>Unnamed: 2_level_7</th>\n",
       "      <th>Unnamed: 3_level_7</th>\n",
       "      <th>Unnamed: 4_level_7</th>\n",
       "      <th>Unnamed: 5_level_7</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_8</th>\n",
       "      <th>Unnamed: 2_level_8</th>\n",
       "      <th>Unnamed: 3_level_8</th>\n",
       "      <th>Unnamed: 4_level_8</th>\n",
       "      <th>Unnamed: 5_level_8</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_9</th>\n",
       "      <th>Unnamed: 2_level_9</th>\n",
       "      <th>Unnamed: 3_level_9</th>\n",
       "      <th>Unnamed: 4_level_9</th>\n",
       "      <th>Unnamed: 5_level_9</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_10</th>\n",
       "      <th>Unnamed: 2_level_10</th>\n",
       "      <th>Unnamed: 3_level_10</th>\n",
       "      <th>Unnamed: 4_level_10</th>\n",
       "      <th>Unnamed: 5_level_10</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Graphs are unavailable due to technical issues.</th>\n",
       "      <th>Unnamed: 1_level_11</th>\n",
       "      <th>Unnamed: 2_level_11</th>\n",
       "      <th>Unnamed: 3_level_11</th>\n",
       "      <th>Unnamed: 4_level_11</th>\n",
       "      <th>Unnamed: 5_level_11</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Graphs are unavailable due to technical issues.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>China[B]</td>\n",
       "      <td>1270</td>\n",
       "      <td>1376</td>\n",
       "      <td>1416</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>India</td>\n",
       "      <td>1053</td>\n",
       "      <td>1311</td>\n",
       "      <td>1528</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>United States</td>\n",
       "      <td>283</td>\n",
       "      <td>322</td>\n",
       "      <td>356</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Indonesia</td>\n",
       "      <td>212</td>\n",
       "      <td>258</td>\n",
       "      <td>295</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>Pakistan</td>\n",
       "      <td>136</td>\n",
       "      <td>208</td>\n",
       "      <td>245</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>Brazil</td>\n",
       "      <td>176</td>\n",
       "      <td>206</td>\n",
       "      <td>228</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>Nigeria</td>\n",
       "      <td>123</td>\n",
       "      <td>182</td>\n",
       "      <td>263</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>Bangladesh</td>\n",
       "      <td>131</td>\n",
       "      <td>161</td>\n",
       "      <td>186</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>Russia</td>\n",
       "      <td>146</td>\n",
       "      <td>146</td>\n",
       "      <td>149</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>Mexico</td>\n",
       "      <td>103</td>\n",
       "      <td>127</td>\n",
       "      <td>148</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World total</td>\n",
       "      <td>6127</td>\n",
       "      <td>7349</td>\n",
       "      <td>8501</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "      <td>Notes: .mw-parser-output .reflist{font-size:90...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a7a5579f-23a8-4822-86c5-8feab5e76135')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-a7a5579f-23a8-4822-86c5-8feab5e76135 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-a7a5579f-23a8-4822-86c5-8feab5e76135');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-5a455418-0777-457d-8764-cf9438d71480\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5a455418-0777-457d-8764-cf9438d71480')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-5a455418-0777-457d-8764-cf9438d71480 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                                    #  \\\n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "      Graphs are unavailable due to technical issues.   \n",
       "0                                                 NaN   \n",
       "1                                                   1   \n",
       "2                                                   2   \n",
       "3                                                   3   \n",
       "4                                                   4   \n",
       "5                                                   5   \n",
       "6                                                   6   \n",
       "7                                                   7   \n",
       "8                                                   8   \n",
       "9                                                   9   \n",
       "10                                                 10   \n",
       "11                                                NaN   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                              Most populous countries  \\\n",
       "                                   Unnamed: 1_level_1   \n",
       "                                   Unnamed: 1_level_2   \n",
       "                                   Unnamed: 1_level_3   \n",
       "                                   Unnamed: 1_level_4   \n",
       "                                   Unnamed: 1_level_5   \n",
       "                                   Unnamed: 1_level_6   \n",
       "                                   Unnamed: 1_level_7   \n",
       "                                   Unnamed: 1_level_8   \n",
       "                                   Unnamed: 1_level_9   \n",
       "                                  Unnamed: 1_level_10   \n",
       "                                  Unnamed: 1_level_11   \n",
       "0     Graphs are unavailable due to technical issues.   \n",
       "1                                            China[B]   \n",
       "2                                               India   \n",
       "3                                       United States   \n",
       "4                                           Indonesia   \n",
       "5                                            Pakistan   \n",
       "6                                              Brazil   \n",
       "7                                             Nigeria   \n",
       "8                                          Bangladesh   \n",
       "9                                              Russia   \n",
       "10                                             Mexico   \n",
       "11                                        World total   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                                                 2000  \\\n",
       "                                   Unnamed: 2_level_1   \n",
       "                                   Unnamed: 2_level_2   \n",
       "                                   Unnamed: 2_level_3   \n",
       "                                   Unnamed: 2_level_4   \n",
       "                                   Unnamed: 2_level_5   \n",
       "                                   Unnamed: 2_level_6   \n",
       "                                   Unnamed: 2_level_7   \n",
       "                                   Unnamed: 2_level_8   \n",
       "                                   Unnamed: 2_level_9   \n",
       "                                  Unnamed: 2_level_10   \n",
       "                                  Unnamed: 2_level_11   \n",
       "0                                                 NaN   \n",
       "1                                                1270   \n",
       "2                                                1053   \n",
       "3                                                 283   \n",
       "4                                                 212   \n",
       "5                                                 136   \n",
       "6                                                 176   \n",
       "7                                                 123   \n",
       "8                                                 131   \n",
       "9                                                 146   \n",
       "10                                                103   \n",
       "11                                               6127   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                                                 2015  \\\n",
       "                                   Unnamed: 3_level_1   \n",
       "                                   Unnamed: 3_level_2   \n",
       "                                   Unnamed: 3_level_3   \n",
       "                                   Unnamed: 3_level_4   \n",
       "                                   Unnamed: 3_level_5   \n",
       "                                   Unnamed: 3_level_6   \n",
       "                                   Unnamed: 3_level_7   \n",
       "                                   Unnamed: 3_level_8   \n",
       "                                   Unnamed: 3_level_9   \n",
       "                                  Unnamed: 3_level_10   \n",
       "                                  Unnamed: 3_level_11   \n",
       "0                                                 NaN   \n",
       "1                                                1376   \n",
       "2                                                1311   \n",
       "3                                                 322   \n",
       "4                                                 258   \n",
       "5                                                 208   \n",
       "6                                                 206   \n",
       "7                                                 182   \n",
       "8                                                 161   \n",
       "9                                                 146   \n",
       "10                                                127   \n",
       "11                                               7349   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "                                              2030[A]  \\\n",
       "                                   Unnamed: 4_level_1   \n",
       "                                   Unnamed: 4_level_2   \n",
       "                                   Unnamed: 4_level_3   \n",
       "                                   Unnamed: 4_level_4   \n",
       "                                   Unnamed: 4_level_5   \n",
       "                                   Unnamed: 4_level_6   \n",
       "                                   Unnamed: 4_level_7   \n",
       "                                   Unnamed: 4_level_8   \n",
       "                                   Unnamed: 4_level_9   \n",
       "                                  Unnamed: 4_level_10   \n",
       "                                  Unnamed: 4_level_11   \n",
       "0                                                 NaN   \n",
       "1                                                1416   \n",
       "2                                                1528   \n",
       "3                                                 356   \n",
       "4                                                 295   \n",
       "5                                                 245   \n",
       "6                                                 228   \n",
       "7                                                 263   \n",
       "8                                                 186   \n",
       "9                                                 149   \n",
       "10                                                148   \n",
       "11                                               8501   \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...   \n",
       "\n",
       "      Graphs are unavailable due to technical issues.  \n",
       "                                   Unnamed: 5_level_1  \n",
       "                                   Unnamed: 5_level_2  \n",
       "                                   Unnamed: 5_level_3  \n",
       "                                   Unnamed: 5_level_4  \n",
       "                                   Unnamed: 5_level_5  \n",
       "                                   Unnamed: 5_level_6  \n",
       "                                   Unnamed: 5_level_7  \n",
       "                                   Unnamed: 5_level_8  \n",
       "                                   Unnamed: 5_level_9  \n",
       "                                  Unnamed: 5_level_10  \n",
       "                                  Unnamed: 5_level_11  \n",
       "0                                                 NaN  \n",
       "1                                                 NaN  \n",
       "2                                                 NaN  \n",
       "3                                                 NaN  \n",
       "4                                                 NaN  \n",
       "5                                                 NaN  \n",
       "6                                                 NaN  \n",
       "7                                                 NaN  \n",
       "8                                                 NaN  \n",
       "9                                                 NaN  \n",
       "10                                                NaN  \n",
       "11                                                NaN  \n",
       "12  Notes: .mw-parser-output .reflist{font-size:90...  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe_list[5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vQpZMmeV3nUu"
   },
   "source": [
    "We can also use the `match` parameter to select the specific table we want. If the table contains a string matching the text it will be read.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 363
    },
    "id": "EnccGsIR3nUu",
    "outputId": "7fadb4f9-aaed-4842-8890-398464ba5eb8"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-7742fea3-467a-404a-835a-5b476fc745fa\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Rank</th>\n",
       "      <th>Country</th>\n",
       "      <th>Population</th>\n",
       "      <th>Area (km2)</th>\n",
       "      <th>Density (pop/km2)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Singapore</td>\n",
       "      <td>5921231</td>\n",
       "      <td>719</td>\n",
       "      <td>8235</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Bangladesh</td>\n",
       "      <td>165650475</td>\n",
       "      <td>148460</td>\n",
       "      <td>1116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Palestine[note 3][103]</td>\n",
       "      <td>5223000</td>\n",
       "      <td>6025</td>\n",
       "      <td>867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Taiwan[note 4]</td>\n",
       "      <td>23580712</td>\n",
       "      <td>35980</td>\n",
       "      <td>655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>South Korea</td>\n",
       "      <td>51844834</td>\n",
       "      <td>99720</td>\n",
       "      <td>520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Lebanon</td>\n",
       "      <td>5296814</td>\n",
       "      <td>10400</td>\n",
       "      <td>509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Rwanda</td>\n",
       "      <td>13173730</td>\n",
       "      <td>26338</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Burundi</td>\n",
       "      <td>12696478</td>\n",
       "      <td>27830</td>\n",
       "      <td>456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>India</td>\n",
       "      <td>1389637446</td>\n",
       "      <td>3287263</td>\n",
       "      <td>423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Netherlands</td>\n",
       "      <td>17400824</td>\n",
       "      <td>41543</td>\n",
       "      <td>419</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7742fea3-467a-404a-835a-5b476fc745fa')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-7742fea3-467a-404a-835a-5b476fc745fa button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-7742fea3-467a-404a-835a-5b476fc745fa');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-c85fbd96-bace-4e6a-98db-6bddcf2c3ce2\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c85fbd96-bace-4e6a-98db-6bddcf2c3ce2')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-c85fbd96-bace-4e6a-98db-6bddcf2c3ce2 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "   Rank                 Country  Population  Area (km2)  Density (pop/km2)\n",
       "0     1               Singapore     5921231         719               8235\n",
       "1     2              Bangladesh   165650475      148460               1116\n",
       "2     3  Palestine[note 3][103]     5223000        6025                867\n",
       "3     4          Taiwan[note 4]    23580712       35980                655\n",
       "4     5             South Korea    51844834       99720                520\n",
       "5     6                 Lebanon     5296814       10400                509\n",
       "6     7                  Rwanda    13173730       26338                500\n",
       "7     8                 Burundi    12696478       27830                456\n",
       "8     9                   India  1389637446     3287263                423\n",
       "9    10             Netherlands    17400824       41543                419"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_html(url, match=\"10 most densely populated countries\", flavor='bs4')[0]"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}