SQL2pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5b60fd57",
   "metadata": {},
   "source": [
    "**SQL commands in pandas**. \n",
    "\n",
    "SQL emulator using Python pandas functions. Has the advantages:\n",
    "\n",
    "• The functions can easily be stored for future use\n",
    "   \n",
    "• Can pass the output of one fucntion in input to another\n",
    "\n",
    "• Sigificantly less code \n",
    "   \n",
    "• Functions can be made interactive \n",
    "\n",
    "More generally:\n",
    "\n",
    "• Can (easily) run on command line and within analysis script for pipelines\n",
    "\n",
    "• The complexity and size of the SQL standard means that most implementers do not support the entire standard – so may be some incomptability with peers\n",
    "\n",
    "\n",
    "\n",
    "See https://github.com/steviecurran/sql2csv to convert .sql to .csv, via the C shell"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64036861",
   "metadata": {},
   "source": [
    "<a id='cont'></a>\n",
    "**Function contents**\n",
    "\n",
    "| SELECT\t\t          | Aggregate              | ORDER and GROUP       | JOIN                       | Others| \n",
    "| :---                 | :---                   | :--- \t           | :---                       | :-- | \n",
    "|**[WHERE](#WHERE)**  |**[COUNT](#COUNT)**  |**[ORDER BY](#OB)**    |**[INNER](#INNER)**  |**[DISTINCT](#DIST)**| \n",
    "|**[OR](#OR)**  |**[SUM](#AGG)**         |**[GROUP COUNT](#GC)** |**[OUTER](#OUTER)**         |**[FILTER](#FIL)**|\n",
    "|**[AND and OR](#AO)** |**[MIN and MAX](#AGG)** |**[GROUP AGG](#GA)** |**[LEFT/RIGHT](#LR)**  |**[CASE](#CAS)**|  \n",
    "| **[IN/NOT IN](#IN)** |**[AVG](#AGG)** |**[HAVING](#HA)**|**[MULTIPLE tables](#MUL)** |**[Window functions](#WF)** |\n",
    "| **[LIKE/NOT LIKE](#LIKE)**      |**[MEDIAN](#AGG)**      |**[DUPLICATES](#DUP)** |**[JOIN WHERE](#JW)**       | |\n",
    "| **[BETWEEN/NOT BETWEEN](#BET)** |                        |                  \t   | **[CROSS JOIN](#CJ)**      | |\n",
    "| **[IS NULL/NOT NULL](#NULL)**   |                        |                       | **[Aggregate and JOIN](#AJ)**|\n",
    "|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1d4a060c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "pd.options.mode.chained_assignment = None"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b764936d",
   "metadata": {},
   "source": [
    "Load some example data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "69ce50db",
   "metadata": {},
   "outputs": [],
   "source": [
    "emp = pd.read_csv(\"employees.csv\");\n",
    "sal = pd.read_csv(\"salaries.csv\");\n",
    "dep = pd.read_csv(\"departments.csv\")\n",
    "man = pd.read_csv(\"dept_manager.csv\")\n",
    "dept_emp = pd.read_csv(\"dept_emp.csv\")\n",
    "titles = pd.read_csv(\"titles.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdfb1909",
   "metadata": {},
   "source": [
    "# SELECT \n",
    "\n",
    "**SELECT FROM**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "562acb01",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT(data,field,entry):\n",
    "    return data[data[field] == entry]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1086,
   "id": "2cae18e8",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>10030</td>\n",
       "      <td>1958-07-14</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Demeyer</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-02-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1049</th>\n",
       "      <td>11050</td>\n",
       "      <td>1952-09-23</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Katiyar</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-05-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2869</th>\n",
       "      <td>12870</td>\n",
       "      <td>1960-03-24</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Pfau</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-04-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3619</th>\n",
       "      <td>13620</td>\n",
       "      <td>1956-10-31</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Dolinsky</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-06-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4433</th>\n",
       "      <td>14434</td>\n",
       "      <td>1954-07-08</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Maginnis</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-11-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>295430</th>\n",
       "      <td>495406</td>\n",
       "      <td>1956-05-04</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Gerlach</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>296527</th>\n",
       "      <td>496503</td>\n",
       "      <td>1956-09-01</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Verhoeff</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-10-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>297934</th>\n",
       "      <td>497910</td>\n",
       "      <td>1961-05-12</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Cannard</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-09-23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298645</th>\n",
       "      <td>498621</td>\n",
       "      <td>1955-09-07</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Zykh</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-08-31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299115</th>\n",
       "      <td>499091</td>\n",
       "      <td>1952-09-24</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Kohling</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-02-25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>246 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name last_name gender   hire_date\n",
       "29       10030  1958-07-14      Elvis   Demeyer      M  1994-02-17\n",
       "1049     11050  1952-09-23      Elvis   Katiyar      M  1986-05-06\n",
       "2869     12870  1960-03-24      Elvis      Pfau      F  1990-04-09\n",
       "3619     13620  1956-10-31      Elvis  Dolinsky      M  1985-06-12\n",
       "4433     14434  1954-07-08      Elvis  Maginnis      M  1987-11-27\n",
       "...        ...         ...        ...       ...    ...         ...\n",
       "295430  495406  1956-05-04      Elvis   Gerlach      M  1990-09-17\n",
       "296527  496503  1956-09-01      Elvis  Verhoeff      M  1986-10-27\n",
       "297934  497910  1961-05-12      Elvis   Cannard      F  1990-09-23\n",
       "298645  498621  1955-09-07      Elvis      Zykh      M  1993-08-31\n",
       "299115  499091  1952-09-24      Elvis   Kohling      F  1986-02-25\n",
       "\n",
       "[246 rows x 6 columns]"
      ]
     },
     "execution_count": 1086,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT(emp,'first_name',\"Elvis\") # E.g. select from employee table anyone with first name Elvis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65a6b751",
   "metadata": {},
   "source": [
    " [Back to contents](#cont)\n",
    " \n",
    " **<a id='WHERE'>SELECT WHERE</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1146,
   "id": "e727129f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_WHERE(data,field1,entry1,field2,entry2): \n",
    "    return data[(data[field1] == entry1) & (data[field2] == entry2)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8b4daca8",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5082</th>\n",
       "      <td>15083</td>\n",
       "      <td>1958-11-24</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Nicolson</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-03-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7223</th>\n",
       "      <td>17224</td>\n",
       "      <td>1955-04-06</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Back</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-07-29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8120</th>\n",
       "      <td>18121</td>\n",
       "      <td>1953-08-26</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Stanfel</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-05-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8186</th>\n",
       "      <td>18187</td>\n",
       "      <td>1965-01-08</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Falster</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-10-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9797</th>\n",
       "      <td>19798</td>\n",
       "      <td>1962-12-22</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Schmezko</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-10-30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>290517</th>\n",
       "      <td>490493</td>\n",
       "      <td>1956-07-06</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Auyong</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-04-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>290863</th>\n",
       "      <td>490839</td>\n",
       "      <td>1958-06-19</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Narwekar</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-12-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>291119</th>\n",
       "      <td>491095</td>\n",
       "      <td>1962-05-30</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Kuszyk</td>\n",
       "      <td>M</td>\n",
       "      <td>1992-12-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>296968</th>\n",
       "      <td>496944</td>\n",
       "      <td>1954-09-02</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Fortenbacher</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-02-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>297316</th>\n",
       "      <td>497292</td>\n",
       "      <td>1964-05-28</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Gustavson</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-07-25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>140 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name     last_name gender   hire_date\n",
       "5082     15083  1958-11-24      Denis      Nicolson      M  1994-03-02\n",
       "7223     17224  1955-04-06      Denis          Back      M  1985-07-29\n",
       "8120     18121  1953-08-26      Denis       Stanfel      M  1990-05-21\n",
       "8186     18187  1965-01-08      Denis       Falster      M  1989-10-12\n",
       "9797     19798  1962-12-22      Denis      Schmezko      M  1986-10-30\n",
       "...        ...         ...        ...           ...    ...         ...\n",
       "290517  490493  1956-07-06      Denis        Auyong      M  1989-04-15\n",
       "290863  490839  1958-06-19      Denis      Narwekar      M  1987-12-09\n",
       "291119  491095  1962-05-30      Denis        Kuszyk      M  1992-12-05\n",
       "296968  496944  1954-09-02      Denis  Fortenbacher      M  1986-02-09\n",
       "297316  497292  1964-05-28      Denis     Gustavson      M  1990-07-25\n",
       "\n",
       "[140 rows x 6 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_WHERE(emp,'first_name',\"Denis\",'gender',\"M\") # E.g. from employee table first name Denis and gender M"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "0207186c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [emp_no, birth_date, first_name, last_name, gender, hire_date]\n",
       "Index: []"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_WHERE(emp,'first_name',\"Elvis\",'last_name',\"Presley\") # Sorry, just had to"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f046dfd5",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**WHERE with comparison operator**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "e6961d21",
   "metadata": {},
   "outputs": [],
   "source": [
    "def FILTER(data,field,entry,action):\n",
    "    act = \"print(data[data[field] %s entry])\" %(action)\n",
    "    exec(act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "b39078ae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date first_name   last_name gender   hire_date\n",
      "37290    47291  1960-09-09        Ulf      Flexer      M  2000-01-12\n",
      "50133    60134  1964-04-21      Seshu    Rathonyi      F  2000-01-02\n",
      "62328    72329  1953-02-09      Randi        Luit      F  2000-01-02\n",
      "105072  205048  1960-09-12      Ennio      Alblas      F  2000-01-06\n",
      "122989  222965  1959-08-07    Volkmar       Perko      F  2000-01-13\n",
      "126657  226633  1958-06-10     Xuejun  Benzmuller      F  2000-01-04\n",
      "127568  227544  1954-11-17     Shahab     Demeyer      M  2000-01-08\n",
      "223014  422990  1953-04-09      Jaana    Verspoor      F  2000-01-11\n",
      "224469  424445  1953-04-27      Jeong     Boreale      M  2000-01-03\n",
      "228401  428377  1957-05-09      Yucai     Gerlach      M  2000-01-23\n",
      "263831  463807  1964-06-12     Bikash      Covnot      M  2000-01-28\n",
      "299577  499553  1954-05-06   Hideyuki   Delgrande      F  2000-01-22\n"
     ]
    }
   ],
   "source": [
    "FILTER(emp,'hire_date','2000-01-01','>') # E.g. employees hired after 2000-01-01 exclusive"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "58fcfbe0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date   first_name   last_name gender   hire_date\n",
      "37290    47291  1960-09-09          Ulf      Flexer      M  2000-01-12\n",
      "50133    60134  1964-04-21        Seshu    Rathonyi      F  2000-01-02\n",
      "62328    72329  1953-02-09        Randi        Luit      F  2000-01-02\n",
      "98200   108201  1955-04-14  Mariangiola     Boreale      M  2000-01-01\n",
      "105072  205048  1960-09-12        Ennio      Alblas      F  2000-01-06\n",
      "122989  222965  1959-08-07      Volkmar       Perko      F  2000-01-13\n",
      "126657  226633  1958-06-10       Xuejun  Benzmuller      F  2000-01-04\n",
      "127568  227544  1954-11-17       Shahab     Demeyer      M  2000-01-08\n",
      "223014  422990  1953-04-09        Jaana    Verspoor      F  2000-01-11\n",
      "224469  424445  1953-04-27        Jeong     Boreale      M  2000-01-03\n",
      "228401  428377  1957-05-09        Yucai     Gerlach      M  2000-01-23\n",
      "263831  463807  1964-06-12       Bikash      Covnot      M  2000-01-28\n",
      "299577  499553  1954-05-06     Hideyuki   Delgrande      F  2000-01-22\n"
     ]
    }
   ],
   "source": [
    "FILTER(emp,'hire_date','2000-01-01','>=') # E.g. employees hired after 2000-01-01 inclusive"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66691a8d",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**WHERE with comparison operator and condition**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "7eaec741",
   "metadata": {},
   "outputs": [],
   "source": [
    "def FILTER_COND(data,field1,entry1,field2,entry2,action): # FILTER with condition\n",
    "    tmp = data[data[field1] == entry1]; \n",
    "    act = \"print(tmp[tmp[field2] %s entry2])\" %(action)\n",
    "    exec(act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "4d65f1e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date first_name  last_name gender   hire_date\n",
      "1        10002  1964-06-02    Bezalel     Simmel      F  1985-11-21\n",
      "5        10006  1953-04-20     Anneke    Preusig      F  1989-06-02\n",
      "6        10007  1957-05-23    Tzvetan  Zielinski      F  1989-02-10\n",
      "8        10009  1952-04-19     Sumant       Peac      F  1985-02-18\n",
      "9        10010  1963-06-01  Duangkaew   Piveteau      F  1989-08-24\n",
      "...        ...         ...        ...        ...    ...         ...\n",
      "300012  499988  1962-09-28   Bangqing    Kleiser      F  1986-06-06\n",
      "300015  499991  1962-02-26      Pohua    Sichman      F  1989-01-12\n",
      "300016  499992  1960-10-12     Siamak   Salverda      F  1987-05-10\n",
      "300018  499994  1952-02-26      Navin    Argence      F  1990-04-24\n",
      "300019  499995  1958-09-24     Dekang   Lichtner      F  1993-01-12\n",
      "\n",
      "[120044 rows x 6 columns]\n"
     ]
    }
   ],
   "source": [
    "FILTER_COND(emp,'gender','F','hire_date','2000','<') \n",
    "# E.g. Female employees who were hired before the year 2000"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d72491f7",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='OR'>SELECT OR</a>** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "d04880e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_OR(data,field,entry1,entry2):\n",
    "    return data[(data[field] == entry1) | (data[field] == entry2)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "9f9955e2",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>10030</td>\n",
       "      <td>1958-07-14</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Demeyer</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-02-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1049</th>\n",
       "      <td>11050</td>\n",
       "      <td>1952-09-23</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Katiyar</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-05-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1687</th>\n",
       "      <td>11688</td>\n",
       "      <td>1958-09-04</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Coullard</td>\n",
       "      <td>F</td>\n",
       "      <td>1994-10-29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2869</th>\n",
       "      <td>12870</td>\n",
       "      <td>1960-03-24</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Pfau</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-04-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3619</th>\n",
       "      <td>13620</td>\n",
       "      <td>1956-10-31</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Dolinsky</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-06-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>297316</th>\n",
       "      <td>497292</td>\n",
       "      <td>1964-05-28</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Gustavson</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-07-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>297934</th>\n",
       "      <td>497910</td>\n",
       "      <td>1961-05-12</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Cannard</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-09-23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298284</th>\n",
       "      <td>498260</td>\n",
       "      <td>1959-04-15</td>\n",
       "      <td>Denis</td>\n",
       "      <td>Fetvedt</td>\n",
       "      <td>F</td>\n",
       "      <td>1988-05-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298645</th>\n",
       "      <td>498621</td>\n",
       "      <td>1955-09-07</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Zykh</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-08-31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299115</th>\n",
       "      <td>499091</td>\n",
       "      <td>1952-09-24</td>\n",
       "      <td>Elvis</td>\n",
       "      <td>Kohling</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-02-25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>478 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name  last_name gender   hire_date\n",
       "29       10030  1958-07-14      Elvis    Demeyer      M  1994-02-17\n",
       "1049     11050  1952-09-23      Elvis    Katiyar      M  1986-05-06\n",
       "1687     11688  1958-09-04      Denis   Coullard      F  1994-10-29\n",
       "2869     12870  1960-03-24      Elvis       Pfau      F  1990-04-09\n",
       "3619     13620  1956-10-31      Elvis   Dolinsky      M  1985-06-12\n",
       "...        ...         ...        ...        ...    ...         ...\n",
       "297316  497292  1964-05-28      Denis  Gustavson      M  1990-07-25\n",
       "297934  497910  1961-05-12      Elvis    Cannard      F  1990-09-23\n",
       "298284  498260  1959-04-15      Denis    Fetvedt      F  1988-05-04\n",
       "298645  498621  1955-09-07      Elvis       Zykh      M  1993-08-31\n",
       "299115  499091  1952-09-24      Elvis    Kohling      F  1986-02-25\n",
       "\n",
       "[478 rows x 6 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_OR(emp,'first_name',\"Denis\",\"Elvis\") # E.g. from employee table first name Denis or Elvis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e9d7464",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='AO'>SELECT AND and OR</a>** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "002d6db3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_AND_OR(data,field1,entry1,field2,entry2,entry3): \n",
    "    return data[(data[field1] == entry1) & ((data[field2] == entry2) | (data[field2] == entry3))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "12ddb59c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>224</th>\n",
       "      <td>10225</td>\n",
       "      <td>1957-02-13</td>\n",
       "      <td>Kellie</td>\n",
       "      <td>Chinen</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-06-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>788</th>\n",
       "      <td>10789</td>\n",
       "      <td>1964-05-19</td>\n",
       "      <td>Aruna</td>\n",
       "      <td>Journel</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-02-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2037</th>\n",
       "      <td>12038</td>\n",
       "      <td>1956-04-13</td>\n",
       "      <td>Aruna</td>\n",
       "      <td>Businaro</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-01-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4079</th>\n",
       "      <td>14080</td>\n",
       "      <td>1957-02-24</td>\n",
       "      <td>Aruna</td>\n",
       "      <td>Motley</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-03-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4458</th>\n",
       "      <td>14459</td>\n",
       "      <td>1964-09-21</td>\n",
       "      <td>Aruna</td>\n",
       "      <td>Boreale</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-12-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>287044</th>\n",
       "      <td>487020</td>\n",
       "      <td>1960-08-12</td>\n",
       "      <td>Kellie</td>\n",
       "      <td>Busillo</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-05-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>288205</th>\n",
       "      <td>488181</td>\n",
       "      <td>1960-05-02</td>\n",
       "      <td>Aruna</td>\n",
       "      <td>Delgrande</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-12-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>292705</th>\n",
       "      <td>492681</td>\n",
       "      <td>1957-04-06</td>\n",
       "      <td>Kellie</td>\n",
       "      <td>Koshino</td>\n",
       "      <td>F</td>\n",
       "      <td>1991-03-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>294769</th>\n",
       "      <td>494745</td>\n",
       "      <td>1962-09-16</td>\n",
       "      <td>Aruna</td>\n",
       "      <td>Massonet</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-09-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>294891</th>\n",
       "      <td>494867</td>\n",
       "      <td>1953-08-24</td>\n",
       "      <td>Kellie</td>\n",
       "      <td>Chenney</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-06-28</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>167 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name  last_name gender   hire_date\n",
       "224      10225  1957-02-13     Kellie     Chinen      F  1986-06-19\n",
       "788      10789  1964-05-19      Aruna    Journel      F  1987-02-02\n",
       "2037     12038  1956-04-13      Aruna   Businaro      F  1987-01-09\n",
       "4079     14080  1957-02-24      Aruna     Motley      F  1987-03-09\n",
       "4458     14459  1964-09-21      Aruna    Boreale      F  1986-12-06\n",
       "...        ...         ...        ...        ...    ...         ...\n",
       "287044  487020  1960-08-12     Kellie    Busillo      F  1985-05-09\n",
       "288205  488181  1960-05-02      Aruna  Delgrande      F  1986-12-15\n",
       "292705  492681  1957-04-06     Kellie    Koshino      F  1991-03-28\n",
       "294769  494745  1962-09-16      Aruna   Massonet      F  1993-09-10\n",
       "294891  494867  1953-08-24     Kellie    Chenney      F  1987-06-28\n",
       "\n",
       "[167 rows x 6 columns]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_AND_OR(emp,'gender',\"F\",'first_name',\"Kellie\",\"Aruna\") # E.g. gender F with first name Kellie or Aruna"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2cbb162",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='IN'>SELECT IN/NOT IN</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "2ad2d673",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_IN(data,field,entry):\n",
    "    return data[data[field].isin(entry)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "85aa3dd7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>10135</td>\n",
       "      <td>1956-12-23</td>\n",
       "      <td>Nathan</td>\n",
       "      <td>Monkewich</td>\n",
       "      <td>M</td>\n",
       "      <td>1988-02-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>414</th>\n",
       "      <td>10415</td>\n",
       "      <td>1957-11-12</td>\n",
       "      <td>Mark</td>\n",
       "      <td>Coorg</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-10-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>427</th>\n",
       "      <td>10428</td>\n",
       "      <td>1957-06-25</td>\n",
       "      <td>Cathie</td>\n",
       "      <td>Brlek</td>\n",
       "      <td>M</td>\n",
       "      <td>1992-06-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1094</th>\n",
       "      <td>11095</td>\n",
       "      <td>1961-12-31</td>\n",
       "      <td>Nathan</td>\n",
       "      <td>Flowers</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-05-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1502</th>\n",
       "      <td>11503</td>\n",
       "      <td>1953-08-01</td>\n",
       "      <td>Nathan</td>\n",
       "      <td>Picht</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-01-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298225</th>\n",
       "      <td>498201</td>\n",
       "      <td>1952-03-22</td>\n",
       "      <td>Mark</td>\n",
       "      <td>Schahn</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-12-24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298880</th>\n",
       "      <td>498856</td>\n",
       "      <td>1952-07-26</td>\n",
       "      <td>Cathie</td>\n",
       "      <td>Wiegley</td>\n",
       "      <td>M</td>\n",
       "      <td>1999-03-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298941</th>\n",
       "      <td>498917</td>\n",
       "      <td>1955-12-28</td>\n",
       "      <td>Nathan</td>\n",
       "      <td>Papadias</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-01-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299858</th>\n",
       "      <td>499834</td>\n",
       "      <td>1956-04-04</td>\n",
       "      <td>Nathan</td>\n",
       "      <td>Chimia</td>\n",
       "      <td>F</td>\n",
       "      <td>1995-05-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300010</th>\n",
       "      <td>499986</td>\n",
       "      <td>1952-07-22</td>\n",
       "      <td>Nathan</td>\n",
       "      <td>Ranta</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-08-11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>691 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name  last_name gender   hire_date\n",
       "134      10135  1956-12-23     Nathan  Monkewich      M  1988-02-19\n",
       "414      10415  1957-11-12       Mark      Coorg      M  1993-10-25\n",
       "427      10428  1957-06-25     Cathie      Brlek      M  1992-06-04\n",
       "1094     11095  1961-12-31     Nathan    Flowers      M  1994-05-28\n",
       "1502     11503  1953-08-01     Nathan      Picht      F  1993-01-25\n",
       "...        ...         ...        ...        ...    ...         ...\n",
       "298225  498201  1952-03-22       Mark     Schahn      F  1990-12-24\n",
       "298880  498856  1952-07-26     Cathie    Wiegley      M  1999-03-09\n",
       "298941  498917  1955-12-28     Nathan   Papadias      F  1987-01-09\n",
       "299858  499834  1956-04-04     Nathan     Chimia      F  1995-05-12\n",
       "300010  499986  1952-07-22     Nathan      Ranta      F  1985-08-11\n",
       "\n",
       "[691 rows x 6 columns]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_IN(emp,'first_name', ['Cathie','Mark','Nathan']) # E.g. first name Cathie, Mark or Nathan in the emplyee table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "23c3d94c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_NOT_IN(data,field,entry):\n",
    "    return data[~data[field].isin(entry)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "83df7afc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>1953-09-02</td>\n",
       "      <td>Georgi</td>\n",
       "      <td>Facello</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-06-26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>1964-06-02</td>\n",
       "      <td>Bezalel</td>\n",
       "      <td>Simmel</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>1959-12-03</td>\n",
       "      <td>Parto</td>\n",
       "      <td>Bamford</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-08-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>1954-05-01</td>\n",
       "      <td>Chirstian</td>\n",
       "      <td>Koblick</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10005</td>\n",
       "      <td>1955-01-21</td>\n",
       "      <td>Kyoichi</td>\n",
       "      <td>Maliniak</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-09-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300019</th>\n",
       "      <td>499995</td>\n",
       "      <td>1958-09-24</td>\n",
       "      <td>Dekang</td>\n",
       "      <td>Lichtner</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-01-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300020</th>\n",
       "      <td>499996</td>\n",
       "      <td>1953-03-07</td>\n",
       "      <td>Zito</td>\n",
       "      <td>Baaz</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300021</th>\n",
       "      <td>499997</td>\n",
       "      <td>1961-08-03</td>\n",
       "      <td>Berhard</td>\n",
       "      <td>Lenart</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300022</th>\n",
       "      <td>499998</td>\n",
       "      <td>1956-09-05</td>\n",
       "      <td>Patricia</td>\n",
       "      <td>Breugel</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-10-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300023</th>\n",
       "      <td>499999</td>\n",
       "      <td>1958-05-01</td>\n",
       "      <td>Sachin</td>\n",
       "      <td>Tsukuda</td>\n",
       "      <td>M</td>\n",
       "      <td>1997-11-30;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>299333 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name last_name gender    hire_date\n",
       "0        10001  1953-09-02     Georgi   Facello      M   1986-06-26\n",
       "1        10002  1964-06-02    Bezalel    Simmel      F   1985-11-21\n",
       "2        10003  1959-12-03      Parto   Bamford      M   1986-08-28\n",
       "3        10004  1954-05-01  Chirstian   Koblick      M   1986-12-01\n",
       "4        10005  1955-01-21    Kyoichi  Maliniak      M   1989-09-12\n",
       "...        ...         ...        ...       ...    ...          ...\n",
       "300019  499995  1958-09-24     Dekang  Lichtner      F   1993-01-12\n",
       "300020  499996  1953-03-07       Zito      Baaz      M   1990-09-27\n",
       "300021  499997  1961-08-03    Berhard    Lenart      M   1986-04-21\n",
       "300022  499998  1956-09-05   Patricia   Breugel      M   1993-10-13\n",
       "300023  499999  1958-05-01     Sachin   Tsukuda      M  1997-11-30;\n",
       "\n",
       "[299333 rows x 6 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_NOT_IN(emp,'first_name', ['Cathie','Mark','Nathan']) # E.g. all first names which are not Cathie, Mark or Nathan"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d3b4c54",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='LIKE'>SELECT LIKE/NOT LIKE</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "bd8d8099",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_LIKE(data,field,entry,action):\n",
    "    act = \"print(data[data[field].str.%s(entry)])\" %(action)\n",
    "    exec(act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "3c82469f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date first_name    last_name gender   hire_date\n",
      "231      10232  1956-03-11      Marko        Auria      F  1992-06-04\n",
      "414      10415  1957-11-12       Mark        Coorg      M  1993-10-25\n",
      "521      10522  1955-07-24      Marke     Cesareni      F  1986-12-12\n",
      "1886     11887  1957-05-25      Marke    Kragelund      M  1994-10-17\n",
      "2006     12007  1952-06-18      Marko     Dehkordi      M  1989-05-17\n",
      "...        ...         ...        ...          ...    ...         ...\n",
      "298713  498689  1954-10-23      Marko     Hatcliff      F  1985-06-03\n",
      "298957  498933  1952-05-24      Marke  Grospietsch      F  1996-06-02\n",
      "299004  498980  1957-05-29      Marko     Eppinger      M  1988-07-13\n",
      "299475  499451  1953-04-28      Marko   Goldhammer      M  1986-10-19\n",
      "299644  499620  1961-07-30      Marke         Ibel      F  1996-01-30\n",
      "\n",
      "[690 rows x 6 columns]\n"
     ]
    }
   ],
   "source": [
    "SELECT_LIKE(emp,'first_name', 'Mark','startswith') # E.g. First name starts with 'Mark'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "11437ae4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date  first_name last_name gender   hire_date\n",
      "2        10003  1959-12-03       Parto   Bamford      M  1986-08-28\n",
      "10       10011  1953-11-07        Mary     Sluis      F  1990-01-22\n",
      "12       10013  1963-06-07   Eberhardt    Terkki      M  1985-10-20\n",
      "28       10029  1956-12-13       Otmar    Herbst      M  1985-11-20\n",
      "30       10031  1959-01-27     Karsten    Joslin      M  1991-09-01\n",
      "...        ...         ...         ...       ...    ...         ...\n",
      "299970  499946  1956-08-10      Barton   Soicher      F  1986-02-21\n",
      "299985  499961  1962-10-02     Holgard    Nanard      F  1988-07-04\n",
      "299992  499968  1959-03-07  Dharmaraja      Ertl      M  1991-10-08\n",
      "300001  499977  1956-06-05     Martial   Weisert      F  1996-09-17\n",
      "300021  499997  1961-08-03     Berhard    Lenart      M  1986-04-21\n",
      "\n",
      "[25828 rows x 6 columns]\n"
     ]
    }
   ],
   "source": [
    "SELECT_LIKE(emp,'first_name','ar','contains') # E.g. First name contains 'ar'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "5ee0df0e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date  first_name  last_name gender   hire_date\n",
      "28       10029  1956-12-13       Otmar     Herbst      M  1985-11-20\n",
      "73       10074  1955-08-28     Mokhtar  Bernatsky      F  1990-08-13\n",
      "174      10175  1960-01-11  Aleksandar  Ananiadou      F  1988-01-11\n",
      "265      10266  1958-02-24     Sukumar    Rassart      M  1990-05-25\n",
      "401      10402  1953-07-23     Volkmar    Ebeling      M  1987-01-02\n",
      "...        ...         ...         ...        ...    ...         ...\n",
      "299676  499652  1963-01-16      Oddvar    Ushiama      F  1993-11-27\n",
      "299786  499762  1952-08-24   Subhankar    Munawer      M  1985-07-06\n",
      "299830  499806  1958-10-11      Ingmar  Barbanera      F  1995-04-14\n",
      "299908  499884  1961-03-15     Adhemar       Gist      F  1988-11-19\n",
      "299951  499927  1953-03-18     Manohar  Heemskerk      M  1988-03-13\n",
      "\n",
      "[5320 rows x 6 columns]\n"
     ]
    }
   ],
   "source": [
    "SELECT_LIKE(emp,'first_name','ar','endswith') # E.g. First name ends with 'ar'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "7d4058d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_NOT_LIKE(data,field,entry,action):\n",
    "    act = \"print(data[~data[field].str.%s(entry)])\" %(action)\n",
    "    exec(act)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50e02569",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='BET'>SELECT BETWEEN/NOT BETWEEN</a>** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "feab60e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_BETWEEN(data,field,entry1,entry2,inc):\n",
    "    if inc == \"y\": # INCLUSIVE\n",
    "        return data[(data[field] >= entry1) & (data[field] <= entry2)]\n",
    "    else: # EXCLUSIVE\n",
    "        return data[(data[field] > entry1) & (data[field] < entry2)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "fad57d78",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10008</td>\n",
       "      <td>1958-02-19</td>\n",
       "      <td>Saniya</td>\n",
       "      <td>Kalloufi</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-09-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10011</td>\n",
       "      <td>1953-11-07</td>\n",
       "      <td>Mary</td>\n",
       "      <td>Sluis</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-01-22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10012</td>\n",
       "      <td>1960-10-04</td>\n",
       "      <td>Patricio</td>\n",
       "      <td>Bridgland</td>\n",
       "      <td>M</td>\n",
       "      <td>1992-12-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>10016</td>\n",
       "      <td>1961-05-02</td>\n",
       "      <td>Kazuhito</td>\n",
       "      <td>Cappelletti</td>\n",
       "      <td>M</td>\n",
       "      <td>1995-01-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>10017</td>\n",
       "      <td>1958-07-06</td>\n",
       "      <td>Cristinel</td>\n",
       "      <td>Bouloucos</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-08-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300018</th>\n",
       "      <td>499994</td>\n",
       "      <td>1952-02-26</td>\n",
       "      <td>Navin</td>\n",
       "      <td>Argence</td>\n",
       "      <td>F</td>\n",
       "      <td>1990-04-24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300019</th>\n",
       "      <td>499995</td>\n",
       "      <td>1958-09-24</td>\n",
       "      <td>Dekang</td>\n",
       "      <td>Lichtner</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-01-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300020</th>\n",
       "      <td>499996</td>\n",
       "      <td>1953-03-07</td>\n",
       "      <td>Zito</td>\n",
       "      <td>Baaz</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300022</th>\n",
       "      <td>499998</td>\n",
       "      <td>1956-09-05</td>\n",
       "      <td>Patricia</td>\n",
       "      <td>Breugel</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-10-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300023</th>\n",
       "      <td>499999</td>\n",
       "      <td>1958-05-01</td>\n",
       "      <td>Sachin</td>\n",
       "      <td>Tsukuda</td>\n",
       "      <td>M</td>\n",
       "      <td>1997-11-30;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>135215 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name    last_name gender    hire_date\n",
       "7        10008  1958-02-19     Saniya     Kalloufi      M   1994-09-15\n",
       "10       10011  1953-11-07       Mary        Sluis      F   1990-01-22\n",
       "11       10012  1960-10-04   Patricio    Bridgland      M   1992-12-18\n",
       "15       10016  1961-05-02   Kazuhito  Cappelletti      M   1995-01-27\n",
       "16       10017  1958-07-06  Cristinel    Bouloucos      F   1993-08-03\n",
       "...        ...         ...        ...          ...    ...          ...\n",
       "300018  499994  1952-02-26      Navin      Argence      F   1990-04-24\n",
       "300019  499995  1958-09-24     Dekang     Lichtner      F   1993-01-12\n",
       "300020  499996  1953-03-07       Zito         Baaz      M   1990-09-27\n",
       "300022  499998  1956-09-05   Patricia      Breugel      M   1993-10-13\n",
       "300023  499999  1958-05-01     Sachin      Tsukuda      M  1997-11-30;\n",
       "\n",
       "[135215 rows x 6 columns]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_BETWEEN(emp,'hire_date','1990-01-01','2000-01-01','y') \n",
    "# E.g. All hire dates between 1990-01-01 and 2000-01-01 inclusive"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5549358",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**E.g. assign emplpyee number 110022 as a manger to all employees from 11001 to 11020 and employee number 110039 to all employees from 11021 to 11040** \n",
    "\n",
    "Takes a lot of SQL!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 455,
   "id": "959370a1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    emp_no  birth_date   first_name     last_name gender   hire_date  \\\n",
      "0    11001  1956-04-16      Baziley       Buchter      F  1987-02-23   \n",
      "1    11002  1952-02-26        Bluma      Ulupinar      M  1996-12-23   \n",
      "2    11003  1960-11-13  Mariangiola         Gulla      M  1987-05-24   \n",
      "3    11004  1954-08-05       JoAnna       Decleir      F  1992-01-19   \n",
      "4    11005  1958-03-12        Byong       Douceur      F  1986-07-27   \n",
      "5    11006  1962-12-26   Christoper   Butterworth      F  1989-08-02   \n",
      "6    11007  1962-03-16      Olivera     Maccarone      M  1991-04-11   \n",
      "7    11008  1962-07-11      Gennady      Menhoudj      M  1988-09-18   \n",
      "8    11009  1954-08-30        Alper      Axelband      F  1986-09-09   \n",
      "9    11010  1963-03-20       Jaques      Narwekar      F  1986-12-18   \n",
      "10   11011  1956-06-13         Moti    Hofstetter      M  1989-11-06   \n",
      "11   11012  1953-09-12      Taegyun     Speckmann      F  1986-12-28   \n",
      "12   11013  1961-03-31     Breannda         Vesel      M  1996-09-13   \n",
      "13   11014  1959-09-22       Baoqiu        Vecchi      M  1994-08-23   \n",
      "14   11015  1965-01-24       Jeanna   Riesenhuber      M  1992-05-29   \n",
      "15   11016  1957-05-25        Sibyl     Rajaraman      M  1988-08-10   \n",
      "16   11017  1959-11-23        Jovan       Sanella      F  1992-07-28   \n",
      "17   11018  1960-08-02     Berthier          Luby      F  1988-07-03   \n",
      "18   11019  1954-09-06         Leon        Huttel      M  1985-02-04   \n",
      "19   11020  1957-03-18        Weiwu   Encarnacion      M  1988-03-25   \n",
      "20   11021  1954-07-13     Srinidhi       Henders      F  1985-05-24   \n",
      "21   11022  1964-02-02      Apostol      Pintelas      F  1988-08-01   \n",
      "22   11023  1964-01-08        Yifei      Driscoll      M  1990-06-12   \n",
      "23   11024  1954-06-17       Manton    Collavizza      M  1995-04-05   \n",
      "24   11025  1960-03-21      Heekeun     Delgrange      M  1989-11-06   \n",
      "25   11026  1957-11-24         Heon  Ossenbruggen      M  1991-11-08   \n",
      "26   11027  1955-12-13        Remzi      Cappello      M  1986-05-07   \n",
      "27   11028  1952-02-23       Miyeon        Krider      M  1986-06-18   \n",
      "28   11029  1962-07-12       Georgi      Itzfeldt      M  1992-12-27   \n",
      "29   11030  1956-06-12         Kwan    Hellwagner      F  1988-05-09   \n",
      "30   11031  1954-12-20      Ottavia       Walstra      M  1985-04-26   \n",
      "31   11032  1953-04-19         Erez        Schaaf      M  1992-03-12   \n",
      "32   11033  1957-03-01      Shushma          Bahk      F  1990-10-02   \n",
      "33   11034  1961-10-23         Moti       Shrader      F  1993-01-01   \n",
      "34   11035  1963-09-25     Bangqing      Atrawala      M  1992-10-06   \n",
      "35   11036  1962-09-04      Muzhong     Greenaway      M  1987-09-02   \n",
      "36   11037  1952-05-22      Lansing      Maliniak      M  1988-01-28   \n",
      "37   11038  1955-10-13        Jinya        Beidas      M  1985-07-15   \n",
      "38   11039  1961-12-04      Raimond      Pintelas      F  1993-07-10   \n",
      "39   11040  1961-11-07    Cristinel         Denna      F  1988-01-22   \n",
      "\n",
      "    manager_ID  \n",
      "0       110022  \n",
      "1       110022  \n",
      "2       110022  \n",
      "3       110022  \n",
      "4       110022  \n",
      "5       110022  \n",
      "6       110022  \n",
      "7       110022  \n",
      "8       110022  \n",
      "9       110022  \n",
      "10      110022  \n",
      "11      110022  \n",
      "12      110022  \n",
      "13      110022  \n",
      "14      110022  \n",
      "15      110022  \n",
      "16      110022  \n",
      "17      110022  \n",
      "18      110022  \n",
      "19      110022  \n",
      "20      110039  \n",
      "21      110039  \n",
      "22      110039  \n",
      "23      110039  \n",
      "24      110039  \n",
      "25      110039  \n",
      "26      110039  \n",
      "27      110039  \n",
      "28      110039  \n",
      "29      110039  \n",
      "30      110039  \n",
      "31      110039  \n",
      "32      110039  \n",
      "33      110039  \n",
      "34      110039  \n",
      "35      110039  \n",
      "36      110039  \n",
      "37      110039  \n",
      "38      110039  \n",
      "39      110039  \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/pb/dhr60dvd19370mv4qdxrn34m0000gn/T/ipykernel_67521/1977768428.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  empA = SELECT_BETWEEN(emp,'emp_no',11001,11020,'y'); empA['manager_ID'] = 110022\n",
      "/var/folders/pb/dhr60dvd19370mv4qdxrn34m0000gn/T/ipykernel_67521/1977768428.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  empB = SELECT_BETWEEN(emp,'emp_no',11021,11040,'y'); empB['manager_ID'] = 110039\n"
     ]
    }
   ],
   "source": [
    "empA = SELECT_BETWEEN(emp,'emp_no',11001,11020,'y'); empA['manager_ID'] = 110022\n",
    "empB = SELECT_BETWEEN(emp,'emp_no',11021,11040,'y'); empB['manager_ID'] = 110039\n",
    "print(pd.concat([empA, empB], ignore_index=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "ef54d621",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_NOT_BETWEEN(data,field,entry1,entry2,inc):\n",
    "    if inc == \"y\":\n",
    "        return data[(data[field] <= entry1) | (data[field] >= entry2)]\n",
    "    else: \n",
    "        return data[(data[field] < entry1) | (data[field] > entry2)]    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "020fa565",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>1953-09-02</td>\n",
       "      <td>Georgi</td>\n",
       "      <td>Facello</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-06-26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>1964-06-02</td>\n",
       "      <td>Bezalel</td>\n",
       "      <td>Simmel</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>1959-12-03</td>\n",
       "      <td>Parto</td>\n",
       "      <td>Bamford</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-08-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>1954-05-01</td>\n",
       "      <td>Chirstian</td>\n",
       "      <td>Koblick</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10005</td>\n",
       "      <td>1955-01-21</td>\n",
       "      <td>Kyoichi</td>\n",
       "      <td>Maliniak</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-09-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300012</th>\n",
       "      <td>499988</td>\n",
       "      <td>1962-09-28</td>\n",
       "      <td>Bangqing</td>\n",
       "      <td>Kleiser</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-06-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300014</th>\n",
       "      <td>499990</td>\n",
       "      <td>1963-11-03</td>\n",
       "      <td>Khaled</td>\n",
       "      <td>Kohling</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-10-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300015</th>\n",
       "      <td>499991</td>\n",
       "      <td>1962-02-26</td>\n",
       "      <td>Pohua</td>\n",
       "      <td>Sichman</td>\n",
       "      <td>F</td>\n",
       "      <td>1989-01-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300016</th>\n",
       "      <td>499992</td>\n",
       "      <td>1960-10-12</td>\n",
       "      <td>Siamak</td>\n",
       "      <td>Salverda</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-05-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300021</th>\n",
       "      <td>499997</td>\n",
       "      <td>1961-08-03</td>\n",
       "      <td>Berhard</td>\n",
       "      <td>Lenart</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-21</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>164809 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name last_name gender   hire_date\n",
       "0        10001  1953-09-02     Georgi   Facello      M  1986-06-26\n",
       "1        10002  1964-06-02    Bezalel    Simmel      F  1985-11-21\n",
       "2        10003  1959-12-03      Parto   Bamford      M  1986-08-28\n",
       "3        10004  1954-05-01  Chirstian   Koblick      M  1986-12-01\n",
       "4        10005  1955-01-21    Kyoichi  Maliniak      M  1989-09-12\n",
       "...        ...         ...        ...       ...    ...         ...\n",
       "300012  499988  1962-09-28   Bangqing   Kleiser      F  1986-06-06\n",
       "300014  499990  1963-11-03     Khaled   Kohling      M  1985-10-10\n",
       "300015  499991  1962-02-26      Pohua   Sichman      F  1989-01-12\n",
       "300016  499992  1960-10-12     Siamak  Salverda      F  1987-05-10\n",
       "300021  499997  1961-08-03    Berhard    Lenart      M  1986-04-21\n",
       "\n",
       "[164809 rows x 6 columns]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_NOT_BETWEEN(emp,'hire_date','1990-01-01','2000-01-01','n')\n",
    "# E.g. All hire dates note between 1990-01-01 and 2000-01-01 exclusive"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4ce592f",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='NULL'>SELECT IS NULL/NOT NULL</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "cbf1cb08",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_NULL(data,field):\n",
    "    return data[data[field] != data[field]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "d9c9c530",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [emp_no, birth_date, first_name, last_name, gender, hire_date]\n",
       "Index: []"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_NULL(emp,'first_name') # E.g. extract rows where the first name is null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "c90dcae5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_NOT_NULL(data,field):\n",
    "    return data[data[field] == data[field]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "38bd96e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>1953-09-02</td>\n",
       "      <td>Georgi</td>\n",
       "      <td>Facello</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-06-26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>1964-06-02</td>\n",
       "      <td>Bezalel</td>\n",
       "      <td>Simmel</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>1959-12-03</td>\n",
       "      <td>Parto</td>\n",
       "      <td>Bamford</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-08-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>1954-05-01</td>\n",
       "      <td>Chirstian</td>\n",
       "      <td>Koblick</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10005</td>\n",
       "      <td>1955-01-21</td>\n",
       "      <td>Kyoichi</td>\n",
       "      <td>Maliniak</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-09-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300019</th>\n",
       "      <td>499995</td>\n",
       "      <td>1958-09-24</td>\n",
       "      <td>Dekang</td>\n",
       "      <td>Lichtner</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-01-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300020</th>\n",
       "      <td>499996</td>\n",
       "      <td>1953-03-07</td>\n",
       "      <td>Zito</td>\n",
       "      <td>Baaz</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300021</th>\n",
       "      <td>499997</td>\n",
       "      <td>1961-08-03</td>\n",
       "      <td>Berhard</td>\n",
       "      <td>Lenart</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300022</th>\n",
       "      <td>499998</td>\n",
       "      <td>1956-09-05</td>\n",
       "      <td>Patricia</td>\n",
       "      <td>Breugel</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-10-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300023</th>\n",
       "      <td>499999</td>\n",
       "      <td>1958-05-01</td>\n",
       "      <td>Sachin</td>\n",
       "      <td>Tsukuda</td>\n",
       "      <td>M</td>\n",
       "      <td>1997-11-30;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>300024 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name last_name gender    hire_date\n",
       "0        10001  1953-09-02     Georgi   Facello      M   1986-06-26\n",
       "1        10002  1964-06-02    Bezalel    Simmel      F   1985-11-21\n",
       "2        10003  1959-12-03      Parto   Bamford      M   1986-08-28\n",
       "3        10004  1954-05-01  Chirstian   Koblick      M   1986-12-01\n",
       "4        10005  1955-01-21    Kyoichi  Maliniak      M   1989-09-12\n",
       "...        ...         ...        ...       ...    ...          ...\n",
       "300019  499995  1958-09-24     Dekang  Lichtner      F   1993-01-12\n",
       "300020  499996  1953-03-07       Zito      Baaz      M   1990-09-27\n",
       "300021  499997  1961-08-03    Berhard    Lenart      M   1986-04-21\n",
       "300022  499998  1956-09-05   Patricia   Breugel      M   1993-10-13\n",
       "300023  499999  1958-05-01     Sachin   Tsukuda      M  1997-11-30;\n",
       "\n",
       "[300024 rows x 6 columns]"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_NOT_NULL(emp,'first_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56a44b30",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='DIST'>DISTINCT</a>** \n",
    "\n",
    "**Show all uniques entries in a field**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0e765642",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SELECT_DISTINCT(data,field): \n",
    "    return data[field].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "af1537ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['M', 'F'], dtype=object)"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SELECT_DISTINCT(emp,'gender')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11b05cf3",
   "metadata": {},
   "source": [
    "**<a id='FIL'>FILTER</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "79de4683",
   "metadata": {},
   "outputs": [],
   "source": [
    "def FILTER(data,field,entry,action):\n",
    "    if action == \">\":\n",
    "        tmp = data[data[field] > entry]\n",
    "    elif action == \">=\":\n",
    "        tmp = data[data[field] >= entry]\n",
    "    elif action == \"<\":\n",
    "        tmp = data[data[field] < entry]\n",
    "    elif action == \"<=\":\n",
    "        tmp = data[data[field] <= entry]\n",
    "    else:\n",
    "        tmp = data[data[field] == entry]   \n",
    "    return tmp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "14f5cbf6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>37290</th>\n",
       "      <td>47291</td>\n",
       "      <td>1960-09-09</td>\n",
       "      <td>Ulf</td>\n",
       "      <td>Flexer</td>\n",
       "      <td>M</td>\n",
       "      <td>2000-01-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50133</th>\n",
       "      <td>60134</td>\n",
       "      <td>1964-04-21</td>\n",
       "      <td>Seshu</td>\n",
       "      <td>Rathonyi</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62328</th>\n",
       "      <td>72329</td>\n",
       "      <td>1953-02-09</td>\n",
       "      <td>Randi</td>\n",
       "      <td>Luit</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105072</th>\n",
       "      <td>205048</td>\n",
       "      <td>1960-09-12</td>\n",
       "      <td>Ennio</td>\n",
       "      <td>Alblas</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122989</th>\n",
       "      <td>222965</td>\n",
       "      <td>1959-08-07</td>\n",
       "      <td>Volkmar</td>\n",
       "      <td>Perko</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126657</th>\n",
       "      <td>226633</td>\n",
       "      <td>1958-06-10</td>\n",
       "      <td>Xuejun</td>\n",
       "      <td>Benzmuller</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127568</th>\n",
       "      <td>227544</td>\n",
       "      <td>1954-11-17</td>\n",
       "      <td>Shahab</td>\n",
       "      <td>Demeyer</td>\n",
       "      <td>M</td>\n",
       "      <td>2000-01-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>223014</th>\n",
       "      <td>422990</td>\n",
       "      <td>1953-04-09</td>\n",
       "      <td>Jaana</td>\n",
       "      <td>Verspoor</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>224469</th>\n",
       "      <td>424445</td>\n",
       "      <td>1953-04-27</td>\n",
       "      <td>Jeong</td>\n",
       "      <td>Boreale</td>\n",
       "      <td>M</td>\n",
       "      <td>2000-01-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>228401</th>\n",
       "      <td>428377</td>\n",
       "      <td>1957-05-09</td>\n",
       "      <td>Yucai</td>\n",
       "      <td>Gerlach</td>\n",
       "      <td>M</td>\n",
       "      <td>2000-01-23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>263831</th>\n",
       "      <td>463807</td>\n",
       "      <td>1964-06-12</td>\n",
       "      <td>Bikash</td>\n",
       "      <td>Covnot</td>\n",
       "      <td>M</td>\n",
       "      <td>2000-01-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299577</th>\n",
       "      <td>499553</td>\n",
       "      <td>1954-05-06</td>\n",
       "      <td>Hideyuki</td>\n",
       "      <td>Delgrande</td>\n",
       "      <td>F</td>\n",
       "      <td>2000-01-22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name   last_name gender   hire_date\n",
       "37290    47291  1960-09-09        Ulf      Flexer      M  2000-01-12\n",
       "50133    60134  1964-04-21      Seshu    Rathonyi      F  2000-01-02\n",
       "62328    72329  1953-02-09      Randi        Luit      F  2000-01-02\n",
       "105072  205048  1960-09-12      Ennio      Alblas      F  2000-01-06\n",
       "122989  222965  1959-08-07    Volkmar       Perko      F  2000-01-13\n",
       "126657  226633  1958-06-10     Xuejun  Benzmuller      F  2000-01-04\n",
       "127568  227544  1954-11-17     Shahab     Demeyer      M  2000-01-08\n",
       "223014  422990  1953-04-09      Jaana    Verspoor      F  2000-01-11\n",
       "224469  424445  1953-04-27      Jeong     Boreale      M  2000-01-03\n",
       "228401  428377  1957-05-09      Yucai     Gerlach      M  2000-01-23\n",
       "263831  463807  1964-06-12     Bikash      Covnot      M  2000-01-28\n",
       "299577  499553  1954-05-06   Hideyuki   Delgrande      F  2000-01-22"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FILTER(emp,'hire_date','2000-01-01','>') # Just a general filter, e,g employess who started after 2000-01-01"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffad47e5",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00408c84",
   "metadata": {},
   "source": [
    "# AGGREGATE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b87167fa",
   "metadata": {},
   "source": [
    "**<a id='COUNT'>COUNT</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "7e72e49e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def COUNT(data,field,unique):\n",
    "    if unique == \"y\":\n",
    "        return len(data[field].unique())\n",
    "    else:    \n",
    "        return data[field].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "29e68094",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1275"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "COUNT(emp,'first_name','y') # E.g. count the number of unique first names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "id": "17d747b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "300024"
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "COUNT(emp,'first_name','n') # E.g. count the number of first names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f57823a1",
   "metadata": {},
   "source": [
    "**COUNT with filter**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "id": "51defa90",
   "metadata": {},
   "outputs": [],
   "source": [
    "def FILTER_COUNT(data,field,entry,action): \n",
    "    act = \"print(data[data[field] %s entry].count())\" %(action)\n",
    "    exec(act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "id": "34016c79",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "emp_no       16\n",
      "salary       16\n",
      "from_date    16\n",
      "to_date      16\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "FILTER_COUNT(sal,'salary',150000,'>') # E.g. count no of salaries greater than 150,000"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8097d57e",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='AGG'>SUM, AVG, MEDIAN, MIN or MAX</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "af826b8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def AGG(data,field,action):\n",
    "    #act = \"print(data[field].%s(()))\" %(action); exec(act) # NOT WORKING IN NOTEBOOK\n",
    "    if action == \"sum\":\n",
    "        return data[field].sum()\n",
    "    elif action == \"ave\":\n",
    "        return data[field].mean()\n",
    "    elif action == \"median\":\n",
    "        return data[field].median()\n",
    "    elif action == \"min\":\n",
    "        return data[field].min()\n",
    "    elif action == \"max\":\n",
    "        return data[field].max() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "81596469",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "63761.2043294429"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "AGG(sal,'salary','ave') # E.g. the average salary from the 'sal' table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 465,
   "id": "6ac1cc4c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "61070.0"
      ]
     },
     "execution_count": 465,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "AGG(sal,'salary','median') # E.g. the median salary from the 'sal' table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 400,
   "id": "03cad412",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "70239.19785541651\n"
     ]
    }
   ],
   "source": [
    "# E.g. #What is the average annual salary paid to employees who started after the 1st of January 2000?\n",
    "tmp = sal[sal['from_date'] > '2000-01-01']\n",
    "df = AGG(tmp,'salary','ave'); \n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a40c219d",
   "metadata": {},
   "source": [
    "# ORDER and GROUP"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23fc8634",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    " \n",
    "**<a id='OB'>ORDER BY</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "id": "1741f109",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ORDER_BY(data,field,ascending):\n",
    "    return data.sort_values(by=field, ascending=ascending)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "id": "c33b8cf6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>290185</th>\n",
       "      <td>490161</td>\n",
       "      <td>1963-01-18</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Rubsam</td>\n",
       "      <td>F</td>\n",
       "      <td>1996-12-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84466</th>\n",
       "      <td>94467</td>\n",
       "      <td>1963-04-24</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Benantar</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-01-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93640</th>\n",
       "      <td>103641</td>\n",
       "      <td>1955-11-11</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Loncour</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-06-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230575</th>\n",
       "      <td>430551</td>\n",
       "      <td>1960-10-22</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Krogh</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-07-29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120795</th>\n",
       "      <td>220771</td>\n",
       "      <td>1956-12-17</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Engberts</td>\n",
       "      <td>M</td>\n",
       "      <td>1998-05-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>275773</th>\n",
       "      <td>475749</td>\n",
       "      <td>1963-01-09</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Bahr</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-03-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96764</th>\n",
       "      <td>106765</td>\n",
       "      <td>1962-05-17</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Salvesen</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-12-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143716</th>\n",
       "      <td>243692</td>\n",
       "      <td>1961-04-19</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Antonisse</td>\n",
       "      <td>F</td>\n",
       "      <td>1994-05-22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132055</th>\n",
       "      <td>232031</td>\n",
       "      <td>1952-03-31</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Alpay</td>\n",
       "      <td>F</td>\n",
       "      <td>1987-02-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30998</th>\n",
       "      <td>40999</td>\n",
       "      <td>1963-08-12</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Cmelik</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-11-13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>300024 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name  last_name gender   hire_date\n",
       "290185  490161  1963-01-18      Aamer     Rubsam      F  1996-12-08\n",
       "84466    94467  1963-04-24      Aamer   Benantar      F  1987-01-28\n",
       "93640   103641  1955-11-11      Aamer    Loncour      M  1987-06-14\n",
       "230575  430551  1960-10-22      Aamer      Krogh      M  1987-07-29\n",
       "120795  220771  1956-12-17      Aamer   Engberts      M  1998-05-14\n",
       "...        ...         ...        ...        ...    ...         ...\n",
       "275773  475749  1963-01-09     Zvonko       Bahr      M  1994-03-05\n",
       "96764   106765  1962-05-17     Zvonko   Salvesen      F  1987-12-11\n",
       "143716  243692  1961-04-19     Zvonko  Antonisse      F  1994-05-22\n",
       "132055  232031  1952-03-31     Zvonko      Alpay      F  1987-02-21\n",
       "30998    40999  1963-08-12     Zvonko     Cmelik      M  1986-11-13\n",
       "\n",
       "[300024 rows x 6 columns]"
      ]
     },
     "execution_count": 258,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ORDER_BY(emp,'first_name',True) # E.g. order employee table in by primary field in ascending order  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "id": "bfcc58e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>59255</th>\n",
       "      <td>69256</td>\n",
       "      <td>1962-04-14</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Anger</td>\n",
       "      <td>M</td>\n",
       "      <td>1998-03-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>286608</th>\n",
       "      <td>486584</td>\n",
       "      <td>1952-08-12</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Armand</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137189</th>\n",
       "      <td>237165</td>\n",
       "      <td>1962-02-23</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Azevdeo</td>\n",
       "      <td>F</td>\n",
       "      <td>1991-06-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>213712</th>\n",
       "      <td>413688</td>\n",
       "      <td>1955-06-26</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Azuma</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-12-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>181387</th>\n",
       "      <td>281363</td>\n",
       "      <td>1956-05-18</td>\n",
       "      <td>Aamer</td>\n",
       "      <td>Baak</td>\n",
       "      <td>F</td>\n",
       "      <td>1994-03-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>270706</th>\n",
       "      <td>470682</td>\n",
       "      <td>1960-03-24</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Yoshizawa</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-03-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130730</th>\n",
       "      <td>230706</td>\n",
       "      <td>1961-07-06</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Yurov</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-09-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99488</th>\n",
       "      <td>109489</td>\n",
       "      <td>1962-11-09</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Zambonelli</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-04-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>182120</th>\n",
       "      <td>282096</td>\n",
       "      <td>1953-04-13</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Zobel</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-04-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147573</th>\n",
       "      <td>247549</td>\n",
       "      <td>1960-01-07</td>\n",
       "      <td>Zvonko</td>\n",
       "      <td>Zuberek</td>\n",
       "      <td>F</td>\n",
       "      <td>1996-06-13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>300024 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name   last_name gender   hire_date\n",
       "59255    69256  1962-04-14      Aamer       Anger      M  1998-03-16\n",
       "286608  486584  1952-08-12      Aamer      Armand      M  1990-09-15\n",
       "137189  237165  1962-02-23      Aamer     Azevdeo      F  1991-06-28\n",
       "213712  413688  1955-06-26      Aamer       Azuma      M  1989-12-10\n",
       "181387  281363  1956-05-18      Aamer        Baak      F  1994-03-10\n",
       "...        ...         ...        ...         ...    ...         ...\n",
       "270706  470682  1960-03-24     Zvonko   Yoshizawa      M  1987-03-12\n",
       "130730  230706  1961-07-06     Zvonko       Yurov      M  1989-09-08\n",
       "99488   109489  1962-11-09     Zvonko  Zambonelli      M  1987-04-11\n",
       "182120  282096  1953-04-13     Zvonko       Zobel      M  1987-04-13\n",
       "147573  247549  1960-01-07     Zvonko     Zuberek      F  1996-06-13\n",
       "\n",
       "[300024 rows x 6 columns]"
      ]
     },
     "execution_count": 262,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ORDER_BY(emp, ['first_name','last_name'],True) \n",
    "# E.g. order employee table in by primary and secondary fields in ascending order "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "094d6ae8",
   "metadata": {},
   "source": [
    "**<a id='GC'>GROUP COUNT</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "2b74dfb7",
   "metadata": {},
   "outputs": [],
   "source": [
    "def GROUP_COUNT(data,field):\n",
    "    return data.groupby(field).agg(Count=(field,'count')).reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "id": "cddd7d38",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first_name</th>\n",
       "      <th>Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Aamer</td>\n",
       "      <td>228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aamod</td>\n",
       "      <td>216</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Abdelaziz</td>\n",
       "      <td>227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Abdelghani</td>\n",
       "      <td>247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Abdelkader</td>\n",
       "      <td>222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1270</th>\n",
       "      <td>Ziyad</td>\n",
       "      <td>229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1271</th>\n",
       "      <td>Zongyan</td>\n",
       "      <td>254</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1272</th>\n",
       "      <td>Zorica</td>\n",
       "      <td>225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1273</th>\n",
       "      <td>Zsolt</td>\n",
       "      <td>236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1274</th>\n",
       "      <td>Zvonko</td>\n",
       "      <td>258</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1275 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      first_name  Count\n",
       "0          Aamer    228\n",
       "1          Aamod    216\n",
       "2      Abdelaziz    227\n",
       "3     Abdelghani    247\n",
       "4     Abdelkader    222\n",
       "...          ...    ...\n",
       "1270       Ziyad    229\n",
       "1271     Zongyan    254\n",
       "1272      Zorica    225\n",
       "1273       Zsolt    236\n",
       "1274      Zvonko    258\n",
       "\n",
       "[1275 rows x 2 columns]"
      ]
     },
     "execution_count": 271,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GROUP_COUNT(emp,'first_name') # E.g. count the employees with the same first names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 389,
   "id": "08e1df93",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  Count\n",
      "9        10010      2\n",
      "17       10018      2\n",
      "28       10029      2\n",
      "39       10040      2\n",
      "49       10050      2\n",
      "...        ...    ...\n",
      "299979  499955      2\n",
      "299988  499964      2\n",
      "299999  499975      2\n",
      "300007  499983      2\n",
      "300016  499992      2\n",
      "\n",
      "[31579 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "# E.g. Select the employee numbers of all individuals who have signed more than 1 contract after the 1st of January 2000.\n",
    "tmp = GROUP_COUNT(dept_emp,'emp_no')\n",
    "tmp2 = tmp[tmp['Count'] >1];\n",
    "print(tmp2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1e411e8",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)   \n",
    "\n",
    "**<a id='GA'>GROUP AGG</a>**\n",
    "\n",
    "**Aggregate functions on groups**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "0ece8117",
   "metadata": {},
   "outputs": [],
   "source": [
    "def GROUP_AGG(data,field,sum_field,action):\n",
    "    astring = 'data.groupby(field).%s(numeric_only=True)' %(action)\n",
    "    return eval(astring)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "40cc306a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>emp_no</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10001</th>\n",
       "      <td>75388.941176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10002</th>\n",
       "      <td>68854.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10003</th>\n",
       "      <td>43030.285714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10004</th>\n",
       "      <td>56512.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10005</th>\n",
       "      <td>87275.769231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201767</th>\n",
       "      <td>47097.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201768</th>\n",
       "      <td>39818.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201769</th>\n",
       "      <td>54001.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201770</th>\n",
       "      <td>75524.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201771</th>\n",
       "      <td>54893.153846</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101796 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              salary\n",
       "emp_no              \n",
       "10001   75388.941176\n",
       "10002   68854.500000\n",
       "10003   43030.285714\n",
       "10004   56512.250000\n",
       "10005   87275.769231\n",
       "...              ...\n",
       "201767  47097.000000\n",
       "201768  39818.000000\n",
       "201769  54001.500000\n",
       "201770  75524.100000\n",
       "201771  54893.153846\n",
       "\n",
       "[101796 rows x 1 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GROUP_AGG(sal,'emp_no','salary','mean') # E.g. the mean salary by employee number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 470,
   "id": "87885e7d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>emp_no</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10001</th>\n",
       "      <td>88958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10002</th>\n",
       "      <td>72527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10003</th>\n",
       "      <td>43699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10004</th>\n",
       "      <td>74057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10005</th>\n",
       "      <td>94692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201767</th>\n",
       "      <td>47097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201768</th>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201769</th>\n",
       "      <td>62287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201770</th>\n",
       "      <td>85571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201771</th>\n",
       "      <td>66792</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101796 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        salary\n",
       "emp_no        \n",
       "10001    88958\n",
       "10002    72527\n",
       "10003    43699\n",
       "10004    74057\n",
       "10005    94692\n",
       "...        ...\n",
       "201767   47097\n",
       "201768   40000\n",
       "201769   62287\n",
       "201770   85571\n",
       "201771   66792\n",
       "\n",
       "[101796 rows x 1 columns]"
      ]
     },
     "execution_count": 470,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GROUP_AGG(sal,'emp_no','salary','max') # E.g. the maximum salary by employee number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a91c695",
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp = GROUP_AGG(sal,'emp_no','salary','mean') # E.g. employees show average salary exceeds 120,000\n",
    "FILTER(tmp,'salary',120000,'>') "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b6b35b8",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Grouping by secondary field and aggregating on another**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2a1c9492",
   "metadata": {},
   "outputs": [],
   "source": [
    "def GROUP_AGG2(data,field1,field2,agg_field,agg):\n",
    "    return data.groupby([field1,field2]).agg(Count=(agg_field,agg)).reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f00743f4",
   "metadata": {},
   "source": [
    "**E.g. the mean of all salaries grouped by the start date. This starts by joining the employee and salary tables ([the function is below](#INNER))**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "80fc95b1",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Start year</th>\n",
       "      <th>Start month</th>\n",
       "      <th>Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1985</td>\n",
       "      <td>01</td>\n",
       "      <td>72435.533333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1985</td>\n",
       "      <td>02</td>\n",
       "      <td>67238.492723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1985</td>\n",
       "      <td>03</td>\n",
       "      <td>66867.920290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1985</td>\n",
       "      <td>04</td>\n",
       "      <td>67261.052013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1985</td>\n",
       "      <td>05</td>\n",
       "      <td>66854.962368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>1999</td>\n",
       "      <td>09</td>\n",
       "      <td>52233.294872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>1999</td>\n",
       "      <td>10</td>\n",
       "      <td>53447.741573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>178</th>\n",
       "      <td>1999</td>\n",
       "      <td>11</td>\n",
       "      <td>56785.260000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>179</th>\n",
       "      <td>1999</td>\n",
       "      <td>12</td>\n",
       "      <td>52060.694444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>180</th>\n",
       "      <td>2000</td>\n",
       "      <td>01</td>\n",
       "      <td>52896.090909</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>181 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    Start year Start month         Count\n",
       "0         1985          01  72435.533333\n",
       "1         1985          02  67238.492723\n",
       "2         1985          03  66867.920290\n",
       "3         1985          04  67261.052013\n",
       "4         1985          05  66854.962368\n",
       "..         ...         ...           ...\n",
       "176       1999          09  52233.294872\n",
       "177       1999          10  53447.741573\n",
       "178       1999          11  56785.260000\n",
       "179       1999          12  52060.694444\n",
       "180       2000          01  52896.090909\n",
       "\n",
       "[181 rows x 3 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = pd.merge(emp,sal,on = 'emp_no', how = 'left') \n",
    "tmp['Start year'] = tmp['hire_date'].str[0:4]\n",
    "tmp['Start month'] = tmp['hire_date'].str[5:7]\n",
    "GROUP_AGG2(tmp,'Start year','Start month','salary','mean') #'sum' in last field will give the total"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd98144b",
   "metadata": {},
   "source": [
    "**<a id='HA'>HAVING</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7a8c09f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "def HAVING (data,field,action,value):\n",
    "      grouped = data.groupby(field).agg(Count=(field,'count'))\n",
    "      act = \"print(grouped[grouped['Count'] %s value])\" %(action)\n",
    "      exec(act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3822ae1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            Count\n",
      "first_name       \n",
      "Adam          251\n",
      "Akemi         259\n",
      "Anyuan        278\n",
      "Arie          255\n",
      "Arno          261\n",
      "...           ...\n",
      "Yurij         266\n",
      "Zhenhua       257\n",
      "Zito          251\n",
      "Zongyan       254\n",
      "Zvonko        258\n",
      "\n",
      "[193 rows x 1 columns]\n"
     ]
    }
   ],
   "source": [
    "HAVING(emp,'first_name','>',250) # E.g. Employees in which over 250 have the same first name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "031b290a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def SAME_COUNT(data,field,action,value):\n",
    "    act = \"print(data[data[field] %s value].groupby(field).agg(same=(field,'count')).reset_index())\" %(action)\n",
    "    exec(act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3388a1b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       salary  same\n",
      "0       80000    14\n",
      "1       80001    12\n",
      "2       80002    10\n",
      "3       80003    10\n",
      "4       80004    13\n",
      "...       ...   ...\n",
      "36613  155190     1\n",
      "36614  155377     1\n",
      "36615  155709     1\n",
      "36616  157821     1\n",
      "36617  158220     1\n",
      "\n",
      "[36618 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "SAME_COUNT(sal,'salary', '>=', 80000) \n",
    "# E.g. count the number of employees with the same salary where this is greater or euqal to 80,0000"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a4e18ba",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) \n",
    "\n",
    "**<a id='DUP'>DUPLICATES</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "id": "5cf532f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def DROP_DUPS(table): # Will drop duplicate rows keeping the first\n",
    "    table['duplicated'] = table.duplicated(); print(table)\n",
    "    del table['duplicated']\n",
    "    return table.drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "id": "7562d195",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    emp_no  birth_date first_name  last_name gender   hire_date  duplicated\n",
      "0    10001  1953-09-02     Georgi    Facello      M  1986-06-26       False\n",
      "1    10002  1964-06-02    Bezalel     Simmel      F  1985-11-21       False\n",
      "2    10003  1959-12-03      Parto    Bamford      M  1986-08-28       False\n",
      "3    10004  1954-05-01  Chirstian    Koblick      M  1986-12-01       False\n",
      "4    10005  1955-01-21    Kyoichi   Maliniak      M  1989-09-12       False\n",
      "5    10006  1953-04-20     Anneke    Preusig      F  1989-06-02       False\n",
      "6    10007  1957-05-23    Tzvetan  Zielinski      F  1989-02-10       False\n",
      "7    10008  1958-02-19     Saniya   Kalloufi      M  1994-09-15       False\n",
      "8    10009  1952-04-19     Sumant       Peac      F  1985-02-18       False\n",
      "9    10010  1963-06-01  Duangkaew   Piveteau      F  1989-08-24       False\n",
      "10   10001  1953-09-02     Georgi    Facello      M  1986-06-26        True\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/pb/dhr60dvd19370mv4qdxrn34m0000gn/T/ipykernel_67521/526296586.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  emp_dup = emp.head(10); emp_dup.loc[len(emp_dup)] = emp_dup.loc[0];\n",
      "/var/folders/pb/dhr60dvd19370mv4qdxrn34m0000gn/T/ipykernel_67521/25059225.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  table['duplicated'] = table.duplicated(); print(table)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>1953-09-02</td>\n",
       "      <td>Georgi</td>\n",
       "      <td>Facello</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-06-26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>1964-06-02</td>\n",
       "      <td>Bezalel</td>\n",
       "      <td>Simmel</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>1959-12-03</td>\n",
       "      <td>Parto</td>\n",
       "      <td>Bamford</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-08-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>1954-05-01</td>\n",
       "      <td>Chirstian</td>\n",
       "      <td>Koblick</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10005</td>\n",
       "      <td>1955-01-21</td>\n",
       "      <td>Kyoichi</td>\n",
       "      <td>Maliniak</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-09-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10006</td>\n",
       "      <td>1953-04-20</td>\n",
       "      <td>Anneke</td>\n",
       "      <td>Preusig</td>\n",
       "      <td>F</td>\n",
       "      <td>1989-06-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10007</td>\n",
       "      <td>1957-05-23</td>\n",
       "      <td>Tzvetan</td>\n",
       "      <td>Zielinski</td>\n",
       "      <td>F</td>\n",
       "      <td>1989-02-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10008</td>\n",
       "      <td>1958-02-19</td>\n",
       "      <td>Saniya</td>\n",
       "      <td>Kalloufi</td>\n",
       "      <td>M</td>\n",
       "      <td>1994-09-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10009</td>\n",
       "      <td>1952-04-19</td>\n",
       "      <td>Sumant</td>\n",
       "      <td>Peac</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-02-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10010</td>\n",
       "      <td>1963-06-01</td>\n",
       "      <td>Duangkaew</td>\n",
       "      <td>Piveteau</td>\n",
       "      <td>F</td>\n",
       "      <td>1989-08-24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   emp_no  birth_date first_name  last_name gender   hire_date\n",
       "0   10001  1953-09-02     Georgi    Facello      M  1986-06-26\n",
       "1   10002  1964-06-02    Bezalel     Simmel      F  1985-11-21\n",
       "2   10003  1959-12-03      Parto    Bamford      M  1986-08-28\n",
       "3   10004  1954-05-01  Chirstian    Koblick      M  1986-12-01\n",
       "4   10005  1955-01-21    Kyoichi   Maliniak      M  1989-09-12\n",
       "5   10006  1953-04-20     Anneke    Preusig      F  1989-06-02\n",
       "6   10007  1957-05-23    Tzvetan  Zielinski      F  1989-02-10\n",
       "7   10008  1958-02-19     Saniya   Kalloufi      M  1994-09-15\n",
       "8   10009  1952-04-19     Sumant       Peac      F  1985-02-18\n",
       "9   10010  1963-06-01  Duangkaew   Piveteau      F  1989-08-24"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Testing - copy first 10 rows of emp table and copy first row to the end\n",
    "emp_dup = emp.head(10); emp_dup.loc[len(emp_dup)] = emp_dup.loc[0];\n",
    "DROP_DUPS(emp_dup)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a231a9a9",
   "metadata": {},
   "source": [
    "# JOINS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5abc0b34",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) \n",
    "\n",
    "**<a id='INNER'>INNER</a>**\n",
    "\n",
    "**The intersection of two tables**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fe585557",
   "metadata": {},
   "outputs": [],
   "source": [
    "def JOIN(table1,table2,on,how):\n",
    "    return pd.merge(table1,table2,on = [on], how = how)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 460,
   "id": "dba7659b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    emp_no  birth_date   first_name     last_name gender   hire_date dept_no  \\\n",
      "0   110022  1956-09-12    Margareta    Markovitch      M  1985-01-01    d001   \n",
      "1   110039  1963-06-21     Vishwani      Minakawa      M  1986-04-12    d001   \n",
      "2   110085  1959-10-28         Ebru         Alpin      M  1985-01-01    d002   \n",
      "3   110114  1957-03-28        Isamu    Legleitner      F  1985-01-14    d002   \n",
      "4   110183  1953-06-24      Shirish  Ossenbruggen      F  1985-01-01    d003   \n",
      "5   110228  1958-12-02      Karsten       Sigstam      F  1985-08-04    d003   \n",
      "6   110303  1956-06-08    Krassimir       Wegerle      F  1985-01-01    d004   \n",
      "7   110344  1961-09-07       Rosine         Cools      F  1985-11-22    d004   \n",
      "8   110386  1953-10-04         Shem        Kieras      M  1988-10-14    d004   \n",
      "9   110420  1963-07-27        Oscar      Ghazalie      M  1992-02-05    d004   \n",
      "10  110511  1957-07-08     DeForest      Hagimont      M  1985-01-01    d005   \n",
      "11  110567  1964-04-25         Leon      DasSarma      F  1986-10-21    d005   \n",
      "12  110725  1961-03-14    Peternela       Onuegbe      F  1985-01-01    d006   \n",
      "13  110765  1954-05-22       Rutger       Hofmeyr      F  1989-01-07    d006   \n",
      "14  110800  1963-02-07       Sanjoy       Quadeer      F  1986-08-12    d006   \n",
      "15  110854  1960-08-19         Dung         Pesch      M  1989-06-09    d006   \n",
      "16  111035  1962-02-24  Przemyslawa     Kaelbling      M  1985-01-01    d007   \n",
      "17  111133  1955-03-16        Hauke         Zhang      M  1986-12-30    d007   \n",
      "18  111400  1959-11-09         Arie       Staelin      M  1985-01-01    d008   \n",
      "19  111534  1952-06-27       Hilary        Kambil      F  1988-01-31    d008   \n",
      "20  111692  1954-10-05        Tonny   Butterworth      F  1985-01-01    d009   \n",
      "21  111784  1956-06-14        Marjo    Giarratana      F  1988-02-12    d009   \n",
      "22  111877  1962-10-18      Xiaobin      Spinelli      F  1991-08-17    d009   \n",
      "23  111939  1960-03-25      Yuchang       Weedman      M  1989-07-10    d009   \n",
      "\n",
      "     from_date      to_date key  \n",
      "0   1985-01-01   1991-10-01   0  \n",
      "1   1991-10-01   9999-01-01   0  \n",
      "2   1985-01-01   1989-12-17   0  \n",
      "3   1989-12-17   9999-01-01   0  \n",
      "4   1985-01-01   1992-03-21   0  \n",
      "5   1992-03-21   9999-01-01   0  \n",
      "6   1985-01-01   1988-09-09   0  \n",
      "7   1988-09-09   1992-08-02   0  \n",
      "8   1992-08-02   1996-08-30   0  \n",
      "9   1996-08-30   9999-01-01   0  \n",
      "10  1985-01-01   1992-04-25   0  \n",
      "11  1992-04-25   9999-01-01   0  \n",
      "12  1985-01-01   1989-05-06   0  \n",
      "13  1989-05-06   1991-09-12   0  \n",
      "14  1991-09-12   1994-06-28   0  \n",
      "15  1994-06-28   9999-01-01   0  \n",
      "16  1985-01-01   1991-03-07   0  \n",
      "17  1991-03-07   9999-01-01   0  \n",
      "18  1985-01-01   1991-04-08   0  \n",
      "19  1991-04-08   9999-01-01   0  \n",
      "20  1985-01-01   1988-10-17   0  \n",
      "21  1988-10-17   1992-09-08   0  \n",
      "22  1992-09-08   1996-01-03   0  \n",
      "23  1996-01-03  9999-01-01;   0  \n"
     ]
    }
   ],
   "source": [
    "print(JOIN(emp,man,'emp_no','inner')) \n",
    "# E.g. Intersect (overlap) of employee number in employee and manager tables - checks also for null values"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78fce7e1",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) \n",
    "\n",
    "**<a id='OUTER'>OUTER</a>**\n",
    "\n",
    "**UNION ALL**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "11aeebdf",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date first_name last_name gender    hire_date dept_no  \\\n",
      "0        10001  1953-09-02     Georgi   Facello      M   1986-06-26     NaN   \n",
      "1        10002  1964-06-02    Bezalel    Simmel      F   1985-11-21     NaN   \n",
      "2        10003  1959-12-03      Parto   Bamford      M   1986-08-28     NaN   \n",
      "3        10004  1954-05-01  Chirstian   Koblick      M   1986-12-01     NaN   \n",
      "4        10005  1955-01-21    Kyoichi  Maliniak      M   1989-09-12     NaN   \n",
      "...        ...         ...        ...       ...    ...          ...     ...   \n",
      "300019  499995  1958-09-24     Dekang  Lichtner      F   1993-01-12     NaN   \n",
      "300020  499996  1953-03-07       Zito      Baaz      M   1990-09-27     NaN   \n",
      "300021  499997  1961-08-03    Berhard    Lenart      M   1986-04-21     NaN   \n",
      "300022  499998  1956-09-05   Patricia   Breugel      M   1993-10-13     NaN   \n",
      "300023  499999  1958-05-01     Sachin   Tsukuda      M  1997-11-30;     NaN   \n",
      "\n",
      "       from_date to_date  key  \n",
      "0            NaN     NaN  NaN  \n",
      "1            NaN     NaN  NaN  \n",
      "2            NaN     NaN  NaN  \n",
      "3            NaN     NaN  NaN  \n",
      "4            NaN     NaN  NaN  \n",
      "...          ...     ...  ...  \n",
      "300019       NaN     NaN  NaN  \n",
      "300020       NaN     NaN  NaN  \n",
      "300021       NaN     NaN  NaN  \n",
      "300022       NaN     NaN  NaN  \n",
      "300023       NaN     NaN  NaN  \n",
      "\n",
      "[300024 rows x 10 columns]\n"
     ]
    }
   ],
   "source": [
    "print(JOIN(emp,man,'emp_no','outer')) \n",
    "# E.g. Union of employee numers in either the employee and manager tables "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9bca81c",
   "metadata": {},
   "source": [
    "**UNION**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 441,
   "id": "a17f9567",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        emp_no  birth_date first_name last_name gender    hire_date dept_no  \\\n",
      "0        10001  1953-09-02     Georgi   Facello      M   1986-06-26     NaN   \n",
      "1        10002  1964-06-02    Bezalel    Simmel      F   1985-11-21     NaN   \n",
      "2        10003  1959-12-03      Parto   Bamford      M   1986-08-28     NaN   \n",
      "3        10004  1954-05-01  Chirstian   Koblick      M   1986-12-01     NaN   \n",
      "4        10005  1955-01-21    Kyoichi  Maliniak      M   1989-09-12     NaN   \n",
      "...        ...         ...        ...       ...    ...          ...     ...   \n",
      "300019  499995  1958-09-24     Dekang  Lichtner      F   1993-01-12     NaN   \n",
      "300020  499996  1953-03-07       Zito      Baaz      M   1990-09-27     NaN   \n",
      "300021  499997  1961-08-03    Berhard    Lenart      M   1986-04-21     NaN   \n",
      "300022  499998  1956-09-05   Patricia   Breugel      M   1993-10-13     NaN   \n",
      "300023  499999  1958-05-01     Sachin   Tsukuda      M  1997-11-30;     NaN   \n",
      "\n",
      "       from_date to_date  key  \n",
      "0            NaN     NaN  NaN  \n",
      "1            NaN     NaN  NaN  \n",
      "2            NaN     NaN  NaN  \n",
      "3            NaN     NaN  NaN  \n",
      "4            NaN     NaN  NaN  \n",
      "...          ...     ...  ...  \n",
      "300019       NaN     NaN  NaN  \n",
      "300020       NaN     NaN  NaN  \n",
      "300021       NaN     NaN  NaN  \n",
      "300022       NaN     NaN  NaN  \n",
      "300023       NaN     NaN  NaN  \n",
      "\n",
      "[300024 rows x 10 columns]\n"
     ]
    }
   ],
   "source": [
    "print(JOIN(emp,man,'emp_no','outer').drop_duplicates()) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89f4621d",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) \n",
    "\n",
    "**<a id='LR'>LEFT/RIGHT</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d29b78da",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(JOIN(emp,man,'emp_no','left')) #E.g. All records from the employee table matched with the manager table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "7d4f1683",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------------------------------------\n",
      "table1 contains null values -  False\n",
      "table2 contains null values -  False\n",
      "------------------------------------\n",
      "    emp_no  birth_date   first_name     last_name gender   hire_date dept_no  \\\n",
      "0   110022  1956-09-12    Margareta    Markovitch      M  1985-01-01    d001   \n",
      "1   110039  1963-06-21     Vishwani      Minakawa      M  1986-04-12    d001   \n",
      "2   110085  1959-10-28         Ebru         Alpin      M  1985-01-01    d002   \n",
      "3   110114  1957-03-28        Isamu    Legleitner      F  1985-01-14    d002   \n",
      "4   110183  1953-06-24      Shirish  Ossenbruggen      F  1985-01-01    d003   \n",
      "5   110228  1958-12-02      Karsten       Sigstam      F  1985-08-04    d003   \n",
      "6   110303  1956-06-08    Krassimir       Wegerle      F  1985-01-01    d004   \n",
      "7   110344  1961-09-07       Rosine         Cools      F  1985-11-22    d004   \n",
      "8   110386  1953-10-04         Shem        Kieras      M  1988-10-14    d004   \n",
      "9   110420  1963-07-27        Oscar      Ghazalie      M  1992-02-05    d004   \n",
      "10  110511  1957-07-08     DeForest      Hagimont      M  1985-01-01    d005   \n",
      "11  110567  1964-04-25         Leon      DasSarma      F  1986-10-21    d005   \n",
      "12  110725  1961-03-14    Peternela       Onuegbe      F  1985-01-01    d006   \n",
      "13  110765  1954-05-22       Rutger       Hofmeyr      F  1989-01-07    d006   \n",
      "14  110800  1963-02-07       Sanjoy       Quadeer      F  1986-08-12    d006   \n",
      "15  110854  1960-08-19         Dung         Pesch      M  1989-06-09    d006   \n",
      "16  111035  1962-02-24  Przemyslawa     Kaelbling      M  1985-01-01    d007   \n",
      "17  111133  1955-03-16        Hauke         Zhang      M  1986-12-30    d007   \n",
      "18  111400  1959-11-09         Arie       Staelin      M  1985-01-01    d008   \n",
      "19  111534  1952-06-27       Hilary        Kambil      F  1988-01-31    d008   \n",
      "20  111692  1954-10-05        Tonny   Butterworth      F  1985-01-01    d009   \n",
      "21  111784  1956-06-14        Marjo    Giarratana      F  1988-02-12    d009   \n",
      "22  111877  1962-10-18      Xiaobin      Spinelli      F  1991-08-17    d009   \n",
      "23  111939  1960-03-25      Yuchang       Weedman      M  1989-07-10    d009   \n",
      "\n",
      "     from_date      to_date key  \n",
      "0   1985-01-01   1991-10-01   0  \n",
      "1   1991-10-01   9999-01-01   0  \n",
      "2   1985-01-01   1989-12-17   0  \n",
      "3   1989-12-17   9999-01-01   0  \n",
      "4   1985-01-01   1992-03-21   0  \n",
      "5   1992-03-21   9999-01-01   0  \n",
      "6   1985-01-01   1988-09-09   0  \n",
      "7   1988-09-09   1992-08-02   0  \n",
      "8   1992-08-02   1996-08-30   0  \n",
      "9   1996-08-30   9999-01-01   0  \n",
      "10  1985-01-01   1992-04-25   0  \n",
      "11  1992-04-25   9999-01-01   0  \n",
      "12  1985-01-01   1989-05-06   0  \n",
      "13  1989-05-06   1991-09-12   0  \n",
      "14  1991-09-12   1994-06-28   0  \n",
      "15  1994-06-28   9999-01-01   0  \n",
      "16  1985-01-01   1991-03-07   0  \n",
      "17  1991-03-07   9999-01-01   0  \n",
      "18  1985-01-01   1991-04-08   0  \n",
      "19  1991-04-08   9999-01-01   0  \n",
      "20  1985-01-01   1988-10-17   0  \n",
      "21  1988-10-17   1992-09-08   0  \n",
      "22  1992-09-08   1996-01-03   0  \n",
      "23  1996-01-03  9999-01-01;   0  \n"
     ]
    }
   ],
   "source": [
    "print(JOIN(emp,man,'emp_no','right')) #E.g. All records from the manager table matched with the employee table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e958d74",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) \n",
    "\n",
    "**<a id='MUL'>MULTIPLE</a>**\n",
    "\n",
    "**Interactive multiple joins**  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "ffac014d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def MULT_JOIN():\n",
    "    t1 = str(input(\"table1 [e.g. man]: \")); table1 = globals()[t1]; \n",
    "    t2 = str(input(\"table2 [e.g. emp]: \")); table2 = globals()[t2]; \n",
    "    print(\"****** %s ******\" %(t1))\n",
    "    print(table1.head(5))\n",
    "    print(\"****** %s ******\" %(t2))     \n",
    "    print(table2.head(5))    \n",
    "    on = str(input(\"\\njoin on [e.g. emp_no]: \"))\n",
    "    how = str(input(\"how [e.g. left]: \"))\n",
    "    tmp = JOIN(table1,table2,on,how); \n",
    "    print(\"\\n****** %s JOIN %s ******\" %(t1,t2))\n",
    "    print(tmp.head(5))\n",
    "    again = str(input(\"Join another table with these [y/n]? \"))\n",
    "    while again != \"n\": \n",
    "        l = str(input(\"List available tables [e.g. y/n]: \"))\n",
    "        if l != \"n\":\n",
    "            %whos DataFrame\n",
    "        t = str(input(\"table [e.g. dep]: \"))\n",
    "        table = globals()[t] \n",
    "        print(\"************ %s ************\" %(t))\n",
    "        print(table.head(5))\n",
    "        on = str(input(\"join on [e.g. dept_no]: \"))\n",
    "        how = str(input(\"how [e.g. left]: \"))\n",
    "        tmp1 = JOIN(tmp,table,on,how)\n",
    "        print(\"\\n************ JOINED ************\")\n",
    "        tmp = tmp1.copy()\n",
    "        print(tmp)       \n",
    "        again = str(input(\"Join another table with these [y/n]? \"))\n",
    "    return tmp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abfc49d9",
   "metadata": {},
   "source": [
    "E.g. you need the first and last names of all managers, their hire dates, \n",
    "the dates when they were promoted to managers and the names of their departments.\n",
    "\n",
    "Run the following cell with replies<br>\n",
    "    table1 [e.g. man]: man<br>\n",
    "    table2 [e.g. emp]: emp<br>\n",
    "    join on [e.g. emp_no]: emp_no<br>\n",
    "    how [e.g. left]: left<br>\n",
    "\n",
    "Join another table with these [y/n]? y<br>\n",
    "    table: dep<br>\n",
    "    join on: dept_no<br>\n",
    "    how [e.g. left]: left<br>\n",
    "\n",
    "Join another table with these [y/n]? y<br>\n",
    "    table [e.g. dep]: titles<br>\n",
    "    join on [e.g. dept_no]: emp_no<br>\n",
    "    how [e.g. left]: left<br>\n",
    "\n",
    "Join another table with these [y/n]? n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 463,
   "id": "80225b04",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "table1 [e.g. man]: man\n",
      "table2 [e.g. emp]: emp\n",
      "****** man ******\n",
      "   emp_no dept_no   from_date     to_date key\n",
      "0  110022    d001  1985-01-01  1991-10-01   0\n",
      "1  110039    d001  1991-10-01  9999-01-01   0\n",
      "2  110085    d002  1985-01-01  1989-12-17   0\n",
      "3  110114    d002  1989-12-17  9999-01-01   0\n",
      "4  110183    d003  1985-01-01  1992-03-21   0\n",
      "****** emp ******\n",
      "   emp_no  birth_date first_name last_name gender   hire_date\n",
      "0   10001  1953-09-02     Georgi   Facello      M  1986-06-26\n",
      "1   10002  1964-06-02    Bezalel    Simmel      F  1985-11-21\n",
      "2   10003  1959-12-03      Parto   Bamford      M  1986-08-28\n",
      "3   10004  1954-05-01  Chirstian   Koblick      M  1986-12-01\n",
      "4   10005  1955-01-21    Kyoichi  Maliniak      M  1989-09-12\n",
      "\n",
      "join on [e.g. emp_no]: emp_no\n",
      "how [e.g. left]: left\n",
      "\n",
      "****** man JOIN emp ******\n",
      "   emp_no dept_no   from_date     to_date key  birth_date first_name  \\\n",
      "0  110022    d001  1985-01-01  1991-10-01   0  1956-09-12  Margareta   \n",
      "1  110039    d001  1991-10-01  9999-01-01   0  1963-06-21   Vishwani   \n",
      "2  110085    d002  1985-01-01  1989-12-17   0  1959-10-28       Ebru   \n",
      "3  110114    d002  1989-12-17  9999-01-01   0  1957-03-28      Isamu   \n",
      "4  110183    d003  1985-01-01  1992-03-21   0  1953-06-24    Shirish   \n",
      "\n",
      "      last_name gender   hire_date  \n",
      "0    Markovitch      M  1985-01-01  \n",
      "1      Minakawa      M  1986-04-12  \n",
      "2         Alpin      M  1985-01-01  \n",
      "3    Legleitner      F  1985-01-14  \n",
      "4  Ossenbruggen      F  1985-01-01  \n",
      "Join another table with these [y/n]? y\n",
      "List available tables [e.g. y/n]: n\n",
      "table [e.g. dep]: dep\n",
      "************ dep ************\n",
      "  dept_no        dept_name key\n",
      "0    d001        Marketing   0\n",
      "1    d002          Finance   0\n",
      "2    d003  Human Resources   0\n",
      "3    d004       Production   0\n",
      "4    d005      Development   0\n",
      "join on [e.g. dept_no]: dept_no\n",
      "how [e.g. left]: left\n",
      "\n",
      "************ JOINED ************\n",
      "    emp_no dept_no   from_date      to_date key_x  birth_date   first_name  \\\n",
      "0   110022    d001  1985-01-01   1991-10-01     0  1956-09-12    Margareta   \n",
      "1   110039    d001  1991-10-01   9999-01-01     0  1963-06-21     Vishwani   \n",
      "2   110085    d002  1985-01-01   1989-12-17     0  1959-10-28         Ebru   \n",
      "3   110114    d002  1989-12-17   9999-01-01     0  1957-03-28        Isamu   \n",
      "4   110183    d003  1985-01-01   1992-03-21     0  1953-06-24      Shirish   \n",
      "5   110228    d003  1992-03-21   9999-01-01     0  1958-12-02      Karsten   \n",
      "6   110303    d004  1985-01-01   1988-09-09     0  1956-06-08    Krassimir   \n",
      "7   110344    d004  1988-09-09   1992-08-02     0  1961-09-07       Rosine   \n",
      "8   110386    d004  1992-08-02   1996-08-30     0  1953-10-04         Shem   \n",
      "9   110420    d004  1996-08-30   9999-01-01     0  1963-07-27        Oscar   \n",
      "10  110511    d005  1985-01-01   1992-04-25     0  1957-07-08     DeForest   \n",
      "11  110567    d005  1992-04-25   9999-01-01     0  1964-04-25         Leon   \n",
      "12  110725    d006  1985-01-01   1989-05-06     0  1961-03-14    Peternela   \n",
      "13  110765    d006  1989-05-06   1991-09-12     0  1954-05-22       Rutger   \n",
      "14  110800    d006  1991-09-12   1994-06-28     0  1963-02-07       Sanjoy   \n",
      "15  110854    d006  1994-06-28   9999-01-01     0  1960-08-19         Dung   \n",
      "16  111035    d007  1985-01-01   1991-03-07     0  1962-02-24  Przemyslawa   \n",
      "17  111133    d007  1991-03-07   9999-01-01     0  1955-03-16        Hauke   \n",
      "18  111400    d008  1985-01-01   1991-04-08     0  1959-11-09         Arie   \n",
      "19  111534    d008  1991-04-08   9999-01-01     0  1952-06-27       Hilary   \n",
      "20  111692    d009  1985-01-01   1988-10-17     0  1954-10-05        Tonny   \n",
      "21  111784    d009  1988-10-17   1992-09-08     0  1956-06-14        Marjo   \n",
      "22  111877    d009  1992-09-08   1996-01-03     0  1962-10-18      Xiaobin   \n",
      "23  111939    d009  1996-01-03  9999-01-01;     0  1960-03-25      Yuchang   \n",
      "\n",
      "       last_name gender   hire_date           dept_name key_y  \n",
      "0     Markovitch      M  1985-01-01           Marketing     0  \n",
      "1       Minakawa      M  1986-04-12           Marketing     0  \n",
      "2          Alpin      M  1985-01-01             Finance     0  \n",
      "3     Legleitner      F  1985-01-14             Finance     0  \n",
      "4   Ossenbruggen      F  1985-01-01     Human Resources     0  \n",
      "5        Sigstam      F  1985-08-04     Human Resources     0  \n",
      "6        Wegerle      F  1985-01-01          Production     0  \n",
      "7          Cools      F  1985-11-22          Production     0  \n",
      "8         Kieras      M  1988-10-14          Production     0  \n",
      "9       Ghazalie      M  1992-02-05          Production     0  \n",
      "10      Hagimont      M  1985-01-01         Development     0  \n",
      "11      DasSarma      F  1986-10-21         Development     0  \n",
      "12       Onuegbe      F  1985-01-01  Quality Management     0  \n",
      "13       Hofmeyr      F  1989-01-07  Quality Management     0  \n",
      "14       Quadeer      F  1986-08-12  Quality Management     0  \n",
      "15         Pesch      M  1989-06-09  Quality Management     0  \n",
      "16     Kaelbling      M  1985-01-01               Sales     0  \n",
      "17         Zhang      M  1986-12-30               Sales     0  \n",
      "18       Staelin      M  1985-01-01            Research     0  \n",
      "19        Kambil      F  1988-01-31            Research     0  \n",
      "20   Butterworth      F  1985-01-01   Customer Service;     0  \n",
      "21    Giarratana      F  1988-02-12   Customer Service;     0  \n",
      "22      Spinelli      F  1991-08-17   Customer Service;     0  \n",
      "23       Weedman      M  1989-07-10   Customer Service;     0  \n",
      "Join another table with these [y/n]? y\n",
      "List available tables [e.g. y/n]: n\n",
      "table [e.g. dep]: titles\n",
      "************ titles ************\n",
      "   emp_no            title   from_date     to_date\n",
      "0   10001  Senior Engineer  1986-06-26  9999-01-01\n",
      "1   10002            Staff  1996-08-03  9999-01-01\n",
      "2   10003  Senior Engineer  1995-12-03  9999-01-01\n",
      "3   10004         Engineer  1986-12-01  1995-12-01\n",
      "4   10004  Senior Engineer  1995-12-01  9999-01-01\n",
      "join on [e.g. dept_no]: emp_no\n",
      "how [e.g. left]: left\n",
      "\n",
      "************ JOINED ************\n",
      "    emp_no dept_no from_date_x    to_date_x key_x  birth_date   first_name  \\\n",
      "0   110022    d001  1985-01-01   1991-10-01     0  1956-09-12    Margareta   \n",
      "1   110022    d001  1985-01-01   1991-10-01     0  1956-09-12    Margareta   \n",
      "2   110039    d001  1991-10-01   9999-01-01     0  1963-06-21     Vishwani   \n",
      "3   110039    d001  1991-10-01   9999-01-01     0  1963-06-21     Vishwani   \n",
      "4   110085    d002  1985-01-01   1989-12-17     0  1959-10-28         Ebru   \n",
      "5   110085    d002  1985-01-01   1989-12-17     0  1959-10-28         Ebru   \n",
      "6   110114    d002  1989-12-17   9999-01-01     0  1957-03-28        Isamu   \n",
      "7   110114    d002  1989-12-17   9999-01-01     0  1957-03-28        Isamu   \n",
      "8   110183    d003  1985-01-01   1992-03-21     0  1953-06-24      Shirish   \n",
      "9   110183    d003  1985-01-01   1992-03-21     0  1953-06-24      Shirish   \n",
      "10  110228    d003  1992-03-21   9999-01-01     0  1958-12-02      Karsten   \n",
      "11  110228    d003  1992-03-21   9999-01-01     0  1958-12-02      Karsten   \n",
      "12  110303    d004  1985-01-01   1988-09-09     0  1956-06-08    Krassimir   \n",
      "13  110303    d004  1985-01-01   1988-09-09     0  1956-06-08    Krassimir   \n",
      "14  110344    d004  1988-09-09   1992-08-02     0  1961-09-07       Rosine   \n",
      "15  110344    d004  1988-09-09   1992-08-02     0  1961-09-07       Rosine   \n",
      "16  110344    d004  1988-09-09   1992-08-02     0  1961-09-07       Rosine   \n",
      "17  110386    d004  1992-08-02   1996-08-30     0  1953-10-04         Shem   \n",
      "18  110386    d004  1992-08-02   1996-08-30     0  1953-10-04         Shem   \n",
      "19  110386    d004  1992-08-02   1996-08-30     0  1953-10-04         Shem   \n",
      "20  110420    d004  1996-08-30   9999-01-01     0  1963-07-27        Oscar   \n",
      "21  110420    d004  1996-08-30   9999-01-01     0  1963-07-27        Oscar   \n",
      "22  110511    d005  1985-01-01   1992-04-25     0  1957-07-08     DeForest   \n",
      "23  110511    d005  1985-01-01   1992-04-25     0  1957-07-08     DeForest   \n",
      "24  110567    d005  1992-04-25   9999-01-01     0  1964-04-25         Leon   \n",
      "25  110567    d005  1992-04-25   9999-01-01     0  1964-04-25         Leon   \n",
      "26  110725    d006  1985-01-01   1989-05-06     0  1961-03-14    Peternela   \n",
      "27  110725    d006  1985-01-01   1989-05-06     0  1961-03-14    Peternela   \n",
      "28  110765    d006  1989-05-06   1991-09-12     0  1954-05-22       Rutger   \n",
      "29  110765    d006  1989-05-06   1991-09-12     0  1954-05-22       Rutger   \n",
      "30  110765    d006  1989-05-06   1991-09-12     0  1954-05-22       Rutger   \n",
      "31  110800    d006  1991-09-12   1994-06-28     0  1963-02-07       Sanjoy   \n",
      "32  110800    d006  1991-09-12   1994-06-28     0  1963-02-07       Sanjoy   \n",
      "33  110800    d006  1991-09-12   1994-06-28     0  1963-02-07       Sanjoy   \n",
      "34  110854    d006  1994-06-28   9999-01-01     0  1960-08-19         Dung   \n",
      "35  110854    d006  1994-06-28   9999-01-01     0  1960-08-19         Dung   \n",
      "36  111035    d007  1985-01-01   1991-03-07     0  1962-02-24  Przemyslawa   \n",
      "37  111035    d007  1985-01-01   1991-03-07     0  1962-02-24  Przemyslawa   \n",
      "38  111133    d007  1991-03-07   9999-01-01     0  1955-03-16        Hauke   \n",
      "39  111133    d007  1991-03-07   9999-01-01     0  1955-03-16        Hauke   \n",
      "40  111400    d008  1985-01-01   1991-04-08     0  1959-11-09         Arie   \n",
      "41  111400    d008  1985-01-01   1991-04-08     0  1959-11-09         Arie   \n",
      "42  111534    d008  1991-04-08   9999-01-01     0  1952-06-27       Hilary   \n",
      "43  111534    d008  1991-04-08   9999-01-01     0  1952-06-27       Hilary   \n",
      "44  111692    d009  1985-01-01   1988-10-17     0  1954-10-05        Tonny   \n",
      "45  111692    d009  1985-01-01   1988-10-17     0  1954-10-05        Tonny   \n",
      "46  111784    d009  1988-10-17   1992-09-08     0  1956-06-14        Marjo   \n",
      "47  111784    d009  1988-10-17   1992-09-08     0  1956-06-14        Marjo   \n",
      "48  111784    d009  1988-10-17   1992-09-08     0  1956-06-14        Marjo   \n",
      "49  111877    d009  1992-09-08   1996-01-03     0  1962-10-18      Xiaobin   \n",
      "50  111877    d009  1992-09-08   1996-01-03     0  1962-10-18      Xiaobin   \n",
      "51  111877    d009  1992-09-08   1996-01-03     0  1962-10-18      Xiaobin   \n",
      "52  111939    d009  1996-01-03  9999-01-01;     0  1960-03-25      Yuchang   \n",
      "53  111939    d009  1996-01-03  9999-01-01;     0  1960-03-25      Yuchang   \n",
      "\n",
      "       last_name gender   hire_date           dept_name key_y  \\\n",
      "0     Markovitch      M  1985-01-01           Marketing     0   \n",
      "1     Markovitch      M  1985-01-01           Marketing     0   \n",
      "2       Minakawa      M  1986-04-12           Marketing     0   \n",
      "3       Minakawa      M  1986-04-12           Marketing     0   \n",
      "4          Alpin      M  1985-01-01             Finance     0   \n",
      "5          Alpin      M  1985-01-01             Finance     0   \n",
      "6     Legleitner      F  1985-01-14             Finance     0   \n",
      "7     Legleitner      F  1985-01-14             Finance     0   \n",
      "8   Ossenbruggen      F  1985-01-01     Human Resources     0   \n",
      "9   Ossenbruggen      F  1985-01-01     Human Resources     0   \n",
      "10       Sigstam      F  1985-08-04     Human Resources     0   \n",
      "11       Sigstam      F  1985-08-04     Human Resources     0   \n",
      "12       Wegerle      F  1985-01-01          Production     0   \n",
      "13       Wegerle      F  1985-01-01          Production     0   \n",
      "14         Cools      F  1985-11-22          Production     0   \n",
      "15         Cools      F  1985-11-22          Production     0   \n",
      "16         Cools      F  1985-11-22          Production     0   \n",
      "17        Kieras      M  1988-10-14          Production     0   \n",
      "18        Kieras      M  1988-10-14          Production     0   \n",
      "19        Kieras      M  1988-10-14          Production     0   \n",
      "20      Ghazalie      M  1992-02-05          Production     0   \n",
      "21      Ghazalie      M  1992-02-05          Production     0   \n",
      "22      Hagimont      M  1985-01-01         Development     0   \n",
      "23      Hagimont      M  1985-01-01         Development     0   \n",
      "24      DasSarma      F  1986-10-21         Development     0   \n",
      "25      DasSarma      F  1986-10-21         Development     0   \n",
      "26       Onuegbe      F  1985-01-01  Quality Management     0   \n",
      "27       Onuegbe      F  1985-01-01  Quality Management     0   \n",
      "28       Hofmeyr      F  1989-01-07  Quality Management     0   \n",
      "29       Hofmeyr      F  1989-01-07  Quality Management     0   \n",
      "30       Hofmeyr      F  1989-01-07  Quality Management     0   \n",
      "31       Quadeer      F  1986-08-12  Quality Management     0   \n",
      "32       Quadeer      F  1986-08-12  Quality Management     0   \n",
      "33       Quadeer      F  1986-08-12  Quality Management     0   \n",
      "34         Pesch      M  1989-06-09  Quality Management     0   \n",
      "35         Pesch      M  1989-06-09  Quality Management     0   \n",
      "36     Kaelbling      M  1985-01-01               Sales     0   \n",
      "37     Kaelbling      M  1985-01-01               Sales     0   \n",
      "38         Zhang      M  1986-12-30               Sales     0   \n",
      "39         Zhang      M  1986-12-30               Sales     0   \n",
      "40       Staelin      M  1985-01-01            Research     0   \n",
      "41       Staelin      M  1985-01-01            Research     0   \n",
      "42        Kambil      F  1988-01-31            Research     0   \n",
      "43        Kambil      F  1988-01-31            Research     0   \n",
      "44   Butterworth      F  1985-01-01   Customer Service;     0   \n",
      "45   Butterworth      F  1985-01-01   Customer Service;     0   \n",
      "46    Giarratana      F  1988-02-12   Customer Service;     0   \n",
      "47    Giarratana      F  1988-02-12   Customer Service;     0   \n",
      "48    Giarratana      F  1988-02-12   Customer Service;     0   \n",
      "49      Spinelli      F  1991-08-17   Customer Service;     0   \n",
      "50      Spinelli      F  1991-08-17   Customer Service;     0   \n",
      "51      Spinelli      F  1991-08-17   Customer Service;     0   \n",
      "52       Weedman      M  1989-07-10   Customer Service;     0   \n",
      "53       Weedman      M  1989-07-10   Customer Service;     0   \n",
      "\n",
      "               title from_date_y   to_date_y  \n",
      "0            Manager  1985-01-01  1991-10-01  \n",
      "1       Senior Staff  1991-10-01  9999-01-01  \n",
      "2            Manager  1991-10-01  9999-01-01  \n",
      "3       Senior Staff  1986-04-12  1991-10-01  \n",
      "4            Manager  1985-01-01  1989-12-17  \n",
      "5       Senior Staff  1989-12-17  9999-01-01  \n",
      "6            Manager  1989-12-17  9999-01-01  \n",
      "7       Senior Staff  1985-01-14  1989-12-17  \n",
      "8            Manager  1985-01-01  1992-03-21  \n",
      "9              Staff  1992-03-21  9999-01-01  \n",
      "10           Manager  1992-03-21  9999-01-01  \n",
      "11      Senior Staff  1985-08-04  1992-03-21  \n",
      "12           Manager  1985-01-01  1988-09-09  \n",
      "13  Technique Leader  1988-09-09  9999-01-01  \n",
      "14           Manager  1988-09-09  1992-08-02  \n",
      "15   Senior Engineer  1985-11-22  1988-09-09  \n",
      "16  Technique Leader  1992-08-02  9999-01-01  \n",
      "17           Manager  1992-08-02  1996-08-30  \n",
      "18  Technique Leader  1988-10-14  1992-08-02  \n",
      "19  Technique Leader  1996-08-30  9999-01-01  \n",
      "20           Manager  1996-08-30  9999-01-01  \n",
      "21   Senior Engineer  1992-02-05  1996-08-30  \n",
      "22           Manager  1985-01-01  1992-04-25  \n",
      "23  Technique Leader  1992-04-25  9999-01-01  \n",
      "24           Manager  1992-04-25  9999-01-01  \n",
      "25  Technique Leader  1986-10-21  1992-04-25  \n",
      "26           Manager  1985-01-01  1989-05-06  \n",
      "27  Technique Leader  1989-05-06  9999-01-01  \n",
      "28           Manager  1989-05-06  1991-09-12  \n",
      "29  Technique Leader  1989-01-07  1989-05-06  \n",
      "30  Technique Leader  1991-09-12  9999-01-01  \n",
      "31           Manager  1991-09-12  1994-06-28  \n",
      "32   Senior Engineer  1986-08-12  1991-09-12  \n",
      "33  Technique Leader  1994-06-28  9999-01-01  \n",
      "34           Manager  1994-06-28  9999-01-01  \n",
      "35  Technique Leader  1989-06-09  1994-06-28  \n",
      "36           Manager  1985-01-01  1991-03-07  \n",
      "37             Staff  1991-03-07  9999-01-01  \n",
      "38           Manager  1991-03-07  9999-01-01  \n",
      "39             Staff  1986-12-30  1991-03-07  \n",
      "40           Manager  1985-01-01  1991-04-08  \n",
      "41             Staff  1991-04-08  9999-01-01  \n",
      "42           Manager  1991-04-08  9999-01-01  \n",
      "43      Senior Staff  1988-01-31  1991-04-08  \n",
      "44           Manager  1985-01-01  1988-10-17  \n",
      "45             Staff  1988-10-17  9999-01-01  \n",
      "46           Manager  1988-10-17  1992-09-08  \n",
      "47      Senior Staff  1988-02-12  1988-10-17  \n",
      "48             Staff  1992-09-08  9999-01-01  \n",
      "49           Manager  1992-09-08  1996-01-03  \n",
      "50      Senior Staff  1991-08-17  1992-09-08  \n",
      "51             Staff  1996-01-03  9999-01-01  \n",
      "52           Manager  1996-01-03  9999-01-01  \n",
      "53      Senior Staff  1989-07-10  1996-01-03  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Join another table with these [y/n]? n\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>emp_no</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>from_date_x</th>\n",
       "      <th>to_date_x</th>\n",
       "      <th>key_x</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "      <th>dept_name</th>\n",
       "      <th>key_y</th>\n",
       "      <th>title</th>\n",
       "      <th>from_date_y</th>\n",
       "      <th>to_date_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>110022</td>\n",
       "      <td>d001</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-10-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1956-09-12</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Marketing</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-10-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>110039</td>\n",
       "      <td>d001</td>\n",
       "      <td>1991-10-01</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1963-06-21</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-12</td>\n",
       "      <td>Marketing</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1991-10-01</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>110085</td>\n",
       "      <td>d002</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1989-12-17</td>\n",
       "      <td>0</td>\n",
       "      <td>1959-10-28</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Finance</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1989-12-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6</td>\n",
       "      <td>110114</td>\n",
       "      <td>d002</td>\n",
       "      <td>1989-12-17</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1957-03-28</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-14</td>\n",
       "      <td>Finance</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1989-12-17</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>110183</td>\n",
       "      <td>d003</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>0</td>\n",
       "      <td>1953-06-24</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Human Resources</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1992-03-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10</td>\n",
       "      <td>110228</td>\n",
       "      <td>d003</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1958-12-02</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-08-04</td>\n",
       "      <td>Human Resources</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>12</td>\n",
       "      <td>110303</td>\n",
       "      <td>d004</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1988-09-09</td>\n",
       "      <td>0</td>\n",
       "      <td>1956-06-08</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Production</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1988-09-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>14</td>\n",
       "      <td>110344</td>\n",
       "      <td>d004</td>\n",
       "      <td>1988-09-09</td>\n",
       "      <td>1992-08-02</td>\n",
       "      <td>0</td>\n",
       "      <td>1961-09-07</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-22</td>\n",
       "      <td>Production</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1988-09-09</td>\n",
       "      <td>1992-08-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>17</td>\n",
       "      <td>110386</td>\n",
       "      <td>d004</td>\n",
       "      <td>1992-08-02</td>\n",
       "      <td>1996-08-30</td>\n",
       "      <td>0</td>\n",
       "      <td>1953-10-04</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>M</td>\n",
       "      <td>1988-10-14</td>\n",
       "      <td>Production</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1992-08-02</td>\n",
       "      <td>1996-08-30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>20</td>\n",
       "      <td>110420</td>\n",
       "      <td>d004</td>\n",
       "      <td>1996-08-30</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1963-07-27</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>M</td>\n",
       "      <td>1992-02-05</td>\n",
       "      <td>Production</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1996-08-30</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>22</td>\n",
       "      <td>110511</td>\n",
       "      <td>d005</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1992-04-25</td>\n",
       "      <td>0</td>\n",
       "      <td>1957-07-08</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Development</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1992-04-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>24</td>\n",
       "      <td>110567</td>\n",
       "      <td>d005</td>\n",
       "      <td>1992-04-25</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1964-04-25</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-10-21</td>\n",
       "      <td>Development</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1992-04-25</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>26</td>\n",
       "      <td>110725</td>\n",
       "      <td>d006</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1989-05-06</td>\n",
       "      <td>0</td>\n",
       "      <td>1961-03-14</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Quality Management</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1989-05-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>28</td>\n",
       "      <td>110765</td>\n",
       "      <td>d006</td>\n",
       "      <td>1989-05-06</td>\n",
       "      <td>1991-09-12</td>\n",
       "      <td>0</td>\n",
       "      <td>1954-05-22</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>F</td>\n",
       "      <td>1989-01-07</td>\n",
       "      <td>Quality Management</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1989-05-06</td>\n",
       "      <td>1991-09-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>31</td>\n",
       "      <td>110800</td>\n",
       "      <td>d006</td>\n",
       "      <td>1991-09-12</td>\n",
       "      <td>1994-06-28</td>\n",
       "      <td>0</td>\n",
       "      <td>1963-02-07</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-08-12</td>\n",
       "      <td>Quality Management</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1991-09-12</td>\n",
       "      <td>1994-06-28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>34</td>\n",
       "      <td>110854</td>\n",
       "      <td>d006</td>\n",
       "      <td>1994-06-28</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1960-08-19</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-06-09</td>\n",
       "      <td>Quality Management</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1994-06-28</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>36</td>\n",
       "      <td>111035</td>\n",
       "      <td>d007</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-03-07</td>\n",
       "      <td>0</td>\n",
       "      <td>1962-02-24</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Sales</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-03-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>38</td>\n",
       "      <td>111133</td>\n",
       "      <td>d007</td>\n",
       "      <td>1991-03-07</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1955-03-16</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-30</td>\n",
       "      <td>Sales</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1991-03-07</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>40</td>\n",
       "      <td>111400</td>\n",
       "      <td>d008</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-04-08</td>\n",
       "      <td>0</td>\n",
       "      <td>1959-11-09</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Research</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-04-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>42</td>\n",
       "      <td>111534</td>\n",
       "      <td>d008</td>\n",
       "      <td>1991-04-08</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>1952-06-27</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>F</td>\n",
       "      <td>1988-01-31</td>\n",
       "      <td>Research</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1991-04-08</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>44</td>\n",
       "      <td>111692</td>\n",
       "      <td>d009</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1988-10-17</td>\n",
       "      <td>0</td>\n",
       "      <td>1954-10-05</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Customer Service;</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1988-10-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>46</td>\n",
       "      <td>111784</td>\n",
       "      <td>d009</td>\n",
       "      <td>1988-10-17</td>\n",
       "      <td>1992-09-08</td>\n",
       "      <td>0</td>\n",
       "      <td>1956-06-14</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>F</td>\n",
       "      <td>1988-02-12</td>\n",
       "      <td>Customer Service;</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1988-10-17</td>\n",
       "      <td>1992-09-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>49</td>\n",
       "      <td>111877</td>\n",
       "      <td>d009</td>\n",
       "      <td>1992-09-08</td>\n",
       "      <td>1996-01-03</td>\n",
       "      <td>0</td>\n",
       "      <td>1962-10-18</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>F</td>\n",
       "      <td>1991-08-17</td>\n",
       "      <td>Customer Service;</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1992-09-08</td>\n",
       "      <td>1996-01-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>52</td>\n",
       "      <td>111939</td>\n",
       "      <td>d009</td>\n",
       "      <td>1996-01-03</td>\n",
       "      <td>9999-01-01;</td>\n",
       "      <td>0</td>\n",
       "      <td>1960-03-25</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-07-10</td>\n",
       "      <td>Customer Service;</td>\n",
       "      <td>0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1996-01-03</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    index  emp_no dept_no from_date_x    to_date_x key_x  birth_date  \\\n",
       "0       0  110022    d001  1985-01-01   1991-10-01     0  1956-09-12   \n",
       "1       2  110039    d001  1991-10-01   9999-01-01     0  1963-06-21   \n",
       "2       4  110085    d002  1985-01-01   1989-12-17     0  1959-10-28   \n",
       "3       6  110114    d002  1989-12-17   9999-01-01     0  1957-03-28   \n",
       "4       8  110183    d003  1985-01-01   1992-03-21     0  1953-06-24   \n",
       "5      10  110228    d003  1992-03-21   9999-01-01     0  1958-12-02   \n",
       "6      12  110303    d004  1985-01-01   1988-09-09     0  1956-06-08   \n",
       "7      14  110344    d004  1988-09-09   1992-08-02     0  1961-09-07   \n",
       "8      17  110386    d004  1992-08-02   1996-08-30     0  1953-10-04   \n",
       "9      20  110420    d004  1996-08-30   9999-01-01     0  1963-07-27   \n",
       "10     22  110511    d005  1985-01-01   1992-04-25     0  1957-07-08   \n",
       "11     24  110567    d005  1992-04-25   9999-01-01     0  1964-04-25   \n",
       "12     26  110725    d006  1985-01-01   1989-05-06     0  1961-03-14   \n",
       "13     28  110765    d006  1989-05-06   1991-09-12     0  1954-05-22   \n",
       "14     31  110800    d006  1991-09-12   1994-06-28     0  1963-02-07   \n",
       "15     34  110854    d006  1994-06-28   9999-01-01     0  1960-08-19   \n",
       "16     36  111035    d007  1985-01-01   1991-03-07     0  1962-02-24   \n",
       "17     38  111133    d007  1991-03-07   9999-01-01     0  1955-03-16   \n",
       "18     40  111400    d008  1985-01-01   1991-04-08     0  1959-11-09   \n",
       "19     42  111534    d008  1991-04-08   9999-01-01     0  1952-06-27   \n",
       "20     44  111692    d009  1985-01-01   1988-10-17     0  1954-10-05   \n",
       "21     46  111784    d009  1988-10-17   1992-09-08     0  1956-06-14   \n",
       "22     49  111877    d009  1992-09-08   1996-01-03     0  1962-10-18   \n",
       "23     52  111939    d009  1996-01-03  9999-01-01;     0  1960-03-25   \n",
       "\n",
       "     first_name     last_name gender   hire_date           dept_name key_y  \\\n",
       "0     Margareta    Markovitch      M  1985-01-01           Marketing     0   \n",
       "1      Vishwani      Minakawa      M  1986-04-12           Marketing     0   \n",
       "2          Ebru         Alpin      M  1985-01-01             Finance     0   \n",
       "3         Isamu    Legleitner      F  1985-01-14             Finance     0   \n",
       "4       Shirish  Ossenbruggen      F  1985-01-01     Human Resources     0   \n",
       "5       Karsten       Sigstam      F  1985-08-04     Human Resources     0   \n",
       "6     Krassimir       Wegerle      F  1985-01-01          Production     0   \n",
       "7        Rosine         Cools      F  1985-11-22          Production     0   \n",
       "8          Shem        Kieras      M  1988-10-14          Production     0   \n",
       "9         Oscar      Ghazalie      M  1992-02-05          Production     0   \n",
       "10     DeForest      Hagimont      M  1985-01-01         Development     0   \n",
       "11         Leon      DasSarma      F  1986-10-21         Development     0   \n",
       "12    Peternela       Onuegbe      F  1985-01-01  Quality Management     0   \n",
       "13       Rutger       Hofmeyr      F  1989-01-07  Quality Management     0   \n",
       "14       Sanjoy       Quadeer      F  1986-08-12  Quality Management     0   \n",
       "15         Dung         Pesch      M  1989-06-09  Quality Management     0   \n",
       "16  Przemyslawa     Kaelbling      M  1985-01-01               Sales     0   \n",
       "17        Hauke         Zhang      M  1986-12-30               Sales     0   \n",
       "18         Arie       Staelin      M  1985-01-01            Research     0   \n",
       "19       Hilary        Kambil      F  1988-01-31            Research     0   \n",
       "20        Tonny   Butterworth      F  1985-01-01   Customer Service;     0   \n",
       "21        Marjo    Giarratana      F  1988-02-12   Customer Service;     0   \n",
       "22      Xiaobin      Spinelli      F  1991-08-17   Customer Service;     0   \n",
       "23      Yuchang       Weedman      M  1989-07-10   Customer Service;     0   \n",
       "\n",
       "      title from_date_y   to_date_y  \n",
       "0   Manager  1985-01-01  1991-10-01  \n",
       "1   Manager  1991-10-01  9999-01-01  \n",
       "2   Manager  1985-01-01  1989-12-17  \n",
       "3   Manager  1989-12-17  9999-01-01  \n",
       "4   Manager  1985-01-01  1992-03-21  \n",
       "5   Manager  1992-03-21  9999-01-01  \n",
       "6   Manager  1985-01-01  1988-09-09  \n",
       "7   Manager  1988-09-09  1992-08-02  \n",
       "8   Manager  1992-08-02  1996-08-30  \n",
       "9   Manager  1996-08-30  9999-01-01  \n",
       "10  Manager  1985-01-01  1992-04-25  \n",
       "11  Manager  1992-04-25  9999-01-01  \n",
       "12  Manager  1985-01-01  1989-05-06  \n",
       "13  Manager  1989-05-06  1991-09-12  \n",
       "14  Manager  1991-09-12  1994-06-28  \n",
       "15  Manager  1994-06-28  9999-01-01  \n",
       "16  Manager  1985-01-01  1991-03-07  \n",
       "17  Manager  1991-03-07  9999-01-01  \n",
       "18  Manager  1985-01-01  1991-04-08  \n",
       "19  Manager  1991-04-08  9999-01-01  \n",
       "20  Manager  1985-01-01  1988-10-17  \n",
       "21  Manager  1988-10-17  1992-09-08  \n",
       "22  Manager  1992-09-08  1996-01-03  \n",
       "23  Manager  1996-01-03  9999-01-01  "
      ]
     },
     "execution_count": 463,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = MULT_JOIN()\n",
    "SELECT(tmp,'title',\"Manager\").reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8387279",
   "metadata": {},
   "source": [
    "**<a id='JW'>JOIN WHERE</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 456,
   "id": "51fb48e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "      <th>title</th>\n",
       "      <th>from_date</th>\n",
       "      <th>to_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>148135</th>\n",
       "      <td>110022</td>\n",
       "      <td>1956-09-12</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Manager</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-10-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148136</th>\n",
       "      <td>110022</td>\n",
       "      <td>1956-09-12</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>Senior Staff</td>\n",
       "      <td>1991-10-01</td>\n",
       "      <td>9999-01-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name   last_name gender   hire_date  \\\n",
       "148135  110022  1956-09-12  Margareta  Markovitch      M  1985-01-01   \n",
       "148136  110022  1956-09-12  Margareta  Markovitch      M  1985-01-01   \n",
       "\n",
       "               title   from_date     to_date  \n",
       "148135       Manager  1985-01-01  1991-10-01  \n",
       "148136  Senior Staff  1991-10-01  9999-01-01  "
      ]
     },
     "execution_count": 456,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# E.g. Select all details of employees whose first name is “Margareta” and have the last name “Markovitch”.\n",
    "tmp = JOIN(emp,titles,'emp_no','inner')\n",
    "SELECT_WHERE(tmp,'first_name',\"Margareta\",'last_name',\"Markovitch\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c9df0f0",
   "metadata": {},
   "source": [
    "[Back to contents](#cont) \n",
    "\n",
    "**<a id='CJ'>CROSS JOIN</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 457,
   "id": "b808b916",
   "metadata": {},
   "outputs": [],
   "source": [
    "def CROSS_JOIN(table1,table2):\n",
    "        #print(table1.head(5)); print(table2.head(5)) \n",
    "        #Create a common 'key' to cross join the two, then outer\n",
    "        #key = str(input(\"Enter key: \"))\n",
    "        key = '0'\n",
    "        table1['key'] = key; table2['key'] = key\n",
    "        tmp = JOIN(table1,table2,'key','outer')\n",
    "        del tmp['key']\n",
    "        return tmp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2296fd99",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     emp_no dept_no_x   from_date      to_date dept_no_y           dept_name\n",
      "0    110022      d001  1985-01-01   1991-10-01      d001           Marketing\n",
      "1    110022      d001  1985-01-01   1991-10-01      d002             Finance\n",
      "2    110022      d001  1985-01-01   1991-10-01      d003     Human Resources\n",
      "3    110022      d001  1985-01-01   1991-10-01      d004          Production\n",
      "4    110022      d001  1985-01-01   1991-10-01      d005         Development\n",
      "..      ...       ...         ...          ...       ...                 ...\n",
      "211  111939      d009  1996-01-03  9999-01-01;      d005         Development\n",
      "212  111939      d009  1996-01-03  9999-01-01;      d006  Quality Management\n",
      "213  111939      d009  1996-01-03  9999-01-01;      d007               Sales\n",
      "214  111939      d009  1996-01-03  9999-01-01;      d008            Research\n",
      "215  111939      d009  1996-01-03  9999-01-01;      d009   Customer Service;\n",
      "\n",
      "[216 rows x 6 columns]\n"
     ]
    }
   ],
   "source": [
    "# E.g. return a list with all possible combinations between managers from the dept_manager \n",
    "# table and department number 9.\n",
    "tmp = CROSS_JOIN(man,dep); print(tmp)\n",
    "field = str(input(\"Which column [e.g. dept_no_y]? \"))\n",
    "value = str(input(\"Which value [e.g. dd009]? \"))\n",
    "SELECT(tmp,field,value)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a5e94e1",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='AJ'>Aggregate and JOIN</a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "id": "efb3c70d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def AGG_JOIN(table1,table2,on,how):\n",
    "    tmp = JOIN_NULL(table1,table2,on,how)\n",
    "    print(tmp.head(5))\n",
    "    field = str(input(\"Which group [e.g. gender]? \"))\n",
    "    value = str(input(\"Which quantity [e.g. salary]? \"))\n",
    "    action = str(input(\"Which function [mean, median, max or min]? \"))\n",
    "    out = GROUP_AGG(tmp,field,value,action); del out[on]\n",
    "    return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "id": "1ff67f8a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   emp_no  birth_date first_name last_name gender   hire_date   salary  \\\n",
      "0   10001  1953-09-02     Georgi   Facello      M  1986-06-26  60117.0   \n",
      "1   10001  1953-09-02     Georgi   Facello      M  1986-06-26  62102.0   \n",
      "2   10001  1953-09-02     Georgi   Facello      M  1986-06-26  66074.0   \n",
      "3   10001  1953-09-02     Georgi   Facello      M  1986-06-26  66596.0   \n",
      "4   10001  1953-09-02     Georgi   Facello      M  1986-06-26  66961.0   \n",
      "\n",
      "    from_date     to_date  \n",
      "0  1986-06-26  1987-06-26  \n",
      "1  1987-06-26  1988-06-25  \n",
      "2  1988-06-25  1989-06-25  \n",
      "3  1989-06-25  1990-06-25  \n",
      "4  1990-06-25  1991-06-25  \n",
      "Which group [e.g. gender]? gender\n",
      "Which quantity [e.g. salary]? salary\n",
      "Which function [mean, max or min]? median\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gender</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>F</th>\n",
       "      <td>61113.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M</th>\n",
       "      <td>61042.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         salary\n",
       "gender         \n",
       "F       61113.0\n",
       "M       61042.0"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "AGG_JOIN(emp,sal,'emp_no','outer') # E.g. The average, minimum or maximum salary by gender"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 428,
   "id": "9f22e49d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dept_name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Customer Service;</th>\n",
       "      <td>54959.672414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Development</th>\n",
       "      <td>59658.117647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finance</th>\n",
       "      <td>70815.888889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Human Resources</th>\n",
       "      <td>58286.055556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Marketing</th>\n",
       "      <td>88371.685714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Production</th>\n",
       "      <td>56233.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Quality Management</th>\n",
       "      <td>67130.935484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Research</th>\n",
       "      <td>77535.181818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sales</th>\n",
       "      <td>85738.764706</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          salary\n",
       "dept_name                       \n",
       "Customer Service;   54959.672414\n",
       "Development         59658.117647\n",
       "Finance             70815.888889\n",
       "Human Resources     58286.055556\n",
       "Marketing           88371.685714\n",
       "Production          56233.400000\n",
       "Quality Management  67130.935484\n",
       "Research            77535.181818\n",
       "Sales               85738.764706"
      ]
     },
     "execution_count": 428,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# E.g. The names of all departments and calculate the average salary paid to the managers in each of them.\n",
    "tmp = JOIN(man,dep,'dept_no', 'left')\n",
    "tmp1 = JOIN(tmp,titles,'emp_no','left')\n",
    "tmp2 = JOIN(tmp1,sal,'emp_no', 'inner');\n",
    "tmp3 = tmp2[tmp2['title'] == \"Manager\"]\n",
    "del tmp3['emp_no']\n",
    "GROUP_AGG(tmp3,'dept_name','salary','mean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "812e41ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "def DATE_RANGES(data,field1,field2):\n",
    "    d1 = AGG(data,field1,'min')\n",
    "    d2 = AGG(data,field1,'max')\n",
    "    d3 = AGG(data,field2,'min')\n",
    "    d4 = AGG(data,field2,'max')\n",
    "    print(field1,'ranges over', d1,d2,' and ', field2, 'ranges over', d3,d4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 360,
   "id": "1f025082",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "from_date ranges over 1985-01-01 2002-07-31  and  to_date ranges over 1986-01-01 9999-01-01\n"
     ]
    }
   ],
   "source": [
    "DATE_RANGES(tmp3,'from_date','to_date')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 727,
   "id": "1d2d7cea",
   "metadata": {},
   "outputs": [],
   "source": [
    "# E.g. Max salary after from_date = 2000-01-01\n",
    "tmp4 = FILTER(tmp3,'from_date','2000-01-01','>')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 726,
   "id": "711fc84c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dept_name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Customer Service;</th>\n",
       "      <td>73953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Development</th>\n",
       "      <td>74510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finance</th>\n",
       "      <td>88443</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Human Resources</th>\n",
       "      <td>79229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Marketing</th>\n",
       "      <td>108407</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Production</th>\n",
       "      <td>75121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Quality Management</th>\n",
       "      <td>93193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Research</th>\n",
       "      <td>103244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sales</th>\n",
       "      <td>101987</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    salary\n",
       "dept_name                 \n",
       "Customer Service;    73953\n",
       "Development          74510\n",
       "Finance              88443\n",
       "Human Resources      79229\n",
       "Marketing           108407\n",
       "Production           75121\n",
       "Quality Management   93193\n",
       "Research            103244\n",
       "Sales               101987"
      ]
     },
     "execution_count": 726,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GROUP_AGG(tmp4,'dept_name','salary','max') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 432,
   "id": "6ad4abb8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "table1 [e.g. man]: man\n",
      "table2 [e.g. emp]: dep\n",
      "****** man ******\n",
      "   emp_no dept_no   from_date     to_date key\n",
      "0  110022    d001  1985-01-01  1991-10-01   0\n",
      "1  110039    d001  1991-10-01  9999-01-01   0\n",
      "2  110085    d002  1985-01-01  1989-12-17   0\n",
      "3  110114    d002  1989-12-17  9999-01-01   0\n",
      "4  110183    d003  1985-01-01  1992-03-21   0\n",
      "****** dep ******\n",
      "  dept_no        dept_name key\n",
      "0    d001        Marketing   0\n",
      "1    d002          Finance   0\n",
      "2    d003  Human Resources   0\n",
      "3    d004       Production   0\n",
      "4    d005      Development   0\n",
      "\n",
      "join on [e.g. emp_no]: dept_no\n",
      "how [e.g. left]: left\n",
      "\n",
      "****** man JOIN dep ******\n",
      "   emp_no dept_no   from_date     to_date key_x        dept_name key_y\n",
      "0  110022    d001  1985-01-01  1991-10-01     0        Marketing     0\n",
      "1  110039    d001  1991-10-01  9999-01-01     0        Marketing     0\n",
      "2  110085    d002  1985-01-01  1989-12-17     0          Finance     0\n",
      "3  110114    d002  1989-12-17  9999-01-01     0          Finance     0\n",
      "4  110183    d003  1985-01-01  1992-03-21     0  Human Resources     0\n",
      "Join another table with these [y/n]? y\n",
      "List available tables [e.g. y/n]: n\n",
      "table [e.g. dep]: titles\n",
      "************ titles ************\n",
      "   emp_no            title   from_date     to_date\n",
      "0   10001  Senior Engineer  1986-06-26  9999-01-01\n",
      "1   10002            Staff  1996-08-03  9999-01-01\n",
      "2   10003  Senior Engineer  1995-12-03  9999-01-01\n",
      "3   10004         Engineer  1986-12-01  1995-12-01\n",
      "4   10004  Senior Engineer  1995-12-01  9999-01-01\n",
      "join on [e.g. dept_no]: emp_no\n",
      "how [e.g. left]: left\n",
      "\n",
      "************ JOINED ************\n",
      "    emp_no dept_no from_date_x    to_date_x key_x           dept_name key_y  \\\n",
      "0   110022    d001  1985-01-01   1991-10-01     0           Marketing     0   \n",
      "1   110022    d001  1985-01-01   1991-10-01     0           Marketing     0   \n",
      "2   110039    d001  1991-10-01   9999-01-01     0           Marketing     0   \n",
      "3   110039    d001  1991-10-01   9999-01-01     0           Marketing     0   \n",
      "4   110085    d002  1985-01-01   1989-12-17     0             Finance     0   \n",
      "5   110085    d002  1985-01-01   1989-12-17     0             Finance     0   \n",
      "6   110114    d002  1989-12-17   9999-01-01     0             Finance     0   \n",
      "7   110114    d002  1989-12-17   9999-01-01     0             Finance     0   \n",
      "8   110183    d003  1985-01-01   1992-03-21     0     Human Resources     0   \n",
      "9   110183    d003  1985-01-01   1992-03-21     0     Human Resources     0   \n",
      "10  110228    d003  1992-03-21   9999-01-01     0     Human Resources     0   \n",
      "11  110228    d003  1992-03-21   9999-01-01     0     Human Resources     0   \n",
      "12  110303    d004  1985-01-01   1988-09-09     0          Production     0   \n",
      "13  110303    d004  1985-01-01   1988-09-09     0          Production     0   \n",
      "14  110344    d004  1988-09-09   1992-08-02     0          Production     0   \n",
      "15  110344    d004  1988-09-09   1992-08-02     0          Production     0   \n",
      "16  110344    d004  1988-09-09   1992-08-02     0          Production     0   \n",
      "17  110386    d004  1992-08-02   1996-08-30     0          Production     0   \n",
      "18  110386    d004  1992-08-02   1996-08-30     0          Production     0   \n",
      "19  110386    d004  1992-08-02   1996-08-30     0          Production     0   \n",
      "20  110420    d004  1996-08-30   9999-01-01     0          Production     0   \n",
      "21  110420    d004  1996-08-30   9999-01-01     0          Production     0   \n",
      "22  110511    d005  1985-01-01   1992-04-25     0         Development     0   \n",
      "23  110511    d005  1985-01-01   1992-04-25     0         Development     0   \n",
      "24  110567    d005  1992-04-25   9999-01-01     0         Development     0   \n",
      "25  110567    d005  1992-04-25   9999-01-01     0         Development     0   \n",
      "26  110725    d006  1985-01-01   1989-05-06     0  Quality Management     0   \n",
      "27  110725    d006  1985-01-01   1989-05-06     0  Quality Management     0   \n",
      "28  110765    d006  1989-05-06   1991-09-12     0  Quality Management     0   \n",
      "29  110765    d006  1989-05-06   1991-09-12     0  Quality Management     0   \n",
      "30  110765    d006  1989-05-06   1991-09-12     0  Quality Management     0   \n",
      "31  110800    d006  1991-09-12   1994-06-28     0  Quality Management     0   \n",
      "32  110800    d006  1991-09-12   1994-06-28     0  Quality Management     0   \n",
      "33  110800    d006  1991-09-12   1994-06-28     0  Quality Management     0   \n",
      "34  110854    d006  1994-06-28   9999-01-01     0  Quality Management     0   \n",
      "35  110854    d006  1994-06-28   9999-01-01     0  Quality Management     0   \n",
      "36  111035    d007  1985-01-01   1991-03-07     0               Sales     0   \n",
      "37  111035    d007  1985-01-01   1991-03-07     0               Sales     0   \n",
      "38  111133    d007  1991-03-07   9999-01-01     0               Sales     0   \n",
      "39  111133    d007  1991-03-07   9999-01-01     0               Sales     0   \n",
      "40  111400    d008  1985-01-01   1991-04-08     0            Research     0   \n",
      "41  111400    d008  1985-01-01   1991-04-08     0            Research     0   \n",
      "42  111534    d008  1991-04-08   9999-01-01     0            Research     0   \n",
      "43  111534    d008  1991-04-08   9999-01-01     0            Research     0   \n",
      "44  111692    d009  1985-01-01   1988-10-17     0   Customer Service;     0   \n",
      "45  111692    d009  1985-01-01   1988-10-17     0   Customer Service;     0   \n",
      "46  111784    d009  1988-10-17   1992-09-08     0   Customer Service;     0   \n",
      "47  111784    d009  1988-10-17   1992-09-08     0   Customer Service;     0   \n",
      "48  111784    d009  1988-10-17   1992-09-08     0   Customer Service;     0   \n",
      "49  111877    d009  1992-09-08   1996-01-03     0   Customer Service;     0   \n",
      "50  111877    d009  1992-09-08   1996-01-03     0   Customer Service;     0   \n",
      "51  111877    d009  1992-09-08   1996-01-03     0   Customer Service;     0   \n",
      "52  111939    d009  1996-01-03  9999-01-01;     0   Customer Service;     0   \n",
      "53  111939    d009  1996-01-03  9999-01-01;     0   Customer Service;     0   \n",
      "\n",
      "               title from_date_y   to_date_y  \n",
      "0            Manager  1985-01-01  1991-10-01  \n",
      "1       Senior Staff  1991-10-01  9999-01-01  \n",
      "2            Manager  1991-10-01  9999-01-01  \n",
      "3       Senior Staff  1986-04-12  1991-10-01  \n",
      "4            Manager  1985-01-01  1989-12-17  \n",
      "5       Senior Staff  1989-12-17  9999-01-01  \n",
      "6            Manager  1989-12-17  9999-01-01  \n",
      "7       Senior Staff  1985-01-14  1989-12-17  \n",
      "8            Manager  1985-01-01  1992-03-21  \n",
      "9              Staff  1992-03-21  9999-01-01  \n",
      "10           Manager  1992-03-21  9999-01-01  \n",
      "11      Senior Staff  1985-08-04  1992-03-21  \n",
      "12           Manager  1985-01-01  1988-09-09  \n",
      "13  Technique Leader  1988-09-09  9999-01-01  \n",
      "14           Manager  1988-09-09  1992-08-02  \n",
      "15   Senior Engineer  1985-11-22  1988-09-09  \n",
      "16  Technique Leader  1992-08-02  9999-01-01  \n",
      "17           Manager  1992-08-02  1996-08-30  \n",
      "18  Technique Leader  1988-10-14  1992-08-02  \n",
      "19  Technique Leader  1996-08-30  9999-01-01  \n",
      "20           Manager  1996-08-30  9999-01-01  \n",
      "21   Senior Engineer  1992-02-05  1996-08-30  \n",
      "22           Manager  1985-01-01  1992-04-25  \n",
      "23  Technique Leader  1992-04-25  9999-01-01  \n",
      "24           Manager  1992-04-25  9999-01-01  \n",
      "25  Technique Leader  1986-10-21  1992-04-25  \n",
      "26           Manager  1985-01-01  1989-05-06  \n",
      "27  Technique Leader  1989-05-06  9999-01-01  \n",
      "28           Manager  1989-05-06  1991-09-12  \n",
      "29  Technique Leader  1989-01-07  1989-05-06  \n",
      "30  Technique Leader  1991-09-12  9999-01-01  \n",
      "31           Manager  1991-09-12  1994-06-28  \n",
      "32   Senior Engineer  1986-08-12  1991-09-12  \n",
      "33  Technique Leader  1994-06-28  9999-01-01  \n",
      "34           Manager  1994-06-28  9999-01-01  \n",
      "35  Technique Leader  1989-06-09  1994-06-28  \n",
      "36           Manager  1985-01-01  1991-03-07  \n",
      "37             Staff  1991-03-07  9999-01-01  \n",
      "38           Manager  1991-03-07  9999-01-01  \n",
      "39             Staff  1986-12-30  1991-03-07  \n",
      "40           Manager  1985-01-01  1991-04-08  \n",
      "41             Staff  1991-04-08  9999-01-01  \n",
      "42           Manager  1991-04-08  9999-01-01  \n",
      "43      Senior Staff  1988-01-31  1991-04-08  \n",
      "44           Manager  1985-01-01  1988-10-17  \n",
      "45             Staff  1988-10-17  9999-01-01  \n",
      "46           Manager  1988-10-17  1992-09-08  \n",
      "47      Senior Staff  1988-02-12  1988-10-17  \n",
      "48             Staff  1992-09-08  9999-01-01  \n",
      "49           Manager  1992-09-08  1996-01-03  \n",
      "50      Senior Staff  1991-08-17  1992-09-08  \n",
      "51             Staff  1996-01-03  9999-01-01  \n",
      "52           Manager  1996-01-03  9999-01-01  \n",
      "53      Senior Staff  1989-07-10  1996-01-03  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Join another table with these [y/n]? y\n",
      "List available tables [e.g. y/n]: n\n",
      "table [e.g. dep]: sal\n",
      "************ sal ************\n",
      "   emp_no  salary   from_date     to_date\n",
      "0   10001   60117  1986-06-26  1987-06-26\n",
      "1   10001   62102  1987-06-26  1988-06-25\n",
      "2   10001   66074  1988-06-25  1989-06-25\n",
      "3   10001   66596  1989-06-25  1990-06-25\n",
      "4   10001   66961  1990-06-25  1991-06-25\n",
      "join on [e.g. dept_no]: emp_no\n",
      "how [e.g. left]: inner\n",
      "\n",
      "************ JOINED ************\n",
      "     emp_no dept_no from_date_x    to_date_x key_x          dept_name key_y  \\\n",
      "0    110022    d001  1985-01-01   1991-10-01     0          Marketing     0   \n",
      "1    110022    d001  1985-01-01   1991-10-01     0          Marketing     0   \n",
      "2    110022    d001  1985-01-01   1991-10-01     0          Marketing     0   \n",
      "3    110022    d001  1985-01-01   1991-10-01     0          Marketing     0   \n",
      "4    110022    d001  1985-01-01   1991-10-01     0          Marketing     0   \n",
      "..      ...     ...         ...          ...   ...                ...   ...   \n",
      "858  111939    d009  1996-01-03  9999-01-01;     0  Customer Service;     0   \n",
      "859  111939    d009  1996-01-03  9999-01-01;     0  Customer Service;     0   \n",
      "860  111939    d009  1996-01-03  9999-01-01;     0  Customer Service;     0   \n",
      "861  111939    d009  1996-01-03  9999-01-01;     0  Customer Service;     0   \n",
      "862  111939    d009  1996-01-03  9999-01-01;     0  Customer Service;     0   \n",
      "\n",
      "            title from_date_y   to_date_y  salary   from_date     to_date  \n",
      "0         Manager  1985-01-01  1991-10-01   71166  1985-01-01  1986-01-01  \n",
      "1         Manager  1985-01-01  1991-10-01   71820  1986-01-01  1987-01-01  \n",
      "2         Manager  1985-01-01  1991-10-01   72970  1987-01-01  1988-01-01  \n",
      "3         Manager  1985-01-01  1991-10-01   76211  1988-01-01  1988-12-31  \n",
      "4         Manager  1985-01-01  1991-10-01   78443  1988-12-31  1989-12-31  \n",
      "..            ...         ...         ...     ...         ...         ...  \n",
      "858  Senior Staff  1989-07-10  1996-01-03   52051  1998-07-08  1999-07-08  \n",
      "859  Senior Staff  1989-07-10  1996-01-03   56454  1999-07-08  2000-07-07  \n",
      "860  Senior Staff  1989-07-10  1996-01-03   58698  2000-07-07  2001-07-07  \n",
      "861  Senior Staff  1989-07-10  1996-01-03   58968  2001-07-07  2002-07-07  \n",
      "862  Senior Staff  1989-07-10  1996-01-03   58745  2002-07-07  9999-01-01  \n",
      "\n",
      "[863 rows x 13 columns]\n",
      "Join another table with these [y/n]? n\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dept_name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Customer Service;</th>\n",
       "      <td>54959.672414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Development</th>\n",
       "      <td>59658.117647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finance</th>\n",
       "      <td>70815.888889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Human Resources</th>\n",
       "      <td>58286.055556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Marketing</th>\n",
       "      <td>88371.685714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Production</th>\n",
       "      <td>56233.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Quality Management</th>\n",
       "      <td>67130.935484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Research</th>\n",
       "      <td>77535.181818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sales</th>\n",
       "      <td>85738.764706</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          salary\n",
       "dept_name                       \n",
       "Customer Service;   54959.672414\n",
       "Development         59658.117647\n",
       "Finance             70815.888889\n",
       "Human Resources     58286.055556\n",
       "Marketing           88371.685714\n",
       "Production          56233.400000\n",
       "Quality Management  67130.935484\n",
       "Research            77535.181818\n",
       "Sales               85738.764706"
      ]
     },
     "execution_count": 432,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Alternatlively\n",
    "tmp = MULT_JOIN()\n",
    "# table1 [e.g. man]: man\n",
    "# table2 [e.g. emp]: dep\n",
    "# join on [e.g. emp_no]: dept_no\n",
    "# how [e.g. left]: left\n",
    "# table [e.g. dep]: titles\n",
    "# join on [e.g. dept_no]: emp_no\n",
    "# how [e.g. left]: left\n",
    "# table [e.g. dep]: sal\n",
    "# join on [e.g. dept_no]: emp_no\n",
    "# how [e.g. left]: inner\n",
    "# Join another table with these [y/n]? n\n",
    "tmp3 = tmp[tmp['title'] == \"Manager\"]\n",
    "del tmp3['emp_no']\n",
    "GROUP_AGG(tmp3,'dept_name','salary','mean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 737,
   "id": "fbcf112b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>F</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>M</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  gender  Count\n",
       "0      F     13\n",
       "1      M     11"
      ]
     },
     "execution_count": 737,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# E.g. what is the gender of the managers?\n",
    "tmp = JOIN(man,emp,'emp_no','left')\n",
    "GROUP_COUNT(tmp,'gender')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60351aa1",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "439285b0",
   "metadata": {},
   "source": [
    "**<a id='CAS'>CASE</a>**\n",
    "\n",
    "**Example 1: Add a new field (named Position) to the employee table stating whether a manager or not**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 548,
   "id": "641a5fb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def CASE_1(table1,table2,field,new,default,case):\n",
    "    table = table1.copy()\n",
    "    table[new] = default\n",
    "    table.loc[table[field].isin(table2[field]), new] = case\n",
    "    return table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 550,
   "id": "f51e9820",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "      <th>Position</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>1953-09-02</td>\n",
       "      <td>Georgi</td>\n",
       "      <td>Facello</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>1964-06-02</td>\n",
       "      <td>Bezalel</td>\n",
       "      <td>Simmel</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-21</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>1959-12-03</td>\n",
       "      <td>Parto</td>\n",
       "      <td>Bamford</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-08-28</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>1954-05-01</td>\n",
       "      <td>Chirstian</td>\n",
       "      <td>Koblick</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-01</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10005</td>\n",
       "      <td>1955-01-21</td>\n",
       "      <td>Kyoichi</td>\n",
       "      <td>Maliniak</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-09-12</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300019</th>\n",
       "      <td>499995</td>\n",
       "      <td>1958-09-24</td>\n",
       "      <td>Dekang</td>\n",
       "      <td>Lichtner</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-01-12</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300020</th>\n",
       "      <td>499996</td>\n",
       "      <td>1953-03-07</td>\n",
       "      <td>Zito</td>\n",
       "      <td>Baaz</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-27</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300021</th>\n",
       "      <td>499997</td>\n",
       "      <td>1961-08-03</td>\n",
       "      <td>Berhard</td>\n",
       "      <td>Lenart</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-21</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300022</th>\n",
       "      <td>499998</td>\n",
       "      <td>1956-09-05</td>\n",
       "      <td>Patricia</td>\n",
       "      <td>Breugel</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-10-13</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300023</th>\n",
       "      <td>499999</td>\n",
       "      <td>1958-05-01</td>\n",
       "      <td>Sachin</td>\n",
       "      <td>Tsukuda</td>\n",
       "      <td>M</td>\n",
       "      <td>1997-11-30;</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>300024 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  birth_date first_name last_name gender    hire_date  \\\n",
       "0        10001  1953-09-02     Georgi   Facello      M   1986-06-26   \n",
       "1        10002  1964-06-02    Bezalel    Simmel      F   1985-11-21   \n",
       "2        10003  1959-12-03      Parto   Bamford      M   1986-08-28   \n",
       "3        10004  1954-05-01  Chirstian   Koblick      M   1986-12-01   \n",
       "4        10005  1955-01-21    Kyoichi  Maliniak      M   1989-09-12   \n",
       "...        ...         ...        ...       ...    ...          ...   \n",
       "300019  499995  1958-09-24     Dekang  Lichtner      F   1993-01-12   \n",
       "300020  499996  1953-03-07       Zito      Baaz      M   1990-09-27   \n",
       "300021  499997  1961-08-03    Berhard    Lenart      M   1986-04-21   \n",
       "300022  499998  1956-09-05   Patricia   Breugel      M   1993-10-13   \n",
       "300023  499999  1958-05-01     Sachin   Tsukuda      M  1997-11-30;   \n",
       "\n",
       "           Position  \n",
       "0       Not manager  \n",
       "1       Not manager  \n",
       "2       Not manager  \n",
       "3       Not manager  \n",
       "4       Not manager  \n",
       "...             ...  \n",
       "300019  Not manager  \n",
       "300020  Not manager  \n",
       "300021  Not manager  \n",
       "300022  Not manager  \n",
       "300023  Not manager  \n",
       "\n",
       "[300024 rows x 7 columns]"
      ]
     },
     "execution_count": 550,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CASE_1(emp,man,'emp_no','Position','Not manager','Manager')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d80b0da",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 2: List managers' total salary increase**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1051,
   "id": "5237ae95",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "===================== Splitting at 20k and 30k =====================\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>Salary increase</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110022</td>\n",
       "      <td>d001</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110039</td>\n",
       "      <td>d001</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110085</td>\n",
       "      <td>d002</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110114</td>\n",
       "      <td>d002</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110183</td>\n",
       "      <td>d003</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110228</td>\n",
       "      <td>d003</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110303</td>\n",
       "      <td>d004</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110344</td>\n",
       "      <td>d004</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110386</td>\n",
       "      <td>d004</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110420</td>\n",
       "      <td>d004</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>Less than 20k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110511</td>\n",
       "      <td>d005</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110567</td>\n",
       "      <td>d005</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110725</td>\n",
       "      <td>d006</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110765</td>\n",
       "      <td>d006</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110800</td>\n",
       "      <td>d006</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110854</td>\n",
       "      <td>d006</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111035</td>\n",
       "      <td>d007</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>By between 20k and 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111133</td>\n",
       "      <td>d007</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111400</td>\n",
       "      <td>d008</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111534</td>\n",
       "      <td>d008</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111692</td>\n",
       "      <td>d009</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>More than 30k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111784</td>\n",
       "      <td>d009</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>Less than 20k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111877</td>\n",
       "      <td>d009</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>Less than 20k</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>111939</td>\n",
       "      <td>d009</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>Less than 20k</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   emp_no dept_no   first_name     last_name         Salary increase\n",
       "0  110022    d001    Margareta    Markovitch           More than 30k\n",
       "0  110039    d001     Vishwani      Minakawa           More than 30k\n",
       "0  110085    d002         Ebru         Alpin  By between 20k and 30k\n",
       "0  110114    d002        Isamu    Legleitner           More than 30k\n",
       "0  110183    d003      Shirish  Ossenbruggen           More than 30k\n",
       "0  110228    d003      Karsten       Sigstam  By between 20k and 30k\n",
       "0  110303    d004    Krassimir       Wegerle           More than 30k\n",
       "0  110344    d004       Rosine         Cools  By between 20k and 30k\n",
       "0  110386    d004         Shem        Kieras  By between 20k and 30k\n",
       "0  110420    d004        Oscar      Ghazalie           Less than 20k\n",
       "0  110511    d005     DeForest      Hagimont  By between 20k and 30k\n",
       "0  110567    d005         Leon      DasSarma           More than 30k\n",
       "0  110725    d006    Peternela       Onuegbe           More than 30k\n",
       "0  110765    d006       Rutger       Hofmeyr           More than 30k\n",
       "0  110800    d006       Sanjoy       Quadeer  By between 20k and 30k\n",
       "0  110854    d006         Dung         Pesch  By between 20k and 30k\n",
       "0  111035    d007  Przemyslawa     Kaelbling  By between 20k and 30k\n",
       "0  111133    d007        Hauke         Zhang           More than 30k\n",
       "0  111400    d008         Arie       Staelin           More than 30k\n",
       "0  111534    d008       Hilary        Kambil           More than 30k\n",
       "0  111692    d009        Tonny   Butterworth           More than 30k\n",
       "0  111784    d009        Marjo    Giarratana           Less than 20k\n",
       "0  111877    d009      Xiaobin      Spinelli           Less than 20k\n",
       "0  111939    d009      Yuchang       Weedman           Less than 20k"
      ]
     },
     "execution_count": 1051,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(man,emp,'emp_no','left')\n",
    "tmp1 = JOIN(tmp,sal,'emp_no','left')\n",
    "IDs = tmp1['emp_no'].unique()\n",
    "columns = ['emp_no','dept_no','first_name','last_name','diff']\n",
    "\n",
    "dfs = []\n",
    "for i in range (0,len(IDs)):\n",
    "    tmp = tmp1[tmp1['emp_no'] == IDs[i]]\n",
    "    tmp.sort_values(by=['salary'], ascending=True);\n",
    "    tmp['diff'] = tmp['salary'].iloc[-1] - tmp['salary'].iloc[0]\n",
    "    tmp = tmp[columns].reset_index(); del tmp['index']\n",
    "    try:\n",
    "        tmp2=tmp.iloc[[0]]; # DOUBLE BRACKETS FORCE SINGLE LINE\n",
    "        dfs.append(tmp2)\n",
    "    except IndexError: \n",
    "        pass\n",
    "out = pd.concat(dfs)\n",
    "out\n",
    "\n",
    "# if a certain manager has a raise higher than $30 000, we will return, \n",
    "# \"Salary was raised by more than $30 000\". If, instead, it’s between $20 000 and $30 000, \n",
    "# we would like to see, \"Salary was raised by more than $20 000\" but less than $30 000 displayed. \n",
    "# Finally, if none of the conditions were met the query would return, \"Salary was raised by less than $20 000\".\n",
    "\n",
    "d1 = 30000; d2 = 20000\n",
    "print('\\n===================== Splitting at %1.0fk and %1.0fk =====================' %(d2/1000,d1/1000))\n",
    "out.loc[out['diff'] > d1, 'Salary increase'] = 'More than %1.0fk' %(d1/1000)\n",
    "out.loc[out['diff'] < d2, 'Salary increase'] = 'Less than %1.0fk' %(d2/1000)\n",
    "out.loc[(out['diff'] >= d2) & (out['diff'] <= d1), 'Salary increase'] = 'By between %1.0fk and %1.0fk' %(d2/1000,d1/1000)\n",
    "del out['diff'] \n",
    "out\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44e8ba72",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 3: Obtain a table containing the employee number, first name, and last name of all employees with a number higher than 109990. Create a column indicating whether this employee is also a manager, according to the dept_manager table, or a regular employee.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 713,
   "id": "8d7fcfaf",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>emp_no</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "      <th>Position</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>99990</td>\n",
       "      <td>109991</td>\n",
       "      <td>1954-10-30</td>\n",
       "      <td>Jinxi</td>\n",
       "      <td>Reistad</td>\n",
       "      <td>M</td>\n",
       "      <td>1987-01-13</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99991</td>\n",
       "      <td>109992</td>\n",
       "      <td>1955-03-21</td>\n",
       "      <td>Cheong</td>\n",
       "      <td>Heering</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-03-25</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>99992</td>\n",
       "      <td>109993</td>\n",
       "      <td>1953-10-19</td>\n",
       "      <td>Danel</td>\n",
       "      <td>Furudate</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-11-04</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>99993</td>\n",
       "      <td>109994</td>\n",
       "      <td>1956-10-24</td>\n",
       "      <td>Guangming</td>\n",
       "      <td>Takkinen</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-05-24</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>99994</td>\n",
       "      <td>109995</td>\n",
       "      <td>1960-02-11</td>\n",
       "      <td>Constantijn</td>\n",
       "      <td>Anido</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-06-11</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200029</th>\n",
       "      <td>300019</td>\n",
       "      <td>499995</td>\n",
       "      <td>1958-09-24</td>\n",
       "      <td>Dekang</td>\n",
       "      <td>Lichtner</td>\n",
       "      <td>F</td>\n",
       "      <td>1993-01-12</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200030</th>\n",
       "      <td>300020</td>\n",
       "      <td>499996</td>\n",
       "      <td>1953-03-07</td>\n",
       "      <td>Zito</td>\n",
       "      <td>Baaz</td>\n",
       "      <td>M</td>\n",
       "      <td>1990-09-27</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200031</th>\n",
       "      <td>300021</td>\n",
       "      <td>499997</td>\n",
       "      <td>1961-08-03</td>\n",
       "      <td>Berhard</td>\n",
       "      <td>Lenart</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-21</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200032</th>\n",
       "      <td>300022</td>\n",
       "      <td>499998</td>\n",
       "      <td>1956-09-05</td>\n",
       "      <td>Patricia</td>\n",
       "      <td>Breugel</td>\n",
       "      <td>M</td>\n",
       "      <td>1993-10-13</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200033</th>\n",
       "      <td>300023</td>\n",
       "      <td>499999</td>\n",
       "      <td>1958-05-01</td>\n",
       "      <td>Sachin</td>\n",
       "      <td>Tsukuda</td>\n",
       "      <td>M</td>\n",
       "      <td>1997-11-30;</td>\n",
       "      <td>Not manager</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200034 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         index  emp_no  birth_date   first_name last_name gender    hire_date  \\\n",
       "0        99990  109991  1954-10-30        Jinxi   Reistad      M   1987-01-13   \n",
       "1        99991  109992  1955-03-21       Cheong   Heering      F   1993-03-25   \n",
       "2        99992  109993  1953-10-19        Danel  Furudate      M   1986-11-04   \n",
       "3        99993  109994  1956-10-24    Guangming  Takkinen      M   1986-05-24   \n",
       "4        99994  109995  1960-02-11  Constantijn     Anido      F   1986-06-11   \n",
       "...        ...     ...         ...          ...       ...    ...          ...   \n",
       "200029  300019  499995  1958-09-24       Dekang  Lichtner      F   1993-01-12   \n",
       "200030  300020  499996  1953-03-07         Zito      Baaz      M   1990-09-27   \n",
       "200031  300021  499997  1961-08-03      Berhard    Lenart      M   1986-04-21   \n",
       "200032  300022  499998  1956-09-05     Patricia   Breugel      M   1993-10-13   \n",
       "200033  300023  499999  1958-05-01       Sachin   Tsukuda      M  1997-11-30;   \n",
       "\n",
       "           Position  \n",
       "0       Not manager  \n",
       "1       Not manager  \n",
       "2       Not manager  \n",
       "3       Not manager  \n",
       "4       Not manager  \n",
       "...             ...  \n",
       "200029  Not manager  \n",
       "200030  Not manager  \n",
       "200031  Not manager  \n",
       "200032  Not manager  \n",
       "200033  Not manager  \n",
       "\n",
       "[200034 rows x 8 columns]"
      ]
     },
     "execution_count": 713,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = CASE_1(emp,man,'emp_no','Position','Not manager','Manager')\n",
    "FILTER(tmp,'emp_no',109990,'>').reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3289589a",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 4: Extract the employee number, first name, and last name of the first 100 employees. Add column stating if the  employee is still working in the company.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 734,
   "id": "e9a4c052",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>from_date</th>\n",
       "      <th>to_date</th>\n",
       "      <th>current_employee</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>Georgi</td>\n",
       "      <td>Facello</td>\n",
       "      <td>d005</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>Bezalel</td>\n",
       "      <td>Simmel</td>\n",
       "      <td>d007</td>\n",
       "      <td>1996-08-03</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>Parto</td>\n",
       "      <td>Bamford</td>\n",
       "      <td>d004</td>\n",
       "      <td>1995-12-03</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>Chirstian</td>\n",
       "      <td>Koblick</td>\n",
       "      <td>d004</td>\n",
       "      <td>1986-12-01</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10005</td>\n",
       "      <td>Kyoichi</td>\n",
       "      <td>Maliniak</td>\n",
       "      <td>d003</td>\n",
       "      <td>1989-09-12</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>10088</td>\n",
       "      <td>Jungsoon</td>\n",
       "      <td>Syrzycki</td>\n",
       "      <td>d007</td>\n",
       "      <td>1988-09-02</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>Is still employed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>10088</td>\n",
       "      <td>Jungsoon</td>\n",
       "      <td>Syrzycki</td>\n",
       "      <td>d009</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>10089</td>\n",
       "      <td>Sudharsan</td>\n",
       "      <td>Flasterstein</td>\n",
       "      <td>d007</td>\n",
       "      <td>1989-01-10</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>10090</td>\n",
       "      <td>Kendra</td>\n",
       "      <td>Hofting</td>\n",
       "      <td>d005</td>\n",
       "      <td>1986-03-14</td>\n",
       "      <td>1999-05-07</td>\n",
       "      <td>Is still employed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>10091</td>\n",
       "      <td>Amabile</td>\n",
       "      <td>Gomatam</td>\n",
       "      <td>d005</td>\n",
       "      <td>1992-11-18</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>Not an employee anymore</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    emp_no first_name     last_name dept_no   from_date     to_date  \\\n",
       "0    10001     Georgi       Facello    d005  1986-06-26  9999-01-01   \n",
       "1    10002    Bezalel        Simmel    d007  1996-08-03  9999-01-01   \n",
       "2    10003      Parto       Bamford    d004  1995-12-03  9999-01-01   \n",
       "3    10004  Chirstian       Koblick    d004  1986-12-01  9999-01-01   \n",
       "4    10005    Kyoichi      Maliniak    d003  1989-09-12  9999-01-01   \n",
       "..     ...        ...           ...     ...         ...         ...   \n",
       "95   10088   Jungsoon      Syrzycki    d007  1988-09-02  1992-03-21   \n",
       "96   10088   Jungsoon      Syrzycki    d009  1992-03-21  9999-01-01   \n",
       "97   10089  Sudharsan  Flasterstein    d007  1989-01-10  9999-01-01   \n",
       "98   10090     Kendra       Hofting    d005  1986-03-14  1999-05-07   \n",
       "99   10091    Amabile       Gomatam    d005  1992-11-18  9999-01-01   \n",
       "\n",
       "           current_employee  \n",
       "0   Not an employee anymore  \n",
       "1   Not an employee anymore  \n",
       "2   Not an employee anymore  \n",
       "3   Not an employee anymore  \n",
       "4   Not an employee anymore  \n",
       "..                      ...  \n",
       "95        Is still employed  \n",
       "96  Not an employee anymore  \n",
       "97  Not an employee anymore  \n",
       "98        Is still employed  \n",
       "99  Not an employee anymore  \n",
       "\n",
       "[100 rows x 7 columns]"
      ]
     },
     "execution_count": 734,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = emp[['emp_no','first_name','last_name']]\n",
    "tmp1= JOIN(tmp,dept_emp,'emp_no','left')\n",
    "tmp2 = tmp1.head(100)\n",
    "Dict = {\"9999-01-01\": 'Not an employee anymore'}\n",
    "tmp2['current_employee']= tmp2['to_date'].map(Dict)\n",
    "tmp2['current_employee'] = tmp2['current_employee'].replace({np.nan: 'Is still employed'})\n",
    "tmp2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e7ddfcc",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "# <a id='WF'>Window Functions</a>\n",
    "\n",
    "| | | |\n",
    "| :---                            | :---                   | :--- |\n",
    "|**[ROW NUMBER](#RN)** | **[PARTITION BY](#PB)** | **[RANK](#RANK)** |\n",
    "| **[LAG and LEAD](#LaL)**|**[Aggregate Functions](#WAF)** |**[Common Table Expressions](#CTE)** |\n",
    "\n",
    "**<a id='RN'>ROW_NUMBER</a>**\n",
    "\n",
    "**Example 1: Write a query that upon execution, assigns a row number to all managers we have information for in the \"employees\" database (regardless of their department).**\n",
    "\n",
    "**Let the numbering disregard the department the managers have worked in. Also, let it start from the value of 1. Assign that value to the manager with the lowest employee number.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 741,
   "id": "a84b0cbe",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>row_number</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110022</td>\n",
       "      <td>d001</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>110039</td>\n",
       "      <td>d001</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>110085</td>\n",
       "      <td>d002</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>110114</td>\n",
       "      <td>d002</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>110183</td>\n",
       "      <td>d003</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>110228</td>\n",
       "      <td>d003</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>110303</td>\n",
       "      <td>d004</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>110344</td>\n",
       "      <td>d004</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>110386</td>\n",
       "      <td>d004</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>110420</td>\n",
       "      <td>d004</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>110511</td>\n",
       "      <td>d005</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>110567</td>\n",
       "      <td>d005</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>110725</td>\n",
       "      <td>d006</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>110765</td>\n",
       "      <td>d006</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>110800</td>\n",
       "      <td>d006</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>110854</td>\n",
       "      <td>d006</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>111035</td>\n",
       "      <td>d007</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>111133</td>\n",
       "      <td>d007</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>111400</td>\n",
       "      <td>d008</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>111534</td>\n",
       "      <td>d008</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>111692</td>\n",
       "      <td>d009</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>111784</td>\n",
       "      <td>d009</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>111877</td>\n",
       "      <td>d009</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>111939</td>\n",
       "      <td>d009</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    emp_no dept_no   first_name     last_name  row_number\n",
       "0   110022    d001    Margareta    Markovitch           1\n",
       "1   110039    d001     Vishwani      Minakawa           2\n",
       "2   110085    d002         Ebru         Alpin           3\n",
       "3   110114    d002        Isamu    Legleitner           4\n",
       "4   110183    d003      Shirish  Ossenbruggen           5\n",
       "5   110228    d003      Karsten       Sigstam           6\n",
       "6   110303    d004    Krassimir       Wegerle           7\n",
       "7   110344    d004       Rosine         Cools           8\n",
       "8   110386    d004         Shem        Kieras           9\n",
       "9   110420    d004        Oscar      Ghazalie          10\n",
       "10  110511    d005     DeForest      Hagimont          11\n",
       "11  110567    d005         Leon      DasSarma          12\n",
       "12  110725    d006    Peternela       Onuegbe          13\n",
       "13  110765    d006       Rutger       Hofmeyr          14\n",
       "14  110800    d006       Sanjoy       Quadeer          15\n",
       "15  110854    d006         Dung         Pesch          16\n",
       "16  111035    d007  Przemyslawa     Kaelbling          17\n",
       "17  111133    d007        Hauke         Zhang          18\n",
       "18  111400    d008         Arie       Staelin          19\n",
       "19  111534    d008       Hilary        Kambil          20\n",
       "20  111692    d009        Tonny   Butterworth          21\n",
       "21  111784    d009        Marjo    Giarratana          22\n",
       "22  111877    d009      Xiaobin      Spinelli          23\n",
       "23  111939    d009      Yuchang       Weedman          24"
      ]
     },
     "execution_count": 741,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(man,emp,'emp_no','left').reset_index();\n",
    "tmp = tmp[['emp_no','dept_no','first_name','last_name']]\n",
    "tmp[\"row_number\"] = tmp.index+1\n",
    "tmp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "547b533a",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 2: Get the employee number of each manager**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "8cac7771",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>emp_no</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>from_date</th>\n",
       "      <th>to_date</th>\n",
       "      <th>birth_date</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>gender</th>\n",
       "      <th>hire_date</th>\n",
       "      <th>row_number</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>110022</td>\n",
       "      <td>d001</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-10-01</td>\n",
       "      <td>1956-09-12</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>110039</td>\n",
       "      <td>d001</td>\n",
       "      <td>1991-10-01</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1963-06-21</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-04-12</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>110085</td>\n",
       "      <td>d002</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1989-12-17</td>\n",
       "      <td>1959-10-28</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>110114</td>\n",
       "      <td>d002</td>\n",
       "      <td>1989-12-17</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1957-03-28</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-14</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>110183</td>\n",
       "      <td>d003</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>1953-06-24</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>110228</td>\n",
       "      <td>d003</td>\n",
       "      <td>1992-03-21</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1958-12-02</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-08-04</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>110303</td>\n",
       "      <td>d004</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1988-09-09</td>\n",
       "      <td>1956-06-08</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>110344</td>\n",
       "      <td>d004</td>\n",
       "      <td>1988-09-09</td>\n",
       "      <td>1992-08-02</td>\n",
       "      <td>1961-09-07</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-11-22</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>110386</td>\n",
       "      <td>d004</td>\n",
       "      <td>1992-08-02</td>\n",
       "      <td>1996-08-30</td>\n",
       "      <td>1953-10-04</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>M</td>\n",
       "      <td>1988-10-14</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>110420</td>\n",
       "      <td>d004</td>\n",
       "      <td>1996-08-30</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1963-07-27</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>M</td>\n",
       "      <td>1992-02-05</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>110511</td>\n",
       "      <td>d005</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1992-04-25</td>\n",
       "      <td>1957-07-08</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>110567</td>\n",
       "      <td>d005</td>\n",
       "      <td>1992-04-25</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1964-04-25</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-10-21</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>110725</td>\n",
       "      <td>d006</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1989-05-06</td>\n",
       "      <td>1961-03-14</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>110765</td>\n",
       "      <td>d006</td>\n",
       "      <td>1989-05-06</td>\n",
       "      <td>1991-09-12</td>\n",
       "      <td>1954-05-22</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>F</td>\n",
       "      <td>1989-01-07</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>110800</td>\n",
       "      <td>d006</td>\n",
       "      <td>1991-09-12</td>\n",
       "      <td>1994-06-28</td>\n",
       "      <td>1963-02-07</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>F</td>\n",
       "      <td>1986-08-12</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>110854</td>\n",
       "      <td>d006</td>\n",
       "      <td>1994-06-28</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1960-08-19</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-06-09</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>111035</td>\n",
       "      <td>d007</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-03-07</td>\n",
       "      <td>1962-02-24</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>111133</td>\n",
       "      <td>d007</td>\n",
       "      <td>1991-03-07</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1955-03-16</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>M</td>\n",
       "      <td>1986-12-30</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>111400</td>\n",
       "      <td>d008</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1991-04-08</td>\n",
       "      <td>1959-11-09</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>M</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>111534</td>\n",
       "      <td>d008</td>\n",
       "      <td>1991-04-08</td>\n",
       "      <td>9999-01-01</td>\n",
       "      <td>1952-06-27</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>F</td>\n",
       "      <td>1988-01-31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>20</td>\n",
       "      <td>111692</td>\n",
       "      <td>d009</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1988-10-17</td>\n",
       "      <td>1954-10-05</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>F</td>\n",
       "      <td>1985-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>111784</td>\n",
       "      <td>d009</td>\n",
       "      <td>1988-10-17</td>\n",
       "      <td>1992-09-08</td>\n",
       "      <td>1956-06-14</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>F</td>\n",
       "      <td>1988-02-12</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>22</td>\n",
       "      <td>111877</td>\n",
       "      <td>d009</td>\n",
       "      <td>1992-09-08</td>\n",
       "      <td>1996-01-03</td>\n",
       "      <td>1962-10-18</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>F</td>\n",
       "      <td>1991-08-17</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>23</td>\n",
       "      <td>111939</td>\n",
       "      <td>d009</td>\n",
       "      <td>1996-01-03</td>\n",
       "      <td>9999-01-01;</td>\n",
       "      <td>1960-03-25</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>M</td>\n",
       "      <td>1989-07-10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    index  emp_no dept_no   from_date      to_date  birth_date   first_name  \\\n",
       "0       0  110022    d001  1985-01-01   1991-10-01  1956-09-12    Margareta   \n",
       "1       1  110039    d001  1991-10-01   9999-01-01  1963-06-21     Vishwani   \n",
       "2       2  110085    d002  1985-01-01   1989-12-17  1959-10-28         Ebru   \n",
       "3       3  110114    d002  1989-12-17   9999-01-01  1957-03-28        Isamu   \n",
       "4       4  110183    d003  1985-01-01   1992-03-21  1953-06-24      Shirish   \n",
       "5       5  110228    d003  1992-03-21   9999-01-01  1958-12-02      Karsten   \n",
       "6       6  110303    d004  1985-01-01   1988-09-09  1956-06-08    Krassimir   \n",
       "7       7  110344    d004  1988-09-09   1992-08-02  1961-09-07       Rosine   \n",
       "8       8  110386    d004  1992-08-02   1996-08-30  1953-10-04         Shem   \n",
       "9       9  110420    d004  1996-08-30   9999-01-01  1963-07-27        Oscar   \n",
       "10     10  110511    d005  1985-01-01   1992-04-25  1957-07-08     DeForest   \n",
       "11     11  110567    d005  1992-04-25   9999-01-01  1964-04-25         Leon   \n",
       "12     12  110725    d006  1985-01-01   1989-05-06  1961-03-14    Peternela   \n",
       "13     13  110765    d006  1989-05-06   1991-09-12  1954-05-22       Rutger   \n",
       "14     14  110800    d006  1991-09-12   1994-06-28  1963-02-07       Sanjoy   \n",
       "15     15  110854    d006  1994-06-28   9999-01-01  1960-08-19         Dung   \n",
       "16     16  111035    d007  1985-01-01   1991-03-07  1962-02-24  Przemyslawa   \n",
       "17     17  111133    d007  1991-03-07   9999-01-01  1955-03-16        Hauke   \n",
       "18     18  111400    d008  1985-01-01   1991-04-08  1959-11-09         Arie   \n",
       "19     19  111534    d008  1991-04-08   9999-01-01  1952-06-27       Hilary   \n",
       "20     20  111692    d009  1985-01-01   1988-10-17  1954-10-05        Tonny   \n",
       "21     21  111784    d009  1988-10-17   1992-09-08  1956-06-14        Marjo   \n",
       "22     22  111877    d009  1992-09-08   1996-01-03  1962-10-18      Xiaobin   \n",
       "23     23  111939    d009  1996-01-03  9999-01-01;  1960-03-25      Yuchang   \n",
       "\n",
       "       last_name gender   hire_date  row_number  \n",
       "0     Markovitch      M  1985-01-01           1  \n",
       "1       Minakawa      M  1986-04-12           1  \n",
       "2          Alpin      M  1985-01-01           1  \n",
       "3     Legleitner      F  1985-01-14           1  \n",
       "4   Ossenbruggen      F  1985-01-01           1  \n",
       "5        Sigstam      F  1985-08-04           1  \n",
       "6        Wegerle      F  1985-01-01           1  \n",
       "7          Cools      F  1985-11-22           1  \n",
       "8         Kieras      M  1988-10-14           1  \n",
       "9       Ghazalie      M  1992-02-05           1  \n",
       "10      Hagimont      M  1985-01-01           1  \n",
       "11      DasSarma      F  1986-10-21           1  \n",
       "12       Onuegbe      F  1985-01-01           1  \n",
       "13       Hofmeyr      F  1989-01-07           1  \n",
       "14       Quadeer      F  1986-08-12           1  \n",
       "15         Pesch      M  1989-06-09           1  \n",
       "16     Kaelbling      M  1985-01-01           1  \n",
       "17         Zhang      M  1986-12-30           1  \n",
       "18       Staelin      M  1985-01-01           1  \n",
       "19        Kambil      F  1988-01-31           1  \n",
       "20   Butterworth      F  1985-01-01           1  \n",
       "21    Giarratana      F  1988-02-12           1  \n",
       "22      Spinelli      F  1991-08-17           1  \n",
       "23       Weedman      M  1989-07-10           1  "
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(man,emp,'emp_no','left')\n",
    "tmp1= tmp.groupby(\"emp_no\").rank(method=\"first\", ascending=True)\n",
    "tmp[\"row_number\"] = tmp1['dept_no'].astype(int) # DOESN'T MATTER WHICH FIELD\n",
    "tmp1 = tmp[tmp[\"row_number\"]==1].reset_index()\n",
    "tmp1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b59efa1",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 3: Compare the salaries between departments**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "fd8e73c7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=============== Over all dates salaries in $000 are ===============\n",
      "\n",
      "  dept_no  Min salary  Ave salary  Max salary           dept_name\n",
      "0    d001     39127.0     71160.0    142753.0           Marketing\n",
      "1    d002     38812.0     69418.0    133230.0             Finance\n",
      "2    d003     38735.0     54600.0    120933.0     Human Resources\n",
      "3    d004     38836.0     58781.0    128773.0          Production\n",
      "4    d005     38850.0     58748.0    140142.0         Development\n",
      "5    d006     38928.0     56539.0    122376.0  Quality Management\n",
      "6    d007     39427.0     80013.0    157821.0               Sales\n",
      "7    d008     38851.0     59104.0    122121.0            Research\n",
      "8    d009     38836.0     57999.0    144088.0   Customer Service;\n"
     ]
    }
   ],
   "source": [
    "#DATE_RANGES(sal,'from_date','to_date')\n",
    "sal_fixed = sal[sal['to_date'] < '9999-01-01'] # These employees have left\n",
    "#DATE_RANGES(sal_fixed,'from_date','to_date') # Latest is now 2002-08-01 # CHOOSE DATE RANGES \n",
    "tmp = JOIN(emp,sal_fixed,'emp_no','left')\n",
    "tmp1 = JOIN(tmp,dept_emp,'emp_no','left')\n",
    "tmp = tmp1.dropna(subset = ['salary'])\n",
    "tmp1 = JOIN(tmp,dep,'dept_no','left')\n",
    "tmp1 = tmp1[['emp_no','first_name','last_name','salary','dept_no','dept_name','from_date_x','to_date_x']]\n",
    "\n",
    "tmp = GROUP_AGG(tmp1,'dept_no','salary','min')\n",
    "tmp = tmp.rename(columns={'salary': 'Min salary'});# print(tmp)\n",
    "\n",
    "tmp2 = GROUP_AGG(tmp1,'dept_no','salary','mean'); \n",
    "tmp2 = tmp2.rename(columns={'salary': 'Ave salary'});\n",
    "tmp2['Ave salary'] = round(tmp2['Ave salary'],0)\n",
    "tmp['Ave salary'] = tmp2['Ave salary'] \n",
    "\n",
    "tmp3 = GROUP_AGG(tmp1,'dept_no','salary','max');\n",
    "tmp3 = tmp3.rename(columns={'salary':'Max salary'})\n",
    "out = JOIN(tmp,tmp3,'dept_no', 'left')\n",
    "del out['emp_no_x']; del out['emp_no_y']\n",
    "out2 = JOIN(out,dep,'dept_no', 'left')\n",
    "print(\"\\n=============== Over all dates salaries in $000 are ===============\\n\") \n",
    "print(out2)\n",
    "\n",
    "###### VERSION WITH HISTOGRAMS - UNCOMMENT AT BOTTOM TO RUN ###\n",
    "def plot_histos(bins,date1,date2):\n",
    "    import matplotlib.pyplot as plt\n",
    "    \n",
    "    tmp = SELECT_BETWEEN(tmp1,'to_date_x',date1,date2,'y'); # FILTER TIME RANGE AS WHOLE SET TAKES A WHILE\n",
    "    depts = np.sort(tmp['dept_no'].unique())\n",
    "    \n",
    "    print(\"\\n=============== Over %s to %s salaries in $000 are ===============\\n\" %(date1,date2))\n",
    "    for i in range (0,len(depts)): \n",
    "        data = tmp[tmp['dept_no'] == depts[i]]\n",
    "        dept_name = data['dept_name'].iloc[0]\n",
    "        print(\"For %s [%s], %d staff -  salary: mean = %1.3f median = %1.3f max = %1.3f min = %1.3f\" \n",
    "              %(depts[i],dept_name, len(data), 1e-3*data['salary'].mean(), 1e-3*data['salary'].median(),\n",
    "                1e-3*max(data['salary']),1e-3*min(data['salary'])))\n",
    "    \n",
    "    font = 14\n",
    "    colors = ['r','orange','gold','g','b','indigo','blueviolet','dimgrey','slategrey']    \n",
    "\n",
    "    for i in range (0,len(depts)): \n",
    "        plt.rcParams.update({'font.size': font})\n",
    "        plt.figure(figsize=(5,4))\n",
    "        ax = plt.gca()\n",
    "        ax.tick_params(direction='in', length=6, width=1.5, which='major')\n",
    "        ax.tick_params(direction='in', length=3, width=1.5, which='minor')\n",
    "        plt.setp(ax.spines.values(),linewidth=2)\n",
    "        \n",
    "        data = tmp[tmp['dept_no'] == depts[i]]\n",
    "        dept_name = data['dept_name'].iloc[0]\n",
    "        plt.ylabel('No. in %s' %(dept_name), size=font)\n",
    "        plt.xlabel(\"Salaries in Dept %s [$000]\" %(depts[i]), size=font)\n",
    "    \n",
    "        x1 = 1e-3*min(data['salary']); x2 = 1e-3*max(data['salary'])\n",
    "        ax.hist(1e-3*data['salary'], bins = bins, color=\"w\", edgecolor=colors[i], linewidth=3);\n",
    "        y1,y2 = ax.get_ylim(); y2 = 1.1*y2 \n",
    "        ax.set_ylim(x1,x2); ax.set_ylim(y1,y2)\n",
    "        xpos = x2-(x2-x1)/4; ypos = y2-(y2-y1)/12; yskip = (y2-y1)/12;\n",
    "        mean = 1e-3*np.mean(data['salary']); std = 1e-3*np.std(data['salary'])\n",
    "        plt.text(xpos,ypos,\"\\u03BC = %1.2fk\" %(mean), fontsize = 0.8*font, c = 'k') \n",
    "        plt.text(xpos,ypos-yskip,\"\\u03C3  = %1.2fk\" %(std), fontsize = 0.8*font, c = 'k') \n",
    "        plt.tight_layout()\n",
    "        plt.show()\n",
    "#plot_histos(10,'2002-07-01','2002-08-01') # Uncomment for histograms of each department"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7914d6f3",
   "metadata": {},
   "source": [
    "See https://github.com/steviecurran/two-sample to apply AB testing between these\n",
    "\n",
    "[Back to contents](#cont)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9573a65e",
   "metadata": {},
   "source": [
    "**<a id='PB'>PARTITION BY</a>**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b2b0805",
   "metadata": {},
   "source": [
    "**Example 1: Retrieve the employee number and the lowest contract salary value for all managers.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "8e990c94",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>emp_no</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>min_sal</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17</td>\n",
       "      <td>110022</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>d001</td>\n",
       "      <td>108407</td>\n",
       "      <td>71166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>110039</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>d001</td>\n",
       "      <td>106491</td>\n",
       "      <td>69941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>52</td>\n",
       "      <td>110085</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>d002</td>\n",
       "      <td>88443</td>\n",
       "      <td>60026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>70</td>\n",
       "      <td>110114</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>d002</td>\n",
       "      <td>83457</td>\n",
       "      <td>52070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>88</td>\n",
       "      <td>110183</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>d003</td>\n",
       "      <td>79229</td>\n",
       "      <td>48291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>106</td>\n",
       "      <td>110228</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>d003</td>\n",
       "      <td>65400</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>124</td>\n",
       "      <td>110303</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>d004</td>\n",
       "      <td>72583</td>\n",
       "      <td>42093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>141</td>\n",
       "      <td>110344</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>d004</td>\n",
       "      <td>75121</td>\n",
       "      <td>50365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>110386</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>d004</td>\n",
       "      <td>66995</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>166</td>\n",
       "      <td>110420</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>d004</td>\n",
       "      <td>56654</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>180</td>\n",
       "      <td>110511</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>d005</td>\n",
       "      <td>74056</td>\n",
       "      <td>48626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200</td>\n",
       "      <td>110567</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>d005</td>\n",
       "      <td>74510</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>218</td>\n",
       "      <td>110725</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>d006</td>\n",
       "      <td>93193</td>\n",
       "      <td>61357</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>232</td>\n",
       "      <td>110765</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>d006</td>\n",
       "      <td>72432</td>\n",
       "      <td>40055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>248</td>\n",
       "      <td>110800</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>d006</td>\n",
       "      <td>77626</td>\n",
       "      <td>50457</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>262</td>\n",
       "      <td>110854</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>d006</td>\n",
       "      <td>72876</td>\n",
       "      <td>45169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>280</td>\n",
       "      <td>111035</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>d007</td>\n",
       "      <td>95873</td>\n",
       "      <td>71612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>296</td>\n",
       "      <td>111133</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>d007</td>\n",
       "      <td>101987</td>\n",
       "      <td>70787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>314</td>\n",
       "      <td>111400</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>d008</td>\n",
       "      <td>103244</td>\n",
       "      <td>72446</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>329</td>\n",
       "      <td>111534</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>d008</td>\n",
       "      <td>79393</td>\n",
       "      <td>48077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>347</td>\n",
       "      <td>111692</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>d009</td>\n",
       "      <td>73953</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>362</td>\n",
       "      <td>111784</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>d009</td>\n",
       "      <td>58189</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>373</td>\n",
       "      <td>111877</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>d009</td>\n",
       "      <td>71406</td>\n",
       "      <td>53846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>386</td>\n",
       "      <td>111939</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>d009</td>\n",
       "      <td>58968</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  emp_no   first_name     last_name dept_no  salary  min_sal\n",
       "0     17  110022    Margareta    Markovitch    d001  108407    71166\n",
       "0     34  110039     Vishwani      Minakawa    d001  106491    69941\n",
       "0     52  110085         Ebru         Alpin    d002   88443    60026\n",
       "0     70  110114        Isamu    Legleitner    d002   83457    52070\n",
       "0     88  110183      Shirish  Ossenbruggen    d003   79229    48291\n",
       "0    106  110228      Karsten       Sigstam    d003   65400    40000\n",
       "0    124  110303    Krassimir       Wegerle    d004   72583    42093\n",
       "0    141  110344       Rosine         Cools    d004   75121    50365\n",
       "0    155  110386         Shem        Kieras    d004   66995    40000\n",
       "0    166  110420        Oscar      Ghazalie    d004   56654    40000\n",
       "0    180  110511     DeForest      Hagimont    d005   74056    48626\n",
       "0    200  110567         Leon      DasSarma    d005   74510    40000\n",
       "0    218  110725    Peternela       Onuegbe    d006   93193    61357\n",
       "0    232  110765       Rutger       Hofmeyr    d006   72432    40055\n",
       "0    248  110800       Sanjoy       Quadeer    d006   77626    50457\n",
       "0    262  110854         Dung         Pesch    d006   72876    45169\n",
       "0    280  111035  Przemyslawa     Kaelbling    d007   95873    71612\n",
       "0    296  111133        Hauke         Zhang    d007  101987    70787\n",
       "0    314  111400         Arie       Staelin    d008  103244    72446\n",
       "0    329  111534       Hilary        Kambil    d008   79393    48077\n",
       "0    347  111692        Tonny   Butterworth    d009   73953    40000\n",
       "0    362  111784        Marjo    Giarratana    d009   58189    40000\n",
       "0    373  111877      Xiaobin      Spinelli    d009   71406    53846\n",
       "0    386  111939      Yuchang       Weedman    d009   58968    40000"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(man,sal,'emp_no','left')\n",
    "tmp1 = JOIN(tmp,emp,'emp_no','left'); #print(tmp1)\n",
    "tmp1 = tmp1[['emp_no','first_name','last_name','dept_no','salary']]\n",
    "\n",
    "def ID_split(data,ID,field,rank,label):\n",
    "    dfs = []\n",
    "    IDs = tmp1['emp_no'].unique()\n",
    "    for i in range (0,len(IDs)):\n",
    "        tmp= data[data[ID] == IDs[i]]\n",
    "        tmp = tmp.sort_values(by=[field], ascending=False).reset_index();\n",
    "        tmp[label] = tmp[field].iloc[rank-1] #min(tmp[field]); \n",
    "        try:\n",
    "            tmp2=tmp.iloc[[0]]; \n",
    "            dfs.append(tmp2)\n",
    "        except IndexError: \n",
    "            pass\n",
    "        #del tmp2[field]\n",
    "        del tmp['index']\n",
    "    out = pd.concat(dfs)\n",
    "    return out\n",
    "ID_split(tmp1,'emp_no','salary',0,'min_sal')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e568bb7",
   "metadata": {},
   "source": [
    "**Example 2: Retrieve the employee number and the highest contract salary value for all managers.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "2c5d925a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>emp_no</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>max_sal</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17</td>\n",
       "      <td>110022</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>d001</td>\n",
       "      <td>108407</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>110039</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>d001</td>\n",
       "      <td>106491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>52</td>\n",
       "      <td>110085</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>d002</td>\n",
       "      <td>88443</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>70</td>\n",
       "      <td>110114</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>d002</td>\n",
       "      <td>83457</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>88</td>\n",
       "      <td>110183</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>d003</td>\n",
       "      <td>79229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>106</td>\n",
       "      <td>110228</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>d003</td>\n",
       "      <td>65400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>124</td>\n",
       "      <td>110303</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>d004</td>\n",
       "      <td>72583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>141</td>\n",
       "      <td>110344</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>d004</td>\n",
       "      <td>75121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>110386</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>d004</td>\n",
       "      <td>66995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>166</td>\n",
       "      <td>110420</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>d004</td>\n",
       "      <td>56654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>180</td>\n",
       "      <td>110511</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>d005</td>\n",
       "      <td>74056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200</td>\n",
       "      <td>110567</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>d005</td>\n",
       "      <td>74510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>218</td>\n",
       "      <td>110725</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>d006</td>\n",
       "      <td>93193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>232</td>\n",
       "      <td>110765</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>d006</td>\n",
       "      <td>72432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>248</td>\n",
       "      <td>110800</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>d006</td>\n",
       "      <td>77626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>262</td>\n",
       "      <td>110854</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>d006</td>\n",
       "      <td>72876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>280</td>\n",
       "      <td>111035</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>d007</td>\n",
       "      <td>95873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>296</td>\n",
       "      <td>111133</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>d007</td>\n",
       "      <td>101987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>314</td>\n",
       "      <td>111400</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>d008</td>\n",
       "      <td>103244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>329</td>\n",
       "      <td>111534</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>d008</td>\n",
       "      <td>79393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>347</td>\n",
       "      <td>111692</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>d009</td>\n",
       "      <td>73953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>362</td>\n",
       "      <td>111784</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>d009</td>\n",
       "      <td>58189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>373</td>\n",
       "      <td>111877</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>d009</td>\n",
       "      <td>71406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>386</td>\n",
       "      <td>111939</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>d009</td>\n",
       "      <td>58968</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  emp_no   first_name     last_name dept_no  max_sal\n",
       "0     17  110022    Margareta    Markovitch    d001   108407\n",
       "0     34  110039     Vishwani      Minakawa    d001   106491\n",
       "0     52  110085         Ebru         Alpin    d002    88443\n",
       "0     70  110114        Isamu    Legleitner    d002    83457\n",
       "0     88  110183      Shirish  Ossenbruggen    d003    79229\n",
       "0    106  110228      Karsten       Sigstam    d003    65400\n",
       "0    124  110303    Krassimir       Wegerle    d004    72583\n",
       "0    141  110344       Rosine         Cools    d004    75121\n",
       "0    155  110386         Shem        Kieras    d004    66995\n",
       "0    166  110420        Oscar      Ghazalie    d004    56654\n",
       "0    180  110511     DeForest      Hagimont    d005    74056\n",
       "0    200  110567         Leon      DasSarma    d005    74510\n",
       "0    218  110725    Peternela       Onuegbe    d006    93193\n",
       "0    232  110765       Rutger       Hofmeyr    d006    72432\n",
       "0    248  110800       Sanjoy       Quadeer    d006    77626\n",
       "0    262  110854         Dung         Pesch    d006    72876\n",
       "0    280  111035  Przemyslawa     Kaelbling    d007    95873\n",
       "0    296  111133        Hauke         Zhang    d007   101987\n",
       "0    314  111400         Arie       Staelin    d008   103244\n",
       "0    329  111534       Hilary        Kambil    d008    79393\n",
       "0    347  111692        Tonny   Butterworth    d009    73953\n",
       "0    362  111784        Marjo    Giarratana    d009    58189\n",
       "0    373  111877      Xiaobin      Spinelli    d009    71406\n",
       "0    386  111939      Yuchang       Weedman    d009    58968"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ID_split(tmp1,'emp_no','salary',1,'max_sal')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c651338",
   "metadata": {},
   "source": [
    "**Example 3: Retrieve the employee number and the third-highest contract salary value\n",
    "for all managers.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "6c6a8984",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>emp_no</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>dept_no</th>\n",
       "      <th>3rd highest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17</td>\n",
       "      <td>110022</td>\n",
       "      <td>Margareta</td>\n",
       "      <td>Markovitch</td>\n",
       "      <td>d001</td>\n",
       "      <td>100592</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>110039</td>\n",
       "      <td>Vishwani</td>\n",
       "      <td>Minakawa</td>\n",
       "      <td>d001</td>\n",
       "      <td>101901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>52</td>\n",
       "      <td>110085</td>\n",
       "      <td>Ebru</td>\n",
       "      <td>Alpin</td>\n",
       "      <td>d002</td>\n",
       "      <td>85394</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>70</td>\n",
       "      <td>110114</td>\n",
       "      <td>Isamu</td>\n",
       "      <td>Legleitner</td>\n",
       "      <td>d002</td>\n",
       "      <td>80423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>88</td>\n",
       "      <td>110183</td>\n",
       "      <td>Shirish</td>\n",
       "      <td>Ossenbruggen</td>\n",
       "      <td>d003</td>\n",
       "      <td>74584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>106</td>\n",
       "      <td>110228</td>\n",
       "      <td>Karsten</td>\n",
       "      <td>Sigstam</td>\n",
       "      <td>d003</td>\n",
       "      <td>62276</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>124</td>\n",
       "      <td>110303</td>\n",
       "      <td>Krassimir</td>\n",
       "      <td>Wegerle</td>\n",
       "      <td>d004</td>\n",
       "      <td>68483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>141</td>\n",
       "      <td>110344</td>\n",
       "      <td>Rosine</td>\n",
       "      <td>Cools</td>\n",
       "      <td>d004</td>\n",
       "      <td>73754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>110386</td>\n",
       "      <td>Shem</td>\n",
       "      <td>Kieras</td>\n",
       "      <td>d004</td>\n",
       "      <td>61207</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>166</td>\n",
       "      <td>110420</td>\n",
       "      <td>Oscar</td>\n",
       "      <td>Ghazalie</td>\n",
       "      <td>d004</td>\n",
       "      <td>53773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>180</td>\n",
       "      <td>110511</td>\n",
       "      <td>DeForest</td>\n",
       "      <td>Hagimont</td>\n",
       "      <td>d005</td>\n",
       "      <td>73350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200</td>\n",
       "      <td>110567</td>\n",
       "      <td>Leon</td>\n",
       "      <td>DasSarma</td>\n",
       "      <td>d005</td>\n",
       "      <td>66307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>218</td>\n",
       "      <td>110725</td>\n",
       "      <td>Peternela</td>\n",
       "      <td>Onuegbe</td>\n",
       "      <td>d006</td>\n",
       "      <td>91012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>232</td>\n",
       "      <td>110765</td>\n",
       "      <td>Rutger</td>\n",
       "      <td>Hofmeyr</td>\n",
       "      <td>d006</td>\n",
       "      <td>65921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>248</td>\n",
       "      <td>110800</td>\n",
       "      <td>Sanjoy</td>\n",
       "      <td>Quadeer</td>\n",
       "      <td>d006</td>\n",
       "      <td>77088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>262</td>\n",
       "      <td>110854</td>\n",
       "      <td>Dung</td>\n",
       "      <td>Pesch</td>\n",
       "      <td>d006</td>\n",
       "      <td>69035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>280</td>\n",
       "      <td>111035</td>\n",
       "      <td>Przemyslawa</td>\n",
       "      <td>Kaelbling</td>\n",
       "      <td>d007</td>\n",
       "      <td>93765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>296</td>\n",
       "      <td>111133</td>\n",
       "      <td>Hauke</td>\n",
       "      <td>Zhang</td>\n",
       "      <td>d007</td>\n",
       "      <td>99072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>314</td>\n",
       "      <td>111400</td>\n",
       "      <td>Arie</td>\n",
       "      <td>Staelin</td>\n",
       "      <td>d008</td>\n",
       "      <td>99971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>329</td>\n",
       "      <td>111534</td>\n",
       "      <td>Hilary</td>\n",
       "      <td>Kambil</td>\n",
       "      <td>d008</td>\n",
       "      <td>78032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>347</td>\n",
       "      <td>111692</td>\n",
       "      <td>Tonny</td>\n",
       "      <td>Butterworth</td>\n",
       "      <td>d009</td>\n",
       "      <td>72421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>362</td>\n",
       "      <td>111784</td>\n",
       "      <td>Marjo</td>\n",
       "      <td>Giarratana</td>\n",
       "      <td>d009</td>\n",
       "      <td>54187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>373</td>\n",
       "      <td>111877</td>\n",
       "      <td>Xiaobin</td>\n",
       "      <td>Spinelli</td>\n",
       "      <td>d009</td>\n",
       "      <td>67470</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>386</td>\n",
       "      <td>111939</td>\n",
       "      <td>Yuchang</td>\n",
       "      <td>Weedman</td>\n",
       "      <td>d009</td>\n",
       "      <td>58698</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  emp_no   first_name     last_name dept_no  3rd highest\n",
       "0     17  110022    Margareta    Markovitch    d001       100592\n",
       "0     34  110039     Vishwani      Minakawa    d001       101901\n",
       "0     52  110085         Ebru         Alpin    d002        85394\n",
       "0     70  110114        Isamu    Legleitner    d002        80423\n",
       "0     88  110183      Shirish  Ossenbruggen    d003        74584\n",
       "0    106  110228      Karsten       Sigstam    d003        62276\n",
       "0    124  110303    Krassimir       Wegerle    d004        68483\n",
       "0    141  110344       Rosine         Cools    d004        73754\n",
       "0    155  110386         Shem        Kieras    d004        61207\n",
       "0    166  110420        Oscar      Ghazalie    d004        53773\n",
       "0    180  110511     DeForest      Hagimont    d005        73350\n",
       "0    200  110567         Leon      DasSarma    d005        66307\n",
       "0    218  110725    Peternela       Onuegbe    d006        91012\n",
       "0    232  110765       Rutger       Hofmeyr    d006        65921\n",
       "0    248  110800       Sanjoy       Quadeer    d006        77088\n",
       "0    262  110854         Dung         Pesch    d006        69035\n",
       "0    280  111035  Przemyslawa     Kaelbling    d007        93765\n",
       "0    296  111133        Hauke         Zhang    d007        99072\n",
       "0    314  111400         Arie       Staelin    d008        99971\n",
       "0    329  111534       Hilary        Kambil    d008        78032\n",
       "0    347  111692        Tonny   Butterworth    d009        72421\n",
       "0    362  111784        Marjo    Giarratana    d009        54187\n",
       "0    373  111877      Xiaobin      Spinelli    d009        67470\n",
       "0    386  111939      Yuchang       Weedman    d009        58698"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ID_split(tmp1,'emp_no','salary',3,'3rd highest')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29953a6d",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='RANK'>RANK</a>**\n",
    "\n",
    "**Example 1: Order and number all contract salary values of employee 10002 from\n",
    "highest to lowest. Store the row numbers in a third column.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1101,
   "id": "d1472492",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>order no</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10002</td>\n",
       "      <td>72527.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>71963.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10002</td>\n",
       "      <td>69366.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10002</td>\n",
       "      <td>67534.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10002</td>\n",
       "      <td>65909.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10002</td>\n",
       "      <td>65828.0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   emp_no   salary  order no\n",
       "0   10002  72527.0         1\n",
       "1   10002  71963.0         2\n",
       "2   10002  69366.0         3\n",
       "3   10002  67534.0         4\n",
       "4   10002  65909.0         5\n",
       "5   10002  65828.0         6"
      ]
     },
     "execution_count": 1101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(emp,sal,'emp_no','left'); #print(tmp)\n",
    "tmp1 = SELECT(tmp,'emp_no',10002); #print(tmp1.columns)\n",
    "tmp1 = tmp1[['emp_no', 'salary']]\n",
    "tmp1 = tmp1.sort_values(by=['salary'], ascending=False).reset_index();#print(tmp1)\n",
    "del tmp1['index']\n",
    "tmp1['order no'] = tmp1.index +1\n",
    "tmp1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc503232",
   "metadata": {},
   "source": [
    "**Example 2: Rank the contract salary values\n",
    "from highest to lowest for employees 10001, 10002, 10003, 10004, 10005, and 10006. Every row in the desired output should contain an employee number obtained from the employees table and a salary value obtained from the salaries table. Additionally, include the salary ranking values\n",
    "between the two columns**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a2822ef1",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>rank</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>88958.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10001</td>\n",
       "      <td>85112.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001</td>\n",
       "      <td>85097.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10001</td>\n",
       "      <td>84917.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10001</td>\n",
       "      <td>81097.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10006</td>\n",
       "      <td>47518.0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10006</td>\n",
       "      <td>45844.0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10006</td>\n",
       "      <td>42629.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10006</td>\n",
       "      <td>42085.0</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10006</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>71 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    emp_no   salary  rank\n",
       "0    10001  88958.0     1\n",
       "1    10001  85112.0     2\n",
       "2    10001  85097.0     3\n",
       "3    10001  84917.0     4\n",
       "4    10001  81097.0     5\n",
       "..     ...      ...   ...\n",
       "7    10006  47518.0     8\n",
       "8    10006  45844.0     9\n",
       "9    10006  42629.0    10\n",
       "10   10006  42085.0    11\n",
       "11   10006  40000.0    12\n",
       "\n",
       "[71 rows x 3 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#pd.set_option('display.max_rows', None)\n",
    "tmp = JOIN(emp,sal,'emp_no','left')\n",
    "IDs = []\n",
    "for i in range (10001,10007):\n",
    "    IDs.append(i)\n",
    "#print(IDs); print(tmp); print(tmp.columns)\n",
    "tmp1 = tmp[['emp_no','salary']]\n",
    "def rank_by(data,ID,field,asc):\n",
    "    dfs = []\n",
    "    for i in range (0,len(IDs)):\n",
    "        tmp= data[data[ID] == IDs[i]]\n",
    "        tmp = tmp.sort_values(by=[field], ascending=asc).reset_index()\n",
    "        tmp['rank'] = tmp.index +1\n",
    "        dfs.append(tmp)\n",
    "    out = pd.concat(dfs)\n",
    "    del out['index']\n",
    "    return out\n",
    "rank_by(tmp1,'emp_no','salary',False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2003c363",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 3: Rank the contract salary\n",
    "values from highest to lowest for employees 10001, 10002, and 10003.**\n",
    "\n",
    "**Every row in the desired output should contain the relevant employee number\n",
    "and the hire date from the employees table, as well as the relevant salary value and the\n",
    "start date from the salaries table. Additionally, include the salaryranking values.**\n",
    "\n",
    "**Retrieve only data for contracts that have started prior to 2000. Sort your data by the\n",
    "emplyee number in ascending order.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "dc87cc77",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>hire_date</th>\n",
       "      <th>from_date</th>\n",
       "      <th>rank</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>84917.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1999-06-23</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10001</td>\n",
       "      <td>81097.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1998-06-23</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001</td>\n",
       "      <td>81025.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1997-06-23</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10001</td>\n",
       "      <td>80013.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1996-06-23</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10001</td>\n",
       "      <td>76884.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1995-06-24</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10001</td>\n",
       "      <td>75994.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1994-06-24</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10001</td>\n",
       "      <td>75286.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1993-06-24</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10001</td>\n",
       "      <td>74333.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1992-06-24</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10001</td>\n",
       "      <td>71046.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1991-06-25</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10001</td>\n",
       "      <td>66961.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1990-06-25</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10001</td>\n",
       "      <td>66596.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1989-06-25</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10001</td>\n",
       "      <td>66074.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1988-06-25</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10001</td>\n",
       "      <td>62102.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1987-06-26</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10001</td>\n",
       "      <td>60117.0</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10002</td>\n",
       "      <td>69366.0</td>\n",
       "      <td>1985-11-21</td>\n",
       "      <td>1999-08-03</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10002</td>\n",
       "      <td>67534.0</td>\n",
       "      <td>1985-11-21</td>\n",
       "      <td>1998-08-03</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10002</td>\n",
       "      <td>65909.0</td>\n",
       "      <td>1985-11-21</td>\n",
       "      <td>1997-08-03</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10002</td>\n",
       "      <td>65828.0</td>\n",
       "      <td>1985-11-21</td>\n",
       "      <td>1996-08-03</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10003</td>\n",
       "      <td>43636.0</td>\n",
       "      <td>1986-08-28</td>\n",
       "      <td>1998-12-02</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10003</td>\n",
       "      <td>43616.0</td>\n",
       "      <td>1986-08-28</td>\n",
       "      <td>1996-12-02</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>43478.0</td>\n",
       "      <td>1986-08-28</td>\n",
       "      <td>1999-12-02</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10003</td>\n",
       "      <td>43466.0</td>\n",
       "      <td>1986-08-28</td>\n",
       "      <td>1997-12-02</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10003</td>\n",
       "      <td>40006.0</td>\n",
       "      <td>1986-08-28</td>\n",
       "      <td>1995-12-03</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    emp_no   salary   hire_date   from_date  rank\n",
       "0    10001  84917.0  1986-06-26  1999-06-23     1\n",
       "1    10001  81097.0  1986-06-26  1998-06-23     2\n",
       "2    10001  81025.0  1986-06-26  1997-06-23     3\n",
       "3    10001  80013.0  1986-06-26  1996-06-23     4\n",
       "4    10001  76884.0  1986-06-26  1995-06-24     5\n",
       "5    10001  75994.0  1986-06-26  1994-06-24     6\n",
       "6    10001  75286.0  1986-06-26  1993-06-24     7\n",
       "7    10001  74333.0  1986-06-26  1992-06-24     8\n",
       "8    10001  71046.0  1986-06-26  1991-06-25     9\n",
       "9    10001  66961.0  1986-06-26  1990-06-25    10\n",
       "10   10001  66596.0  1986-06-26  1989-06-25    11\n",
       "11   10001  66074.0  1986-06-26  1988-06-25    12\n",
       "12   10001  62102.0  1986-06-26  1987-06-26    13\n",
       "13   10001  60117.0  1986-06-26  1986-06-26    14\n",
       "0    10002  69366.0  1985-11-21  1999-08-03     1\n",
       "1    10002  67534.0  1985-11-21  1998-08-03     2\n",
       "2    10002  65909.0  1985-11-21  1997-08-03     3\n",
       "3    10002  65828.0  1985-11-21  1996-08-03     4\n",
       "0    10003  43636.0  1986-08-28  1998-12-02     1\n",
       "1    10003  43616.0  1986-08-28  1996-12-02     2\n",
       "2    10003  43478.0  1986-08-28  1999-12-02     3\n",
       "3    10003  43466.0  1986-08-28  1997-12-02     4\n",
       "4    10003  40006.0  1986-08-28  1995-12-03     5"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(emp,sal,'emp_no','left')\n",
    "tmp1 = FILTER(tmp,'from_date','2000-01-01','<') \n",
    "IDs = [10001,10002,10003]\n",
    "#print(tmp.columns)\n",
    "tmp = tmp1[['emp_no','salary', 'hire_date','from_date']]\n",
    "rank_by(tmp,'emp_no','salary',False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07b623d6",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='LaL'>LAG and LEAD</a>**\n",
    "\n",
    "**Example 1: For employees with employee numbers between 10003 and 10008, inclusive, and their salary contracts with values less than $70 000, retrieve the following data from the salaries table:<br>\n",
    "• employee number<br>\n",
    "• salary<br>\n",
    "• previous salary<br>\n",
    "• next salary<br>\n",
    "• the difference between the current salary of a certain employee and their previous salary<br>\n",
    "• the difference between the next salary of a certain employee and their current salary<br>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "58a1f010",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>previous</th>\n",
       "      <th>next</th>\n",
       "      <th>$\\Delta$ previous</th>\n",
       "      <th>$\\Delta$ next</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10003</td>\n",
       "      <td>40006.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>43311.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3305.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10003</td>\n",
       "      <td>43311.0</td>\n",
       "      <td>40006.0</td>\n",
       "      <td>43466.0</td>\n",
       "      <td>3305.0</td>\n",
       "      <td>155.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10003</td>\n",
       "      <td>43466.0</td>\n",
       "      <td>43311.0</td>\n",
       "      <td>43478.0</td>\n",
       "      <td>155.0</td>\n",
       "      <td>12.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10003</td>\n",
       "      <td>43478.0</td>\n",
       "      <td>43466.0</td>\n",
       "      <td>43616.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>138.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10003</td>\n",
       "      <td>43616.0</td>\n",
       "      <td>43478.0</td>\n",
       "      <td>43636.0</td>\n",
       "      <td>138.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10003</td>\n",
       "      <td>43636.0</td>\n",
       "      <td>43616.0</td>\n",
       "      <td>43699.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>63.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10003</td>\n",
       "      <td>43699.0</td>\n",
       "      <td>43636.0</td>\n",
       "      <td>40054.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>-3645.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10004</td>\n",
       "      <td>40054.0</td>\n",
       "      <td>43699.0</td>\n",
       "      <td>42283.0</td>\n",
       "      <td>-3645.0</td>\n",
       "      <td>2229.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10004</td>\n",
       "      <td>42283.0</td>\n",
       "      <td>40054.0</td>\n",
       "      <td>42542.0</td>\n",
       "      <td>2229.0</td>\n",
       "      <td>259.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10004</td>\n",
       "      <td>42542.0</td>\n",
       "      <td>42283.0</td>\n",
       "      <td>46065.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3523.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10004</td>\n",
       "      <td>46065.0</td>\n",
       "      <td>42542.0</td>\n",
       "      <td>48271.0</td>\n",
       "      <td>3523.0</td>\n",
       "      <td>2206.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10004</td>\n",
       "      <td>48271.0</td>\n",
       "      <td>46065.0</td>\n",
       "      <td>50594.0</td>\n",
       "      <td>2206.0</td>\n",
       "      <td>2323.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10004</td>\n",
       "      <td>50594.0</td>\n",
       "      <td>48271.0</td>\n",
       "      <td>52119.0</td>\n",
       "      <td>2323.0</td>\n",
       "      <td>1525.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10004</td>\n",
       "      <td>52119.0</td>\n",
       "      <td>50594.0</td>\n",
       "      <td>54693.0</td>\n",
       "      <td>1525.0</td>\n",
       "      <td>2574.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10004</td>\n",
       "      <td>54693.0</td>\n",
       "      <td>52119.0</td>\n",
       "      <td>58326.0</td>\n",
       "      <td>2574.0</td>\n",
       "      <td>3633.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10004</td>\n",
       "      <td>58326.0</td>\n",
       "      <td>54693.0</td>\n",
       "      <td>60770.0</td>\n",
       "      <td>3633.0</td>\n",
       "      <td>2444.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10004</td>\n",
       "      <td>60770.0</td>\n",
       "      <td>58326.0</td>\n",
       "      <td>62566.0</td>\n",
       "      <td>2444.0</td>\n",
       "      <td>1796.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10004</td>\n",
       "      <td>62566.0</td>\n",
       "      <td>60770.0</td>\n",
       "      <td>64340.0</td>\n",
       "      <td>1796.0</td>\n",
       "      <td>1774.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10004</td>\n",
       "      <td>64340.0</td>\n",
       "      <td>62566.0</td>\n",
       "      <td>67096.0</td>\n",
       "      <td>1774.0</td>\n",
       "      <td>2756.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10004</td>\n",
       "      <td>67096.0</td>\n",
       "      <td>64340.0</td>\n",
       "      <td>69722.0</td>\n",
       "      <td>2756.0</td>\n",
       "      <td>2626.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10004</td>\n",
       "      <td>69722.0</td>\n",
       "      <td>67096.0</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>2626.0</td>\n",
       "      <td>-29722.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10006</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>69722.0</td>\n",
       "      <td>42085.0</td>\n",
       "      <td>-29722.0</td>\n",
       "      <td>2085.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10006</td>\n",
       "      <td>42085.0</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>42629.0</td>\n",
       "      <td>2085.0</td>\n",
       "      <td>544.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10006</td>\n",
       "      <td>42629.0</td>\n",
       "      <td>42085.0</td>\n",
       "      <td>45844.0</td>\n",
       "      <td>544.0</td>\n",
       "      <td>3215.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10006</td>\n",
       "      <td>45844.0</td>\n",
       "      <td>42629.0</td>\n",
       "      <td>47518.0</td>\n",
       "      <td>3215.0</td>\n",
       "      <td>1674.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10006</td>\n",
       "      <td>47518.0</td>\n",
       "      <td>45844.0</td>\n",
       "      <td>47917.0</td>\n",
       "      <td>1674.0</td>\n",
       "      <td>399.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10006</td>\n",
       "      <td>47917.0</td>\n",
       "      <td>47518.0</td>\n",
       "      <td>52255.0</td>\n",
       "      <td>399.0</td>\n",
       "      <td>4338.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10006</td>\n",
       "      <td>52255.0</td>\n",
       "      <td>47917.0</td>\n",
       "      <td>53747.0</td>\n",
       "      <td>4338.0</td>\n",
       "      <td>1492.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10006</td>\n",
       "      <td>53747.0</td>\n",
       "      <td>52255.0</td>\n",
       "      <td>56032.0</td>\n",
       "      <td>1492.0</td>\n",
       "      <td>2285.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10006</td>\n",
       "      <td>56032.0</td>\n",
       "      <td>53747.0</td>\n",
       "      <td>58299.0</td>\n",
       "      <td>2285.0</td>\n",
       "      <td>2267.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10006</td>\n",
       "      <td>58299.0</td>\n",
       "      <td>56032.0</td>\n",
       "      <td>59755.0</td>\n",
       "      <td>2267.0</td>\n",
       "      <td>1456.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10006</td>\n",
       "      <td>59755.0</td>\n",
       "      <td>58299.0</td>\n",
       "      <td>60098.0</td>\n",
       "      <td>1456.0</td>\n",
       "      <td>343.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10006</td>\n",
       "      <td>60098.0</td>\n",
       "      <td>59755.0</td>\n",
       "      <td>56724.0</td>\n",
       "      <td>343.0</td>\n",
       "      <td>-3374.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10007</td>\n",
       "      <td>56724.0</td>\n",
       "      <td>60098.0</td>\n",
       "      <td>60740.0</td>\n",
       "      <td>-3374.0</td>\n",
       "      <td>4016.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10007</td>\n",
       "      <td>60740.0</td>\n",
       "      <td>56724.0</td>\n",
       "      <td>62745.0</td>\n",
       "      <td>4016.0</td>\n",
       "      <td>2005.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10007</td>\n",
       "      <td>62745.0</td>\n",
       "      <td>60740.0</td>\n",
       "      <td>63208.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>463.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10007</td>\n",
       "      <td>63208.0</td>\n",
       "      <td>62745.0</td>\n",
       "      <td>63475.0</td>\n",
       "      <td>463.0</td>\n",
       "      <td>267.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10007</td>\n",
       "      <td>63475.0</td>\n",
       "      <td>63208.0</td>\n",
       "      <td>64563.0</td>\n",
       "      <td>267.0</td>\n",
       "      <td>1088.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10007</td>\n",
       "      <td>64563.0</td>\n",
       "      <td>63475.0</td>\n",
       "      <td>68833.0</td>\n",
       "      <td>1088.0</td>\n",
       "      <td>4270.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10007</td>\n",
       "      <td>68833.0</td>\n",
       "      <td>64563.0</td>\n",
       "      <td>46671.0</td>\n",
       "      <td>4270.0</td>\n",
       "      <td>-22162.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10008</td>\n",
       "      <td>46671.0</td>\n",
       "      <td>68833.0</td>\n",
       "      <td>48584.0</td>\n",
       "      <td>-22162.0</td>\n",
       "      <td>1913.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10008</td>\n",
       "      <td>48584.0</td>\n",
       "      <td>46671.0</td>\n",
       "      <td>52668.0</td>\n",
       "      <td>1913.0</td>\n",
       "      <td>4084.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10008</td>\n",
       "      <td>52668.0</td>\n",
       "      <td>48584.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4084.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    emp_no   salary  previous     next  $\\Delta$ previous  $\\Delta$ next\n",
       "0    10003  40006.0       NaN  43311.0                NaN         3305.0\n",
       "1    10003  43311.0   40006.0  43466.0             3305.0          155.0\n",
       "2    10003  43466.0   43311.0  43478.0              155.0           12.0\n",
       "3    10003  43478.0   43466.0  43616.0               12.0          138.0\n",
       "4    10003  43616.0   43478.0  43636.0              138.0           20.0\n",
       "5    10003  43636.0   43616.0  43699.0               20.0           63.0\n",
       "6    10003  43699.0   43636.0  40054.0               63.0        -3645.0\n",
       "0    10004  40054.0   43699.0  42283.0            -3645.0         2229.0\n",
       "1    10004  42283.0   40054.0  42542.0             2229.0          259.0\n",
       "2    10004  42542.0   42283.0  46065.0              259.0         3523.0\n",
       "3    10004  46065.0   42542.0  48271.0             3523.0         2206.0\n",
       "4    10004  48271.0   46065.0  50594.0             2206.0         2323.0\n",
       "5    10004  50594.0   48271.0  52119.0             2323.0         1525.0\n",
       "6    10004  52119.0   50594.0  54693.0             1525.0         2574.0\n",
       "7    10004  54693.0   52119.0  58326.0             2574.0         3633.0\n",
       "8    10004  58326.0   54693.0  60770.0             3633.0         2444.0\n",
       "9    10004  60770.0   58326.0  62566.0             2444.0         1796.0\n",
       "10   10004  62566.0   60770.0  64340.0             1796.0         1774.0\n",
       "11   10004  64340.0   62566.0  67096.0             1774.0         2756.0\n",
       "12   10004  67096.0   64340.0  69722.0             2756.0         2626.0\n",
       "13   10004  69722.0   67096.0  40000.0             2626.0       -29722.0\n",
       "0    10006  40000.0   69722.0  42085.0           -29722.0         2085.0\n",
       "1    10006  42085.0   40000.0  42629.0             2085.0          544.0\n",
       "2    10006  42629.0   42085.0  45844.0              544.0         3215.0\n",
       "3    10006  45844.0   42629.0  47518.0             3215.0         1674.0\n",
       "4    10006  47518.0   45844.0  47917.0             1674.0          399.0\n",
       "5    10006  47917.0   47518.0  52255.0              399.0         4338.0\n",
       "6    10006  52255.0   47917.0  53747.0             4338.0         1492.0\n",
       "7    10006  53747.0   52255.0  56032.0             1492.0         2285.0\n",
       "8    10006  56032.0   53747.0  58299.0             2285.0         2267.0\n",
       "9    10006  58299.0   56032.0  59755.0             2267.0         1456.0\n",
       "10   10006  59755.0   58299.0  60098.0             1456.0          343.0\n",
       "11   10006  60098.0   59755.0  56724.0              343.0        -3374.0\n",
       "0    10007  56724.0   60098.0  60740.0            -3374.0         4016.0\n",
       "1    10007  60740.0   56724.0  62745.0             4016.0         2005.0\n",
       "2    10007  62745.0   60740.0  63208.0             2005.0          463.0\n",
       "3    10007  63208.0   62745.0  63475.0              463.0          267.0\n",
       "4    10007  63475.0   63208.0  64563.0              267.0         1088.0\n",
       "5    10007  64563.0   63475.0  68833.0             1088.0         4270.0\n",
       "6    10007  68833.0   64563.0  46671.0             4270.0       -22162.0\n",
       "0    10008  46671.0   68833.0  48584.0           -22162.0         1913.0\n",
       "1    10008  48584.0   46671.0  52668.0             1913.0         4084.0\n",
       "2    10008  52668.0   48584.0      NaN             4084.0            NaN"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(emp,sal,'emp_no','left')\n",
    "tmp1 = FILTER(tmp,'salary',70000,'<'); #print(tmp1)\n",
    "IDs = []\n",
    "for i in range (10003,10009):\n",
    "    IDs.append(i)\n",
    "tmp = rank_by(tmp1,'emp_no','salary',True)\n",
    "tmp = tmp[['emp_no','salary']]\n",
    "tmp['previous'] = tmp.salary.shift(1)\n",
    "tmp['next'] = tmp.salary.shift(-1)\n",
    "tmp['$\\Delta$ previous'] = tmp['salary'] - tmp['previous']\n",
    "tmp['$\\Delta$ next'] = tmp['next'] - tmp['salary']\n",
    "tmp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d02d3ff",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 2: Retrieve the following data from the salaries table:<br>\n",
    "• employee number<br>\n",
    "• salary<br>\n",
    "• the salary value of three contracts prior to the given employee contract salary value, if applicable<br>\n",
    "• the salary value of three contracts after the given employee contract salary value, if applicable<br>\n",
    "Retrieve only the first one hundred rows of data.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "e306ff33",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>3_previous</th>\n",
       "      <th>3_next</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>60117.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>66596.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10001</td>\n",
       "      <td>62102.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>66961.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001</td>\n",
       "      <td>66074.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>71046.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10001</td>\n",
       "      <td>66596.0</td>\n",
       "      <td>60117.0</td>\n",
       "      <td>74333.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10001</td>\n",
       "      <td>66961.0</td>\n",
       "      <td>62102.0</td>\n",
       "      <td>75286.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10009</td>\n",
       "      <td>74612.0</td>\n",
       "      <td>69042.0</td>\n",
       "      <td>80944.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10009</td>\n",
       "      <td>76518.0</td>\n",
       "      <td>70889.0</td>\n",
       "      <td>82507.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10009</td>\n",
       "      <td>78335.0</td>\n",
       "      <td>71434.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10009</td>\n",
       "      <td>80944.0</td>\n",
       "      <td>74612.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10009</td>\n",
       "      <td>82507.0</td>\n",
       "      <td>76518.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    emp_no   salary  3_previous   3_next\n",
       "0    10001  60117.0         NaN  66596.0\n",
       "1    10001  62102.0         NaN  66961.0\n",
       "2    10001  66074.0         NaN  71046.0\n",
       "3    10001  66596.0     60117.0  74333.0\n",
       "4    10001  66961.0     62102.0  75286.0\n",
       "..     ...      ...         ...      ...\n",
       "7    10009  74612.0     69042.0  80944.0\n",
       "8    10009  76518.0     70889.0  82507.0\n",
       "9    10009  78335.0     71434.0      NaN\n",
       "10   10009  80944.0     74612.0      NaN\n",
       "11   10009  82507.0     76518.0      NaN\n",
       "\n",
       "[100 rows x 4 columns]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(emp,sal,'emp_no','left')\n",
    "tmp1 = tmp.head(100)\n",
    "IDs = tmp1['emp_no'].unique() # NEED FOR rank_by FUNCTION\n",
    "tmp = rank_by(tmp1,'emp_no','salary',True)\n",
    "tmp = tmp[['emp_no','salary']]\n",
    "no = 3\n",
    "for i in range(1,no):\n",
    "    tmp['%d_previous' %(no)] = tmp.salary.shift(no)\n",
    "    tmp['%d_next' %(no)] = tmp.salary.shift(-no)\n",
    "tmp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f51aff86",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='WAF'>Aggregate Functions</a>**\n",
    "\n",
    "**Example 1: Retrieve the employee number, salary value, start date  and end date of the latest contract of all employees according to the data stored in the salaries table.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0b21210f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>emp_no</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10001</th>\n",
       "      <td>88958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10002</th>\n",
       "      <td>72527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10003</th>\n",
       "      <td>43699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10004</th>\n",
       "      <td>74057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10005</th>\n",
       "      <td>94692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201767</th>\n",
       "      <td>47097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201768</th>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201769</th>\n",
       "      <td>62287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201770</th>\n",
       "      <td>85571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201771</th>\n",
       "      <td>66792</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101796 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        salary\n",
       "emp_no        \n",
       "10001    88958\n",
       "10002    72527\n",
       "10003    43699\n",
       "10004    74057\n",
       "10005    94692\n",
       "...        ...\n",
       "201767   47097\n",
       "201768   40000\n",
       "201769   62287\n",
       "201770   85571\n",
       "201771   66792\n",
       "\n",
       "[101796 rows x 1 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#ID_split(sal,'emp_no','from_date',1,'latest'); # TAKES WAY TO LONG, SO USE\n",
    "latest_sals = GROUP_AGG(sal,'emp_no','from_date','max')\n",
    "#latest_sals = latest_sals.rename(columns={'salary':'Latest salary'})\n",
    "latest_sals"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c49235dc",
   "metadata": {},
   "source": [
    "**Example 2: Join the salaries and dept_emp tables ON employee number, as well as the dept_emp and departments tables ON department number to select the department number from the dept_emp table, the department name from the departments table, and the salary from the salaries table. Use this subset in an outer query to retrieve:<br>\n",
    "• the distinct department numbers<br>\n",
    "• the relevant department name<br>\n",
    "• the smallest salary value recorded for the given department<br>\n",
    "• the highest salary value recorded for the given department<br>\n",
    "• the average salary value recorded for the given department, rounded to the nearest dollar<br>\n",
    "Sort the final output by department number in ascending order.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0f605a42",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dept_no</th>\n",
       "      <th>dept_name</th>\n",
       "      <th>salary</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>ave</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>d001</td>\n",
       "      <td>Marketing</td>\n",
       "      <td>99651</td>\n",
       "      <td>40000</td>\n",
       "      <td>143644</td>\n",
       "      <td>77960.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>d002</td>\n",
       "      <td>Finance</td>\n",
       "      <td>95035</td>\n",
       "      <td>40000</td>\n",
       "      <td>134662</td>\n",
       "      <td>76187.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>d003</td>\n",
       "      <td>Human Resources</td>\n",
       "      <td>94692</td>\n",
       "      <td>40000</td>\n",
       "      <td>123674</td>\n",
       "      <td>61560.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>d004</td>\n",
       "      <td>Production</td>\n",
       "      <td>43699</td>\n",
       "      <td>40000</td>\n",
       "      <td>132552</td>\n",
       "      <td>65725.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>d005</td>\n",
       "      <td>Development</td>\n",
       "      <td>88958</td>\n",
       "      <td>40000</td>\n",
       "      <td>144434</td>\n",
       "      <td>65761.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>d006</td>\n",
       "      <td>Quality Management</td>\n",
       "      <td>94443</td>\n",
       "      <td>40000</td>\n",
       "      <td>122376</td>\n",
       "      <td>63398.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>d007</td>\n",
       "      <td>Sales</td>\n",
       "      <td>72527</td>\n",
       "      <td>40000</td>\n",
       "      <td>158220</td>\n",
       "      <td>86965.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>d008</td>\n",
       "      <td>Research</td>\n",
       "      <td>88070</td>\n",
       "      <td>40000</td>\n",
       "      <td>124181</td>\n",
       "      <td>66066.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>d009</td>\n",
       "      <td>Customer Service;</td>\n",
       "      <td>56753</td>\n",
       "      <td>40000</td>\n",
       "      <td>144866</td>\n",
       "      <td>64905.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  dept_no           dept_name  salary    min     max      ave\n",
       "0    d001           Marketing   99651  40000  143644  77960.0\n",
       "1    d002             Finance   95035  40000  134662  76187.0\n",
       "2    d003     Human Resources   94692  40000  123674  61560.0\n",
       "3    d004          Production   43699  40000  132552  65725.0\n",
       "4    d005         Development   88958  40000  144434  65761.0\n",
       "5    d006  Quality Management   94443  40000  122376  63398.0\n",
       "6    d007               Sales   72527  40000  158220  86965.0\n",
       "7    d008            Research   88070  40000  124181  66066.0\n",
       "8    d009   Customer Service;   56753  40000  144866  64905.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(dept_emp,dep,'dept_no','left')\n",
    "#tmp1 = JOIN(tmp,sal,'emp_no','inner') # ALL\n",
    "tmp1 = JOIN(tmp,latest_sals,'emp_no','inner') # LATEST\n",
    "\n",
    "IDs = tmp1['dept_no'].unique()\n",
    "tmp1 = tmp1[['dept_no','dept_name','salary']]\n",
    "\n",
    "dfs = []\n",
    "data = tmp1; ID = 'dept_no'\n",
    "for i in range (0,len(IDs)):\n",
    "    tmp= data[data[ID] == IDs[i]]\n",
    "    tmp['min'] = min(tmp['salary'])\n",
    "    tmp['max'] = max(tmp['salary'])\n",
    "    tmp['ave'] = round(tmp['salary'].mean(),0)\n",
    "    tmp1 = tmp.iloc[[0]]\n",
    "    dfs.append(tmp1)\n",
    "tmp = pd.concat(dfs)\n",
    "out = tmp.sort_values(by=['dept_no'], ascending=True).reset_index()\n",
    "del out['index']\n",
    "out"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2b7425f",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**<a id='CTE'>Common Table Expressions</a>**\n",
    "\n",
    "**Example 1: Considering the salary contracts signed by female employees in the company,\n",
    "how many have been signed for a value below the average? Also provide the total number of contracts signed by women.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "06d8d9e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "63761.2043294429\n",
      "Total female salaries = 40832, number below average = 31854\n"
     ]
    }
   ],
   "source": [
    "# START WITH AVERAGE SALARY\n",
    "ave_sal = AGG(sal,'salary','ave'); print(ave_sal)\n",
    "tmp1 = JOIN(emp,sal,'emp_no','inner')\n",
    "tmp2 = FILTER(tmp1,'gender','F','=='); #print(tmp2)\n",
    "ave =  GROUP_COUNT(tmp2,'emp_no');\n",
    "tmp3 = FILTER(tmp2,'salary',ave_sal,'<'); #print(tmp3)\n",
    "bel_ave =  GROUP_COUNT(tmp3,'emp_no')\n",
    "print('Total female salaries = %d, number below average = %d' %(len(ave),len(bel_ave)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e21478a",
   "metadata": {},
   "source": [
    "**Exmple 2: Create a temporary table containing the employee number and the lowest salary for each male employee in the company and retrieve all the information stored about employees whose employee numbers are less than 10007.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "481d9b43",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "      <th>emp_no</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>emp_no</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10001</th>\n",
       "      <td>60117</td>\n",
       "      <td>10001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10003</th>\n",
       "      <td>40006</td>\n",
       "      <td>10003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10004</th>\n",
       "      <td>40054</td>\n",
       "      <td>10004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10005</th>\n",
       "      <td>78228</td>\n",
       "      <td>10005</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        salary  emp_no\n",
       "emp_no                \n",
       "10001    60117   10001\n",
       "10003    40006   10003\n",
       "10004    40054   10004\n",
       "10005    78228   10005"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp1 = JOIN(emp,sal,'emp_no','inner')\n",
    "tmp2 = FILTER(tmp1,'gender','M','==')\n",
    "tmp3 = GROUP_AGG(tmp2,'emp_no','salary','min')\n",
    "tmp3['emp_no'] = tmp3.index\n",
    "tmp = FILTER(tmp3,'emp_no',10007,'<='); tmp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1506343e",
   "metadata": {},
   "source": [
    "[Back to contents](#cont)\n",
    "\n",
    "**Example 3: Execute the following two queries**<br>\n",
    "1. Create a temporary table based on the data in the salaries table. It should contain the following five fields for all employees:<br>\n",
    "\n",
    "    (a) Employee number<br>\n",
    "\n",
    "    (b) Salary<br>\n",
    "\n",
    "    (c) A field named containing the salary rounded to the nearest cent, which should be:<br>\n",
    "\n",
    "        • Multiplied by 6.5 if the contract start date was between January 1, 1970, and December 31, 1989, inclusive\n",
    "    \n",
    "        • Multiplied by 2.8 if the contract start date was between January 1, 1990, and December 31, 1999, inclusive\n",
    "    \n",
    "        • Multiplied by 3 for the rest of the contracts\n",
    "\n",
    "    (d) The contract start date<br>\n",
    "\n",
    "    (e) The contract end date<br>\n",
    "    \n",
    "\n",
    "2. Select all the data from the temporary table just created.<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "ab2b69ce",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>emp_no</th>\n",
       "      <th>salary</th>\n",
       "      <th>from_date</th>\n",
       "      <th>to_date</th>\n",
       "      <th>Adj. salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10001</td>\n",
       "      <td>60117</td>\n",
       "      <td>1986-06-26</td>\n",
       "      <td>1987-06-26</td>\n",
       "      <td>390760.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10001</td>\n",
       "      <td>62102</td>\n",
       "      <td>1987-06-26</td>\n",
       "      <td>1988-06-25</td>\n",
       "      <td>403663.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001</td>\n",
       "      <td>66074</td>\n",
       "      <td>1988-06-25</td>\n",
       "      <td>1989-06-25</td>\n",
       "      <td>429481.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10001</td>\n",
       "      <td>66596</td>\n",
       "      <td>1989-06-25</td>\n",
       "      <td>1990-06-25</td>\n",
       "      <td>432874.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10001</td>\n",
       "      <td>66961</td>\n",
       "      <td>1990-06-25</td>\n",
       "      <td>1991-06-25</td>\n",
       "      <td>187490.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>967325</th>\n",
       "      <td>201771</td>\n",
       "      <td>56736</td>\n",
       "      <td>1998-04-02</td>\n",
       "      <td>1999-04-02</td>\n",
       "      <td>158860.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>967326</th>\n",
       "      <td>201771</td>\n",
       "      <td>56577</td>\n",
       "      <td>1999-04-02</td>\n",
       "      <td>2000-04-01</td>\n",
       "      <td>158415.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>967327</th>\n",
       "      <td>201771</td>\n",
       "      <td>60056</td>\n",
       "      <td>2000-04-01</td>\n",
       "      <td>2001-04-02</td>\n",
       "      <td>180168.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>967328</th>\n",
       "      <td>201771</td>\n",
       "      <td>62899</td>\n",
       "      <td>2001-04-02</td>\n",
       "      <td>2002-04-01</td>\n",
       "      <td>188697.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>967329</th>\n",
       "      <td>201771</td>\n",
       "      <td>66792</td>\n",
       "      <td>2002-04-01</td>\n",
       "      <td>9999-01-01;</td>\n",
       "      <td>200376.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>967330 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        emp_no  salary   from_date      to_date  Adj. salary\n",
       "0        10001   60117  1986-06-26   1987-06-26     390760.5\n",
       "1        10001   62102  1987-06-26   1988-06-25     403663.0\n",
       "2        10001   66074  1988-06-25   1989-06-25     429481.0\n",
       "3        10001   66596  1989-06-25   1990-06-25     432874.0\n",
       "4        10001   66961  1990-06-25   1991-06-25     187490.8\n",
       "...        ...     ...         ...          ...          ...\n",
       "967325  201771   56736  1998-04-02   1999-04-02     158860.8\n",
       "967326  201771   56577  1999-04-02   2000-04-01     158415.6\n",
       "967327  201771   60056  2000-04-01   2001-04-02     180168.0\n",
       "967328  201771   62899  2001-04-02   2002-04-01     188697.0\n",
       "967329  201771   66792  2002-04-01  9999-01-01;     200376.0\n",
       "\n",
       "[967330 rows x 5 columns]"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp = JOIN(emp,sal,'emp_no','inner'); \n",
    "#print(tmp.columns)\n",
    "tmp = tmp[['emp_no','salary', 'from_date', 'to_date']]\n",
    "def sal_adj(date1,date2,factor):\n",
    "    tmp.loc[(tmp['from_date'] >= date1) & (tmp['from_date'] <= date2), 'Adj. salary'] = factor*tmp['salary']\n",
    "sal_adj(\"1970-01-01\",\"1989-12-31\",6.5)\n",
    "sal_adj(\"1990-01-01\",\"1999-12-31\",2.8)    \n",
    "tmp.loc[(tmp['from_date'] < \"1970-01-01\") | (tmp['from_date'] > \"1999-12-31\"), 'Adj. salary'] = 3*tmp['salary']\n",
    "tmp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd764d71",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}