hypothesis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pqsKy--8l_vR"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/googlecolab/colabtools/blob/master/notebooks/colab-github-demo.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gTbjVf82xMvk"
   },
   "source": [
    "# Results:\n",
    "\n",
    "By Friday, April 29 at 11:59PM, you must submit a draft of your results for at least one research question (we recommend trying to have a draft of both done by this time). If (and only if) you address all the criteria in the corresponding Results section above, you will receive full credit on the\n",
    "checkpoint.\n",
    "- MULTIPLE HYPOTHESIS TESTING: \n",
    "  - Summarize and interpret the results from the hypothesis tests themselves.\n",
    "  - For the two correction methods you chose, clearly explain what kind of error rate is being controlled by each one\n",
    "- CAUSAL INFERENCE:\n",
    "  - Summarize and interpret your results, providing a clear statement about causality (or a lack thereof) including any assumptions necessary.\n",
    "  - Where possible, discuss the uncertainty in your estimate and/or the evidence against the hypotheses you are investigating.\n",
    "  \n",
    "You are free to change your results section or add (or remove) content between the checkpoint and your final submission. Course staff will not provide any feedback on the research question checkpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xBvu7KuSyMpP"
   },
   "source": [
    "## Multiple Hypothesis Testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "import statsmodels.api as sm\n",
    "from statsmodels.graphics.regressionplots import abline_plot\n",
    "import datetime\n",
    "import math\n",
    "\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random \n",
    "df = pd.read_csv('https://raw.githubusercontent.com/haroldcha/gunviolencedata/master/gun_violence.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "s9MiGlX_qhBZ"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>incident_id</th>\n",
       "      <th>date</th>\n",
       "      <th>state</th>\n",
       "      <th>city_or_county</th>\n",
       "      <th>address</th>\n",
       "      <th>n_killed</th>\n",
       "      <th>n_injured</th>\n",
       "      <th>congressional_district</th>\n",
       "      <th>gun_stolen</th>\n",
       "      <th>...</th>\n",
       "      <th>participant_status</th>\n",
       "      <th>participant_type</th>\n",
       "      <th>state_house_district</th>\n",
       "      <th>state_senate_district</th>\n",
       "      <th>datetime</th>\n",
       "      <th>year</th>\n",
       "      <th>state_pop</th>\n",
       "      <th>city_pop</th>\n",
       "      <th>area (sq. mi)</th>\n",
       "      <th>pop_density</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>461105</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Pennsylvania</td>\n",
       "      <td>Mckeesport</td>\n",
       "      <td>1506 Versailles Avenue and Coursin Street</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Arrested||1::Injured||2::Injured||3::Injure...</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Victim||4:...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>12776309</td>\n",
       "      <td>NaN</td>\n",
       "      <td>46058</td>\n",
       "      <td>277.396088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>460726</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>California</td>\n",
       "      <td>Hawthorne</td>\n",
       "      <td>13500 block of Cerise Avenue</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>43.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Injured||2::Injured||3::Injured</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Victim||4:...</td>\n",
       "      <td>62.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>38260787</td>\n",
       "      <td>85863.0</td>\n",
       "      <td>163707</td>\n",
       "      <td>233.715034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>478855</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Ohio</td>\n",
       "      <td>Lorain</td>\n",
       "      <td>1776 East 28th Street</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured, Unharmed, Arrested||1::Unharmed, A...</td>\n",
       "      <td>0::Subject-Suspect||1::Subject-Suspect||2::Vic...</td>\n",
       "      <td>56.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>11576684</td>\n",
       "      <td>63735.0</td>\n",
       "      <td>44828</td>\n",
       "      <td>258.246721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>478925</td>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>Colorado</td>\n",
       "      <td>Aurora</td>\n",
       "      <td>16000 block of East Ithaca Place</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Killed||2::Killed||3::Killed</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Subject-Su...</td>\n",
       "      <td>40.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>2013</td>\n",
       "      <td>5269035</td>\n",
       "      <td>345613.0</td>\n",
       "      <td>104100</td>\n",
       "      <td>50.615130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>478959</td>\n",
       "      <td>2013-01-07</td>\n",
       "      <td>North Carolina</td>\n",
       "      <td>Greensboro</td>\n",
       "      <td>307 Mourning Dove Terrace</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured||1::Injured||2::Killed||3::Killed</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Subject-Su...</td>\n",
       "      <td>62.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>2013-01-07</td>\n",
       "      <td>2013</td>\n",
       "      <td>9843336</td>\n",
       "      <td>279244.0</td>\n",
       "      <td>53821</td>\n",
       "      <td>182.890247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239672</th>\n",
       "      <td>239672</td>\n",
       "      <td>1083142</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Rayne</td>\n",
       "      <td>North Riceland Road and Highway 90</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Unharmed, Arrested</td>\n",
       "      <td>0::Subject-Suspect</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239673</th>\n",
       "      <td>239673</td>\n",
       "      <td>1083139</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Natchitoches</td>\n",
       "      <td>247 Keyser Ave</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Unharmed, Arrested</td>\n",
       "      <td>0::Victim||1::Subject-Suspect</td>\n",
       "      <td>23.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239674</th>\n",
       "      <td>239674</td>\n",
       "      <td>1083151</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Gretna</td>\n",
       "      <td>1300 block of Cook Street</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured</td>\n",
       "      <td>0::Victim</td>\n",
       "      <td>85.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239675</th>\n",
       "      <td>239675</td>\n",
       "      <td>1082514</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Texas</td>\n",
       "      <td>Houston</td>\n",
       "      <td>12630 Ashford Point Dr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed</td>\n",
       "      <td>0::Victim</td>\n",
       "      <td>149.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>28628666</td>\n",
       "      <td>2318573.0</td>\n",
       "      <td>268601</td>\n",
       "      <td>106.584361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239676</th>\n",
       "      <td>239676</td>\n",
       "      <td>1081940</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Maine</td>\n",
       "      <td>Norridgewock</td>\n",
       "      <td>434 Skowhegan Rd</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Killed</td>\n",
       "      <td>0::Victim||1::Subject-Suspect</td>\n",
       "      <td>111.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>1339057</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35387</td>\n",
       "      <td>37.840365</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>239677 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Unnamed: 0  incident_id        date           state city_or_county  \\\n",
       "0                0       461105  2013-01-01    Pennsylvania     Mckeesport   \n",
       "1                1       460726  2013-01-01      California      Hawthorne   \n",
       "2                2       478855  2013-01-01            Ohio         Lorain   \n",
       "3                3       478925  2013-01-05        Colorado         Aurora   \n",
       "4                4       478959  2013-01-07  North Carolina     Greensboro   \n",
       "...            ...          ...         ...             ...            ...   \n",
       "239672      239672      1083142  2018-03-31       Louisiana          Rayne   \n",
       "239673      239673      1083139  2018-03-31       Louisiana   Natchitoches   \n",
       "239674      239674      1083151  2018-03-31       Louisiana         Gretna   \n",
       "239675      239675      1082514  2018-03-31           Texas        Houston   \n",
       "239676      239676      1081940  2018-03-31           Maine   Norridgewock   \n",
       "\n",
       "                                          address  n_killed  n_injured  \\\n",
       "0       1506 Versailles Avenue and Coursin Street         0          4   \n",
       "1                    13500 block of Cerise Avenue         1          3   \n",
       "2                           1776 East 28th Street         1          3   \n",
       "3                16000 block of East Ithaca Place         4          0   \n",
       "4                       307 Mourning Dove Terrace         2          2   \n",
       "...                                           ...       ...        ...   \n",
       "239672         North Riceland Road and Highway 90         0          0   \n",
       "239673                             247 Keyser Ave         1          0   \n",
       "239674                  1300 block of Cook Street         0          1   \n",
       "239675                     12630 Ashford Point Dr         1          0   \n",
       "239676                           434 Skowhegan Rd         2          0   \n",
       "\n",
       "        congressional_district              gun_stolen  ...  \\\n",
       "0                         14.0                     NaN  ...   \n",
       "1                         43.0                     NaN  ...   \n",
       "2                          9.0  0::Unknown||1::Unknown  ...   \n",
       "3                          6.0                     NaN  ...   \n",
       "4                          6.0  0::Unknown||1::Unknown  ...   \n",
       "...                        ...                     ...  ...   \n",
       "239672                     NaN              0::Unknown  ...   \n",
       "239673                     4.0              0::Unknown  ...   \n",
       "239674                     2.0              0::Unknown  ...   \n",
       "239675                     9.0              0::Unknown  ...   \n",
       "239676                     2.0  0::Unknown||1::Unknown  ...   \n",
       "\n",
       "                                       participant_status  \\\n",
       "0       0::Arrested||1::Injured||2::Injured||3::Injure...   \n",
       "1           0::Killed||1::Injured||2::Injured||3::Injured   \n",
       "2       0::Injured, Unharmed, Arrested||1::Unharmed, A...   \n",
       "3              0::Killed||1::Killed||2::Killed||3::Killed   \n",
       "4            0::Injured||1::Injured||2::Killed||3::Killed   \n",
       "...                                                   ...   \n",
       "239672                              0::Unharmed, Arrested   \n",
       "239673                   0::Killed||1::Unharmed, Arrested   \n",
       "239674                                         0::Injured   \n",
       "239675                                          0::Killed   \n",
       "239676                               0::Killed||1::Killed   \n",
       "\n",
       "                                         participant_type  \\\n",
       "0       0::Victim||1::Victim||2::Victim||3::Victim||4:...   \n",
       "1       0::Victim||1::Victim||2::Victim||3::Victim||4:...   \n",
       "2       0::Subject-Suspect||1::Subject-Suspect||2::Vic...   \n",
       "3       0::Victim||1::Victim||2::Victim||3::Subject-Su...   \n",
       "4       0::Victim||1::Victim||2::Victim||3::Subject-Su...   \n",
       "...                                                   ...   \n",
       "239672                                 0::Subject-Suspect   \n",
       "239673                      0::Victim||1::Subject-Suspect   \n",
       "239674                                          0::Victim   \n",
       "239675                                          0::Victim   \n",
       "239676                      0::Victim||1::Subject-Suspect   \n",
       "\n",
       "       state_house_district  state_senate_district    datetime  year  \\\n",
       "0                       NaN                    NaN  2013-01-01  2013   \n",
       "1                      62.0                   35.0  2013-01-01  2013   \n",
       "2                      56.0                   13.0  2013-01-01  2013   \n",
       "3                      40.0                   28.0  2013-01-05  2013   \n",
       "4                      62.0                   27.0  2013-01-07  2013   \n",
       "...                     ...                    ...         ...   ...   \n",
       "239672                  NaN                    NaN  2018-03-31  2018   \n",
       "239673                 23.0                   31.0  2018-03-31  2018   \n",
       "239674                 85.0                    7.0  2018-03-31  2018   \n",
       "239675                149.0                   17.0  2018-03-31  2018   \n",
       "239676                111.0                    3.0  2018-03-31  2018   \n",
       "\n",
       "       state_pop   city_pop area (sq. mi) pop_density  \n",
       "0       12776309        NaN         46058  277.396088  \n",
       "1       38260787    85863.0        163707  233.715034  \n",
       "2       11576684    63735.0         44828  258.246721  \n",
       "3        5269035   345613.0        104100   50.615130  \n",
       "4        9843336   279244.0         53821  182.890247  \n",
       "...          ...        ...           ...         ...  \n",
       "239672   4659690        NaN         51843   89.880794  \n",
       "239673   4659690        NaN         51843   89.880794  \n",
       "239674   4659690        NaN         51843   89.880794  \n",
       "239675  28628666  2318573.0        268601  106.584361  \n",
       "239676   1339057        NaN         35387   37.840365  \n",
       "\n",
       "[239677 rows x 28 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reminder of cleaned dataset\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "Sr96yCcr2D7P"
   },
   "outputs": [],
   "source": [
    "def test_statistic(df, column):\n",
    "    means_table = df.groupby(column).mean()\n",
    "    if 1 not in means_table.index:\n",
    "        diff = 0 - means_table.loc[0].values\n",
    "    elif 0 not in means_table.index:\n",
    "        diff = means_table.loc[1].values - 0\n",
    "    else:\n",
    "        diff = means_table.loc[1].values - means_table.loc[0].values\n",
    "    \n",
    "    return diff\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "3uA6ZD2m4bH4"
   },
   "outputs": [],
   "source": [
    "def simulated_test_stat(df, column):\n",
    "    shuffled_labels = df.sample(frac=1, replace=True)[column]\n",
    "    df[\"shuffled_labels\"] = shuffled_labels.to_numpy()\n",
    "    df = df.drop(column,axis=1)\n",
    "    \n",
    "    return test_statistic(df, 'shuffled_labels')   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "CvbC35Vx8gie"
   },
   "outputs": [],
   "source": [
    "def p_value(df, column, observed_difference, comparison):\n",
    "    differences = []\n",
    "\n",
    "    repetitions = 1000\n",
    "    for i in np.arange(repetitions):\n",
    "        new_difference = simulated_test_stat(df, column)\n",
    "        differences = np.append(differences, new_difference) \n",
    "    if comparison==\"greater\":\n",
    "        #indicates that higher values of the difference favor the alternative hypothesis that the treatment incidents were higher on average\n",
    "        empirical_p = np.count_nonzero(differences >= observed_difference) / repetitions\n",
    "    else:\n",
    "        #indicates that lower values of the difference favor the alternative hypothesis that the treatment incidents were lower on average\n",
    "        empirical_p = np.count_nonzero(differences <= observed_difference) / repetitions  \n",
    "    return empirical_p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EdReM-jIEzlx",
    "tags": []
   },
   "source": [
    "### 1. Are more populated cities associated with deadlier instances of gun violence?\n",
    "\n",
    "Null hypothesis: In the U.S., the distribution of deadly instances of gun violence (according to our definition of deadly above) is the same for cities that are populated and cities that are not. The difference in the sample is due to chance.\n",
    "\n",
    "Alternative hypothesis: In the U.S., the instances of gun violence are deadlier for cities that are populated vs. cities that are not. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "Y6EXEIRoyPpE"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_83/3574120211.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  most_populous['populous?'] = 1\n",
      "/tmp/ipykernel_83/3574120211.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  least_populous['populous?'] = 0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>populous?</th>\n",
       "      <th>num_incidents</th>\n",
       "      <th>num_victims</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>city_or_county</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Cerritos</th>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Charleston</th>\n",
       "      <td>0</td>\n",
       "      <td>519</td>\n",
       "      <td>350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Chicago</th>\n",
       "      <td>1</td>\n",
       "      <td>10814</td>\n",
       "      <td>12531</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coral Gables</th>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dallas</th>\n",
       "      <td>1</td>\n",
       "      <td>1179</td>\n",
       "      <td>999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enid</th>\n",
       "      <td>0</td>\n",
       "      <td>60</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Houston</th>\n",
       "      <td>1</td>\n",
       "      <td>2501</td>\n",
       "      <td>2400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lakewood</th>\n",
       "      <td>0</td>\n",
       "      <td>113</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Los Angeles</th>\n",
       "      <td>1</td>\n",
       "      <td>1066</td>\n",
       "      <td>1189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New York</th>\n",
       "      <td>1</td>\n",
       "      <td>377</td>\n",
       "      <td>327</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Niagara Falls</th>\n",
       "      <td>0</td>\n",
       "      <td>182</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philadelphia</th>\n",
       "      <td>1</td>\n",
       "      <td>2963</td>\n",
       "      <td>3227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Phoenix</th>\n",
       "      <td>1</td>\n",
       "      <td>973</td>\n",
       "      <td>886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Saginaw</th>\n",
       "      <td>0</td>\n",
       "      <td>242</td>\n",
       "      <td>218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>San Antonio</th>\n",
       "      <td>1</td>\n",
       "      <td>1628</td>\n",
       "      <td>1545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>San Diego</th>\n",
       "      <td>1</td>\n",
       "      <td>375</td>\n",
       "      <td>285</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>San Jose</th>\n",
       "      <td>1</td>\n",
       "      <td>237</td>\n",
       "      <td>208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Texas City</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Troy</th>\n",
       "      <td>0</td>\n",
       "      <td>167</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Twin Falls</th>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                populous?  num_incidents  num_victims\n",
       "city_or_county                                       \n",
       "Cerritos                0              7            6\n",
       "Charleston              0            519          350\n",
       "Chicago                 1          10814        12531\n",
       "Coral Gables            0             14           12\n",
       "Dallas                  1           1179          999\n",
       "Enid                    0             60           24\n",
       "Houston                 1           2501         2400\n",
       "Lakewood                0            113           85\n",
       "Los Angeles             1           1066         1189\n",
       "New York                1            377          327\n",
       "Niagara Falls           0            182           75\n",
       "Philadelphia            1           2963         3227\n",
       "Phoenix                 1            973          886\n",
       "Saginaw                 0            242          218\n",
       "San Antonio             1           1628         1545\n",
       "San Diego               1            375          285\n",
       "San Jose                1            237          208\n",
       "Texas City              0             24           20\n",
       "Troy                    0            167           89\n",
       "Twin Falls              0             21           19"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "most_populous = df.loc[df['city_or_county'].isin(['New York', 'Los Angeles', 'Chicago', 'Houston', 'Phoenix', 'Philadelphia', 'San Antonio', 'San Diego', 'Dallas', 'San Jose'])]\n",
    "most_populous['populous?'] = 1\n",
    "least_populous = df.loc[df['city_or_county'].isin(['Charleston', 'Niagara Falls', 'Saginaw', 'Troy', 'Lakewood', 'Enid', 'Coral Gables', 'Cerritos', 'Texas City', 'Twin Falls'])]\n",
    "least_populous['populous?'] = 0\n",
    "populous = pd.concat([most_populous, least_populous])\n",
    "populous_df = populous.groupby(['city_or_county', 'populous?']).sum()\n",
    "populous_df['num_incidents'] = populous.groupby(['city_or_county', 'populous?']).count()['incident_id'].values\n",
    "populous_df['num_victims'] = populous_df['n_killed'].values + populous_df['n_injured'].values\n",
    "populous_df = populous_df.loc[:,['num_incidents', 'num_victims']]\n",
    "populous_df = populous_df.reset_index('populous?')\n",
    "populous_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.01"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "victim_populous = populous_df.loc[:,['populous?', 'num_victims']]\n",
    "test_victim_populous = test_statistic(victim_populous, 'populous?')\n",
    "p_value(victim_populous, 'populous?', test_victim_populous, 'greater')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tCIM_5suFEa2",
    "tags": []
   },
   "source": [
    "### 2. Are more populated cities associated with more instances of gun violence?\n",
    "\n",
    "Null hypothesis: In the U.S., the distribution of instances of gun violence is the same for cities that are populated and cities that are not. The difference in the sample is due to chance.\n",
    "\n",
    "Alternative hypothesis: In the U.S., the number of instances of gun violence (according to our definition of deadly above) is higher for cities that are populated vs. cities that are not. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "BmtsMPscFINA"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.005"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "incidents_populous = populous_df.loc[:,['populous?', 'num_incidents']]\n",
    "test_incidents_populous = test_statistic(incidents_populous, 'populous?')\n",
    "p_value(incidents_populous, 'populous?', test_incidents_populous, 'greater')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sDRU6kf_FI5N"
   },
   "source": [
    "### 3. Are more densely populated states associated with deadlier instances of gun violence?\n",
    "\n",
    "Null hypothesis: In the U.S., the distribution of deadly instances of gun violence (according to our definition of deadly above) is the same for cities that are densely populated and cities that are not. The difference in the sample is due to chance.\n",
    "\n",
    "Alternative hypothesis: In the U.S., the instances of gun violence are deadlier for cities that are densely populated vs. cities that are not. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>incident_id</th>\n",
       "      <th>date</th>\n",
       "      <th>state</th>\n",
       "      <th>city_or_county</th>\n",
       "      <th>address</th>\n",
       "      <th>n_killed</th>\n",
       "      <th>n_injured</th>\n",
       "      <th>congressional_district</th>\n",
       "      <th>gun_stolen</th>\n",
       "      <th>...</th>\n",
       "      <th>participant_status</th>\n",
       "      <th>participant_type</th>\n",
       "      <th>state_house_district</th>\n",
       "      <th>state_senate_district</th>\n",
       "      <th>datetime</th>\n",
       "      <th>year</th>\n",
       "      <th>state_pop</th>\n",
       "      <th>city_pop</th>\n",
       "      <th>area (sq. mi)</th>\n",
       "      <th>pop_density</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>461105</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Pennsylvania</td>\n",
       "      <td>Mckeesport</td>\n",
       "      <td>1506 Versailles Avenue and Coursin Street</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Arrested||1::Injured||2::Injured||3::Injure...</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Victim||4:...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>12776309</td>\n",
       "      <td>NaN</td>\n",
       "      <td>46058</td>\n",
       "      <td>277.396088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>460726</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>California</td>\n",
       "      <td>Hawthorne</td>\n",
       "      <td>13500 block of Cerise Avenue</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>43.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Injured||2::Injured||3::Injured</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Victim||4:...</td>\n",
       "      <td>62.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>38260787</td>\n",
       "      <td>85863.0</td>\n",
       "      <td>163707</td>\n",
       "      <td>233.715034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>478855</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Ohio</td>\n",
       "      <td>Lorain</td>\n",
       "      <td>1776 East 28th Street</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured, Unharmed, Arrested||1::Unharmed, A...</td>\n",
       "      <td>0::Subject-Suspect||1::Subject-Suspect||2::Vic...</td>\n",
       "      <td>56.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>11576684</td>\n",
       "      <td>63735.0</td>\n",
       "      <td>44828</td>\n",
       "      <td>258.246721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>478925</td>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>Colorado</td>\n",
       "      <td>Aurora</td>\n",
       "      <td>16000 block of East Ithaca Place</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Killed||2::Killed||3::Killed</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Subject-Su...</td>\n",
       "      <td>40.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>2013</td>\n",
       "      <td>5269035</td>\n",
       "      <td>345613.0</td>\n",
       "      <td>104100</td>\n",
       "      <td>50.615130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>478959</td>\n",
       "      <td>2013-01-07</td>\n",
       "      <td>North Carolina</td>\n",
       "      <td>Greensboro</td>\n",
       "      <td>307 Mourning Dove Terrace</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured||1::Injured||2::Killed||3::Killed</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Subject-Su...</td>\n",
       "      <td>62.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>2013-01-07</td>\n",
       "      <td>2013</td>\n",
       "      <td>9843336</td>\n",
       "      <td>279244.0</td>\n",
       "      <td>53821</td>\n",
       "      <td>182.890247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239672</th>\n",
       "      <td>239672</td>\n",
       "      <td>1083142</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Rayne</td>\n",
       "      <td>North Riceland Road and Highway 90</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Unharmed, Arrested</td>\n",
       "      <td>0::Subject-Suspect</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239673</th>\n",
       "      <td>239673</td>\n",
       "      <td>1083139</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Natchitoches</td>\n",
       "      <td>247 Keyser Ave</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Unharmed, Arrested</td>\n",
       "      <td>0::Victim||1::Subject-Suspect</td>\n",
       "      <td>23.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239674</th>\n",
       "      <td>239674</td>\n",
       "      <td>1083151</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Gretna</td>\n",
       "      <td>1300 block of Cook Street</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured</td>\n",
       "      <td>0::Victim</td>\n",
       "      <td>85.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239675</th>\n",
       "      <td>239675</td>\n",
       "      <td>1082514</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Texas</td>\n",
       "      <td>Houston</td>\n",
       "      <td>12630 Ashford Point Dr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed</td>\n",
       "      <td>0::Victim</td>\n",
       "      <td>149.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>28628666</td>\n",
       "      <td>2318573.0</td>\n",
       "      <td>268601</td>\n",
       "      <td>106.584361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239676</th>\n",
       "      <td>239676</td>\n",
       "      <td>1081940</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Maine</td>\n",
       "      <td>Norridgewock</td>\n",
       "      <td>434 Skowhegan Rd</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Killed</td>\n",
       "      <td>0::Victim||1::Subject-Suspect</td>\n",
       "      <td>111.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>1339057</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35387</td>\n",
       "      <td>37.840365</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>239677 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Unnamed: 0  incident_id        date           state city_or_county  \\\n",
       "0                0       461105  2013-01-01    Pennsylvania     Mckeesport   \n",
       "1                1       460726  2013-01-01      California      Hawthorne   \n",
       "2                2       478855  2013-01-01            Ohio         Lorain   \n",
       "3                3       478925  2013-01-05        Colorado         Aurora   \n",
       "4                4       478959  2013-01-07  North Carolina     Greensboro   \n",
       "...            ...          ...         ...             ...            ...   \n",
       "239672      239672      1083142  2018-03-31       Louisiana          Rayne   \n",
       "239673      239673      1083139  2018-03-31       Louisiana   Natchitoches   \n",
       "239674      239674      1083151  2018-03-31       Louisiana         Gretna   \n",
       "239675      239675      1082514  2018-03-31           Texas        Houston   \n",
       "239676      239676      1081940  2018-03-31           Maine   Norridgewock   \n",
       "\n",
       "                                          address  n_killed  n_injured  \\\n",
       "0       1506 Versailles Avenue and Coursin Street         0          4   \n",
       "1                    13500 block of Cerise Avenue         1          3   \n",
       "2                           1776 East 28th Street         1          3   \n",
       "3                16000 block of East Ithaca Place         4          0   \n",
       "4                       307 Mourning Dove Terrace         2          2   \n",
       "...                                           ...       ...        ...   \n",
       "239672         North Riceland Road and Highway 90         0          0   \n",
       "239673                             247 Keyser Ave         1          0   \n",
       "239674                  1300 block of Cook Street         0          1   \n",
       "239675                     12630 Ashford Point Dr         1          0   \n",
       "239676                           434 Skowhegan Rd         2          0   \n",
       "\n",
       "        congressional_district              gun_stolen  ...  \\\n",
       "0                         14.0                     NaN  ...   \n",
       "1                         43.0                     NaN  ...   \n",
       "2                          9.0  0::Unknown||1::Unknown  ...   \n",
       "3                          6.0                     NaN  ...   \n",
       "4                          6.0  0::Unknown||1::Unknown  ...   \n",
       "...                        ...                     ...  ...   \n",
       "239672                     NaN              0::Unknown  ...   \n",
       "239673                     4.0              0::Unknown  ...   \n",
       "239674                     2.0              0::Unknown  ...   \n",
       "239675                     9.0              0::Unknown  ...   \n",
       "239676                     2.0  0::Unknown||1::Unknown  ...   \n",
       "\n",
       "                                       participant_status  \\\n",
       "0       0::Arrested||1::Injured||2::Injured||3::Injure...   \n",
       "1           0::Killed||1::Injured||2::Injured||3::Injured   \n",
       "2       0::Injured, Unharmed, Arrested||1::Unharmed, A...   \n",
       "3              0::Killed||1::Killed||2::Killed||3::Killed   \n",
       "4            0::Injured||1::Injured||2::Killed||3::Killed   \n",
       "...                                                   ...   \n",
       "239672                              0::Unharmed, Arrested   \n",
       "239673                   0::Killed||1::Unharmed, Arrested   \n",
       "239674                                         0::Injured   \n",
       "239675                                          0::Killed   \n",
       "239676                               0::Killed||1::Killed   \n",
       "\n",
       "                                         participant_type  \\\n",
       "0       0::Victim||1::Victim||2::Victim||3::Victim||4:...   \n",
       "1       0::Victim||1::Victim||2::Victim||3::Victim||4:...   \n",
       "2       0::Subject-Suspect||1::Subject-Suspect||2::Vic...   \n",
       "3       0::Victim||1::Victim||2::Victim||3::Subject-Su...   \n",
       "4       0::Victim||1::Victim||2::Victim||3::Subject-Su...   \n",
       "...                                                   ...   \n",
       "239672                                 0::Subject-Suspect   \n",
       "239673                      0::Victim||1::Subject-Suspect   \n",
       "239674                                          0::Victim   \n",
       "239675                                          0::Victim   \n",
       "239676                      0::Victim||1::Subject-Suspect   \n",
       "\n",
       "       state_house_district  state_senate_district    datetime  year  \\\n",
       "0                       NaN                    NaN  2013-01-01  2013   \n",
       "1                      62.0                   35.0  2013-01-01  2013   \n",
       "2                      56.0                   13.0  2013-01-01  2013   \n",
       "3                      40.0                   28.0  2013-01-05  2013   \n",
       "4                      62.0                   27.0  2013-01-07  2013   \n",
       "...                     ...                    ...         ...   ...   \n",
       "239672                  NaN                    NaN  2018-03-31  2018   \n",
       "239673                 23.0                   31.0  2018-03-31  2018   \n",
       "239674                 85.0                    7.0  2018-03-31  2018   \n",
       "239675                149.0                   17.0  2018-03-31  2018   \n",
       "239676                111.0                    3.0  2018-03-31  2018   \n",
       "\n",
       "       state_pop   city_pop area (sq. mi) pop_density  \n",
       "0       12776309        NaN         46058  277.396088  \n",
       "1       38260787    85863.0        163707  233.715034  \n",
       "2       11576684    63735.0         44828  258.246721  \n",
       "3        5269035   345613.0        104100   50.615130  \n",
       "4        9843336   279244.0         53821  182.890247  \n",
       "...          ...        ...           ...         ...  \n",
       "239672   4659690        NaN         51843   89.880794  \n",
       "239673   4659690        NaN         51843   89.880794  \n",
       "239674   4659690        NaN         51843   89.880794  \n",
       "239675  28628666  2318573.0        268601  106.584361  \n",
       "239676   1339057        NaN         35387   37.840365  \n",
       "\n",
       "[239677 rows x 28 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "ROIqvswTFMXa"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_83/404976281.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  most_dense['dense?'] = 1\n",
      "/tmp/ipykernel_83/404976281.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  least_dense['dense?'] = 0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dense?</th>\n",
       "      <th>num_incidents</th>\n",
       "      <th>num_victims</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>city_or_county</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Aurora</th>\n",
       "      <td>0</td>\n",
       "      <td>373</td>\n",
       "      <td>273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Boston</th>\n",
       "      <td>1</td>\n",
       "      <td>1737</td>\n",
       "      <td>762</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Chicago</th>\n",
       "      <td>1</td>\n",
       "      <td>10814</td>\n",
       "      <td>12531</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jacksonville</th>\n",
       "      <td>0</td>\n",
       "      <td>2448</td>\n",
       "      <td>1643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kansas City</th>\n",
       "      <td>0</td>\n",
       "      <td>1381</td>\n",
       "      <td>1442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Long Beach</th>\n",
       "      <td>1</td>\n",
       "      <td>343</td>\n",
       "      <td>339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Los Angeles</th>\n",
       "      <td>1</td>\n",
       "      <td>1066</td>\n",
       "      <td>1189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Memphis</th>\n",
       "      <td>0</td>\n",
       "      <td>2386</td>\n",
       "      <td>2313</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Miami</th>\n",
       "      <td>1</td>\n",
       "      <td>846</td>\n",
       "      <td>922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Nashville</th>\n",
       "      <td>0</td>\n",
       "      <td>1329</td>\n",
       "      <td>1068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New Orleans</th>\n",
       "      <td>0</td>\n",
       "      <td>3071</td>\n",
       "      <td>2595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>New York</th>\n",
       "      <td>1</td>\n",
       "      <td>377</td>\n",
       "      <td>327</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Oklahoma City</th>\n",
       "      <td>0</td>\n",
       "      <td>815</td>\n",
       "      <td>663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philadelphia</th>\n",
       "      <td>1</td>\n",
       "      <td>2963</td>\n",
       "      <td>3227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>San Francisco</th>\n",
       "      <td>1</td>\n",
       "      <td>421</td>\n",
       "      <td>413</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Seattle</th>\n",
       "      <td>1</td>\n",
       "      <td>732</td>\n",
       "      <td>409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tucson</th>\n",
       "      <td>0</td>\n",
       "      <td>333</td>\n",
       "      <td>312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tulsa</th>\n",
       "      <td>0</td>\n",
       "      <td>1260</td>\n",
       "      <td>845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Virginia Beach</th>\n",
       "      <td>0</td>\n",
       "      <td>289</td>\n",
       "      <td>249</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                dense?  num_incidents  num_victims\n",
       "city_or_county                                    \n",
       "Aurora               0            373          273\n",
       "Boston               1           1737          762\n",
       "Chicago              1          10814        12531\n",
       "Jacksonville         0           2448         1643\n",
       "Kansas City          0           1381         1442\n",
       "Long Beach           1            343          339\n",
       "Los Angeles          1           1066         1189\n",
       "Memphis              0           2386         2313\n",
       "Miami                1            846          922\n",
       "Nashville            0           1329         1068\n",
       "New Orleans          0           3071         2595\n",
       "New York             1            377          327\n",
       "Oklahoma City        0            815          663\n",
       "Philadelphia         1           2963         3227\n",
       "San Francisco        1            421          413\n",
       "Seattle              1            732          409\n",
       "Tucson               0            333          312\n",
       "Tulsa                0           1260          845\n",
       "Virginia Beach       0            289          249"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "most_dense = df.loc[df['city_or_county'].isin(['New York','San Francisco','Boston','Miami','Chicago', 'Philadelphia', 'District of Columbia', 'Long Beach', 'Seattle', 'Los Angeles'])]\n",
    "most_dense['dense?'] = 1\n",
    "least_dense = df.loc[df['city_or_county'].isin(['Oklahoma City', 'Jacksonville', 'Nashville', 'Kansas City', 'Virginia Beach', 'Tulsa', 'Memphis', 'Tucson', 'New Orleans', 'Aurora'])]\n",
    "least_dense['dense?'] = 0\n",
    "dense = pd.concat([most_dense, least_dense])\n",
    "dense_df = dense.groupby(['city_or_county', 'dense?']).sum()\n",
    "dense_df['num_incidents'] = dense.groupby(['city_or_county', 'dense?']).count()['incident_id'].values\n",
    "dense_df['num_victims'] = dense_df['n_killed'].values + dense_df['n_injured'].values\n",
    "dense_df = dense_df.loc[:,['num_incidents', 'num_victims']]\n",
    "dense_df = dense_df.reset_index('dense?')\n",
    "dense_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.291"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "victims_dense = dense_df.loc[:,['dense?', 'num_victims']]\n",
    "test_victim_dense = test_statistic(victims_dense, 'dense?')\n",
    "p_value(victims_dense, 'dense?', test_victim_dense, 'greater')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J6IQUmLQFM8c"
   },
   "source": [
    "### 4. Are more densely populated states associated with more instances of gun violence?\n",
    "\n",
    "Null hypothesis: In the U.S., the distribution of instances of gun violence is the same for cities that are densely populated and cities that are not. The difference in the sample is due to chance.\n",
    "\n",
    "Alternative hypothesis: In the U.S., the number of instances of gun violence is higher for cities that are densely populated vs. cities that are not. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "paylHrOiFO2m"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.302"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "incidents_dense = dense_df.loc[:,['dense?', 'num_victims']]\n",
    "test_incidents_dense = test_statistic(incidents_dense, 'dense?')\n",
    "p_value(incidents_dense, 'dense?', test_incidents_dense, 'greater')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VszSP9OjFPWZ"
   },
   "source": [
    "### 5. Are crimes getting less violent over time (number of casualties)?\n",
    "\n",
    "Null hypothesis: In the U.S., the distribution of instances of deadly gun violence is the same before as it was after. The difference in the sample is due to chance.\n",
    "\n",
    "Alternative hypothesis: In the U.S., the number of instances of deadly gun is higher before than it was after. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "if before == 1, we predict a greater number of casualties. if before == 0, it would be less because the number went down. therefore mean(1) - mean(0) would be positive. we want to see how likely it is to observe values more extreme (greater than this)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "AaJij5OvFRPE"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>incident_id</th>\n",
       "      <th>date</th>\n",
       "      <th>state</th>\n",
       "      <th>city_or_county</th>\n",
       "      <th>address</th>\n",
       "      <th>n_killed</th>\n",
       "      <th>n_injured</th>\n",
       "      <th>congressional_district</th>\n",
       "      <th>gun_stolen</th>\n",
       "      <th>...</th>\n",
       "      <th>participant_status</th>\n",
       "      <th>participant_type</th>\n",
       "      <th>state_house_district</th>\n",
       "      <th>state_senate_district</th>\n",
       "      <th>datetime</th>\n",
       "      <th>year</th>\n",
       "      <th>state_pop</th>\n",
       "      <th>city_pop</th>\n",
       "      <th>area (sq. mi)</th>\n",
       "      <th>pop_density</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>461105</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Pennsylvania</td>\n",
       "      <td>Mckeesport</td>\n",
       "      <td>1506 Versailles Avenue and Coursin Street</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Arrested||1::Injured||2::Injured||3::Injure...</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Victim||4:...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>12776309</td>\n",
       "      <td>NaN</td>\n",
       "      <td>46058</td>\n",
       "      <td>277.396088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>460726</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>California</td>\n",
       "      <td>Hawthorne</td>\n",
       "      <td>13500 block of Cerise Avenue</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>43.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Injured||2::Injured||3::Injured</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Victim||4:...</td>\n",
       "      <td>62.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>38260787</td>\n",
       "      <td>85863.0</td>\n",
       "      <td>163707</td>\n",
       "      <td>233.715034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>478855</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Ohio</td>\n",
       "      <td>Lorain</td>\n",
       "      <td>1776 East 28th Street</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured, Unharmed, Arrested||1::Unharmed, A...</td>\n",
       "      <td>0::Subject-Suspect||1::Subject-Suspect||2::Vic...</td>\n",
       "      <td>56.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013</td>\n",
       "      <td>11576684</td>\n",
       "      <td>63735.0</td>\n",
       "      <td>44828</td>\n",
       "      <td>258.246721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>478925</td>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>Colorado</td>\n",
       "      <td>Aurora</td>\n",
       "      <td>16000 block of East Ithaca Place</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Killed||2::Killed||3::Killed</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Subject-Su...</td>\n",
       "      <td>40.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>2013</td>\n",
       "      <td>5269035</td>\n",
       "      <td>345613.0</td>\n",
       "      <td>104100</td>\n",
       "      <td>50.615130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>478959</td>\n",
       "      <td>2013-01-07</td>\n",
       "      <td>North Carolina</td>\n",
       "      <td>Greensboro</td>\n",
       "      <td>307 Mourning Dove Terrace</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured||1::Injured||2::Killed||3::Killed</td>\n",
       "      <td>0::Victim||1::Victim||2::Victim||3::Subject-Su...</td>\n",
       "      <td>62.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>2013-01-07</td>\n",
       "      <td>2013</td>\n",
       "      <td>9843336</td>\n",
       "      <td>279244.0</td>\n",
       "      <td>53821</td>\n",
       "      <td>182.890247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239672</th>\n",
       "      <td>239672</td>\n",
       "      <td>1083142</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Rayne</td>\n",
       "      <td>North Riceland Road and Highway 90</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Unharmed, Arrested</td>\n",
       "      <td>0::Subject-Suspect</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239673</th>\n",
       "      <td>239673</td>\n",
       "      <td>1083139</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Natchitoches</td>\n",
       "      <td>247 Keyser Ave</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Unharmed, Arrested</td>\n",
       "      <td>0::Victim||1::Subject-Suspect</td>\n",
       "      <td>23.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239674</th>\n",
       "      <td>239674</td>\n",
       "      <td>1083151</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Louisiana</td>\n",
       "      <td>Gretna</td>\n",
       "      <td>1300 block of Cook Street</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Injured</td>\n",
       "      <td>0::Victim</td>\n",
       "      <td>85.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>4659690</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51843</td>\n",
       "      <td>89.880794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239675</th>\n",
       "      <td>239675</td>\n",
       "      <td>1082514</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Texas</td>\n",
       "      <td>Houston</td>\n",
       "      <td>12630 Ashford Point Dr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed</td>\n",
       "      <td>0::Victim</td>\n",
       "      <td>149.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>28628666</td>\n",
       "      <td>2318573.0</td>\n",
       "      <td>268601</td>\n",
       "      <td>106.584361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239676</th>\n",
       "      <td>239676</td>\n",
       "      <td>1081940</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>Maine</td>\n",
       "      <td>Norridgewock</td>\n",
       "      <td>434 Skowhegan Rd</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0::Unknown||1::Unknown</td>\n",
       "      <td>...</td>\n",
       "      <td>0::Killed||1::Killed</td>\n",
       "      <td>0::Victim||1::Subject-Suspect</td>\n",
       "      <td>111.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2018-03-31</td>\n",
       "      <td>2018</td>\n",
       "      <td>1339057</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35387</td>\n",
       "      <td>37.840365</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>239677 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Unnamed: 0  incident_id        date           state city_or_county  \\\n",
       "0                0       461105  2013-01-01    Pennsylvania     Mckeesport   \n",
       "1                1       460726  2013-01-01      California      Hawthorne   \n",
       "2                2       478855  2013-01-01            Ohio         Lorain   \n",
       "3                3       478925  2013-01-05        Colorado         Aurora   \n",
       "4                4       478959  2013-01-07  North Carolina     Greensboro   \n",
       "...            ...          ...         ...             ...            ...   \n",
       "239672      239672      1083142  2018-03-31       Louisiana          Rayne   \n",
       "239673      239673      1083139  2018-03-31       Louisiana   Natchitoches   \n",
       "239674      239674      1083151  2018-03-31       Louisiana         Gretna   \n",
       "239675      239675      1082514  2018-03-31           Texas        Houston   \n",
       "239676      239676      1081940  2018-03-31           Maine   Norridgewock   \n",
       "\n",
       "                                          address  n_killed  n_injured  \\\n",
       "0       1506 Versailles Avenue and Coursin Street         0          4   \n",
       "1                    13500 block of Cerise Avenue         1          3   \n",
       "2                           1776 East 28th Street         1          3   \n",
       "3                16000 block of East Ithaca Place         4          0   \n",
       "4                       307 Mourning Dove Terrace         2          2   \n",
       "...                                           ...       ...        ...   \n",
       "239672         North Riceland Road and Highway 90         0          0   \n",
       "239673                             247 Keyser Ave         1          0   \n",
       "239674                  1300 block of Cook Street         0          1   \n",
       "239675                     12630 Ashford Point Dr         1          0   \n",
       "239676                           434 Skowhegan Rd         2          0   \n",
       "\n",
       "        congressional_district              gun_stolen  ...  \\\n",
       "0                         14.0                     NaN  ...   \n",
       "1                         43.0                     NaN  ...   \n",
       "2                          9.0  0::Unknown||1::Unknown  ...   \n",
       "3                          6.0                     NaN  ...   \n",
       "4                          6.0  0::Unknown||1::Unknown  ...   \n",
       "...                        ...                     ...  ...   \n",
       "239672                     NaN              0::Unknown  ...   \n",
       "239673                     4.0              0::Unknown  ...   \n",
       "239674                     2.0              0::Unknown  ...   \n",
       "239675                     9.0              0::Unknown  ...   \n",
       "239676                     2.0  0::Unknown||1::Unknown  ...   \n",
       "\n",
       "                                       participant_status  \\\n",
       "0       0::Arrested||1::Injured||2::Injured||3::Injure...   \n",
       "1           0::Killed||1::Injured||2::Injured||3::Injured   \n",
       "2       0::Injured, Unharmed, Arrested||1::Unharmed, A...   \n",
       "3              0::Killed||1::Killed||2::Killed||3::Killed   \n",
       "4            0::Injured||1::Injured||2::Killed||3::Killed   \n",
       "...                                                   ...   \n",
       "239672                              0::Unharmed, Arrested   \n",
       "239673                   0::Killed||1::Unharmed, Arrested   \n",
       "239674                                         0::Injured   \n",
       "239675                                          0::Killed   \n",
       "239676                               0::Killed||1::Killed   \n",
       "\n",
       "                                         participant_type  \\\n",
       "0       0::Victim||1::Victim||2::Victim||3::Victim||4:...   \n",
       "1       0::Victim||1::Victim||2::Victim||3::Victim||4:...   \n",
       "2       0::Subject-Suspect||1::Subject-Suspect||2::Vic...   \n",
       "3       0::Victim||1::Victim||2::Victim||3::Subject-Su...   \n",
       "4       0::Victim||1::Victim||2::Victim||3::Subject-Su...   \n",
       "...                                                   ...   \n",
       "239672                                 0::Subject-Suspect   \n",
       "239673                      0::Victim||1::Subject-Suspect   \n",
       "239674                                          0::Victim   \n",
       "239675                                          0::Victim   \n",
       "239676                      0::Victim||1::Subject-Suspect   \n",
       "\n",
       "       state_house_district  state_senate_district    datetime  year  \\\n",
       "0                       NaN                    NaN  2013-01-01  2013   \n",
       "1                      62.0                   35.0  2013-01-01  2013   \n",
       "2                      56.0                   13.0  2013-01-01  2013   \n",
       "3                      40.0                   28.0  2013-01-05  2013   \n",
       "4                      62.0                   27.0  2013-01-07  2013   \n",
       "...                     ...                    ...         ...   ...   \n",
       "239672                  NaN                    NaN  2018-03-31  2018   \n",
       "239673                 23.0                   31.0  2018-03-31  2018   \n",
       "239674                 85.0                    7.0  2018-03-31  2018   \n",
       "239675                149.0                   17.0  2018-03-31  2018   \n",
       "239676                111.0                    3.0  2018-03-31  2018   \n",
       "\n",
       "       state_pop   city_pop area (sq. mi) pop_density  \n",
       "0       12776309        NaN         46058  277.396088  \n",
       "1       38260787    85863.0        163707  233.715034  \n",
       "2       11576684    63735.0         44828  258.246721  \n",
       "3        5269035   345613.0        104100   50.615130  \n",
       "4        9843336   279244.0         53821  182.890247  \n",
       "...          ...        ...           ...         ...  \n",
       "239672   4659690        NaN         51843   89.880794  \n",
       "239673   4659690        NaN         51843   89.880794  \n",
       "239674   4659690        NaN         51843   89.880794  \n",
       "239675  28628666  2318573.0        268601  106.584361  \n",
       "239676   1339057        NaN         35387   37.840365  \n",
       "\n",
       "[239677 rows x 28 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_83/762332063.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  before['before?'] = 1\n",
      "/tmp/ipykernel_83/762332063.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  after['before?'] = 0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>before?</th>\n",
       "      <th>num_victims</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2013</th>\n",
       "      <td>1</td>\n",
       "      <td>1296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014</th>\n",
       "      <td>1</td>\n",
       "      <td>35559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015</th>\n",
       "      <td>1</td>\n",
       "      <td>40451</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016</th>\n",
       "      <td>0</td>\n",
       "      <td>45646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017</th>\n",
       "      <td>0</td>\n",
       "      <td>46214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018</th>\n",
       "      <td>0</td>\n",
       "      <td>9704</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      before?  num_victims\n",
       "year                      \n",
       "2013        1         1296\n",
       "2014        1        35559\n",
       "2015        1        40451\n",
       "2016        0        45646\n",
       "2017        0        46214\n",
       "2018        0         9704"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "before = df.loc[df['year'].isin([2013, 2014, 2015])]\n",
    "before['before?'] = 1\n",
    "after = df.loc[df['year'].isin([2016, 2017, 2018])]\n",
    "after['before?'] = 0\n",
    "time = pd.concat([after, before])\n",
    "time_df = time.groupby(['year', 'before?']).sum()\n",
    "time_df['num_victims'] = time_df['n_killed'].values + time_df['n_injured'].values\n",
    "time_df = time_df.loc[:,['num_victims']]\n",
    "time_df = time_df.reset_index('before?')\n",
    "time_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.65"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_time = test_statistic(time_df, 'before?')\n",
    "p_value(time_df, 'before?', test_time, 'greater')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary and Interpretation of the Result on 5 Hypothesis Based on P-Value testing\n",
    "\n",
    "For the procedure of multiple hypothesis testing, we gave 5 hypothesis regarding the count of deadly gun-violence incident, and the count of gun-violence incidents based off of the factos on the population and density of the cities. \n",
    "\n",
    "For our first hypothesis question regarding the correlation between population of the city and deadly gun-violence incident, our p-value was 0.01. This p-value was statistically significant as to show that our test rejects the null hypothesis. This also signifies that there are some positive correlation between the populous city and the deadly gun-violence incidents. \n",
    "\n",
    "Simmilarly, we discovered a similar result for our second hypothesis regarding the correlation between population of the city and total count of gun-vioence incidents. The p-value between the two factors was 0.005, which demonstrated statistically significant correlation between the two factors. The result was naturally predictable as the factor on deadly gun-violence incidents and total count of gun-violence incidents showed similar numbers. Therefore, our p-value indicated statistically significant relationship between total gun-violence incident and populous cities. Once again, we rejected our null hypothesis. \n",
    "\n",
    "For our third hypothesis, we studied the correlation between density of the cities against the gun-violence cases that led to death or severe injuries. The p-value that we derived from studying their correlation was 0.291, which was statistically not significant. This showed that there wasn't a strong correlation between the two factors and it failed to reject the null hypothesis. The results were expected due to multiple confounding variables involved in strictness on regulatioins of gun-violence.\n",
    "\n",
    "Correspondingly, our fourth hypothesis, which studied the correlation between the density of the cities against the total count of gun-violence, had a close result as our third hypothesis. The resulting p-value between the two factos was 0.302, which was statistically insignificant. The p-value showed that this study also failed to reject the null hypothesis for a similar reason from our third hypothesis. \n",
    "\n",
    "The fifth hypothesis studied the correlation between the total count of gun-violence and the time. We tried to discover if the total count of gun-violences have gone down over time. Different from our expectation, the p-value that resulted from our study was 0.65 which was, again, statistically insignificant. This demonstrated that our study failed to reject the null hypothesis. This was unexpected as we were able to observe that the total number of gun-violence have gone down overtime from our previous analysis on the gun-violence case reports between Philladelphia and Camden."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Corrections"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bonferroni(df, column, test, alpha):\n",
    "    \"\"\"\n",
    "    Returns decisions on p-values using the Bonferroni correction.\n",
    "    \n",
    "    Inputs:\n",
    "        p_values: array of p-values\n",
    "        alpha_total: desired family-wise error rate (FWER = P(at least one false discovery))\n",
    "    \n",
    "    Returns:\n",
    "        decisions: binary array of same length as p-values, where `decisions[i]` is 1\n",
    "        if `p_values[i]` is deemed significant, and 0 otherwise\n",
    "    \"\"\"\n",
    "    p_values = []\n",
    "\n",
    "    for i in range(10):\n",
    "        # test = test_statistic(df, column)\n",
    "        p = p_value(df, column, test, 'greater')\n",
    "        p_values.append(p)\n",
    "    \n",
    "    decisions = [(p_value <= alpha/len(p_values)) for p_value in p_values]\n",
    "    return np.sum(decisions)/len(decisions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def benjamini_hochberg(df, column, test, alpha):\n",
    "    \"\"\"\n",
    "    Returns decisions on p-values using Benjamini-Hochberg.\n",
    "    \n",
    "    Inputs:\n",
    "        p_values: array of p-values\n",
    "        alpha: desired FDR (FDR = E[# false positives / # positives])\n",
    "    \n",
    "    Returns:\n",
    "        decisions: binary array of same length as p-values, where `decisions[i]` is 1\n",
    "        if `p_values[i]` is deemed significant, and 0 otherwise\n",
    "    \"\"\"\n",
    "    p_values = []\n",
    "    for i in range(10):\n",
    "        # test = test_statistic(df, column)\n",
    "        p = p_value(df, column, test, 'greater')\n",
    "        p_values.append(p)\n",
    "        \n",
    "    p_list = np.array(p_values)\n",
    "    p_list.sort()\n",
    "    k = np.arange(len(p_values)) + 1\n",
    "    pk = k*alpha/len(p_values)\n",
    "    boolean_array = list(p_list <= pk)\n",
    "    if sum(boolean_array) == 0:\n",
    "        return 0\n",
    "    p_max = np.max(p_list[boolean_array])\n",
    "    decisions = [(p_value <= p_max) for p_value in p_values]\n",
    "    return np.sum(decisions)/len(decisions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 1. Are more populated cities associated with deadlier instances of gun violence?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.05\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bonferroni_decisions_1 = bonferroni(victim_populous, 'populous?', test_victim_populous, alpha)\n",
    "bonferroni_decisions_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bh_decisions_1 = benjamini_hochberg(incidents_populous, 'populous?', test_victim_populous, alpha)\n",
    "bh_decisions_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Are more populated cities associated with more instances of gun violence?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bonferroni_decisions_2 = bonferroni(incidents_populous, 'populous?', test_incidents_populous, alpha)\n",
    "bonferroni_decisions_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bh_decisions_2 = benjamini_hochberg(incidents_populous, 'populous?', test_incidents_populous, alpha)\n",
    "bh_decisions_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Are more densely populated states associated with deadlier instances of gun violence?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bonferroni_decisions_3 = bonferroni(victims_dense, 'dense?', test_victim_dense, alpha)\n",
    "bonferroni_decisions_3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bh_decisions_3 = benjamini_hochberg(victims_dense, 'dense?', test_victim_dense, alpha)\n",
    "bh_decisions_3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Are more densely populated states associated with more instances of gun violence?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bonferroni_decisions_4 = bonferroni(victims_dense, 'dense?', test_incidents_dense, alpha)\n",
    "bonferroni_decisions_4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bh_decisions_4 = benjamini_hochberg(incidents_dense, 'dense?', test_incidents_dense, alpha)\n",
    "bh_decisions_4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Are crimes getting less violent over time (number of casualties)?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bonferroni_decisions_5 = bonferroni(time_df, 'before?', test_time, alpha)\n",
    "bonferroni_decisions_5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bh_decisions_5 = benjamini_hochberg(time_df, 'before?', test_time, alpha)\n",
    "bh_decisions_5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary and Interpretation of the Result\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our correction in the process of multiple hypothesis, we utilized Bonferroni Correction and Benjamini-Hochberg (B-H). For each hypothesis tests that we had at the earlier part of our experiment, we derived the decisions for each tests using both Bonferroni and Benjamini-Hochberg. However, the results were different from our previous part of the experiment. Before the correction, we were able to find some statistically significant relation between the factors for few of the hypothesis that we had, but we weren't able to find any statistic significance between the variables that we listed for all five of our hypothesis. Even some of the decisions that we derived from the first hypothesis (which was the only hypothesis that showed some sign of statistic significance) had only 1 decision value out of 10 simulations. For this discovery, we denoted the multiple confounding variables that are behind the correlation between the two compared factors as the reason for low significance. \n",
    "\n",
    "Before we derived the results, we expected that we would have most of our simulations to have the same discovery for each individual tests. However, the aggregate result demonstrated a completely different discovery as none of the decisions were withled with our expectations. Most of our hypothesis demonstrated 0 decisions for both Bonferroni and Benhamini experiment. \n",
    "\n",
    "For some of the limitations that we had in our analysis, as it was mentioned before, there were countless number of confounding variables that we could count into our analysis. Therefore, we were never able to discover the clear correlation between the factors for each of our hypothesis. The p-values/decisions that were made for each tests did not show any sign of significance. The way we avoided p-hacking was we added more data sets and city variables to outweigh the confounding variables that were getting in our way of experiment. \n",
    "\n",
    "If we had more data, the addimitional tests that we would have conducted is generalized linear model. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "– After applying your correction procedures, which discoveries remained significant? If none\n",
    "did, explain why.\n",
    "\n",
    "~  \n",
    "\n",
    "\n",
    "– What decisions can or should be made from the individual tests? What about from the\n",
    "results in aggregate?\n",
    "\n",
    "~\n",
    "\n",
    "– Discuss any limitations in your analysis, and if relevant, how you avoided p-hacking.\n",
    "\n",
    "~\n",
    "\n",
    "– What additional tests would you conduct if you had more data?\n",
    "\n",
    "~\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Citations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- https://inferentialthinking.com/chapters/12/1/AB_Testing.html\n",
    "- 'lab01', Data 102 https://data102.datahub.berkeley.edu/user/doud/notebooks/sp22/labs/lab01/lab01.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "data102final.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}