ls88_c06_confidence.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.5.10'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# HIDDEN\n",
    "# This useful nonsense should just go at the top of your notebook.\n",
    "from datascience import *\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plots\n",
    "import numpy as np\n",
    "plots.style.use('fivethirtyeight')\n",
    "# datascience version number of last run of this notebook\n",
    "version.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code that sets the random number seed should make our RNG results uniform instead of jumping around. But remember this probably needs to be rerun each time you rerun the RNG!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "np.random.seed(seed=30716)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<h1>Class 6: Permutations and Confidence Intervals with Cat People</h1>\n",
    "\n",
    "*Very special thanks to Kyle Nguyen, who wrote this exercise.*\n",
    "\n",
    "<h3>\"How different is that?\"</h3>\n",
    "\n",
    "When data is inherently messy, how different do things have to be before we say that the differences between them are *statistically significant*?\n",
    "\n",
    "For example, suppose we ran a randomized controlled trial to examine the effect, $b$, of *eating breakfast*, $X$, on *body mass index*, $Y$. Formally, we can imagine $Y$ is a linear function of $X$:\n",
    "\n",
    "$$Y = a + b X + \\epsilon$$\n",
    "\n",
    "and we'd like to estimate $b$.  When $X$ is a binary 0/1 treatment, we can compare the average $Y$ given $X = 1$ to the average $Y$ given $X = 0$, where $E[\\cdot]$ is the *expectations operator* that produces the average:\n",
    "\n",
    "$$b = E [Y \\ | \\ X = 1] - E[Y \\ | \\ X = 0]$$\n",
    "\n",
    "Answering this question &mdash; how do we assess how big a $b$ is \"big enough\" &mdash; is critical for statistical inference!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are at least two ways of doing this when we have data on $Y$ and $X$:\n",
    "<ol>\n",
    "<li>  Using the *bootstrap* &mdash; which is like rolling our dice a bunch of times and comparing what we see in the data to what randomness alone gets us &mdash; to estimate *standard errors* and *confidence intervals*\n",
    "</li>\n",
    "\n",
    "<li>  Using a combination of empirics and theory to estimate *standard errors* and *confidence intervals*\n",
    "</li>\n",
    "</ol>\n",
    "\n",
    "In DS 8, you'll mostly use the bootstrap.  In other classes, and with other statistical packages other than Python, you may estimate standard errors instead. These methods are formally and computationally very different, but they are meant to answer the same question: \"how different is that?\"\n",
    "\n",
    "Let's reexamine the HRS pets data from class 4."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Data source</h2>\n",
    "\n",
    "As you may recall, the U.S. <a href=\"http://hrsonline.isr.umich.edu\">Health and Retirement Study</a> (HRS) is a panel survey of thousands of Americans aged 50 and over and their spouses that is conducted biennially (every 2 years) starting in 1992.  In its 11th wave in 2012, a subsample of about 1,700 individuals were asked about <b>pet ownership</b>. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>hhidpn</th> <th>age</th> <th>sex</th> <th>edyrs</th> <th>health</th> <th>anypets</th> <th>numdogs</th> <th>numcats</th> <th>height</th> <th>weight</th> <th>bmi</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>1.0059e+07</td> <td>84  </td> <td>1   </td> <td>17   </td> <td>2     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.905 </td> <td>89.3572</td> <td>24.6</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.021e+07 </td> <td>73  </td> <td>1   </td> <td>6    </td> <td>5     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.6256</td> <td>77.1103</td> <td>29.2</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0372e+07</td> <td>76  </td> <td>2   </td> <td>10   </td> <td>3     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.4986</td> <td>67.5849</td> <td>30.1</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0395e+07</td> <td>74  </td> <td>2   </td> <td>15   </td> <td>5     </td> <td>1      </td> <td>1      </td> <td>0      </td> <td>1.7272</td> <td>92.0788</td> <td>30.9</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0458e+07</td> <td>68  </td> <td>2   </td> <td>16   </td> <td>4     </td> <td>1      </td> <td>0      </td> <td>1      </td> <td>1.6002</td> <td>66.6777</td> <td>26  </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0475e+07</td> <td>81  </td> <td>2   </td> <td>17   </td> <td>3     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.524 </td> <td>41.2767</td> <td>17.8</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0648e+07</td> <td>71  </td> <td>1   </td> <td>12   </td> <td>4     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.8034</td> <td>68.0385</td> <td>20.9</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0773e+07</td> <td>83  </td> <td>1   </td> <td>17   </td> <td>2     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.6764</td> <td>63.5026</td> <td>22.6</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.0818e+07</td> <td>63  </td> <td>2   </td> <td>13   </td> <td>2     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.6383</td> <td>70.3064</td> <td>26.2</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>1.1141e+07</td> <td>78  </td> <td>2   </td> <td>16   </td> <td>2     </td> <td>0      </td> <td>0      </td> <td>0      </td> <td>1.7526</td> <td>74.8423</td> <td>24.4</td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>\n",
       "<p>... (1684 rows omitted)</p"
      ],
      "text/plain": [
       "hhidpn     | age  | sex  | edyrs | health | anypets | numdogs | numcats | height | weight  | bmi\n",
       "1.0059e+07 | 84   | 1    | 17    | 2      | 0       | 0       | 0       | 1.905  | 89.3572 | 24.6\n",
       "1.021e+07  | 73   | 1    | 6     | 5      | 0       | 0       | 0       | 1.6256 | 77.1103 | 29.2\n",
       "1.0372e+07 | 76   | 2    | 10    | 3      | 0       | 0       | 0       | 1.4986 | 67.5849 | 30.1\n",
       "1.0395e+07 | 74   | 2    | 15    | 5      | 1       | 1       | 0       | 1.7272 | 92.0788 | 30.9\n",
       "1.0458e+07 | 68   | 2    | 16    | 4      | 1       | 0       | 1       | 1.6002 | 66.6777 | 26\n",
       "1.0475e+07 | 81   | 2    | 17    | 3      | 0       | 0       | 0       | 1.524  | 41.2767 | 17.8\n",
       "1.0648e+07 | 71   | 1    | 12    | 4      | 0       | 0       | 0       | 1.8034 | 68.0385 | 20.9\n",
       "1.0773e+07 | 83   | 1    | 17    | 2      | 0       | 0       | 0       | 1.6764 | 63.5026 | 22.6\n",
       "1.0818e+07 | 63   | 2    | 13    | 2      | 0       | 0       | 0       | 1.6383 | 70.3064 | 26.2\n",
       "1.1141e+07 | 78   | 2    | 16    | 2      | 0       | 0       | 0       | 1.7526 | 74.8423 | 24.4\n",
       "... (1684 rows omitted)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#importing the data set that we will be using \n",
    "HRSpets = Table.read_table(\"http://demog.berkeley.edu/~redwards/Courses/LS88/c04_hrspets.csv\")\n",
    "HRSpets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Formulate a question</h2>\n",
    "\n",
    "There are many questions we can ask with data. Let's just pick one.  How about: \"Do female respondents report owning more cats ($Y$) than males?\"  Reformulating this, we might also say, \"Is the average number of cats owned by female respondents, $E[Y \\ | \\ F = 1]$, greater than the average number of cats owned by male respondents, $E[Y \\ | \\ F = 0]$?\"\n",
    "\n",
    "Here I've defined the new female indicator variable $F =$ `sex` $- \\ 1$, where our variable `sex` = 1 for males and 2 for females. At the end of this notebook, you'll see why this is useful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>How to formulate and complete a Hypothesis Test</h2>\n",
    "\n",
    "Whenever you get a data set and are looking to draw a conclusion about something from the data set, you will be running a hypothesis test! In this example, we would like to determine whether or not there is an actual difference between the average number of cats owned by men and women. In order to come up with a conclusion, we will need these important pieces (taken from the Fall 2015 Data 8 Course):\n",
    "\n",
    "<b>Null Hypothesis</b>: This says that the data were generated at random under clearly specified assumptions that make it possible to compute chances. The word \"null\" reinforces the idea that if the data look different from what the null hypothesis predicts, the difference is due to nothing but chance.\n",
    "\n",
    "<b>Alternative Hypothesis</b>: This says that some reason other than chance made the data differ from what was predicted by the null hypothesis.\n",
    "\n",
    "<b>Test Statistic</b>: In order to decide between the two hypothesis, we must choose a statistic upon which we will base our decision. This is called the test statistic. The test statistic can be anything you want it to be as long as it is repeatable.  \n",
    "\n",
    "<b>P-value</b>: The P-value is the probability, under the null hypothesis, that the test statistic is equal to the value that was observed or is even further in the direction of the alternative. \n",
    "\n",
    "If your P-value is under 0.05, the observed test statistic is said to be statistically significant at the 5% level, a conventional level of significance. If your P-value is under 0.01, the observed test statistic is statistically significant at the 1% level. If you obtain a highly significant result, with a very low P-value, you can reject the null hypothesis in favor of the alternative hypothesis.  Recall that rejecting a \"null\" hypothesis means you're finding a pattern, typically a good thing!\n",
    "\n",
    "The P-value and thus the choice between the null and alternative hypotheses depends on the comparison between the observed test statistic and its *simulated distribution* (basically the list of simulated values we will come up with) as predicted by the null hypothesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q1: State in words both (1) a good null hypothesis and (2) an alternative hypothesis regarding the number of cats reported by sex of the respondent.</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we are comparing the average number of cats owned by males and females, what do you think our test statistic should be? \n",
    "\n",
    "We will be taking the difference between the mean numbers of cats owned by males and females! \n",
    "\n",
    "*(Since the P-value is based on proportion of the value you get from the data set, we will not have to take the absolute value of the difference! That will actually skew our dataset set, by turning it into a one tailed test (making a P-value using only half of a normal distribution) rather than a two tailed test (the full normal distribution). This does not matter too much in the end, as you can always multiply a one tailed test value by 2 to turn it into a two tailed test value, we will not worry about this for now.)*\n",
    "\n",
    "In the cell below, we will be solving for the test statistic of the empirical dataset. In order to solve for the test statistic, we will:\n",
    "<ol>\n",
    "<li>Find the average number of cats owned by males: $E[Y \\ | \\ F = 0]$\n",
    "</li>\n",
    "\n",
    "<li>Find the average number of cats owned by females: $E[Y \\ | \\ F = 1]$\n",
    "</li>\n",
    "\n",
    "<li>Find the difference. You can examine either male - female, or female - male, *as long as you keep the process consistent*\n",
    "</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q2: Select males from HRSpets using `.where()` remembering that `sex` == 1 for males, and create a new Table:</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Males = HRSpets.where('sex',1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*(If you want to use `.mean()` from numpy on `Males`, that's cool. Otherwise, construct the mean manually as shown here.)*\n",
    "\n",
    "<br>\n",
    "<font color=\"blue\">Q3: Calculate the number of males, for example using `len(Males.rows)`. This will be our denominator.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "641"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num_males = len(Males.rows)\n",
    "num_males"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q4: Calculate the total number of cats owned by males using `sum()` and a call to `Males` and its appropriate column. This will be our numerator.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "204"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "total_cats_males = sum(Males['numcats'])\n",
    "total_cats_males"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q5: Now calculate the average number of cats owned by males, which equals the total number of cats owned by males divided by the number of males.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.31825273010920435"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "avg_cats_males = total_cats_males / num_males\n",
    "avg_cats_males "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q6: Repeat this process for females.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.47768281101614435"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Females = HRSpets.where('sex',2)\n",
    "num_females = len(Females.rows)\n",
    "total_cats_females = sum(Females['numcats'])\n",
    "avg_cats_females = total_cats_females / num_females\n",
    "avg_cats_females"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q7: Construct the <b>test statistic</b>.  Let's measure the average number of cats among women minus the average number of cats among men: $E[Y \\ | \\ F = 1] - E[Y \\ | \\ F = 0]$</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.15943008090694"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff_avg_cats = avg_cats_females - avg_cats_males\n",
    "diff_avg_cats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "<h3>Is this difference statistically significant?</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our test statistic, `diff_avg_cats`, how will we be able to tell whether or not it is statistically significant? \n",
    "\n",
    "According to the null hypothesis, the two variables are not related. If that were true, we could jumble around and play with the values all we want. If on the other hand they were related, then altering the values would alter the outcome.\n",
    "\n",
    "*If it helps, think about the example from Class 5 with 4 dice. When we run the bootstrap, we're basically taking the dataset, tossing it up in the air like dice, and when things land randomly, we're seeing if things look the same or different from our original dataset.  If our dataset looks like 4 \"boxcars\" (i.e., number 6's), or like a real outlier, then we know something special is happening.  If it looks like two 3's and two 4's for 14, then we know it's not very special.*\n",
    "\n",
    "<h4>Some terminology:</h4>\n",
    "The process of resampling from the same data set with replacement is called the *bootstrap*.\n",
    "\n",
    "The process of resampling from the same data set *without* replacement is called *permuting*.\n",
    "\n",
    "You will see permutation and bootstrap tests many times in the class. For the purpose of the class, there isn't an enormous difference between the two;  you can use them interchangeably. That is because when you have a large sample size, the difference between sampling with and without replacement is close to negligible. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we care only about gender and number of cats owned we will select only those columns for simplicity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cat_ownership_data = HRSpets.select(['sex', 'numcats'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's set `repetitions` to determine how many iterations to make. The more the better; 1,000 is a good round number.\n",
    "\n",
    "And `diffs` stores the simulated test statistic which we will use later to compare to our empirical test statistic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "repetitions = 1000\n",
    "diffs = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now for the guts of the bootstrap.\n",
    "\n",
    "Let's write a `for`-loop that will loop through all `repetitions` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.1970487334931153,\n",
       " 0.19959534959249692,\n",
       " 0.17797206131254617,\n",
       " 0.21600789674478954,\n",
       " 0.15002443443228686,\n",
       " 0.044795865908710431,\n",
       " 0.21242490413534687,\n",
       " 0.18306345286204501,\n",
       " 0.10250463821892397,\n",
       " 0.11745720204994003,\n",
       " 0.15929229761783065,\n",
       " 0.055569358178053863,\n",
       " 0.097396804800908232,\n",
       " 0.19514432485322891,\n",
       " 0.21147260273972601,\n",
       " 0.2345977011494253,\n",
       " 0.096602387511478427,\n",
       " 0.057808359888410266,\n",
       " 0.22584179237746133,\n",
       " 0.24550669790352619,\n",
       " 0.10986548437688504,\n",
       " 0.25608488893778802,\n",
       " 0.20363408521303261,\n",
       " 0.14777390317087019,\n",
       " 0.15448472499011023,\n",
       " 0.22267042323806752,\n",
       " 0.11549165120593691,\n",
       " 0.2147654231993753,\n",
       " 0.16628221301135337,\n",
       " 0.13210724942039986,\n",
       " 0.114957974942972,\n",
       " 0.18829944002357796,\n",
       " 0.19277360066833754,\n",
       " 0.048133715225523843,\n",
       " 0.16382046244521686,\n",
       " 0.28019597845123823,\n",
       " 0.172940957155783,\n",
       " 0.16758115737473539,\n",
       " 0.17794060501866393,\n",
       " 0.074713161530606087,\n",
       " 0.11411353646165984,\n",
       " 0.063772918345631979,\n",
       " 0.14977683646240919,\n",
       " 0.18484186286949578,\n",
       " 0.20687134502923982,\n",
       " 0.12304613377609108,\n",
       " 0.12052655176955074,\n",
       " 0.13604112623337244,\n",
       " 0.19963692637343017,\n",
       " 0.17162402403651222,\n",
       " 0.21936308621738371,\n",
       " 0.12255639097744359,\n",
       " 0.16667558345519617,\n",
       " 0.1952086432787829,\n",
       " 0.19161208100915461,\n",
       " 0.21390429246251796,\n",
       " 0.18209274832147399,\n",
       " 0.10191042709218517,\n",
       " 0.12581413227643551,\n",
       " 0.11008298450466325,\n",
       " 0.16397679813702515,\n",
       " 0.1779395178344847,\n",
       " 0.22577976755218215,\n",
       " 0.086042124145609744,\n",
       " 0.17213250517598344,\n",
       " 0.044172932330827086,\n",
       " 0.20193997257630386,\n",
       " 0.056941440598690385,\n",
       " 0.1734814694671965,\n",
       " 0.20825644954836914,\n",
       " 0.10412131154809673,\n",
       " 0.18140712606257914,\n",
       " 0.064384293107697388,\n",
       " 0.19142561615008741,\n",
       " 0.14310121858395963,\n",
       " 0.10271017822488904,\n",
       " 0.19529477131828282,\n",
       " 0.17575569358178056,\n",
       " 0.14641821946169775,\n",
       " 0.17226644397853669,\n",
       " 0.23256340875718406,\n",
       " 0.22178552038335336,\n",
       " 0.21015502473157194,\n",
       " 0.16538562138127,\n",
       " 0.18629079480407745,\n",
       " 0.16855080690941532,\n",
       " 0.081274835779742149,\n",
       " 0.22295112645159954,\n",
       " 0.1469760900140647,\n",
       " 0.13149499160442363,\n",
       " 0.16674397031539889,\n",
       " 0.19453016888175262,\n",
       " 0.187838155728064,\n",
       " 0.18662153449387492,\n",
       " 0.11832662824834184,\n",
       " 0.10906891324626866,\n",
       " 0.12020688425182807,\n",
       " 0.20511387163561073,\n",
       " 0.18336375145924794,\n",
       " 0.30246847362273244,\n",
       " 0.18952506773550681,\n",
       " 0.13247765231710185,\n",
       " 0.15739088202252999,\n",
       " 0.14044752983408615,\n",
       " 0.20711896762486431,\n",
       " 0.20921052631578949,\n",
       " 0.1581652961779717,\n",
       " 0.16234575308938287,\n",
       " 0.19278548687451191,\n",
       " 0.12601343902644746,\n",
       " 0.1238527431408159,\n",
       " 0.078583831207519383,\n",
       " 0.17421209384078312,\n",
       " 0.15175717928305238,\n",
       " 0.25788497217068651,\n",
       " 0.11235406032000161,\n",
       " 0.14963940403197451,\n",
       " 0.16795496517811448,\n",
       " 0.24484203739522892,\n",
       " 0.083926027106908607,\n",
       " 0.046119685497961582,\n",
       " 0.16529494481601403,\n",
       " 0.20651192778852351,\n",
       " 0.15645359267302428,\n",
       " 0.19573098332753963,\n",
       " 0.19676491510221755,\n",
       " 0.20927125358921239,\n",
       " 0.13520913983570959,\n",
       " 0.16030893560431692,\n",
       " 0.072769324334522545,\n",
       " 0.15524790794823135,\n",
       " 0.21200967504469448,\n",
       " 0.19721075030026936,\n",
       " 0.12620820271682337,\n",
       " 0.12330088109800402,\n",
       " 0.25374423288601605,\n",
       " 0.19069869633067421,\n",
       " 0.14642956583627775,\n",
       " 0.11422048242575095,\n",
       " 0.12209302325581395,\n",
       " 0.21394038723624229,\n",
       " 0.10564837244118225,\n",
       " 0.14733786369334306,\n",
       " 0.26653723847391142,\n",
       " 0.16866458167487225,\n",
       " 0.20854019345959018,\n",
       " 0.12143171323368246,\n",
       " 0.19301534060079906,\n",
       " 0.088689192427510166,\n",
       " 0.22617409867172672,\n",
       " 0.18881524804905664,\n",
       " 0.22396892201971208,\n",
       " 0.24896480331262943,\n",
       " 0.21051756136849109,\n",
       " 0.079213215377640311,\n",
       " 0.09360135031456196,\n",
       " 0.16930071444035505,\n",
       " 0.19269763708896914,\n",
       " 0.1538531347338345,\n",
       " 0.23440043407487793,\n",
       " 0.14553190227703983,\n",
       " 0.14397477935654646,\n",
       " 0.15042243759037299,\n",
       " 0.19301379916026035,\n",
       " 0.16717123433184622,\n",
       " 0.23414741710171877,\n",
       " 0.12293233082706767,\n",
       " 0.15087206191817959,\n",
       " 0.19488417457004026,\n",
       " 0.25387616639282912,\n",
       " 0.12729063749705954,\n",
       " 0.23523371395711817,\n",
       " 0.12839602925809823,\n",
       " 0.10950276000087966,\n",
       " 0.16821224334510348,\n",
       " 0.16891851675495506,\n",
       " 0.096583850931676984,\n",
       " 0.16549828797981619,\n",
       " 0.17497305236724592,\n",
       " 0.19149938907313668,\n",
       " 0.13311918769023473,\n",
       " 0.19627261509863309,\n",
       " 0.13906013092747999,\n",
       " 0.10518903646098587,\n",
       " 0.1209182340647858,\n",
       " 0.11168461201331392,\n",
       " 0.24166595941412689,\n",
       " 0.20622820533614472,\n",
       " 0.12028123616293929,\n",
       " 0.2440939832089552,\n",
       " 0.078237203562216584,\n",
       " 0.17659669349736956,\n",
       " 0.13736783557029536,\n",
       " 0.28067951961903909,\n",
       " 0.15730807702841038,\n",
       " 0.16117810760667906,\n",
       " 0.17219947713707889,\n",
       " 0.15285158897171597,\n",
       " 0.20410135899084397,\n",
       " 0.13085863377609108,\n",
       " 0.13914684819956874,\n",
       " 0.15602799341331453,\n",
       " 0.11248073959938365,\n",
       " 0.13199288885192384,\n",
       " 0.11276562395037953,\n",
       " 0.16347221893980574,\n",
       " 0.10942540322580646,\n",
       " 0.19168163489870882,\n",
       " 0.10122210534394566,\n",
       " 0.099579510703363905,\n",
       " 0.21979527746799604,\n",
       " 0.21916850187488846,\n",
       " 0.19615666140888049,\n",
       " 0.14025222381163777,\n",
       " 0.15439213207417712,\n",
       " 0.20928935494968998,\n",
       " 0.10237203495630459,\n",
       " 0.17380507904844705,\n",
       " 0.19503988776735509,\n",
       " 0.10923399898958924,\n",
       " 0.1319460407718831,\n",
       " 0.12060041407867494,\n",
       " 0.13371010803836481,\n",
       " 0.17230259300475048,\n",
       " 0.16316710151802655,\n",
       " 0.19574468085106389,\n",
       " 0.054357091142509528,\n",
       " 0.17837367936415627,\n",
       " 0.1590390496353799,\n",
       " 0.11627605222609683,\n",
       " 0.19200515796260481,\n",
       " 0.15628647214854108,\n",
       " 0.092208132769755258,\n",
       " 0.1528001836958186,\n",
       " 0.13689422373297455,\n",
       " 0.094369871082315093,\n",
       " 0.16127074760737897,\n",
       " 0.19027560083594564,\n",
       " 0.13738436907020873,\n",
       " 0.17666916124912674,\n",
       " 0.16654868743101214,\n",
       " 0.22030118690190637,\n",
       " 0.1577060503579068,\n",
       " 0.23669368605134827,\n",
       " 0.1693611908417329,\n",
       " 0.19621865203761751,\n",
       " 0.20305689340458755,\n",
       " 0.11883563908338174,\n",
       " 0.13595625276710821,\n",
       " 0.18919241287019256,\n",
       " 0.090966077190286654,\n",
       " 0.17192960752172698,\n",
       " 0.21099679791271342,\n",
       " 0.16808405158780948,\n",
       " 0.12241933719785625,\n",
       " 0.15003448812940834,\n",
       " 0.18777855473431737,\n",
       " 0.22095760504933643,\n",
       " 0.21510110893672535,\n",
       " 0.1473717094986764,\n",
       " 0.14244963084205692,\n",
       " 0.10862835611127264,\n",
       " 0.11958478631092373,\n",
       " 0.09300973062328588,\n",
       " 0.12941193976549015,\n",
       " 0.098480372462363797,\n",
       " 0.12136046414856561,\n",
       " 0.16925449669736647,\n",
       " 0.11190107731682641,\n",
       " 0.15068150705315159,\n",
       " 0.09627671293236395,\n",
       " 0.1691190493217955,\n",
       " 0.089400697429243392,\n",
       " 0.14618399575826252,\n",
       " 0.22407732994394819,\n",
       " 0.1376111034840104,\n",
       " 0.11419549093045456,\n",
       " 0.16817437650123684,\n",
       " 0.13675785063977675,\n",
       " 0.1637009375956206,\n",
       " 0.15161290322580645,\n",
       " 0.16052995056702529,\n",
       " 0.19405839063819524,\n",
       " 0.23703631138975961,\n",
       " 0.13842075636430473,\n",
       " 0.18214491086831514,\n",
       " 0.096763729820096345,\n",
       " 0.098846646979151742,\n",
       " 0.15349716501616745,\n",
       " 0.20603006168317067,\n",
       " 0.096982758620689669,\n",
       " 0.14964347586863258,\n",
       " 0.089341692789968674,\n",
       " 0.12943262411347517,\n",
       " 0.19268984627564822,\n",
       " 0.18806215722120662,\n",
       " 0.14932516679426883,\n",
       " 0.038721804511278157,\n",
       " 0.089478584729981381,\n",
       " 0.19235447617661428,\n",
       " 0.16159659180344027,\n",
       " 0.18599086812144211,\n",
       " 0.18798662508922864,\n",
       " 0.18848136135749533,\n",
       " 0.18278058998079943,\n",
       " 0.18281753707285625,\n",
       " 0.28714464455432442,\n",
       " 0.054336921370063518,\n",
       " 0.10678629897152236,\n",
       " 0.21102167445427389,\n",
       " 0.15126637711762725,\n",
       " 0.096731814126996996,\n",
       " 0.16084226232435733,\n",
       " 0.19136041263700837,\n",
       " 0.12516474958063739,\n",
       " 0.25007474303858807,\n",
       " 0.1509682516858894,\n",
       " 0.20045585869904942,\n",
       " 0.10666159116863339,\n",
       " 0.17074032873465228,\n",
       " 0.10475827963080991,\n",
       " 0.19908651937012939,\n",
       " 0.12405182919246932,\n",
       " 0.19024935309338981,\n",
       " 0.23517871568102894,\n",
       " 0.15218041087640244,\n",
       " 0.14603169961398993,\n",
       " 0.19391025641025639,\n",
       " 0.15100213472485768,\n",
       " 0.19864322610681273,\n",
       " 0.18682358322850284,\n",
       " 0.10815313064188625,\n",
       " 0.15333333333333332,\n",
       " 0.10541393592416615,\n",
       " 0.08951055991850565,\n",
       " 0.26015655157938616,\n",
       " 0.089501684341090959,\n",
       " 0.14461697722567285,\n",
       " 0.12849774339136039,\n",
       " 0.21097308488612837,\n",
       " 0.19821454202692246,\n",
       " 0.22453255963894264,\n",
       " 0.15222367191415015,\n",
       " 0.14107903849445469,\n",
       " 0.18546359610117302,\n",
       " 0.20845683368619147,\n",
       " 0.13492063492063494,\n",
       " 0.18933747412008284,\n",
       " 0.19215613948804422,\n",
       " 0.22765155568237211,\n",
       " 0.14935751312331069,\n",
       " 0.13156302420321114,\n",
       " 0.14718921635495569,\n",
       " 0.056380248788851672,\n",
       " 0.23989654057549309,\n",
       " 0.22860677638350296,\n",
       " 0.24124631941199592,\n",
       " 0.26502089864158829,\n",
       " 0.12178290359936667,\n",
       " 0.065997540558740075,\n",
       " 0.1674167954201094,\n",
       " 0.15202335577001053,\n",
       " 0.15592186953967863,\n",
       " 0.11166129833501598,\n",
       " 0.15729493207941486,\n",
       " 0.20474805853705236,\n",
       " 0.14769374541085373,\n",
       " 0.11371599072639754,\n",
       " 0.15994400235779543,\n",
       " 0.11211831008292472,\n",
       " 0.20681604328393316,\n",
       " 0.16475107374024561,\n",
       " 0.11518518518518517,\n",
       " 0.22942986718496927,\n",
       " 0.25925229741019218,\n",
       " 0.23906546489563574,\n",
       " 0.16767682974107345,\n",
       " 0.13590062111801243,\n",
       " 0.22056526701507778,\n",
       " 0.14258904377120818,\n",
       " 0.15047652464031741,\n",
       " 0.087619817876827211,\n",
       " 0.17346769906255444,\n",
       " 0.062778671655753071,\n",
       " 0.099613946193750746,\n",
       " 0.26525066151755705,\n",
       " 0.060583554376657855,\n",
       " 0.11452099636847224,\n",
       " 0.099885865106823124,\n",
       " 0.19701429741987403,\n",
       " 0.18850289312080165,\n",
       " 0.12512938132204193,\n",
       " 0.10573543224255177,\n",
       " 0.1763877820849335,\n",
       " 0.19033126293995856,\n",
       " 0.11782697239968604,\n",
       " 0.076640157871103454,\n",
       " 0.19653904963537994,\n",
       " 0.087174250049592616,\n",
       " 0.17286723058113135,\n",
       " 0.30238151820885628,\n",
       " 0.23950693059178563,\n",
       " 0.17643592643295042,\n",
       " 0.19011872733653062,\n",
       " 0.10555778925555936,\n",
       " 0.20052954070301587,\n",
       " 0.19813664596273289,\n",
       " 0.15371107399642281,\n",
       " 0.16637719100771819,\n",
       " 0.21624883718191773,\n",
       " 0.088851880877742928,\n",
       " 0.25185226762460705,\n",
       " 0.09545081125487781,\n",
       " 0.22682868085906294,\n",
       " 0.17286094881424313,\n",
       " 0.16914462022265242,\n",
       " 0.1611124612673917,\n",
       " 0.19390450469658743,\n",
       " 0.062241509701510211,\n",
       " 0.12018822566620635,\n",
       " 0.11117376775271515,\n",
       " 0.20964525921076133,\n",
       " 0.29703031298775978,\n",
       " 0.13556708340226409,\n",
       " 0.085624324451336997,\n",
       " 0.084678267598154111,\n",
       " 0.17364096537077706,\n",
       " 0.12625492096540669,\n",
       " 0.17155388471177946,\n",
       " 0.15945674871339544,\n",
       " 0.063617006185741232,\n",
       " 0.12308188265635073,\n",
       " 0.23386886102403343,\n",
       " 0.044791752238560745,\n",
       " 0.15123111524557542,\n",
       " 0.17715004844846738,\n",
       " 0.16604823747680891,\n",
       " 0.22149872031426698,\n",
       " 0.078706828811973772,\n",
       " 0.1599327197564645,\n",
       " 0.11766561514195584,\n",
       " 0.16478836287313431,\n",
       " 0.15931612664388295,\n",
       " 0.14389749412050928,\n",
       " 0.200587084148728,\n",
       " 0.17082512058715121,\n",
       " 0.18801264317608229,\n",
       " 0.10137189469781238,\n",
       " 0.15653956969130023,\n",
       " 0.22192188133531554,\n",
       " 0.15868375885003216,\n",
       " 0.22131372617753942,\n",
       " 0.11368985688192179,\n",
       " 0.13799491887471232,\n",
       " 0.13772145521541038,\n",
       " 0.16815527006817943,\n",
       " 0.22329443175914854,\n",
       " 0.25532797809440727,\n",
       " 0.15805531730846767,\n",
       " 0.061310808066464217,\n",
       " 0.2396421845574388,\n",
       " 0.09824629930838924,\n",
       " 0.12362718974152281,\n",
       " 0.16667821467752175,\n",
       " 0.20168231992516372,\n",
       " 0.1711059780320256,\n",
       " 0.23728823499226614,\n",
       " 0.17594884824199125,\n",
       " 0.1184656072696037,\n",
       " 0.14407619856950638,\n",
       " 0.089056570208728703,\n",
       " 0.13671330324013892,\n",
       " 0.1995087467050084,\n",
       " 0.21819923371647515,\n",
       " 0.10426667147892832,\n",
       " 0.12694137395125432,\n",
       " 0.19806310527960019,\n",
       " 0.10256580588684233,\n",
       " 0.15904099533511112,\n",
       " 0.16952752090227735,\n",
       " 0.15042243759037299,\n",
       " 0.19906767558104715,\n",
       " 0.074854190275193078,\n",
       " 0.14453092801714496,\n",
       " 0.061688311688311681,\n",
       " 0.20639097744360901,\n",
       " 0.14326105920652926,\n",
       " 0.09043780836758003,\n",
       " 0.14508838891120934,\n",
       " 0.10509018894181876,\n",
       " 0.15187196977899686,\n",
       " 0.064429024319950345,\n",
       " 0.19045092838196287,\n",
       " 0.20286733566220916,\n",
       " 0.1301418439716312,\n",
       " 0.21139217120772091,\n",
       " 0.11355325581673259,\n",
       " 0.19559832205746602,\n",
       " 0.1512240663585977,\n",
       " 0.21125289456022689,\n",
       " 0.12456350105856084,\n",
       " 0.18970538455253233,\n",
       " 0.20308115185951175,\n",
       " 0.18460549360409689,\n",
       " 0.13054939358596135,\n",
       " 0.24341623115276026,\n",
       " 0.21533511922362325,\n",
       " 0.15745820271682337,\n",
       " 0.14919214157673749,\n",
       " 0.16730273091510034,\n",
       " 0.18526743358633779,\n",
       " 0.16027211482776565,\n",
       " 0.13496345283675598,\n",
       " 0.12596306803603763,\n",
       " 0.1430867144229076,\n",
       " 0.1283566089809649,\n",
       " 0.20863882528259364,\n",
       " 0.17519862005835235,\n",
       " 0.13366459627329191,\n",
       " 0.22357962453221164,\n",
       " 0.14266588296760713,\n",
       " 0.17496363530575432,\n",
       " 0.085739997501914966,\n",
       " 0.047387560365392489,\n",
       " 0.19149522998778984,\n",
       " 0.24238394210249592,\n",
       " 0.19255839817115422,\n",
       " 0.20830111650635574,\n",
       " 0.11561852539150214,\n",
       " 0.19537329219533817,\n",
       " 0.18035756641366224,\n",
       " 0.23720565541732314,\n",
       " 0.18817397555715315,\n",
       " 0.11334600072385093,\n",
       " 0.1376215738284704,\n",
       " 0.16989272388059701,\n",
       " 0.11483693971378811,\n",
       " 0.15477964406260497,\n",
       " 0.18232146723162096,\n",
       " 0.13704425138322918,\n",
       " 0.19937206118683409,\n",
       " 0.028107790978305791,\n",
       " 0.097435187428610959,\n",
       " 0.044867289243895658,\n",
       " 0.1702797022974234,\n",
       " 0.077145740386658168,\n",
       " 0.15902643455834947,\n",
       " 0.12209785109539512,\n",
       " 0.28187340605087252,\n",
       " 0.11351581682519246,\n",
       " 0.12735630969207806,\n",
       " 0.16086110380583835,\n",
       " 0.16388174055445082,\n",
       " 0.24351797441210732,\n",
       " 0.16661684994758091,\n",
       " 0.073339780786589259,\n",
       " 0.14910538846205829,\n",
       " 0.21341441824233409,\n",
       " 0.080682926114080011,\n",
       " 0.21564922231446854,\n",
       " 0.11203999334258652,\n",
       " 0.12678428477860831,\n",
       " 0.13020228811676837,\n",
       " 0.25402819126313869,\n",
       " 0.19943095638159497,\n",
       " 0.15027151211361739,\n",
       " 0.10877293100783492,\n",
       " 0.1559030297698899,\n",
       " 0.1347024645264398,\n",
       " 0.15023536627722228,\n",
       " 0.21239594736571427,\n",
       " 0.16070393374741199,\n",
       " 0.041798619809514825,\n",
       " 0.14856685479253573,\n",
       " 0.15732087227414332,\n",
       " 0.14415891294540156,\n",
       " 0.16548714653531177,\n",
       " 0.13276538177310543,\n",
       " 0.25357269924098669,\n",
       " 0.18795177550241166,\n",
       " 0.037857353374594727,\n",
       " 0.26287892011403968,\n",
       " 0.15386511131191982,\n",
       " 0.15442221759675467,\n",
       " 0.16383607054896054,\n",
       " 0.23919371912491172,\n",
       " 0.16502390524909255,\n",
       " 0.2733629259515365,\n",
       " 0.18897489088651448,\n",
       " 0.24290702620943916,\n",
       " 0.13728063173958316,\n",
       " 0.13897783965657268,\n",
       " 0.26121302057260004,\n",
       " 0.14656175102213448,\n",
       " 0.11274683171234895,\n",
       " 0.11769900599687838,\n",
       " 0.14997212304695828,\n",
       " 0.17098634693853132,\n",
       " 0.18181258384464827,\n",
       " 0.043166171091288119,\n",
       " 0.13513399094026501,\n",
       " 0.20535355564483732,\n",
       " 0.13896565278983003,\n",
       " 0.16357178089772745,\n",
       " 0.11848497691329657,\n",
       " 0.15504116824607739,\n",
       " 0.25984400675128072,\n",
       " 0.20607047139874052,\n",
       " 0.17200551455680807,\n",
       " 0.20690218877435768,\n",
       " 0.15875423292769025,\n",
       " 0.15527152641878667,\n",
       " 0.17080021593058625,\n",
       " 0.095316133253876445,\n",
       " 0.13167525447421058,\n",
       " 0.14949130388705062,\n",
       " 0.097543277551265184,\n",
       " 0.16604043977697663,\n",
       " 0.15278181641950178,\n",
       " 0.15164894575599208,\n",
       " 0.077499467016919787,\n",
       " 0.080206318504190854,\n",
       " 0.19236744302768849,\n",
       " 0.26203171998906205,\n",
       " 0.2071150097465887,\n",
       " 0.16560356774812551,\n",
       " 0.20757043966323668,\n",
       " 0.17831496622822612,\n",
       " 0.072851358411703204,\n",
       " 0.20631774738942588,\n",
       " 0.15049251331765606,\n",
       " 0.032876199308057186,\n",
       " 0.18413662486938348,\n",
       " 0.18079834304483289,\n",
       " 0.23374799119100054,\n",
       " 0.1475462077756533,\n",
       " 0.21043643263757111,\n",
       " 0.13914656771799627,\n",
       " 0.24545939123526345,\n",
       " 0.17771526418786693,\n",
       " 0.1576579944726676,\n",
       " 0.2717291696038347,\n",
       " 0.095950053396861912,\n",
       " 0.14527684148212583,\n",
       " 0.05212035275394572,\n",
       " 0.083225418863894285,\n",
       " 0.087796096859352024,\n",
       " 0.095341214765252424,\n",
       " 0.21634399904625795,\n",
       " 0.093048997326990346,\n",
       " 0.16621232024738147,\n",
       " 0.17617426518033952,\n",
       " 0.20044752808313809,\n",
       " 0.27437893956247683,\n",
       " 0.24112289912136792,\n",
       " 0.10995367788658145,\n",
       " 0.076808442625534101,\n",
       " 0.21009765387576484,\n",
       " 0.16402168234064785,\n",
       " 0.236761536921682,\n",
       " 0.21859684812618069,\n",
       " 0.1834383778297099,\n",
       " 0.05478676488116524,\n",
       " 0.050111583615018329,\n",
       " 0.10346008289781944,\n",
       " 0.15485498628169048,\n",
       " 0.20250517598343687,\n",
       " 0.14015832390929112,\n",
       " 0.10745243246860431,\n",
       " 0.10227609840928847,\n",
       " 0.11442686087248644,\n",
       " 0.21408272908291692,\n",
       " 0.14134843453510437,\n",
       " 0.15223520724549222,\n",
       " 0.20145108003914886,\n",
       " 0.11229795232269241,\n",
       " 0.23307050092764378,\n",
       " 0.073662079510703304,\n",
       " 0.13423224973855774,\n",
       " 0.22499152849347404,\n",
       " 0.15392961081421858,\n",
       " 0.12045867035952956,\n",
       " 0.13314364075740048,\n",
       " 0.25866183179319219,\n",
       " 0.033942014563142442,\n",
       " 0.16852029473652902,\n",
       " 0.13560144561002252,\n",
       " 0.22724484597710709,\n",
       " 0.1127424275850063,\n",
       " 0.13088392343711491,\n",
       " 0.1418462323090654,\n",
       " 0.09927339670135632,\n",
       " 0.23383492521174992,\n",
       " 0.10869574940402804,\n",
       " 0.15826237196100207,\n",
       " 0.20314654490381789,\n",
       " 0.0010402919951016942,\n",
       " 0.19592281168713321,\n",
       " 0.10473223666326253,\n",
       " 0.14897115987734544,\n",
       " 0.17061173665698842,\n",
       " 0.089559285225314589,\n",
       " 0.19437960960649142,\n",
       " 0.27364389233954445,\n",
       " 0.17951127819548873,\n",
       " 0.15690062999133469,\n",
       " 0.17789427461171065,\n",
       " 0.13851580640743766,\n",
       " 0.17401288644138357,\n",
       " 0.21199775491113193,\n",
       " 0.11309447915338516,\n",
       " 0.15380947172700427,\n",
       " 0.10755159072337617,\n",
       " 0.14536125330728028,\n",
       " 0.17365853089143346,\n",
       " 0.10015177667995956,\n",
       " 0.14598741752340033,\n",
       " 0.23452822164165499,\n",
       " 0.19149068322981366,\n",
       " 0.11171591381486434,\n",
       " 0.089501795648722182,\n",
       " 0.097170686456400757,\n",
       " 0.19817867984048571,\n",
       " 0.15379091277503637,\n",
       " 0.12595859788856745,\n",
       " 0.1829522977824552,\n",
       " 0.15930313342675129,\n",
       " 0.14620199377775811,\n",
       " 0.13179535696091987,\n",
       " 0.18737490240350357,\n",
       " 0.10805023309763329,\n",
       " 0.17891368089823356,\n",
       " 0.20791166205600498,\n",
       " 0.23564321871181709,\n",
       " 0.12455664133188565,\n",
       " 0.075185229631407502,\n",
       " 0.21893423361108189,\n",
       " 0.095186781609195414,\n",
       " 0.17145533494414683,\n",
       " 0.22959836671789124,\n",
       " 0.19483514776353056,\n",
       " 0.036677569365640927,\n",
       " 0.14383278888609685,\n",
       " 0.080823286563002472,\n",
       " 0.15663731245923024,\n",
       " 0.1606240563065473,\n",
       " 0.16608503421323112,\n",
       " 0.26930553257864015,\n",
       " 0.18325992500446409,\n",
       " 0.1715725663507669,\n",
       " 0.17447700884865219,\n",
       " 0.19218273589549556,\n",
       " 0.072074993281218969,\n",
       " 0.10351812239337488,\n",
       " 0.1483553241513324,\n",
       " 0.19625851553304569,\n",
       " 0.1128853187042842,\n",
       " 0.1848223615464995,\n",
       " 0.17051698900993428,\n",
       " 0.23576352282864324,\n",
       " 0.23962354159236371,\n",
       " 0.15662145007815864,\n",
       " 0.10256252877090682,\n",
       " 0.25803902814309632,\n",
       " 0.12686238050966653,\n",
       " 0.12761094755206631,\n",
       " 0.16909343652552883,\n",
       " 0.22258340234629304,\n",
       " 0.072029014529515156,\n",
       " 0.2494197292069632,\n",
       " 0.20637447255045405,\n",
       " 0.10730897401175182,\n",
       " 0.046252635755166804,\n",
       " 0.17153667474289214,\n",
       " 0.13651165179947383,\n",
       " 0.18276515151515155,\n",
       " 0.12679089026915114,\n",
       " 0.076976005645730372,\n",
       " 0.18669531411690959,\n",
       " 0.16965230754241528,\n",
       " 0.23388472335737892,\n",
       " 0.13958630307470288,\n",
       " 0.1858151309428509,\n",
       " 0.17838949843260188,\n",
       " 0.056265343509185717,\n",
       " 0.06114457831325304,\n",
       " 0.040363438989389766,\n",
       " 0.1350138330131806,\n",
       " 0.18826868495742671,\n",
       " 0.13007118089918113,\n",
       " 0.26415855632163526,\n",
       " 0.095953893201599638,\n",
       " 0.1471631205673759,\n",
       " 0.21632845251837901,\n",
       " 0.1275284406366744,\n",
       " 0.16819203513501918,\n",
       " 0.10341715022566089,\n",
       " 0.17815965337445844,\n",
       " 0.1350231756196959,\n",
       " 0.16073924267087591,\n",
       " 0.13569565568772085,\n",
       " 0.083199241936637358,\n",
       " 0.21902242927452381,\n",
       " 0.15667064119943241,\n",
       " 0.13610373774099765,\n",
       " 0.071011454224559678,\n",
       " 0.20837098496672968,\n",
       " 0.10268734003928337,\n",
       " 0.11499820273184763,\n",
       " 0.11382909903552108,\n",
       " 0.1696625178102516,\n",
       " 0.035725027369738094,\n",
       " 0.11597262837758798,\n",
       " 0.11806792607742667,\n",
       " 0.064568638090070574,\n",
       " 0.11752624508090503,\n",
       " 0.068856875584658583,\n",
       " 0.21701954873161095,\n",
       " 0.19573283858998142,\n",
       " 0.086571104103995267,\n",
       " 0.26328320802005012,\n",
       " 0.15267433586337759,\n",
       " 0.10111222838083089,\n",
       " 0.12395162473165594,\n",
       " 0.20108411703239293,\n",
       " 0.21714985067599485,\n",
       " 0.19079399905123345,\n",
       " 0.14512045492950498,\n",
       " 0.063153978929825605,\n",
       " -0.010330262242558064,\n",
       " 0.21814867090096446,\n",
       " 0.17314741999879013,\n",
       " 0.2197504069451981,\n",
       " 0.19293074144890587,\n",
       " 0.13261992391674138,\n",
       " 0.17218648946377169,\n",
       " 0.096102388526842497,\n",
       " 0.28669367725440464,\n",
       " 0.2182439376264802,\n",
       " 0.17888266897095839,\n",
       " 0.11463004773245083,\n",
       " 0.13941458816111935,\n",
       " 0.19862633094728793,\n",
       " 0.1586241780475468,\n",
       " 0.19217488991141246,\n",
       " 0.21851096266072872,\n",
       " 0.15570761578079845,\n",
       " 0.25185717709082195,\n",
       " 0.17334614308749247,\n",
       " 0.07672312397083042,\n",
       " 0.21190062530697218,\n",
       " 0.13478260869565217,\n",
       " 0.27317814949116853,\n",
       " 0.22167136284195882,\n",
       " 0.12965828497743387,\n",
       " 0.066929414702000922,\n",
       " 0.23349136242689461,\n",
       " 0.096727655257917877,\n",
       " 0.10161445621263354,\n",
       " 0.25662621416585923,\n",
       " 0.10343626314433596,\n",
       " 0.123805096664998,\n",
       " 0.15902862488306824,\n",
       " 0.16919439008480103,\n",
       " 0.089170287482887911,\n",
       " 0.16386903430166155,\n",
       " 0.093407007773523565,\n",
       " 0.14308905380333953,\n",
       " 0.21799628942486088,\n",
       " 0.143940466916885,\n",
       " 0.14540172701751897,\n",
       " 0.22745426484145298,\n",
       " 0.1177050270367781,\n",
       " 0.093650507598966759,\n",
       " 0.23465256008359459,\n",
       " 0.16326530612244899,\n",
       " 0.185225793289095,\n",
       " 0.16383393412575398,\n",
       " 0.18561186891920123,\n",
       " 0.07751190083353926,\n",
       " 0.192436179619698,\n",
       " 0.15128768028704298,\n",
       " 0.20644987001944171,\n",
       " 0.19791512520758314,\n",
       " 0.14597008062063796,\n",
       " 0.068116151929696778,\n",
       " 0.16672594633614124,\n",
       " 0.18740767798955948,\n",
       " 0.20156697697090747,\n",
       " 0.18281546489563572,\n",
       " 0.19201988754209054,\n",
       " 0.13815498199136061,\n",
       " 0.13815249092315934,\n",
       " 0.18737998420755098,\n",
       " 0.20435261341550831,\n",
       " 0.091764092723546908,\n",
       " 0.18477491588351314,\n",
       " 0.14872403525787231,\n",
       " 0.21292696555741814,\n",
       " 0.15588852611940296,\n",
       " 0.21190476190476187,\n",
       " 0.043182259382491428,\n",
       " 0.12508631875261156,\n",
       " 0.25420289855072464,\n",
       " 0.17930544461165709,\n",
       " 0.17613082965688992,\n",
       " 0.17298529532572088,\n",
       " 0.17056941604592929,\n",
       " 0.19157257115972709,\n",
       " 0.12118050227609839,\n",
       " 0.16101528130655396,\n",
       " 0.18798851222809171,\n",
       " 0.15181550250515768,\n",
       " 0.10179384731745017,\n",
       " 0.1703976012486651,\n",
       " 0.22886583319547177,\n",
       " 0.11420758677618603,\n",
       " 0.10187668359198077,\n",
       " 0.11307866341802991,\n",
       " 0.22551204593298052,\n",
       " 0.10351556352878233,\n",
       " 0.12839610541044777,\n",
       " 0.2137740604677153,\n",
       " 0.15868988582631766,\n",
       " 0.1715746745617745,\n",
       " 0.11076604554865427,\n",
       " 0.18110826736503782,\n",
       " 0.077210239834931804,\n",
       " 0.15875739653792081,\n",
       " 0.12592885091718414,\n",
       " 0.16006558882752608,\n",
       " 0.17545248619999559,\n",
       " 0.22329495339579009,\n",
       " 0.16227248152820328,\n",
       " 0.16426972738554907,\n",
       " 0.14666417212347987,\n",
       " 0.23250688705234157,\n",
       " 0.1327823691460055,\n",
       " 0.12378167498108134,\n",
       " 0.22508214108240812,\n",
       " 0.12818744473916888,\n",
       " 0.20425453348877998,\n",
       " 0.14441274110691094,\n",
       " 0.1218244201338633,\n",
       " 0.1124448725404672,\n",
       " 0.12497092476291644,\n",
       " 0.18973853721161094,\n",
       " 0.091656154262029166,\n",
       " 0.10521737190783159,\n",
       " 0.17067924304262644,\n",
       " 0.18229500649468089,\n",
       " 0.22630385084633337,\n",
       " 0.063006248809803211,\n",
       " 0.17621311621999697,\n",
       " 0.13704115684093437,\n",
       " 0.16770657660474175,\n",
       " 0.13228197750068771,\n",
       " 0.053432743087915513,\n",
       " 0.21278857657273498,\n",
       " 0.18054634120160479,\n",
       " 0.12976652552340628,\n",
       " 0.15150320208728651,\n",
       " 0.12694283953918228,\n",
       " 0.19361518167911856,\n",
       " 0.093466279415464093,\n",
       " 0.20216007750036791,\n",
       " 0.17291788513276402,\n",
       " 0.059032162360809437,\n",
       " 0.24438755559721331,\n",
       " 0.172930013786392,\n",
       " 0.1642171550454522,\n",
       " 0.21071955098222639,\n",
       " 0.23210724721732734,\n",
       " 0.15135956625318331,\n",
       " 0.13415397268111279,\n",
       " 0.25447712149839813,\n",
       " 0.16255404994474482,\n",
       " 0.26485261434085866,\n",
       " 0.16233801198980508,\n",
       " 0.15310559006211177,\n",
       " 0.28880058389425028,\n",
       " 0.13146997929606624,\n",
       " 0.17376568559160799,\n",
       " 0.18927932075085674,\n",
       " 0.19802977370723568,\n",
       " 0.21024161235711636,\n",
       " 0.17073156554794755,\n",
       " 0.12975416276894292,\n",
       " 0.13161949558024488,\n",
       " 0.19361515201348795,\n",
       " 0.20004485045912712,\n",
       " 0.1290034043121287,\n",
       " 0.17819953743232025,\n",
       " 0.15728779531793358,\n",
       " 0.23162978062512135,\n",
       " 0.15850653084695637,\n",
       " 0.17937855004963543,\n",
       " 0.125,\n",
       " 0.22845800988002818,\n",
       " 0.28134602063509423]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for i in np.arange(repetitions):\n",
    "\n",
    "    # Call .sample() to draw a random sample with replacement and call it \"resample\"\n",
    "    \n",
    "    resample = cat_ownership_data.sample(with_replacement=True)\n",
    "    \n",
    "    # Compute the difference of the means of the cat ownership between males and females\n",
    "    # First, generate the mean for males\n",
    "    male_ownership = resample.where('sex', 1)\n",
    "    male_mean = sum(male_ownership['numcats']) / len(male_ownership.rows)\n",
    "    \n",
    "    # Then for females\n",
    "    female_ownership = resample.where('sex', 2)\n",
    "    female_mean = sum(female_ownership['numcats']) / len(female_ownership.rows)\n",
    "    \n",
    "    # Then compute the difference between the means\n",
    "    diff_ownership = female_mean - male_mean\n",
    "    \n",
    "    #Finally, append to the list \"diffs\" the test statistic generated during this run\n",
    "    diffs.append(diff_ownership)\n",
    "\n",
    "diffs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot a histogram of these differences in the average number of cats by sex."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([   2.,   21.,   55.,  136.,  176.,  249.,  189.,  115.,   42.,   15.]),\n",
       " array([-0.01033026,  0.02094961,  0.05222948,  0.08350936,  0.11478923,\n",
       "         0.14606911,  0.17734898,  0.20862885,  0.23990873,  0.2711886 ,\n",
       "         0.30246847]),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbQAAAEWCAYAAAAO4GKjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHblJREFUeJzt3X9QVXX+x/HXlauh9GXA+KUY1QAKOSy4/likJsTWbHYR\nddwWadapdKzVplwbFSwdtqEGMFPXRonZ1GrNNvPHtsZaTRu2uqLkrO62lXjN1AmKe/2BEgsRcL9/\ntN3tJsoFLpfLh+djhhk553PO533ecnnNPffeD5a6ujqnAADo4wb0dgEAAHgDgQYAMAKBBgAwAoEG\nADACgQYAMAKBBgAwAoEGADBCh4G2Zs0aTZ48WTExMYqLi9Ps2bP1ySefuI1ZuHChQkND3b7uuusu\ntzHNzc1aunSpYmNjFR0drZycHNXU1Hj3agAA/VaHgXbw4EHNnz9f77zzjvbs2SOr1aoZM2aorq7O\nbVxGRoZsNptOnDihEydOaPv27W778/LyVFZWps2bN2vv3r2qr69Xdna2nE4+1w0A6D5LZ1cKaWho\nUExMjLZt26apU6dK+vYZ2oULF/THP/6x3WMuX76suLg4lZSUaNasWZKk6upqJSUlaefOncrIyOjm\nZQAA+rtOv4ZWX1+vtrY2hYSEuG0/dOiQ4uPjNW7cOC1atEjnzp1z7Tt27JhaWlrcgis6OlqjRo3S\n4cOHu1E+AADfsnb2gLy8PCUnJ2vChAmubVOmTFFWVpZuuukmnT17VgUFBcrKytL777+vgQMHym63\nKyAgQEOHDnU7V3h4uOx2e/evAgDQ73Uq0B5//HFVVlbqrbfeksVicW2fOXOm69+JiYlKTk5WUlKS\n3n77bWVmZnqvWgAArsLjW47Lly/X7t27tWfPHsXExFxzbFRUlIYPH65Tp05JkiIiItTa2qoLFy64\njXM4HIqIiOhC2QAAuPMo0HJzc11hFhsb2+H4c+fO6YsvvlBkZKQkKSUlRVarVeXl5a4x1dXVqqqq\nUmpqahdLBwDgfzq85bhkyRJt375dr7zyioKDg12veQUFBSkoKEgNDQ0qKipSVlaWIiMjdebMGRUU\nFCgiIsJ1uzE4OFhz5sxRfn6+wsLCFBISohUrVigpKUnp6ek9e4W9wGazKT4+vrfL6BJq942j55q1\n+GBdxwO9aG1aiMaEDfL6eftS33+I2s3SYaBt2rRJFotF06dPd9uem5ur3NxcBQQE6OOPP9Zrr72m\nS5cuKTIyUnfccYdefPFFBQUFucYXFRXJarVq7ty5ampqUnp6ukpLS91eiwMAoKs6DLSLFy9ec39g\nYKB27tzZ4UQDBw5UcXGxiouLPa8OAAAPsZYjAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDACAQa\nAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDA\nCAQaAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgE\nGgDACAQaAMAIBBoAwAgEGgDACAQaAMAIBBoAwAgEGgDACAQaAMAIHQbamjVrNHnyZMXExCguLk6z\nZ8/WJ598csW4wsJCJSYmatiwYcrMzNTx48fd9jc3N2vp0qWKjY1VdHS0cnJyVFNT470rAQD0ax0G\n2sGDBzV//ny988472rNnj6xWq2bMmKG6ujrXmHXr1qmkpETPPPOMysvLFR4erpkzZ6qhocE1Ji8v\nT2VlZdq8ebP27t2r+vp6ZWdny+l09syVAQD6FWtHA3bs2OH2fWlpqWJiYnT48GFNnTpVkvT8889r\n8eLFyszMlCSVlJQoPj5eO3bs0H333afLly9r69atKikpUXp6uus8SUlJ2rdvnzIyMrx9XQCAfqbT\nr6HV19erra1NISEhkqTTp0+rtrbWLZQCAwOVlpamw4cPS5KOHj2qlpYWtzHR0dEaNWqUawwAAN3R\n6UDLy8tTcnKyJkyYIEmy2+2yWCwKDw93GxceHi673S5JcjgcCggI0NChQ686BgCA7ujwluP3Pf74\n46qsrNRbb70li8XSUzUBANBpHgfa8uXL9ac//UlvvvmmYmJiXNsjIiLkdDrlcDgUHR3t2u5wOBQR\nEeEa09raqgsXLrg9S3M4HEpLS7vmvDabzeOL8Sd9tW6J2n2hMTDC93M2NspmO9Mj5+4rfW8PtftW\nfHx8j53bo0DLzc3VG2+8oTfffFOxsbFu+26++WZFRkaqvLxcKSkpkqSmpiZVVFToqaeekiSlpKTI\narWqvLxcs2bNkiRVV1erqqpKqamp15y7Jy++p9hstj5Zt0TtvvLVuWZJX/t0zsGDByv+Ru/3py/1\n/Yeo3SwdBtqSJUu0fft2vfLKKwoODna95hUUFKSgoCBJ0oIFC7RmzRrFxcUpNjZWq1ev1vXXX+8K\nr+DgYM2ZM0f5+fkKCwtTSEiIVqxYoaSkJNe7HgEA6I4OA23Tpk2yWCyaPn262/bc3Fzl5uZKkhYt\nWqSmpiYtW7ZMdXV1Gjt2rHbt2uUKPEkqKiqS1WrV3Llz1dTUpPT0dJWWlvJaHADAKzoMtIsXL3p0\nou8HXHsGDhyo4uJiFRcXe14dAAAeYi1HAIARCDQAgBEINACAEQg0AIARCDQAgBE6tfQVgL4rwCId\nPdfs9fM2Bkb894Pi7YscPEDDg/hVg57HTxnQT1xqbtPKDy730NmvvurJ2rQQDQ+66m7Aa7jlCAAw\nAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACazmi36tp\naFFtY5tP52xudfp0PqA/INDQ79U2tmnxwTqfzlkwPtin8wH9AbccAQBGINAAAEYg0AAARiDQAABG\nINAAAEYg0AAARiDQAABGINAAAEYg0AAARiDQAABGINAAAEYg0AAARiDQAABGINAAAEYg0AAARiDQ\nAABGINAAAEbwKNAOHjyonJwc3XrrrQoNDdWrr77qtn/hwoUKDQ11+7rrrrvcxjQ3N2vp0qWKjY1V\ndHS0cnJyVFNT470rAQD0ax4FWkNDg0aPHq2ioiINGTKk3TEZGRmy2Ww6ceKETpw4oe3bt7vtz8vL\nU1lZmTZv3qy9e/eqvr5e2dnZcjqd3b8KAEC/Z/Vk0JQpUzRlyhRJ3z4ba8+gQYMUFhbW7r7Lly9r\n69atKikpUXp6uiSptLRUSUlJ2rdvnzIyMrpSOwAALl57De3QoUOKj4/XuHHjtGjRIp07d86179ix\nY2ppaXELrujoaI0aNUqHDx/2VgkAgH7Mo2doHZkyZYqysrJ000036ezZsyooKFBWVpbef/99DRw4\nUHa7XQEBARo6dKjbceHh4bLb7d4oAQDQz3kl0GbOnOn6d2JiopKTk5WUlKS3335bmZmZ3Tq3zWbr\nbnm9oq/WLfW/2hsDI3qgkmtra2vrF3NKUmNjo2y2M70ytyf62897b4uPj++xc3sl0H4oKipKw4cP\n16lTpyRJERERam1t1YULF9yepTkcDqWlpV3zXD158T3FZrP1ybql/ln7V+eaJX3t/YKuYcAA339i\npjfmlKTBgwcr/kb//Jnqjz/vJuuRn/Bz587piy++UGRkpCQpJSVFVqtV5eXlrjHV1dWqqqpSampq\nT5QAAOhnPHqG1tDQoFOnTsnpdKqtrU2ff/65PvzwQ9dnzoqKipSVlaXIyEidOXNGBQUFioiIcN1u\nDA4O1pw5c5Sfn6+wsDCFhIRoxYoVSkpKcr3rEQCA7vAo0I4ePapp06bJYrFIkgoLC1VYWKicnBw9\n++yz+vjjj/Xaa6/p0qVLioyM1B133KEXX3xRQUFBrnMUFRXJarVq7ty5ampqUnp6ukpLS13nBACg\nOzwKtNtvv10XL1686v6dO3d2eI6BAwequLhYxcXFnlcHAICHWMsRAGAEAg0AYAQCDQBgBAINAGAE\nAg0AYAQCDQBgBAINAGAEAg0AYIQeWZwY6IqahhbVNnZ9RfjGwIj/LjTcOc2t/NV0wAQEGvxGbWOb\nFh+s6+ZZOr9qfsH44G7OCcAfcMsRAGAEAg0AYAQCDQBgBAINAGAEAg0AYAQCDQBgBAINAGAEAg0A\nYAQCDQBgBAINAGAEAg0AYAQCDQBgBAINAGAEAg0AYAQCDQBgBAINAGAEAg0AYAQCDQBgBGtvFwDA\nbAEW6ei5Zp/OGTl4gIYH8eutv+F/HECPutTcppUfXPbpnGvTQjQ8yKdTwg9wyxEAYAQCDQBgBAIN\nAGAEAg0AYAQCDQBgBAINAGAEAg0AYASPAu3gwYPKycnRrbfeqtDQUL366qtXjCksLFRiYqKGDRum\nzMxMHT9+3G1/c3Ozli5dqtjYWEVHRysnJ0c1NTXeuQoAQL/nUaA1NDRo9OjRKioq0pAhQ67Yv27d\nOpWUlOiZZ55ReXm5wsPDNXPmTDU0NLjG5OXlqaysTJs3b9bevXtVX1+v7OxsOZ1O710NAKDf8ijQ\npkyZohUrVigrK0sWi+WK/c8//7wWL16szMxMJSQkqKSkRF999ZV27NghSbp8+bK2bt2qgoICpaen\n60c/+pFKS0v10Ucfad++fV69IABA/9Tt19BOnz6t2tpaZWRkuLYFBgYqLS1Nhw8fliQdPXpULS0t\nbmOio6M1atQo1xgAALqj24Fmt9tlsVgUHh7utj08PFx2u12S5HA4FBAQoKFDh151DAAA3eH3ixPb\nbLbeLqFL+mrdUu/V3hgY0SvztrW1Madh8zY2NspmO+PRWB6rvhUfH99j5+52oEVERMjpdMrhcCg6\nOtq13eFwKCIiwjWmtbVVFy5ccHuW5nA4lJaWds3z9+TF9xSbzdYn65Z6t/avzjVL+trn8w4Y4PtP\nr/SXOXtr3sGDByv+xo5/jnmsmqXbP2k333yzIiMjVV5e7trW1NSkiooKpaamSpJSUlJktVrdxlRX\nV6uqqso1BgCA7vDoGVpDQ4NOnTolp9OptrY2ff755/rwww8VGhqqESNGaMGCBVqzZo3i4uIUGxur\n1atX6/rrr9esWbMkScHBwZozZ47y8/MVFhamkJAQrVixQklJSUpPT+/RCwQA9A8eBdrRo0c1bdo0\n11v2CwsLVVhYqJycHG3YsEGLFi1SU1OTli1bprq6Oo0dO1a7du1SUND//sJeUVGRrFar5s6dq6am\nJqWnp6u0tLTdjwEAANBZHgXa7bffrosXL15zTG5urnJzc6+6f+DAgSouLlZxcXHnKgQAwAOs5QgA\nMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADAC\ngQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEG\nADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAwAoEGADACgQYAMAKBBgAw\nAoEGADCCVwKtqKhIoaGhbl8JCQluYwoLC5WYmKhhw4YpMzNTx48f98bUAABIkqzeOtHIkSNVVlYm\np9MpSQoICHDtW7dunUpKSrRx40bFxcWpuLhYM2fO1JEjRxQUFOStEuBFNQ0tqm1s8+mcza1On84H\nwCxeC7SAgACFhYW1u+/555/X4sWLlZmZKUkqKSlRfHy8duzYofvuu89bJcCLahvbtPhgnU/nLBgf\n7NP5AJjFa6+hnTlzRomJiUpOTta8efN0+vRpSdLp06dVW1urjIwM19jAwEClpaXp8OHD3poeANDP\neSXQxo8fr40bN2rnzp1av369amtrdffdd6uurk52u10Wi0Xh4eFux4SHh8tut3tjegAAvHPL8c47\n73T7fvz48UpOTta2bds0btw4b0wBAMA1ee01tO8bMmSIEhISdOrUKf3sZz+T0+mUw+FQdHS0a4zD\n4VBERESH57LZbD1RYo/rq3VL39beGNjx/423tbX59k0ovTlvf5mzt+ZtbGyUzXbGo7F9/bHa18TH\nx/fYuXsk0JqammSz2ZSenq6bb75ZkZGRKi8vV0pKimt/RUWFnnrqqQ7P1ZMX31NsNlufrFv6X+1f\nnWuW9LVP5x4woHc+Ftkb8/aXOXtr3sGDByv+xo4fgyY8VvE/Xgm0lStX6u6779aIESPkcDj0zDPP\n6D//+Y9mz54tSVqwYIHWrFmjuLg4xcbGavXq1br++us1a9Ysb0wPAIB3Aq2mpkbz58/X+fPnFRYW\npnHjxundd9/ViBEjJEmLFi1SU1OTli1bprq6Oo0dO1a7du3iM2gAAK/xSqBt2rSpwzG5ubnKzc31\nxnQAAFyBtRwBAEYg0AAARiDQAABG6JG37QNAbwqwSEfPNXc4rjEw4r8fUfGOyMEDNDyIX6u9hc4D\nMM6l5jat/OCyh6O993nLtWkhGs6bt3sNtxwBAEYg0AAARiDQAABGINAAAEYg0AAARiDQAABGINAA\nAEYg0AAARiDQAABGINAAAEYg0AAARiDQAABGINAAAEYg0AAARiDQAABGINAAAEYg0AAARiDQAABG\nsPZ2Abi2moYW1Ta2+Wy+xsAIfXWuWc2tTp/NCQDeQKD5udrGNi0+WOfjWb9WwfhgH88JAN3DLUcA\ngBEINACAEQg0AIAReA0NALwkwCIdPdfsk7m+ewNX5OABGh7Er3KJQAMAr7nU3KaVH1z24Yxfa21a\niIYH+XBKP8YtRwCAEQg0AIARCDQAgBEINACAEQg0AIARCDQAgBEINACAEXweaC+88IKSk5MVFRWl\nSZMmqaKiwtclAAAM5NNA27Vrl5YvX64lS5Zo//79mjBhgu655x5VV1f7sgwAgIF8GmgbN27Ur371\nK82ZM0fx8fFatWqVIiMjtXnzZl+WAQAwkM+Wvvrmm2907NgxPfLII27bJ0+erMOHD/uqjG6pb26T\nJ3/2csCQYF1u9s4f5Wxz8oc2AcATPgu08+fPq7W1VREREW7bw8PD9f777/uqjG55/dR/9O7nX3sw\n0ip9ctErcy4czSJtAOAJS11dnU+eAnz55ZdKTEzUX/7yF02cONG1fdWqVdqxY4cqKyt9UQYAwFA+\new3thhtuUEBAgOx2u9t2h8NxxbM2AAA6y2eBNnDgQKWkpGjfvn1u28vLy5WamuqrMgAAhvLp30N7\n+OGH9etf/1pjxoxRamqqNm3apNraWt1///2+LAMAYCCfBtrMmTN18eJFPfvss6qtrVViYqJef/11\njRgxwpdlAAAM5LM3hQAA0JN6ZS3H5uZmLV26VLGxsYqOjlZOTo5qamo6PO6NN95QamqqIiMjNXHi\nRL355ptu+4uKihQaGur2lZCQ0K1aO7tU18cff6yf//znGjZsmEaPHq1Vq1ZdMebAgQOaNGmSoqKi\nNGbMGG3ZsqVbNfqq9gMHDlzR36FDh+rkyZO9WvvXX3+thQsX6rbbblN4eLimTZvW7jh/7Lsntftr\n3w8cOKB7771XCQkJGj58uG677TZt3bq13XH+1ndPavfXvldVVWnatGkaOXKkoqKilJKSooKCAn3z\nzTdX1O9vffek9u70vVcCLS8vT2VlZdq8ebP27t2r+vp6ZWdny3mNDxFXVlZq3rx5ys7O1oEDB/SL\nX/xC999/v/7xj3+4jRs5cqRsNptOnDihEydO6ODBg12us7NLddXX12vmzJmKiorSvn37VFhYqOee\ne04bNmxwjTlz5oyys7OVmpqq/fv3a/HixVq2bJn27NnT5Tp9VbskWSwWVVZWuvpbVVWl2NjYXq29\ntbVVgwcP1kMPPaSpU6e2O8Zf++5J7ZJ/9r2yslKjR4/Wyy+/rIqKCs2bN0+/+c1vtHPnTtcYf+27\nJ7VL/tn3QYMG6d5779Xu3bt15MgRFRUV6Q9/+IOefvpp1xh/7bsntUtd77vPbzlevnxZcXFxKikp\n0axZsyRJ1dXVSkpK0s6dO5WRkdHucXPnzlVdXZ127drl2jZjxgyFh4fr97//vaRvn6H9+c9/7laI\nfd9Pf/pTJSUlae3ata5tY8eO1YwZM7Ry5corxm/atElPPvmkTp48qUGDBkmSVq9erS1btuijjz6S\nJOXn56usrExHjhxxHffoo4+qqqpKb7/9tlfq7qnaDxw4oKysLH366acKDQ31Wq3drf37li5dquPH\nj1/xwPXXvntSe1/o+3ceeOABtbW16aWXXpLUN/p+tdr7Ut+feOIJHTlyxNXTvtT3H9benb77/Bna\nsWPH1NLS4hZc0dHRGjVq1DWXwKqsrLwi7O68884rjjlz5owSExOVnJysefPm6fTp012q87uluiZN\nmuS2/VpLdX3wwQeaOHGiKxC+q/GLL77Q2bNnXWPau46jR4+qtbW1S7X6qnZJcjqdmjRpkhISEjR9\n+nTt37/fKzV3p3ZP+GvfPdVX+l5fX6+QkBDX932p7z+sXeobfT916pT++te/6vbbb3dt6yt9b692\nqet993mg2e12BQQEaOjQoW7bw8PDr/jQ9Q+Pa2/ZrO8fM378eG3cuFE7d+7U+vXrVVtbq6lTp6qu\nrq7TdV5rqa6r1Xm1Gp1Op+uYq41paWnR+fPnO12nL2uPiorS2rVr9fLLL2vr1q2Kj4/X9OnTdejQ\nIa/U3dXaPeGvffdEX+n7W2+9pb/97W964IEHXNv6St/bq93f+z516lRFRUVp3LhxmjhxotszIn/v\n+7Vq707fvfa2/aeeekrPPvvsVfdbLBav37/9oTvvvNPt+/Hjxys5OVnbtm3TwoULe3Tu/iAuLk5x\ncXGu78eNG6ezZ89q/fr1fDi+B/WFvh86dEgPPvigVq1apZSUlN4up1OuVru/933Lli366quv9O9/\n/1srV67U2rVrtXjx4t4uyyPXqr07ffdaoD388MOaPXv2NceMGDFClZWVam1t1YULF9yepTkcDqWl\npV312IiIiE4vmzVkyBAlJCTo1KlTHl7F/3Rlqa6r1WixWFzHXG2M1WrVDTfc0Ok6fVl7e8aOHavd\nu3d3v+j/6qkl0vy1713lT32vqKhQdna2nnjiiSsWSfD3vl+r9vb4U9+HDx8u6ds3wrW0tOjRRx/V\nokWLNGDAAL/v+7Vqb4+nfffaLcfQ0FBXsl7tKzAwUCkpKbJarSovL3cdW11draqqqmum74QJE9pd\nNusnP/nJVY9pamqSzWZTZGRkp6+nK0t1TZgwQRUVFWpubnZte++99zRs2DDFxMRc9Tree+89jRkz\nRgEBAZ2u05e1t+df//pXl/p7NT21RJq/9r2r/KXvf//73/XLX/5Sy5cv10MPPXTFfn/ue0e1t8df\n+v5Dra2tri/Jv/v+Qz+svT2e9j0gLy/vtx7P7AXXXXedvvzyS73wwgsaPXq0Ll26pMcee0whISH6\n7W9/K4vFIknKysrSyZMnlZ6eLkkaNmyYCgsLNWjQIN1www166aWXtG3bNq1fv17Dhg2TJK1cuVLX\nXXednE6nTp48qaVLl+qzzz7TunXrFBwc3Ola/+///k+FhYWKjIzU4MGDtWrVKh06dEgbNmxQcHCw\nnnzySa1du1Y5OTmSpNjYWL344ov68MMPFR8fr4qKCuXn5+uxxx7T+PHjJUm33HKLfve738nhcCgm\nJkZlZWVau3atnn76aY0cOdIbLe6x2ktKSmS322W1WlVbW6sNGzZo69atvV679O3nW2pqarRv3z7Z\n7Xb9+Mc/lt1udz0I/LXvntTur33fv3+/srOzNW/ePN13331qaGhQQ0ODmpqaNGTIEEn+23dPavfX\nvr/22mv69NNPFRAQoPr6epWXlys/P1933XWXZsyYIcl/++5J7d3pu0+XvvpOUVGRrFar5s6dq6am\nJqWnp6u0tNQVZtK371b8/jODCRMmaNOmTXr66adVWFioW265RVu2bNGYMWNcY2pqajR//nydP39e\nYWFhGjdunN59990uL611taW6oqOjJUm1tbU6c+aMa3xwcLB2796tJUuWaPLkyQoJCdEjjzzi9vrd\nTTfdpO3bt+vxxx/Xli1bFBUVpVWrVikzM7NLNfqy9m+++Ub5+fmqqalRYGCgEhIS9Prrr1/x2qWv\na5eke+65R59//rnr+zvuuEMWi0UXLlyQ5L9996R2f+37q6++qsbGRj333HN67rnnXNtvvPFG/fOf\n/5Tkv333pHZ/7bvVatWaNWv02Wefyel06sYbb9SDDz6oBQsWuMb4a989qb07fWfpKwCAEXplpRAA\nALyNQAMAGIFAAwAYgUADABiBQAMAGIFAAwAYgUADABiBQAMAGIFAAwAY4f8B7Fj0TdASrkwAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f907207ab38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plots.hist(diffs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>The P-value</h3>\n",
    "\n",
    "The P-value is the probability of obtaining a result like the sample test statistic &mdash; which in this case is the difference in the mean number of cats between men and women &mdash; under the null hypothesis.\n",
    "\n",
    "Under the null hypothesis, there is no relationship between cat ownership and gender, so this difference should be zero: \n",
    "\n",
    "$$H_0: \\ \\ E[Y \\ | \\ F = 1] - E[Y \\ | \\ F = 0] = 0$$ \n",
    "\n",
    "So the P-value equals the proportion of our simulated, \"toss-em-in-the-air\" differences that are zero or \"worse,\" in this case, negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.001"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_value = np.count_nonzero(diffs < np.zeros(len(diffs)) ) / len(diffs)\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the P-value is very low, the probability of obtaining a result like our sample test statistic from randomness (the null hypothesis) is also very low. If it's low enough, commonly when it's under 0.05, we reject the null and conclude that the test statistic is *significantly different from zero*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Other common methods</h2>\n",
    "\n",
    "Another common way of assessing statistical significance is by constructing a *confidence interval* with the *standard error*.  \n",
    "\n",
    "Recall the definition of the mean of $X$, often denoted $\\bar{X}$ and also called the *expectation* of $X$, $E[X_i]$ (where we can write $i = 1, 2, ... N$ to index the $N$ observations we have of $X$):\n",
    "\n",
    "$$\\bar{X_i} \\equiv E[X_i] = \\frac{1}{N} \\sum_i X_i$$\n",
    "\n",
    "The <b>standard error of a mean</b>, or \"SEM\", is a function of the sample standard deviation, $STD$, of the underlying data, $X_i$, and the sample size, $N$.\n",
    "\n",
    "$$SEM = \\frac{STD[X_i]}{\\sqrt{N}} = \\frac{ \\sqrt{ \\frac{1}{N} \\sum_i (X_i - E[X_i])^2 } }{\\sqrt{N}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q8: Calculate the standard error of the mean cats owned by males.</font> \n",
    "\n",
    "You will want to calculate the standard deviation of `Males['numcats']` and then divide by the square root of its length.\n",
    "*(There is also a function in scipy.stats that does this directly.)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.035679975821773582"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SEM_males_numcats = Males['numcats'].std() / len(Males.rows)**0.5\n",
    "SEM_males_numcats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q9: Calculate the standard error of the mean cats owned by females.</font> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.038396713523280661"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SEM_females_numcats = Females['numcats'].std() / len(Females.rows)**0.5\n",
    "SEM_females_numcats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What now? A standard approach is to generate a 95% *confidence interval* around each mean.\n",
    "\n",
    "A **confidence interval** is usually stated as either (a) a beginning number and an ending number, or (b) the midpoint number plus-or-minus half of the interval often called the \"margin of error.\"\n",
    "\n",
    "For example, the results of public opinion polls are often stated with a margin of error.  <a href=\"http://www.wbur.org/2016/02/25/wbur-poll-trump-clinton-sanders\">A WBUR poll</a> of likely primary voters in Massachusetts prior to Super Tuesday 2016 provides a helpful example:\n",
    "\n",
    ">Among [the $N = 418$ surveyed] likely Democratic voters, former Secretary of State Hillary Clinton is ahead of Vermont U.S. Sen. Bernie Sanders by 5 percentage points — 49-44 — just outside the survey’s 4.9 percent margin of error.\n",
    "\n",
    "\n",
    "*(An aside: the eventual Super Tuesday result was <a href=\"http://www.nytimes.com/elections/results/massachusetts\">Clinton with 50.1%</a>.)*\n",
    "\n",
    "When the $Y$ variable that we're talking about is a dichotomous yes-or-no kind of question, then its mean is simply the share of \"yes\" answers, here $p = 0.49$, and the standard deviation equals $\\sqrt{p\\times(1-p)}$.\n",
    "\n",
    "So our formula showing the standard error of the mean equal to the standard deviation of $Y$ divided by $\\sqrt{N}$ implies that here, the standard error of the mean equals \n",
    "\n",
    "$$SEM_{y/n} = \\sqrt{\\frac{p\\times(1-p)}{N}}$$\n",
    "\n",
    "When $p = 0.49$ and $N = 418$, $SEM_{y/n} = $?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.024450907753131645"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SEM = (0.49*(1-0.49)/418)**0.5\n",
    "SEM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://en.wikipedia.org/wiki/1.96\">From statistical theory and the great Ronald Fisher,</a> we know that 95% of the draws from a normal distribution will fall within **1.96** standard deviations of the mean. Thus a 95% confidence interval for an opinion poll like this is defined as $\\pm 1.96 \\times SEM_{y/n}$, here equal to what?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.04792377919613802"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1.96*SEM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Wait a minute!!* Didn't the WBUR report state that the margin of error was 4.9%?  Try this math:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.04890181550626329"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2*SEM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the real world, folks often just use 2 instead of 1.96 as the critical value, and you may see 2 instead of 1.96 in DS 8 as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q10: Calculate the *lower* 95% confidence interval of the mean cats owned by females, and compare it to the *upper* 95% confidence interval of the mean cats owned by males.</font> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.40242525251051425"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Females['numcats'].mean() - 1.96*SEM_females_numcats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.38818548271988057"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Males['numcats'].mean() + 1.96*SEM_males_numcats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"blue\">Q11: Do these confidence interval overlap?</font>\n",
    "\n",
    "If they do overlap, then we can't reject the hypothesis that males and females do not differ in their cat ownership. \n",
    "\n",
    "If the confidence intervals do NOT overlap, then we can reject the hypothesis that males and females do not differ.  Here, we would conclude that females report owning more cats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Confidence Intervals and the Bootstrap</h3>\n",
    "\n",
    "You can also calculate confidence intervals with the bootstrap results too. In order to solve for the confidence interval, find two percentiles from the list of the simulated values you created. Which two percentiles do you need? It depends on the level of confidence you want. \n",
    "\n",
    "In general, you will need to find the $\\frac{c}{2}$ and $1 - \\frac{c}{2}$ percentiles, where $c$ is a number like 0.10, 0.05, or 0.01. For example, if you were trying to find the 95% confidence interval, where $x = 0.05$, you'll want to find the 2.5 and 97.5 percentiles. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.054356586898198375, 0.26018296330421647]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the confidence interval\n",
    "low_bound_of_ci = np.percentile(diffs, 2.5)\n",
    "high_bound_of_ci = np.percentile(diffs, 97.5)\n",
    "\n",
    "# This is the 95% confidence interval of the test statistic\n",
    "[low_bound_of_ci, high_bound_of_ci]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Ordinary Least Squares Regression</h3>\n",
    "\n",
    "Another very common and very flexible tool for hypothesis testing and creating confidence intervals is *ordinary least squares regression* or OLS. You will likely see this in DS 8 on Wednesday, March 9, 2016, but it's likely to be coded up from scratch, more or less.\n",
    "\n",
    "Rather than step you through all the nuts and bolts of OLS, let's instead try to keep things simpler albeit more mysterious. OLS is a technique for estimating equations like:\n",
    "\n",
    "$$Y = a + b X + \\epsilon$$\n",
    "\n",
    "Look familiar?\n",
    "\n",
    "Here, let's set $Y = $ number of cats (`numcats`), and $X = 1$ if female. We could also set $X = sex$, which equals 1 for men and 2 for women, and our estimate and standard error of $b$ would be the same! \n",
    "\n",
    "In Python, the `stats.linregress(X,Y)` function from the `scipy` package allows us to estimate this equation and produce standard errors and P-values using empirics and theory. The arguments X and Y need to be of type `numpy.ndarray`. Columns of the Table `HRSpets` fulfill this requirement!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from scipy import stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(HRSpets['sex'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(HRSpets['numcats'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.159430080907\n",
      "0.0565742907197\n",
      "0.00488756900363\n",
      "\n"
     ]
    }
   ],
   "source": [
    "slope, intercept, r_value, p_value, std_err = stats.linregress(HRSpets['sex']-[1],HRSpets['numcats'])\n",
    "print(slope)\n",
    "print(std_err)\n",
    "print(p_value)\n",
    "print(\"\")\n",
    "#print(intercept)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<h1>Try It For Yourself! </h1>\n",
    "\n",
    "Now it's your turn to try out writing a program to:\n",
    "1. Run your own hypothesis test\n",
    "2. Determine whether or not it is statistically significant\n",
    "3. Create your own confidence interval.\n",
    "\n",
    "Time permitting, let's split into workgroups and divvy up the following questions that we can answer with this dataset.  Go ahead and copy this code, copy the entire notebook if you like, and alter it to answer the question that you choose.\n",
    "\n",
    "<br>\n",
    "<font color=\"purple\">Q1:  Are men aged 50 and over (in the HRS) taller or shorter than women?\n",
    "</font>\n",
    "\n",
    "&nbsp;\n",
    "\n",
    "<font color=\"purple\">Q2:  (\"Crazy Dog Dudes\") Is the number of dogs owned by men larger than the number of dogs owned by women?\n",
    "</font>  \n",
    "\n",
    "&nbsp;\n",
    "\n",
    "<font color=\"purple\">Q3:  Is health better for high school graduates (`edyrs` $\\geq 12$)?\n",
    "</font>  \n",
    "\n",
    "&nbsp;\n",
    "\n",
    "<font color=\"purple\">Q4:  Is health better for college graduates (`edyrs` $\\geq 16$)?\n",
    "</font>  \n",
    "\n",
    "*(Or choose another)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}