statistics/1-Random-Variables-Mean-Median-Variance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To open this notebook in Google Colab and start coding, click on the Colab icon below.\n",
    "\n",
    "<table style=\"border:2px solid orange\" align=\"left\">\n",
    "  <td style=\"border:2px solid orange \">\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/neuefische/ds-welcome-package/blob/main/statistics/1-Random-Variables-Mean-Median-Variance.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Introduction to Random variables, Mean, Median, Mode, Variance and Standard deviation\n",
    "In this notebook you will find the statistics basics that we require for the bootcamp.\n",
    "If you still have problems with one or the other term, we strongly recommend you to work on it a bit more. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Random variables\n",
    "In probability and statistics, a random variable, random quantity, aleatory variable, or stochastic variable is described informally as a variable whose values depend on outcomes of a random phenomenon.The formal mathematical treatment of random variables is a topic in probability theory. In that context, a random variable is understood as a measurable function defined on a probability space that maps from the sample space to the real numbers.\n",
    "\n",
    "A random variable is a measurable function $X\\colon \\Omega \\to E$ from a set of possible outcomes $\\Omega$  to a measurable space $E$. The technical axiomatic definition requires $\\Omega$ to be a sample space of a probability triple $(\\Omega ,{\\mathcal {F}},\\operatorname P)$. A random variable is often denoted by capital roman letters such as $X, Y, Z, T$.\n",
    "\n",
    "The probability that $X$ takes on a value in a measurable set $S$ $\\subseteq$ $E$ is written as:\n",
    "$$P (X \\in S) = P(\\{ \\omega \\in \\Omega | X(\\omega) \\in S \\}) $$  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "x = []\n",
    "y = [] \n",
    "for i in range(50):\n",
    "    x.append(random.randrange(0, 101))\n",
    "    y.append(random.randrange(0, 101))\n",
    "\n",
    "plt.scatter(x,y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mean, Median, Mode\n",
    "\n",
    "### Mean\n",
    "There are several kinds of mean in mathematics, especially in statistics.\n",
    "\n",
    "For a data set, the arithmetic mean, also called the expected value or average, is the central value of a discrete set of numbers: specifically, the sum of the values divided by the number of values. The arithmetic mean of a set of numbers $x1, x2, ..., xn$ is typically denoted by $ \\bar x$. If the data set were based on a series of observations obtained by sampling from a statistical population, the arithmetic mean is the sample mean (denoted $\\bar x$) to distinguish it from the mean of the underlying distribution, the population mean (denoted $\\mu$ or $\\mu_{x}$.\n",
    "\n",
    "In probability and statistics, the population mean, or expected value, is a measure of the central tendency either of a probability distribution or of the random variable characterized by that distribution. In a discrete probability distribution of a random variable X, the mean is equal to the sum over every possible value weighted by the probability of that value; that is, it is computed by taking the product of each possible value x of X and its probability p(x), and then adding all these products together, giving $\\mu =\\sum xp(x)....$ An analogous formula applies to the case of a continuous probability distribution. Not every probability distribution has a defined mean (see the Cauchy distribution for an example). Moreover, the mean can be infinite for some distributions.\n",
    "\n",
    "For a finite population, the population mean of a property is equal to the arithmetic mean of the given property, while considering every member of the population. For example, the population mean height is equal to the sum of the heights of every individual—divided by the total number of individuals. The sample mean may differ from the population mean, especially for small samples. The law of large numbers states that the larger the size of the sample, the more likely it is that the sample mean will be close to the population mean.\n",
    "\n",
    "#### Arithmetic Mean (AM)\n",
    "Main article: Arithmetic mean\n",
    "The arithmetic mean (or simply mean) of a list of numbers, is the sum of all of the numbers divided by the amount of numbers. Similarly, the mean of a sample $x_{1},x_{2},\\ldots ,x_{n}$, usually denoted by $\\bar x$, is the sum of the sampled values divided by the number of items in the sample:\n",
    "\n",
    "$$\\bar x = \\frac {1}{n} (\\sum_{i=1}^{n} x_{i})= \\frac {x_{1}+x_{2}+\\cdots +x_{n}}{n}$$\n",
    "\n",
    "For example, the arithmetic mean of five values: 4, 36, 45, 50, 75 is:\n",
    "\n",
    "$$\\frac {4+36+45+50+75}{5}=\\frac {210}{5}=42$$\n",
    "\n",
    "### Median\n",
    "\n",
    "In statistics and probability theory, a median is a value separating the higher half from the lower half of a data sample, a population, or a probability distribution. For a data set, it may be thought of as \"the middle\" value. The basic advantage of the median in describing data compared to the mean (often simply described as the \"average\") is that it is not skewed so much by a small proportion of extremely large or small values, and so it may give a better idea of a \"typical\" value. For example, in understanding statistics like household income or assets, which vary greatly, the mean may be skewed by a small number of extremely high or low values. Median income, for example, may be a better way to suggest what a \"typical\" income is. Because of this, the median is of central importance in robust statistics, as it is the most resistant statistic, having a breakdown point of 50%: so long as no more than half the data are contaminated, the median will not give an arbitrarily large or small result.\n",
    "\n",
    "The median of a finite list of numbers is the \"middle\" number, when those numbers are listed in order from smallest to greatest.\n",
    "\n",
    "If there is an odd number of observations, the middle one is picked. For example, consider the list of numbers\n",
    "\n",
    "> 1, 3, 3, **6**, 7, 8, 9\n",
    "\n",
    "This list contains seven numbers. The median is the fourth of them, which is 6.\n",
    "\n",
    "In general, for a set $x$ of $n$ elements, this can be written as:\n",
    "\n",
    "$$median (x)=x_{(n+1)/2}$$\n",
    "\n",
    "If there is an even number of observations, then there is no single middle value; the median is then usually defined to be the mean of the two middle values.For example, in the data set\n",
    "\n",
    "> 1, 2, 3, **4**, **5**, 6, 8, 9\n",
    "\n",
    "The median is the mean of the middle two numbers: this is $\\frac {(4+5)}{2}$, which is $4.5$. (In more technical terms, this interprets the median as the fully trimmed mid-range). With this convention, the median can be described in a formula as follows (for **even** number of observations):\n",
    "\n",
    "$$median (x)= \\frac {x_{n/2}+x_{n/2+1}}{2}$$\n",
    "\n",
    "\n",
    "### Mode\n",
    "\n",
    "The mode is the value that appears most often in a set of data values. If X is a discrete random variable, the mode is the value x (i.e, **X** = x) at which the probability mass function takes its maximum value. In other words, it is the value that is most likely to be sampled.\n",
    "\n",
    "Like the statistical mean and median, the mode is a way of expressing, in a (usually) single number, important information about a random variable or a population. The numerical value of the mode is the same as that of the mean and median in a normal distribution, and it may be very different in highly skewed distributions.\n",
    "\n",
    "The mode is not necessarily unique to a given discrete distribution, since the probability mass function may take the same maximum value at several points x1, x2, etc. The most extreme case occurs in uniform distributions, where all values occur equally frequently.\n",
    "\n",
    "The mode of a sample is the element that occurs most often in the collection. For example, the mode of the sample [1, 3, 6, 6, 6, 6, 7, 7, 12, 12, 17] is 6. Given the list of data [1, 1, 2, 4, 4] the mode is not unique – the dataset may be said to be bimodal, while a set with more than two modes may be described as multimodal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "from scipy import stats\n",
    "\n",
    "speed = [99,86,87,88,111,86,103,87,94,78,77,85,86]\n",
    "\n",
    "mean = numpy.mean(speed)\n",
    "median = numpy.median(speed)\n",
    "mode = stats.mode(speed)\n",
    "\n",
    "\n",
    "print(f\"The mean is: {round(mean,2)}\")\n",
    "print(f\"The median is: {round(median,2)}\")\n",
    "print(f\"The mode is: {mode[0]} and it appeared {mode[1]} times.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variance, Autocovariance and Autocorrelation\n",
    "\n",
    "### Variance\n",
    "In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its mean. Informally, it measures how far a set of numbers is spread out from their average value. Variance has a central role in statistics, where some ideas that use it include descriptive statistics, statistical inference, hypothesis testing, goodness of fit, and Monte Carlo sampling. Variance is an important tool in the sciences, where statistical analysis of data is common. The variance is the square of the standard deviation, the second central moment of a distribution, and the covariance of the random variable with itself, and it is often represented by $ \\sigma ^{2}, s^{2}$, or $\\operatorname {Var} (X)$.\n",
    "\n",
    "A random variable $X$ with a finite or countably infinite range of values $\\mathcal {T}=\\{x_{1}, x_{2},\\dotsc ,x_{k},\\dotsc \\}$ is called discrete. Its variance is then calculated as the weighted sum of the squares of deviations (from the expected value):\n",
    "\n",
    "$$\\sigma^{2} = (x_1 - \\mu)^{2} p_1 + (x_2 - \\mu)^{2} p_2 + \\dotsc + (x_k - \\mu)^{2} p_k + \\dotsc = \\sum_{i \\geq 1} (x_i - \\mu)^{2} p_i $$\n",
    "\n",
    "Here $ p_{i}=P(X=x_{i})$ is the probability that $X$ takes the value $x_{i}$. Thus, in the above sum, each possible expression $(x_{i}-\\mu )^{2}$ is weighted by the probability of its occurrence $p_{i}$. Thus, for discrete random variables, the variance is a weighted sum with the weights $ p_{i} \\;(i=1,\\ldots ,n)$. The expected value of a discrete random variable $X$ also represents a weighted sum given by\n",
    "\n",
    "$$ \\mu =x_{1}p_{1}+x_{2}p_{2}+\\ldots +x_{k}p_{k}+\\ldots =\\sum _{i\\geq 1}x_{i}p_{i}$$\n",
    "\n",
    "In statistics, variance measures variability from the average or mean. It is calculated by taking the differences between each number in the data set and the mean, then squaring the differences to make them positive, and finally dividing the sum of the squares by the number of values in the data set.\n",
    "\n",
    "\n",
    "Variance is calculated using the following formula:\n",
    "\n",
    "$$ s^{2} = \\frac {\\sum_{i=1}^{n}(x_i - \\bar x)^{2}}{n - 1} $$\n",
    "\n",
    "$$ \\sigma^{2} = \\frac {\\sum_{i=1}^{n}(x_i - \\mu)^{2}}{n} $$\n",
    "\n",
    "where:\n",
    "\n",
    "$$ x_i = \\textrm{the}\\ i^{th} \\; \\textrm{data point}\\  $$ \n",
    "$$ \\bar x = \\textrm{the mean of all data points} \\; \\textrm{or}\\; \\mu = \\textrm{the mean of the population} $$ \n",
    "$$ n = \\textrm{the number of data points} $$ \n",
    "\n",
    "\n",
    "\n",
    "### Autocovariance\n",
    "\n",
    "In probability theory and statistics, given a stochastic process, the autocovariance is a function that gives the covariance of the process with itself at pairs of time points. Autocovariance is closely related to the autocorrelation of the process in question.\n",
    "\n",
    "### Autocorrelation\n",
    "\n",
    "Autocorrelation, also known as serial correlation, is the correlation of a signal with a delayed copy of itself as a function of delay. Informally, it is the similarity between observations as a function of the time lag between them. The analysis of autocorrelation is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise, or identifying the missing fundamental frequency in a signal implied by its harmonic frequencies. It is often used in signal processing for analyzing functions or series of values, such as time domain signals."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example\n",
    "\n",
    "Say we have a dataset [3, 5, 2, 7, 1, 3]. To find its variance, we need to calculate the mean which is:\n",
    "\n",
    "(3+5+2+7+1+3) / 6 = 3.5 \n",
    "\n",
    "Then, we need to calculate the sum of the square deviation from the mean of all the observations. Here's how:\n",
    "\n",
    "(3−3.5)^2+(5−3.5)^2+(2−3.5)^2+(7−3.5)^2+(1−3.5)^2+(3−3.5)^2 = 23.5 \n",
    "\n",
    "To find the variance, we just need to divide this result by the number of observations like this:\n",
    "\n",
    "23.5 / 6 = 3.916666667 \n",
    "\n",
    "That's all. The variance of our data is 3.916666667.\n",
    "\n",
    "Python includes a standard module called statistics that provides some functions for calculating basic statistics of data. In this case, the statistics.pvariance() and statistics.variance() are the functions that we can use to calculate the variance of a population and of a sample respectively.\n",
    "\n",
    "We just need to import the statistics module and then call pvariance() or variance().\n",
    "\n",
    "pvariance(): Return the population variance of data. When called with the entire population, this gives the population variance σ². When called on a sample instead, this is the biased sample variance s², also known as variance with N degrees of freedom. If you somehow know the true population mean μ, you may use this function to calculate the variance of a sample, giving the known population mean as the second argument. Provided the data points are a random sample of the population, the result will be an unbiased estimate of the population variance.\n",
    "\n",
    "variance(): Return the sample variance of data. This is the sample variance s² with Bessel’s correction, also known as variance with N-1 degrees of freedom. Provided that the data points are representative (e.g. independent and identically distributed), the result should be an unbiased estimate of the true population variance. If you somehow know the actual population mean μ you should pass it to the pvariance() function as the mu parameter to get the variance of a sample."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standard Deviation\n",
    "\n",
    "Standard deviation is a measure that quantifies the amount of dispersion of the observations in a dataset. The low standard deviation is an indicator of the closeness of the scores to the arithmetic mean and a high standard deviation represents; the scores are dispersed over a higher range of values.\n",
    "\n",
    "$$ s = \\sqrt{ \\frac {\\sum_{i=1}^{n} (x_i-\\ \\bar x)^{2}}{n-1}} $$\n",
    "$$ \\sigma = \\sqrt{ \\frac {\\sum_{i=1}^{n} (x_i-\\ \\mu)^{2}}{n}} $$\n",
    "\n",
    "where:\n",
    "\n",
    "$$ x_i = \\textrm{the}\\ i^{th} \\; \\textrm{data point}\\  $$ \n",
    "$$ \\bar x = \\textrm{the mean of all data points} \\; \\textrm{or}\\; \\mu = \\textrm{the mean of the population} $$ \n",
    "$$ n = \\textrm{the number of data points} $$ \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The difference between Variance and Standard Deviation\n",
    "|      Variance                 |        Standard Deviation         |\n",
    "|:-----------------------------:|:---------------------------------:|\n",
    "| Variance is a numerical value that describes <br> the variability of observations from its arithmetic mean.|Standard deviation is a measure of  the dispersion <br> of observations within a data set relative to their mean.| \n",
    "|An average of squared deviations | The root mean square deviation |\n",
    "|Expressed in square units which are usually larger than <br> the values in the given dataset|Expressed in the same units as the values <br> in the set of data.|\n",
    "|Measures how far individuals in a group are spread out <br> in the set of data from the average|Measures how much the observations of a data set <br> differ from its mean|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import statistics\n",
    "import numpy\n",
    "\n",
    "data = [3, 5, 2, 7, 1, 3]\n",
    "\n",
    "mean = numpy.mean(data)\n",
    "pvar = statistics.pvariance(data)\n",
    "var = statistics.variance(data)\n",
    "std = statistics.stdev(data)\n",
    "\n",
    "print(f\"The mean is: {round(mean,2)}\")\n",
    "print(f\"The pvariance is: {round(pvar,2)}\")\n",
    "print(f\"The variance is: {var}\")\n",
    "print(f\"The standard deviation is: {std}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise: Try it yourself!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Find the mean and median of the following data.\n",
    "# \n",
    "# (a) 9, 7, 11, 13, 2, 4, 5, 5\n",
    "# \n",
    "# (b) 16, 18, 19, 21, 23, 23, 27, 29, 29, 35\n",
    "# \n",
    "# (c) 2.2, 10.2, 14.7, 5.9, 4.9, 11.1, 10.5\n",
    "# \n",
    "# (d) 11/4, 21/2, 51/2, 31/4, 21/2\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. Find the mode of the following data.\n",
    "# \n",
    "# (a) 12, 8, 4, 8, 1, 8, 9, 11, 9, 10, 12, 8\n",
    "# \n",
    "# (b) 15, 22, 17, 19, 22, 17, 29, 24, 17, 15\n",
    "# \n",
    "# (c) 0, 3, 2, 1, 3, 5, 4, 3, 42, 1, 2, 0\n",
    "# \n",
    "# (d) 1, 7, 2, 4, 5, 9, 8, 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 3. Calculate the mean, variance and standard deviation for \n",
    "# both groups: \n",
    "# Body weight (kgs)\t\t\t\t\t\t\t\t\t\t\t\n",
    "# Group 1 (heart failure)\t54.6\t73.5\t50.3\t44.6\t58.1\t61.3\t75.3\t41.1\t51.5\t41.7\t59.7\n",
    "# Group 2 (controls)\t40.8\t67.4\t53.3\t62.2\t65.5\t47.5\t51.2\t74.9\t59\t40.5\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}