AR typos

Author: elizavetasemenova

16_AR.ipynb

@@ -14,7 +14,18 @@
         "\n",
         "ARIMA is an abbreviation for  <font color='orange'>autoregressive integrated moving average</font> and these models are what the tin says: they include autoregressive, differencing, and moving average components.\n",
         "\n",
-        "In this lecture we are interested in looking at time series structure through the temporal random effects lens. We will consider a specific subclass of ARIMA family. Namely, the autoregressive models.\n",
+        "A common way to work with time series data, is to decompose it into the trend and seasonality components, and then model thre residual random effect with ARIMA. We can view the trend and seasonality parts as the fixed effect of the model, and the residual effect as the random effect:\n",
+        "\n",
+        "$$\n",
+        "\\begin{align*}\n",
+        "y_t &= \\mu_t + \\epsilon_t, \\epsilon_t \\sim \\mathcal{N}(0, \\sigma^2),\\\\\n",
+        "\\mu_t &= \\beta_1 t + \\beta_2 \\sin\\left(\\frac{2\\pi}{r}t\\right) +  \\beta_3 \\sin\\left(\\frac{2\\pi}{r}t\\right).\n",
+        "\\end{align*}\n",
+        "$$\n",
+        "\n",
+        "Here $r$ is the seasonality period.\n",
+        "\n",
+        "In this lecture we are interested in looking at time series structure through the temporal random effects lens. I.e. we assume that that trend and seasonality have already been deducted $x_t := y_t - \\mu_t$. We will consider a specific subclass of ARIMA family. Namely, the autoregressive models.\n",
         "\n",
         "<font color='orange'>Autoregressive</font> models are statistical models that use a linear combination of past observations of a time series $x_{t-1}, x_{t-2},... $ to predict future values, where each observation is regressed on previous observations.\n",
         "\n",
@@ -31,7 +42,7 @@
         "\n",
         "The innovations $\\epsilon_t$ are independent from each other.\n",
         "\n",
-        "The condition $\\mid \\phi \\mid <1$ here is required for stationarity. In order to ensure stationarity, we need to check that all roots of the characterictic polynomial $\\lambda_i$ satisfy $|\\lambda_i|<1.$ If $B$ is the backshift operator, such that $B^k = X_{t-k}$, we can rewrite the AR(1) model as \n",
+        "The condition $\\mid \\phi \\mid <1$ here is required for stationarity. In order to ensure stationarity, we need to check that all roots of the characterictic polynomial $\\lambda_i$ satisfy $|\\lambda_i|>1.$ If $B$ is the backshift operator, such that $B^k = X_{t-k}$, we can rewrite the AR(1) model as \n",
         "\n",
         "$$\n",
         "x_t (1 - \\phi B) = \\epsilon_t.\n",
@@ -46,9 +57,18 @@
     },
     {
       "cell_type": "code",
-      "execution_count": 2,
+      "execution_count": 1,
       "metadata": {},
-      "outputs": [],
+      "outputs": [
+        {
+          "name": "stderr",
+          "output_type": "stream",
+          "text": [
+            "/opt/anaconda3/envs/aims/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
+            "  from .autonotebook import tqdm as notebook_tqdm\n"
+          ]
+        }
+      ],
       "source": [
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
@@ -318,17 +338,17 @@
         "Joint density of $x_1, ..., x_T$, therefore, has the form\n",
         "\n",
         "```{margin}\n",
-        "recall the pdf of the multivariate Normal distribution which is $\\propto \\exp{(-x \\Sigma^{-1} x^T)}= \\exp{(-x Q x^T)}$ if we denote $Q = \\Sigma^{=1}$. The matrix $Q$ is called *precision matrix*.\n",
+        "recall the pdf of the multivariate Normal distribution which is $\\propto \\exp{(-\\frac{1}{2}x^T \\Sigma^{-1} x)}= \\exp{(-\\frac{1}{2}x^T Q x)}$ if we denote $Q = \\Sigma^{-1}$. The matrix $Q$ is called *precision matrix*.\n",
         "```\n",
         "\n",
         "$$\n",
         "\\begin{align*}\n",
         "p(x_1, x_2, ..., x_T) &= p(x_1)p(x_2 \\mid x_1) p(x_3 \\mid x_2)... p(x_T \\mid x_{T-1})\\\\\n",
         "& = \\mathcal{N}_{x_1} \\left(0, \\frac{\\sigma^2}{1-\\phi^2} \\right) \\times \\mathcal{N}_{x_2} \\left(\\phi x_1,\\sigma^2 \\right) \\times ... \\times \\mathcal{N}_{x_T} \\left(\\phi x_{T-1},\\sigma^2 \\right) = \\\\\n",
-        "& = \\frac{1}{\\sqrt{2 \\pi * \\frac{\\sigma^2}{1-\\phi^2}}} \\exp{\\left( - \\frac{x_1^2(1-\\phi^2)}{\\sigma^2} \\right)} \\times \\frac{1}{\\sqrt{2 \\pi *\\sigma^2}} \\exp{\\left( - \\frac{(x_2 - \\phi x_1)^2}{\\sigma^2} \\right)} \\times ... \\\\\n",
-        "&\\times \\frac{1}{\\sqrt{2 \\pi *\\sigma^2}} \\exp{\\left( - \\frac{(x_T - \\phi x_{T-1})^2}{\\sigma^2} \\right)} \\\\\n",
-        "& = \\sqrt{\\frac{1-\\phi^2}{(2 \\pi *\\sigma^2)^T}}\\exp{ \\left( -\\frac{x_1^2(1-\\phi^2) + (x_2 - \\phi x_1)^2 + ... + (x_T - \\phi x_{T-1})^2}{\\sigma^2} \\right)} \\\\\n",
-        "& \\propto \\exp{\\left( - (x_1, x_2, ..., x_T) * Q * (x_1, x_2, ..., x_T)^T \\right)} = \\exp{ (- x Q x^T)}\n",
+        "& = \\frac{1}{\\sqrt{2 \\pi * \\frac{\\sigma^2}{1-\\phi^2}}} \\exp{\\left( - \\frac{x_1^2(1-\\phi^2)}{2\\sigma^2} \\right)} \\times \\frac{1}{\\sqrt{2 \\pi *\\sigma^2}} \\exp{\\left( - \\frac{(x_2 - \\phi x_1)^2}{2\\sigma^2} \\right)} \\times ... \\\\\n",
+        "&\\times \\frac{1}{\\sqrt{2 \\pi *\\sigma^2}} \\exp{\\left( - \\frac{(x_T - \\phi x_{T-1})^2}{2\\sigma^2} \\right)} \\\\\n",
+        "& = \\sqrt{\\frac{1-\\phi^2}{(2 \\pi *\\sigma^2)^T}}\\exp{ \\left( -\\frac{x_1^2(1-\\phi^2) + (x_2 - \\phi x_1)^2 + ... + (x_T - \\phi x_{T-1})^2}{2\\sigma^2} \\right)} \\\\\n",
+        "& \\propto \\exp{\\left( - (x_1, x_2, ..., x_T) * Q * (x_1, x_2, ..., x_T)^T \\right)} = \\exp{ (- x^T Q x)}\n",
         "\\end{align*}\n",
         "$$\n",
         "\n",
@@ -387,7 +407,7 @@
         "AR(2) process $x_t$ can be formulated mathematically as\n",
         "\n",
         "$$\n",
-        "x_t = \\gamma_0 + \\gamma_1 \\alpha_{t-1} + \\gamma_2 \\alpha_{t-2} + \\epsilon_t, \\\\\n",
+        "x_t = \\gamma_0 + \\gamma_1 x_{t-1} + \\gamma_2 x_{t-2} + \\epsilon_t, \\\\\n",
         "\\epsilon_t \\sim(0, \\sigma^2).\n",
         "$$\n",
         "\n",