-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathvartransform.html
179 lines (168 loc) · 14 KB
/
vartransform.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
<!DOCTYPE html>
<html lang="en" data-content_root="./">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Is VAR model fitting invariant to PCA transformations? — SCoT</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="_static/classic.css?v=514cf933" />
<link rel="stylesheet" type="text/css" href="_static/plot_directive.css" />
<script src="_static/documentation_options.js?v=2b62b5d0"></script>
<script src="_static/doctools.js?v=888ff710"></script>
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Testing SPHINX Features" href="mytest.html" />
<link rel="prev" title="Miscellaneous Topics" href="misc.html" />
</head><body>
<div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px">
<a href="index.html"><img src="_static/logo.png" border="0" alt="py4sci"/></a>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="mytest.html" title="Testing SPHINX Features"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="misc.html" title="Miscellaneous Topics"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">SCoT</a> »</li>
<li class="nav-item nav-item-1"><a href="misc.html" accesskey="U">Miscellaneous Topics</a> »</li>
<li class="nav-item nav-item-this"><a href="">Is VAR model fitting invariant to PCA transformations?</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="body" role="main">
<section id="is-var-model-fitting-invariant-to-pca-transformations">
<h1>Is VAR model fitting invariant to PCA transformations?<a class="headerlink" href="#is-var-model-fitting-invariant-to-pca-transformations" title="Link to this heading">¶</a></h1>
<p><a class="reference internal" href="glossary.html#term-MVARICA"><span class="xref std std-term">MVARICA</span></a> usually applies a PCA transform to the EEG prior to VAR model fitting. This is intended as a
dimensionality reduction step; PCA components that contribute little to total EEG variance are removed. However, the PCA
produces orthogonal components. In other words, PCA transformed signals are uncorrelated.</p>
<p>The question was raised whether it is possible to reconstruct (fit) a VAR model from PCA transformed signals. Here we
show that this is, in fact, the case.</p>
<p>We will denote a var model with coefficients <span class="math notranslate nohighlight">\(\mathbf{C}\)</span> and innovation process <span class="math notranslate nohighlight">\(\vec{\epsilon}\)</span> as
<span class="math notranslate nohighlight">\(\mathrm{VAR}(\mathbf{C},\vec{\epsilon})\)</span>.</p>
<p>Let’s start with a VAR process <span class="math notranslate nohighlight">\(\vec{x}_n = \mathrm{VAR}(\mathbf{A},\vec{e})\)</span>. If the model
contains causal structure, elements of <span class="math notranslate nohighlight">\(\vec{x}\)</span> will in most cases show some degree of correlation. Let
<span class="math notranslate nohighlight">\(\vec{y}_n = \mathbf{W} \vec{x}_n\)</span> be the PCA transformed signal. Furthermore, assume that <span class="math notranslate nohighlight">\(\vec{y}\)</span> is a
VAR process too: <span class="math notranslate nohighlight">\(\vec{y}_n = \mathrm{VAR}(\mathbf{B},\vec{r})\)</span>.</p>
<p>In order to reconstruct the original VAR model <span class="math notranslate nohighlight">\(\mathrm{VAR}(\mathbf{A},\vec{e})\)</span> from
<span class="math notranslate nohighlight">\(\mathrm{VAR}(\mathbf{B},\vec{r})\)</span> the following requirements need to be met:</p>
<blockquote>
<div><ol class="arabic simple">
<li><p><span class="math notranslate nohighlight">\(\mathrm{VAR}(\mathbf{B},\vec{r})\)</span> can be transformed into <span class="math notranslate nohighlight">\(\mathrm{VAR}(\mathbf{A},\vec{e})\)</span> when the PCA transform <span class="math notranslate nohighlight">\(\mathbf{W}\)</span> is known.</p></li>
<li><p>A VAR model can have zero cross-correlation despite having causal structure.</p></li>
</ol>
</div></blockquote>
<p>The first requirement is obvious. Only when the models can be transformed into each other it is possible to reconstruct one model from another.
Since the PCA transformation <span class="math notranslate nohighlight">\(\mathbf{W}\)</span> is a rotation matrix, its inverse is the transpose <span class="math notranslate nohighlight">\(\mathbf{W}^{-1} = \mathbf{W}^\intercal\)</span>.
In section <a class="reference internal" href="#lintransvar"><span class="std std-ref">Linear transformation of a VAR model</span></a> we show that transformation of VAR models is possible if <span class="math notranslate nohighlight">\(\mathbf{S} \mathbf{R} = \mathbf{I}\)</span> and <span class="math notranslate nohighlight">\(\mathbf{R} \mathbf{S} = \mathbf{I}\)</span>.
This is the case with PCA since <span class="math notranslate nohighlight">\(\mathbf{W}^\intercal \mathbf{W} = \mathbf{W} \mathbf{W}^\intercal = \mathbf{I}\)</span>.</p>
<p>The second requirement relates to the fact that in order to reconstruct model A from model B all information about A must be present in B.
Thus, information about the causal structure of A must be preserved in B, although <span class="math notranslate nohighlight">\(\vec{y}_n = \mathrm{VAR}(\mathbf{B},\vec{r})\)</span> is uncorrelated.
<a class="reference internal" href="#covbivar1"><span class="std std-ref">Covariance of a bivariate AR(1) process</span></a> shows that it is possible to construct models where causal structure cancels cross-correlation.</p>
<p>In conclusion, it is possible to fit VAR models on PCA transformed signals and reconstruct the original model.</p>
<section id="linear-transformation-of-a-var-model">
<span id="lintransvar"></span><h2>Linear transformation of a VAR model<a class="headerlink" href="#linear-transformation-of-a-var-model" title="Link to this heading">¶</a></h2>
<p>We start with a two VAR models; one for each vector signal <span class="math notranslate nohighlight">\(\vec{x}\)</span> and <span class="math notranslate nohighlight">\(\vec{y}\)</span>:</p>
<div class="math notranslate nohighlight">
\[\begin{split}\vec{x}_n &= \sum_{k=1}^{p}\mathbf{A}^{(k)}\vec{x}_{n-k} + \vec{e}_n \\
\vec{y}_n &= \sum_{k=1}^{p}\mathbf{B}^{(k)}\vec{y}_{n-k} + \vec{r}_n\end{split}\]</div>
<p>Now assume that <span class="math notranslate nohighlight">\(\vec{x}\)</span> and <span class="math notranslate nohighlight">\(\vec{y}\)</span> can be transformed into each other by linear transformations:</p>
<div class="math notranslate nohighlight">
\[\begin{split}\vec{y}_n &= \mathbf{R} \vec{x}_n \\
\vec{x}_n &= \mathbf{S} \vec{y}_n\end{split}\]</div>
<p>Note that</p>
<div class="math notranslate nohighlight">
\[\begin{split}\left.
\begin{matrix}
\vec{y}_n = \mathbf{R} \mathbf{S} \vec{y}_n \\
\vec{x}_n = \mathbf{S} \mathbf{R} \vec{x}_n
\end{matrix}
\right\} \Rightarrow \mathbf{R} \mathbf{S} = \mathbf{I}, \mathbf{S} \mathbf{R} = \mathbf{I}\end{split}\]</div>
<p>By substituting the transformations into the VAR model equations we obtain</p>
<div class="math notranslate nohighlight">
\[\begin{split}\vec{x}_n &= \sum_{k=1}^{p}\mathbf{S}\mathbf{B}^{(k)}\mathbf{R} \vec{x}_{n-k} + \mathbf{S} \vec{r}_n \\
\vec{y}_n &= \sum_{k=1}^{p}\mathbf{R}\mathbf{A}^{(k)}\mathbf{S} \vec{y}_{n-k} + \mathbf{R} \vec{e}_n\end{split}\]</div>
<p>Thus, each model can be transformed into the other by</p>
<div class="math notranslate nohighlight">
\[\begin{split}\mathbf{A}^{(k)} &= \mathbf{S}\mathbf{B}^{(k)}\mathbf{R},\; \vec{e}_n = \mathbf{S} \vec{r}_n \\
\mathbf{B}^{(k)} &= \mathbf{R}\mathbf{A}^{(k)}\mathbf{S},\; \vec{r}_n = \mathbf{R} \vec{e}_n\end{split}\]</div>
<p>Conclusion: We can equivalently formulate VAR models for vector signals, if these signals are related by linear
transformations that satisfy <span class="math notranslate nohighlight">\(\mathbf{S} \mathbf{R} = \mathbf{I}\)</span> and <span class="math notranslate nohighlight">\(\mathbf{R} \mathbf{S} = \mathbf{I}\)</span>.</p>
</section>
<section id="covariance-of-a-bivariate-ar-1-process">
<span id="covbivar1"></span><h2>Covariance of a bivariate AR(1) process<a class="headerlink" href="#covariance-of-a-bivariate-ar-1-process" title="Link to this heading">¶</a></h2>
<p>Consider the bivariate AR(1) process given by</p>
<div class="math notranslate nohighlight">
\[\begin{split}\left[ \begin{matrix} x_1(n) \\ x_2(n) \end{matrix} \right] =
\left[ \begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix} \right]
\left[ \begin{matrix} x_1(n-1) \\ x_2(n-1) \end{matrix} \right] +
\left[ \begin{matrix} c_{11} & c_{12} \\ c_{21} & c_{22} \end{matrix} \right]
\left[ \begin{matrix} e_1(n) \\ e_2(n) \end{matrix} \right]\end{split}\]</div>
<p>where <span class="math notranslate nohighlight">\(e_1\)</span> and <span class="math notranslate nohighlight">\(e_2\)</span> are uncorrelated Gaussian white noise processes with zero mean and unit variance.</p>
<p>The process variances <span class="math notranslate nohighlight">\(s_i^2\)</span> and covariance <span class="math notranslate nohighlight">\(r\)</span> are obtained by solving the following system of equations <a class="footnote-reference brackets" href="#id2" id="id1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>:</p>
<div class="math notranslate nohighlight">
\[\begin{split}\left( \mathbf{I} - \left[ \begin{matrix} a_{11}^2 & a_{12}^2 & 2a_{11}a_{12} \\
a_{21}^2 & a_{22}^2 & 2a_{21}a_{22} \\
a_{11}a_{21} & a_{12}a_{22} & a_{11}a_{22} + a_{12}a_{21}
\end{matrix} \right]
\right)
\left[ \begin{matrix} s_1^2 \\ s_2^2 \\ r \end{matrix} \right] = \left[ \begin{matrix} c_{11}^2 + c_{12}^2 \\ c_{21}^2 + c_{22}^2 \\ c_{11}c_{21} + c_{12}c_{22} \end{matrix} \right]\end{split}\]</div>
<p>In general, a VAR model with causal structure (<span class="math notranslate nohighlight">\(a_{12} \neq 0\)</span> and/or <span class="math notranslate nohighlight">\(a_{21} \neq 0\)</span>) has some instantaneous correlation (non-zero covariance <span class="math notranslate nohighlight">\(r \neq 0\)</span>) between signals.</p>
<p>Now, let’s constrain the system to zero covariance <span class="math notranslate nohighlight">\(r = 0\)</span>.</p>
<div class="math notranslate nohighlight">
\[ \begin{align}\begin{aligned}c_{11}c_{21} + c_{12}c_{22} + a_{11}a_{21}s_1^2 + a_{12}a_{22}s_2^2 = 0\\\begin{split}\left( \mathbf{I} - \left[ \begin{matrix} a_{11}^2 & a_{12}^2 \\
a_{21}^2 & a_{22}^2
\end{matrix} \right]
\right)
\left[ \begin{matrix} s_1^2 \\ s_2^2 \end{matrix} \right] = \left[ \begin{matrix} c_{11}^2 + c_{12}^2 \\ c_{21}^2 + c_{22}^2 \end{matrix} \right]\end{split}\end{aligned}\end{align} \]</div>
<p>Conclusion: it is possible to construct special cases where VAR processes with causal structure have no instantaneous correlation.</p>
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="id2" role="doc-footnote">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id1">1</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://books.google.at/books?id=_VHxE26QvXgC&pg=PA230&lpg=PA230&dq=cross-covariance+of+bivariate+AR%281%29+process&source=bl&ots=EiwYr1CA6x&sig=zMJwf8s1MXk5CTyf6CKw9JfKBDU&hl=en&sa=X&ei=cLnqUsqRO6ve7Aan84DYDQ&ved=0CDIQ6AEwAQ#v=onepage&q=cross-covariance%20of%20bivariate%20AR%281%29%20process&f=false">http://books.google.at/books?id=_VHxE26QvXgC&pg=PA230&lpg=PA230&dq=cross-covariance+of+bivariate+AR%281%29+process&source=bl&ots=EiwYr1CA6x&sig=zMJwf8s1MXk5CTyf6CKw9JfKBDU&hl=en&sa=X&ei=cLnqUsqRO6ve7Aan84DYDQ&ved=0CDIQ6AEwAQ#v=onepage&q=cross-covariance%20of%20bivariate%20AR%281%29%20process&f=false</a></p>
</aside>
</aside>
</section>
</section>
<div class="clearer"></div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="mytest.html" title="Testing SPHINX Features"
>next</a> |</li>
<li class="right" >
<a href="misc.html" title="Miscellaneous Topics"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">SCoT</a> »</li>
<li class="nav-item nav-item-1"><a href="misc.html" >Miscellaneous Topics</a> »</li>
<li class="nav-item nav-item-this"><a href="">Is VAR model fitting invariant to PCA transformations?</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2013-2016 SCoT Development Team.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 7.2.6.
</div>
</body>
</html>