Merge pull request #1 from OnnoKampman/corrections

Corrections

.gitignore

@@ -1,4 +1,5 @@
 .DS_Store
+.vscode/
 
 ## Core latex/pdflatex auxiliary files:
 *.aux

README.md

@@ -1,5 +1,24 @@
 # Robust time-varying functional connectivity estimation and its relevance for depression
 
+Thesis defended on December 1st, 2022.
+
+## Abstract
+
+This thesis investigates how to robustly estimate time-varying functional connectivity (TVFC), a construct in neuroimaging research that looks at changes in functional coupling (correlation between time series) between brain regions during a functional magnetic resonance imaging (fMRI) scan, and how it can be used as a lens through which to study depression as a functional disorder.
+
+Unfortunately, the field of TVFC is still riddled with uncertainty, especially regarding its estimation. This is mainly due to the absence of a ground truth. Without resolving this first, the value of any study, including this depression study, is significantly undermined and conclusions made therein less trustworthy. Therefore, I propose a novel and principled method for estimating TVFC, based on the Wishart process (WP), a covariance matrix stochastic process that has recently become computationally tractable, and introduce a comprehensive benchmarking framework based on machine learning principles to make sure it performs better than existing methods in the field. These benchmarks include simulations, subject phenotype prediction, test-retest studies, brain state analyses, external task prediction, and a range of qualitative method comparisons. Furthermore, I introduce a benchmark based on cross-validation, that can be run on any data set. The WP model is found to outperform other common estimation methods, such as sliding-windows (SW) approaches and dynamic conditional correlation (DCC).
+
+Returning to the depression study, several differences are found between depressed and healthy control cohorts. The study is run on thousands of participants from the UK Biobank, yielding unprecedented statistical power and robustness. I investigate connectivity between individual brain regions as well as functional networks (FNs). Depressed participants show decreased global connectivity, and increased connectivity instability (as measured by the temporal characteristics of estimated TVFC). By defining multiple depression phenotypes, I find that brain dynamics are affected especially when patients have been professionally diagnosed or indicated to be depressed during their fMRI scan, but were less or not at all affected based on self-reported past instances and genetic predisposition. I show that choosing a different TVFC estimation method would have changed our scientific conclusions. This sensitivity to seemingly arbitrary researcher choices highlights the need for robust method development and the importance of community-approved benchmarking.
+
+I wrap up this thesis with a discussion of results and how this style of work fits into the bigger picture of neuroscientific research, reflect on what has been learned about depression, and posit promising directions for future work.
+## Source code for experiments
+
+https://github.com/OnnoKampman/neuro-dynamic-covariance
+
+## Word count
+
+`ps2ascii main.pdf | wc -w`
+
 ## LaTeX
 
 This document is generated using `Latexmk` version 4.77 and Biber version 2.17.
@@ -9,5 +28,7 @@ You may need to clear the cache by running `rm -rf $(biber --cache)` if you enco
 ## Inspiration
 
 [1] https://github.com/kks32/phd-thesis-template
+
 [1] https://github.com/duvenaud/phd-thesis
+
 [1] https://jwalton.info/Embed-Publication-Matplotlib-Latex/

appendix/03_extra_benchmarking_results.tex

@@ -0,0 +1,333 @@
+\chapter{More benchmarking results}\label{appendix:more-benchmarking-results}
+%%%%%
+
+%%
+\section{Simulations: Optimal learned window lengths}\label{appendix:sim-optimal-window-lengths}
+%%
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N0120_T0200/no_noise/SW_cross_validated_optimal_window_lengths}
+  \caption{
+    Simulations benchmark optimal cross-validated window lengths learned from bivariate ($D = 2$) noiseless data for $N = 120$.
+    Each dot represents one of $T = 200$ trials.
+  }\label{fig:sim-optimal-window-lengths-N120}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N0200_T0200/no_noise/SW_cross_validated_optimal_window_lengths}
+  \caption{
+    Simulations benchmark optimal cross-validated window lengths learned from bivariate ($D = 2$) noiseless data for $N = 200$.
+    Each dot represents one of $T = 200$ trials.
+  }\label{fig:sim-optimal-window-lengths-N200}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N1200_T0200/no_noise/SW_cross_validated_optimal_window_lengths}
+  \caption{
+    Simulations benchmark optimal cross-validated window lengths learned from bivariate ($D = 2$) noiseless data for $N = 1200$.
+    Each dot represents one of $T = 200$ trials.
+  }\label{fig:sim-optimal-window-lengths-N1200}
+\end{figure}
+
+
+%%
+\clearpage
+\section{Simulations: Learned kernel lengthscales}\label{appendix:sim-kernel-lengthscales}
+%%
+
+
+\begin{figure}[ht]
+    \centering
+    \includegraphics[width=\textwidth]{fig/sim/d2/N0120_T0200/no_noise/SVWP_kernel_lengthscales}
+    \caption{
+        Simulations benchmark SVWP kernel lengthscales $l$ learned from bivariate ($D = 2$) noiseless data for $N = 120$.
+        Each dot represents one of $T = 200$ trials.
+    }\label{fig:sim-learned-kernel-lengthscales-N120}
+\end{figure}
+
+
+\begin{figure}[ht]
+    \centering
+    \includegraphics[width=\textwidth]{fig/sim/d2/N0200_T0200/no_noise/SVWP_kernel_lengthscales}
+    \caption{
+        Simulations benchmark SVWP kernel lengthscales $l$ learned from bivariate ($D = 2$) noiseless data for $N = 200$.
+        Each dot represents one of $T = 200$ trials.
+    }\label{fig:sim-learned-kernel-lengthscales-N200}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N1200_T0200/no_noise/SVWP_kernel_lengthscales}
+  \caption{
+    Simulations benchmark SVWP kernel lengthscales $l$ learned from bivariate ($D = 2$) noiseless data for $N = 1200$.
+    Each dot represents one of $T = 200$ trials.
+  }\label{fig:sim-learned-kernel-lengthscales-N1200}
+\end{figure}
+
+
+%%
+\clearpage
+\section{Simulations: Impact of noise}\label{ch:appendix-impact-of-noise}
+%%
+
+%%
+\subsection{Bivariate TVFC estimates}\label{ch:appendix-d2-impact-of-noise}
+%%
+
+
+\begin{figure}[h]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N0400_T0200/no_noise/all_covs_types_correlations}
+  \caption{
+    Model TVFC predictions on bivariate data for $N = 400$ data points.
+    No noise added.
+  }\label{fig:results-all-covariance-structures-tvfc-predictions}
+\end{figure}
+
+
+\begin{figure}[h]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N0400_T0200/HCP_noise_snr_6/all_covs_types_correlations}
+  \caption{
+    Model TVFC predictions on bivariate data for $N=400$ data points.
+    HCP noise with SNR of 6 added.
+  }\label{fig:results-all-covariance-structures-tvfc-predictions-snr-6}
+\end{figure}
+
+
+\begin{figure}[h]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N0400_T0200/HCP_noise_snr_2/all_covs_types_correlations}
+  \caption{
+    Model TVFC predictions on bivariate data for $N = 400$ data points.
+    HCP noise with SNR of 2 added.
+  }\label{fig:results-all-covariance-structures-tvfc-predictions-snr-2}
+\end{figure}
+
+
+\begin{figure}[h]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d2/N0400_T0200/HCP_noise_snr_1/all_covs_types_correlations}
+  \caption{
+    Model TVFC predictions on bivariate data for $N = 400$ data points.
+    HCP noise with SNR of 1 added.
+  }\label{fig:results-all-covariance-structures-tvfc-predictions-snr-1}
+\end{figure}
+
+
+%%
+\clearpage
+\subsection{Bivariate quantitative results}
+%%
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N0400_T0200/no_noise/correlation_RMSE}
+  \caption{
+    Performance of models on all bivariate synthetic covariance structures without noise for $N = 400$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-400-all-correlation-RMSE-no-noise}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N0400_T0200/HCP_noise_snr_6/correlation_RMSE}
+  \caption{
+    Performance of models on all bivariate synthetic covariance structures with HCP noise with SNR of 6 added for $N = 400$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-400-all-correlation-RMSE-snr-6}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N0400_T0200/HCP_noise_snr_2/correlation_RMSE}
+  \caption{
+    Performance of models on all bivariate synthetic covariance structures with HCP noise with SNR of 2 added for $N = 400$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-400-all-correlation-RMSE-snr-2}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N0400_T0200/HCP_noise_snr_1/correlation_RMSE}
+  \caption{
+    Performance of models on all bivariate synthetic covariance structures with HCP noise with SNR of 1 added for $N = 400$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-400-all-correlation-RMSE-snr-1}
+\end{figure}
+
+
+%%
+\clearpage
+\subsection{Trivariate TVFC estimates}\label{ch:appendix-d3d-impact-of-noise}
+%%
+
+
+\begin{figure}[h]
+    \centering
+    \includegraphics[width=\textwidth]{fig/sim/d3d/N0400_T0003/no_noise/periodic_1_correlations}
+    \caption{
+        Model TVFC estimates on dense trivariate data for $N = 400$ data points.
+        No noise added.
+    }\label{fig:results-d3d-periodic-1-tvfc-predictions-no-noise}
+\end{figure}
+
+
+\begin{figure}[h]
+    \centering
+    \includegraphics[width=\textwidth]{fig/sim/d3d/N0400_T0003/HCP_noise_snr_6/periodic_1_correlations}
+    \caption{
+        Model TVFC estimates on dense trivariate data for $N = 400$ data points.
+        HCP noise with SNR of 6 added.
+    }\label{fig:results-d3d-periodic-1-tvfc-predictions-snr-6}
+\end{figure}
+
+
+\begin{figure}[h]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d3d/N0400_T0003/HCP_noise_snr_2/periodic_1_correlations}
+  \caption{
+    Model TVFC estimates on dense trivariate data for $N = 400$ data points.
+    HCP noise with SNR of 2 added.
+  }\label{fig:results-d3d-periodic-1-tvfc-predictions-snr-2}
+\end{figure}
+
+
+\begin{figure}[h]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d3d/N0400_T0003/HCP_noise_snr_1/periodic_1_correlations}
+  \caption{
+    Model TVFC estimates on dense trivariate data for $N = 400$ data points.
+    HCP noise with SNR of 1 added.
+  }\label{fig:results-d3d-periodic-1-tvfc-predictions-snr-1}
+\end{figure}
+
+
+%%
+\clearpage
+\section{Simulations: Trivariate TVFC estimates}\label{ch:appendix-d3-tvfc-estimates}
+%%
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d3s/N0400_T0003/no_noise/null_correlations}
+  \caption{
+    Simulations benchmark single trial TVFC estimates for null covariance structure, for trivariate ($D = 3$) data for $N = 400$.
+  }\label{fig:results-d3-no-noise-null-covariance}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d3d/N0400_T0003/no_noise/periodic_1_correlations}
+  \caption{
+    Simulations benchmark single trial TVFC estimates for periodic (fast) covariance structure, for dense trivariate ($D = 3$) data for $N = 400$.
+  }\label{fig:results-d3s-no-noise-periodic-3-covariance}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{fig/sim/d3s/N0400_T0003/no_noise/periodic_3_correlations}
+  \caption{
+    Simulations benchmark single trial TVFC estimates for periodic (fast) covariance structure, for dense trivariate ($D = 3$) data for $N = 400$.
+  }\label{fig:results-d3s-no-noise-stepwise-covariance}
+\end{figure}
+
+
+%%
+\clearpage
+\section{Simulations: More quantitative results}\label{appendix:sim-more-quantitative-results}
+%%
+
+%%
+\subsection{Bivariate}
+%%
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N0120_T0200/no_noise/correlation_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all bivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 120$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-120-no-noise-all-correlation-RMSE}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N0200_T0200/no_noise/correlation_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all bivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 200$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-200-no-noise-all-correlation-RMSE}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d2/N1200_T0200/no_noise/correlation_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all bivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 1200$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d2-1200-no-noise-all-correlation-RMSE}
+\end{figure}
+
+
+%%
+\clearpage
+\subsection{Trivariate}
+%%
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d3d/N0120_T0200/no_noise/correlation_matrix_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all dense trivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 120$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d3d-120-no-noise-all-correlation-matrix-RMSE}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d3d/N0200_T0200/no_noise/correlation_matrix_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all dense trivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 200$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d3d-200-no-noise-all-correlation-matrix-RMSE}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d3s/N0120_T0200/no_noise/correlation_matrix_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all sparse trivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 120$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d3s-120-no-noise-all-correlation-matrix-RMSE}
+\end{figure}
+
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=0.84\textwidth]{fig/sim/d3s/N0200_T0200/no_noise/correlation_matrix_RMSE}
+  \caption{
+    Simulations benchmark RMSE between model TVFC estimates and ground truth on all sparse trivariate covariance structures with added rs-fMRI noise (SNR of 2) for $N = 200$.
+    Means and standard deviations are shown across $T = 200$ trials.
+  }\label{fig:results-sim-d3s-200-no-noise-all-correlation-matrix-RMSE}
+\end{figure}