[ENH] Add support for multivariate coherency method (canonical coherence)

[orabe]

This PR adds support for another multivariate connectivity method (canonical coherence; CaCoh), taking advantage of the multivariate framework implemented in v0.6. It is authored by myself and @tsbinns.

Method paper: Vidaurre et al. (2019)

As mentioned in #155, canonical coherence is a multivariate method based on coherency. It shares a number of similarities with the recently added methods MIC & MIM, but it is a more appropriate method to use for datasets where you don't expect volume conduction artefacts to be a big issue (e.g. connectivity between: cortical and subcortical recordings; cortex & EMG recordings, etc...).

Members of my own and other groups would be very interested in having this method implemented in MNE, as currently there is no implementation in any signal processing package (Python or MATLAB; only some MATLAB scripts provided by the authors on request).

Changes

Because CaCoh and MIC/MIM are based on similar principles, several bits of code can be shared between them. Accordingly, we:

Refactored the _MultivariateCohEstBase class to accommodate all multivariate coherency methods (CaCoh, MIC, & MIM):

mne-connectivity/mne_connectivity/spectral/epochs_multivariate.py

Lines 179 to 185 in f9ff2ac

	class _MultivariateCohEstBase(_EpochMeanMultivariateConEstBase):
	"""Base estimator for multivariate coherency methods.
	See:
	- Ewald et al. (2012). NeuroImage. DOI: 10.1016/j.neuroimage.2011.11.084
	- Vidaurre et al. (2019). NeuroImage. DOI: 10.1016/j.neuroimage.2019.116009
	"""

Introduced a new class _MultivariateImCohEstBase for code specific to MIC & MIM

mne-connectivity/mne_connectivity/spectral/epochs_multivariate.py

Lines 341 to 346 in f9ff2ac

	class _MultivariateImCohEstBase(_MultivariateCohEstBase):
	"""Base estimator for multivariate imag. part of coherency methods.
	See Ewald et al. (2012). NeuroImage. DOI: 10.1016/j.neuroimage.2011.11.084
	for equation references.
	"""

Introduced a new class _CaCohEst for code specific to CaCoh

mne-connectivity/mne_connectivity/spectral/epochs_multivariate.py

Lines 470 to 475 in f9ff2ac

	class _CaCohEst(_MultivariateCohEstBase):
	"""Canonical coherence (CaCoh) estimator.
	See Vidaurre et al. (2019). NeuroImage. DOI:
	10.1016/j.neuroimage.2019.116009 for equation references.
	"""

There are no API changes (beyond the ability to specify a new method).

Unit Tests

Unit tests for CaCoh have been implemented similarly to the other multivariate methods, which in most cases was just the addition of the "cacoh" method parameter to existing tests.

Notably, a regression test based on results from MATLAB code of the implementation was added, as for the other multivariate methods.

Outstanding Tasks

So far, the only thing missing is an example notebook. The format would be similar to that for MIC & MIM, and would ideally involve a dataset with recordings from two distinct sites (e.g. EEG & EMG, or EEG & subcortical LFPs).

We had a brief look through the MNE datasets, but didn't notice anything that quite fit this. Do you know if there is one like this we missed? If not, we can always show an example with some simulated data.

Any suggestions for the example data or code changes are of course very welcome!

[tsbinns]

Just to add, we started work on this when v0.6 was still in development, so we had to format the already-written code with black, hence the new entry in .git-blame-ignore-revs:

mne-connectivity/.git-blame-ignore-revs

Lines 1 to 2 in f9ff2ac

	0c216d35127f48792caabdc3cca170874a443eee # black, isort, ruff
	e93fb47b3c5bb20164d5bb347b40692a52cfe430 # black

Regarding the unit tests:

• linux py3.9 failing due to a h5py warning - UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2, this may cause problems
• ubuntu (latest) py3.11 failing due to an error in the main MNE package - ModuleNotFoundError: No module named 'mne.viz.backends.tests'

All unit tests are passing on our local machines, and we haven't made any changes to the dependencies or the visualisation, so I'm not sure where this comes from.

I anybody has come across this before and knows the solution that would be great, otherwise I will try to address this in the coming weeks.

[adam2392]

Just to add, we started work on this when v0.6 was still in development, so we had to format the already-written code with black, hence the new entry in .git-blame-ignore-revs:

mne-connectivity/.git-blame-ignore-revs

Lines 1 to 2 in f9ff2ac

	0c216d35127f48792caabdc3cca170874a443eee # black, isort, ruff
	e93fb47b3c5bb20164d5bb347b40692a52cfe430 # black

Is it possible to rebase and/or just start a new branch based off main and copy over the changes, so this is unnecessary?

[adam2392]

• linux py3.9 failing due to a h5py warning - UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2, this may cause problems
• ubuntu (latest) py3.11 failing due to an error in the main MNE package - ModuleNotFoundError: No module named 'mne.viz.backends.tests'

For mne main, https://github.com/mne-tools/mne-python/tree/main/mne/viz/backends it seems there was some restructuring.

I think in general, we can just maintain main consistency with mne main.

[tsbinns]

Just to add, we started work on this when v0.6 was still in development, so we had to format the already-written code with black, hence the new entry in .git-blame-ignore-revs:

mne-connectivity/.git-blame-ignore-revs

Lines 1 to 2 in f9ff2ac

	0c216d35127f48792caabdc3cca170874a443eee # black, isort, ruff
	e93fb47b3c5bb20164d5bb347b40692a52cfe430 # black

Is it possible to rebase and/or just start a new branch based off main and copy over the changes, so this is unnecessary?

@adam2392 Trying to rebase this was a mess, was much easier for me to just revert the change (authorship for some of the CaCoh is a bit messed up, but the rest of the package is unaffected).

[adam2392]

Just looked through the existing examples and don't seem to be others using the same approach. Essentially though it's the same code as for many of the unit tests, just with some extra options for controlling time delays:

mne-connectivity/mne_connectivity/spectral/tests/test_spectral.py

Lines 24 to 81 in 8652da4

	def create_test_dataset(
	sfreq, n_signals, n_epochs, n_times, tmin, tmax, fstart, fend, trans_bandwidth=2.0
	):
	"""Create test dataset with no spurious correlations.
	Parameters
	----------
	sfreq : float
	The simulated data sampling rate.
	n_signals : int
	The number of channels/signals to simulate.
	n_epochs : int
	The number of Epochs to simulate.
	n_times : int
	The number of time points at which the Epoch data is "sampled".
	tmin : int
	The start time of the Epoch data.
	tmax : int
	The end time of the Epoch data.
	fstart : int
	The frequency at which connectivity starts. The lower end of the
	spectral connectivity.
	fend : int
	The frequency at which connectivity ends. The upper end of the
	spectral connectivity.
	trans_bandwidth : int, optional
	The bandwidth of the filtering operation, by default 2.
	Returns
	-------
	data : np.ndarray of shape (n_epochs, n_signals, n_times)
	The epoched dataset.
	times_data : np.ndarray of shape (n_times, )
	The times at which each sample of the ``data`` occurs at.
	"""
	# Use a case known to have no spurious correlations (it would bad if
	# tests could randomly fail):
	rng = np.random.RandomState(0)
	data = rng.randn(n_signals, n_epochs * n_times)
	times_data = np.linspace(tmin, tmax, n_times)
	# simulate connectivity from fstart to fend
	data[1, :] = filter_data(
	data[0, :],
	sfreq,
	fstart,
	fend,
	filter_length="auto",
	fir_design="firwin2",
	l_trans_bandwidth=trans_bandwidth,
	h_trans_bandwidth=trans_bandwidth,
	)
	# add some noise, so the spectrum is not exactly zero
	data[1, :] += 1e-2 * rng.randn(n_times * n_epochs)
	data = data.reshape(n_signals, n_epochs, n_times)
	data = np.transpose(data, [1, 0, 2])
	return data, times_data

In theory this could be abstracted out not only for examples but could also be used for some of the existing simulations in the unit tests. Suppose this is a discussion that could be had in the new PR though.

Agreed. Feel free to open up a separate GH Issue or PR then?

[adam2392]

I've merged in #173, which I think will simplify the diff now

[tsbinns]

Thanks @adam2392. I merged main into this branch now so diff for examples is simplified.

[adam2392]

I’m okay merging this now and slating a release for the next version!

[tsbinns]

I suppose one outstanding point would be whether people still feel the extended examples we have written be instead labelled as 'tutorials'.

[drammock]

I suppose one outstanding point would be whether people still feel the extended examples we have written be instead labelled as 'tutorials'.

MNE-Connectivity doesn't separate "examples" from "tutorials" the way that MNE-Python does. So I'd say that's a larger conversation about docs restructuring that should be done in a separate issue / PR.