1-From_raw_to_epochs_evoked_ERF_ERP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNE : From raw data to epochs and evoked responses (ERF/ERP)\n",
    "\n",
    "`\n",
    "Authors:\n",
    "Alexandre Gramfort\n",
    "Denis A. Engemann\n",
    "`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, load the _mne_ package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mne"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We set the log-level to 'warning' so the output is less verbose"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "mne.set_log_level('warning')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remember if you need help just ask... the machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "mne.pick_types?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Access raw data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should have downloaded the `ds000117-practical` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Change the following path to where the folder ds000117-practical is on your disk\n",
    "data_path = os.path.expanduser(\"~/work/data/ds000117-practical/\")\n",
    "\n",
    "raw_fname = os.path.join(data_path,\n",
    "    'derivatives/meg_derivatives/sub-01/ses-meg/meg/sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/claire/work/data/ds000117-practical/derivatives/meg_derivatives/sub-01/ses-meg/meg/sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif\n"
     ]
    }
   ],
   "source": [
    "print(raw_fname)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31m/Users/claire/work/data/ds000117-practical/derivatives/meg_derivatives/sub-01/ses-meg/meg/sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif\u001b[m\u001b[m*\r\n"
     ]
    }
   ],
   "source": [
    "ls $raw_fname"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read data from file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "mne.io.read_raw_fif?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Raw  |  sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif, n_channels x n_times : 404 x 540100 (491.0 sec), ~7.4 MB, data not loaded>\n"
     ]
    }
   ],
   "source": [
    "raw = mne.io.read_raw_fif(raw_fname, preload=False)\n",
    "print(raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the `preload=False` which states that no data is actually in memory.\n",
    "\n",
    "For general info on importing MEG see:\n",
    "https://mne.tools/stable/auto_tutorials/io/plot_10_reading_meg_data.html\n",
    "or EEG see:\n",
    "https://mne.tools/stable/auto_tutorials/io/plot_20_reading_eeg_data.html\n",
    "\n",
    "Now let's look at the measurement info. It will give details about:\n",
    "\n",
    "   - sampling rate\n",
    "   - filtering parameters\n",
    "   - available channel types\n",
    "   - bad channels\n",
    "   - etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.20.dev0'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mne.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Info | 27 non-empty fields\n",
      "    acq_pars : str | 21833 items\n",
      "    bads : list | 0 items\n",
      "    ch_names : list | MEG0113, MEG0112, MEG0111, MEG0122, MEG0123, MEG0121, ...\n",
      "    chs : list | 404 items (GRAD: 204, MAG: 102, EEG: 74, STIM: 3, MISC: 12, CHPI: 9)\n",
      "    comps : list | 0 items\n",
      "    custom_ref_applied : NamedInt | 0 (FIFFV_MNE_CUSTOM_REF_OFF)\n",
      "    description : str | 36 items\n",
      "    dev_head_t : Transform | 3 items\n",
      "    dig : list | 137 items (3 Cardinal, 5 HPI, 75 EEG, 54 Extra)\n",
      "    events : list | 1 items\n",
      "    experimenter : str | 3 items\n",
      "    file_id : dict | 4 items\n",
      "    highpass : float | 0.0 Hz\n",
      "    hpi_meas : list | 1 items\n",
      "    hpi_results : list | 1 items\n",
      "    hpi_subsystem : dict | 2 items\n",
      "    line_freq : float | 50.0\n",
      "    lowpass : float | 356.3999938964844 Hz\n",
      "    meas_date : tuple | 1941-03-22 11:04:14 GMT\n",
      "    meas_id : dict | 4 items\n",
      "    nchan : int | 404\n",
      "    proc_history : list | 1 items\n",
      "    proj_id : ndarray | 1 items\n",
      "    proj_name : str | 11 items\n",
      "    projs : list | 0 items\n",
      "    sfreq : float | 1100.0 Hz\n",
      "    subject_info : dict | 2 items\n",
      "    acq_stim : NoneType\n",
      "    ctf_head_t : NoneType\n",
      "    dev_ctf_t : NoneType\n",
      "    device_info : NoneType\n",
      "    gantry_angle : NoneType\n",
      "    helium_info : NoneType\n",
      "    kit_system_id : NoneType\n",
      "    utc_offset : NoneType\n",
      "    xplotter_layout : NoneType\n",
      ">\n"
     ]
    }
   ],
   "source": [
    "print(raw.info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>How many channels do you have for each type of sensors?</li>\n",
    "    <li>What is the sampling frequency?</li>\n",
    "    <li>Have the data been filtered?</li>\n",
    "    <li>What is the frequency of the line noise?</li>\n",
    "    <li>Is there any bad channel?</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "INSERT ANSWERS HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "raw.info is just a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "isinstance(raw.info, dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we can access its elements this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1100.0"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info['sfreq']  # Sampling frequency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info['bads']  # list of marked bad channels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.0"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info['line_freq']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next let's see what channels are present. It is available via the `raw.ch_names` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(raw.ch_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['MEG0113',\n",
       " 'MEG0112',\n",
       " 'MEG0111',\n",
       " 'MEG0122',\n",
       " 'MEG0123',\n",
       " 'MEG0121',\n",
       " 'MEG0132',\n",
       " 'MEG0133',\n",
       " 'MEG0131',\n",
       " 'MEG0143']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.ch_names[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can index it as a list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'MEG0432'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.ch_names[42]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['MEG0113', 'MEG0112', 'MEG0111']"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.ch_names[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Channel type of a specific channel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Channel #75 is of type: grad\n",
      "Channel #320 is of type: eeg\n"
     ]
    }
   ],
   "source": [
    "channel_type = mne.io.pick.channel_type(raw.info, 75)\n",
    "print('Channel #75 is of type:', channel_type)\n",
    "\n",
    "channel_type = mne.io.pick.channel_type(raw.info, 320)\n",
    "print('Channel #320 is of type:', channel_type)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Info contains all the details about the sensors (type, locations, coordinate frame etc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "404"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(raw.info['chs'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(raw.info['chs'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'scanno': 1,\n",
       " 'logno': 113,\n",
       " 'kind': 1,\n",
       " 'range': 1.9073486328125e-05,\n",
       " 'cal': 3.250000046861601e-09,\n",
       " 'coil_type': 3012,\n",
       " 'loc': array([-0.1066    ,  0.0464    , -0.0604    , -0.01532829,  0.00619847,\n",
       "        -0.99986327, -0.18597366, -0.98255992, -0.00331254, -0.98243302,\n",
       "         0.185894  ,  0.016216  ]),\n",
       " 'unit': 201,\n",
       " 'unit_mul': 0,\n",
       " 'ch_name': 'MEG0113',\n",
       " 'coord_frame': 1 (FIFFV_COORD_DEVICE)}"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info['chs'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'scanno': 331,\n",
       " 'logno': 25,\n",
       " 'kind': 2,\n",
       " 'range': 0.00030517578125,\n",
       " 'cal': 0.00019999999494757503,\n",
       " 'coil_type': 1,\n",
       " 'loc': array([ 5.63842431e-02,  3.68367434e-02,  9.40217227e-02,  8.26010015e-04,\n",
       "         1.14762366e-01, -2.10680366e-02,  0.00000000e+00,  1.00000000e+00,\n",
       "         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  1.00000000e+00]),\n",
       " 'unit': 107,\n",
       " 'unit_mul': 0,\n",
       " 'ch_name': 'EEG025',\n",
       " 'coord_frame': 4 (FIFFV_COORD_HEAD)}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info['chs'][330]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot_sensors(kind='topomap', ch_type='grad');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot_sensors(kind='topomap', ch_type='eeg');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Info | 27 non-empty fields\n",
       "    acq_pars : str | 21833 items\n",
       "    bads : list | 0 items\n",
       "    ch_names : list | MEG0113, MEG0112, MEG0111, MEG0122, MEG0123, MEG0121, ...\n",
       "    chs : list | 404 items (GRAD: 204, MAG: 102, EEG: 74, STIM: 3, MISC: 12, CHPI: 9)\n",
       "    comps : list | 0 items\n",
       "    custom_ref_applied : NamedInt | 0 (FIFFV_MNE_CUSTOM_REF_OFF)\n",
       "    description : str | 36 items\n",
       "    dev_head_t : Transform | 3 items\n",
       "    dig : list | 137 items (3 Cardinal, 5 HPI, 75 EEG, 54 Extra)\n",
       "    events : list | 1 items\n",
       "    experimenter : str | 3 items\n",
       "    file_id : dict | 4 items\n",
       "    highpass : float | 0.0 Hz\n",
       "    hpi_meas : list | 1 items\n",
       "    hpi_results : list | 1 items\n",
       "    hpi_subsystem : dict | 2 items\n",
       "    line_freq : float | 50.0\n",
       "    lowpass : float | 356.3999938964844 Hz\n",
       "    meas_date : tuple | 1941-03-22 11:04:14 GMT\n",
       "    meas_id : dict | 4 items\n",
       "    nchan : int | 404\n",
       "    proc_history : list | 1 items\n",
       "    proj_id : ndarray | 1 items\n",
       "    proj_name : str | 11 items\n",
       "    projs : list | 0 items\n",
       "    sfreq : float | 1100.0 Hz\n",
       "    subject_info : dict | 2 items\n",
       "    acq_stim : NoneType\n",
       "    ctf_head_t : NoneType\n",
       "    dev_ctf_t : NoneType\n",
       "    device_info : NoneType\n",
       "    gantry_angle : NoneType\n",
       "    helium_info : NoneType\n",
       "    kit_system_id : NoneType\n",
       "    utc_offset : NoneType\n",
       "    xplotter_layout : NoneType\n",
       ">"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting channel types\n",
    "\n",
    "Some channels are wrongly defined as EEG in the file. 2 of these are EOG (EEG061 and EEG062) and EEG063 is actually an ECG channel. EEG064 was recording but not connected to anything, so we'll not make it `'misc'`. We will now set the channel types. This will be useful for automatic artifact rejection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.set_channel_types?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.set_channel_types({'EEG061': 'eog',\n",
    "                       'EEG062': 'eog',\n",
    "                       'EEG063': 'ecg',\n",
    "                       'EEG064': 'misc'})  # EEG064 free-floating el.\n",
    "\n",
    "raw.rename_channels({'EEG061': 'EOG061',\n",
    "                     'EEG062': 'EOG062',\n",
    "                     'EEG063': 'ECG063'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Info | 27 non-empty fields\n",
       "    acq_pars : str | 21833 items\n",
       "    bads : list | 0 items\n",
       "    ch_names : list | MEG0113, MEG0112, MEG0111, MEG0122, MEG0123, MEG0121, ...\n",
       "    chs : list | 404 items (GRAD: 204, MAG: 102, EEG: 70, EOG: 2, ECG: 1, MISC: 13, STIM: 3, CHPI: 9)\n",
       "    comps : list | 0 items\n",
       "    custom_ref_applied : NamedInt | 0 (FIFFV_MNE_CUSTOM_REF_OFF)\n",
       "    description : str | 36 items\n",
       "    dev_head_t : Transform | 3 items\n",
       "    dig : list | 137 items (3 Cardinal, 5 HPI, 75 EEG, 54 Extra)\n",
       "    events : list | 1 items\n",
       "    experimenter : str | 3 items\n",
       "    file_id : dict | 4 items\n",
       "    highpass : float | 0.0 Hz\n",
       "    hpi_meas : list | 1 items\n",
       "    hpi_results : list | 1 items\n",
       "    hpi_subsystem : dict | 2 items\n",
       "    line_freq : float | 50.0\n",
       "    lowpass : float | 356.3999938964844 Hz\n",
       "    meas_date : tuple | 1941-03-22 11:04:14 GMT\n",
       "    meas_id : dict | 4 items\n",
       "    nchan : int | 404\n",
       "    proc_history : list | 1 items\n",
       "    proj_id : ndarray | 1 items\n",
       "    proj_name : str | 11 items\n",
       "    projs : list | 0 items\n",
       "    sfreq : float | 1100.0 Hz\n",
       "    subject_info : dict | 2 items\n",
       "    acq_stim : NoneType\n",
       "    ctf_head_t : NoneType\n",
       "    dev_ctf_t : NoneType\n",
       "    device_info : NoneType\n",
       "    gantry_angle : NoneType\n",
       "    helium_info : NoneType\n",
       "    kit_system_id : NoneType\n",
       "    utc_offset : NoneType\n",
       "    xplotter_layout : NoneType\n",
       ">"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot_sensors(kind='topomap', ch_type='eeg');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing the data\n",
    "\n",
    "To access the data just use the [] syntax as to access any element of a list, dict etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(404, 10)\n",
      "(10,)\n"
     ]
    }
   ],
   "source": [
    "start, stop = 0, 10\n",
    "data, times = raw[:, start:stop]  # fetch all channels and the first 10 time points\n",
    "print(data.shape)\n",
    "print(times.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.        , 0.00090909, 0.00181818, 0.00272727, 0.00363636,\n",
       "       0.00454545, 0.00545455, 0.00636364, 0.00727273, 0.00818182])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "times  # always starts at 0 by convention"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `raw[]` returns both the data and the times array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Resampling the data\n",
    "\n",
    "We will now change the sampling frequency of the data to speed up the computations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Raw  |  sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif, n_channels x n_times : 404 x 147300 (491.0 sec), ~461.4 MB, data loaded>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.load_data()  # it is required to load data in memory\n",
    "raw.resample(300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's remove the unecessary channels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.drop_channels?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "to_drop = ['STI201', 'STI301', 'MISC201', 'MISC202', 'MISC203',\n",
    "           'MISC204', 'MISC205', 'MISC206', 'MISC301', 'MISC302',\n",
    "           'MISC303', 'MISC304', 'MISC305', 'MISC306', 'CHPI001',\n",
    "           'CHPI002', 'CHPI003', 'CHPI004', 'CHPI005', 'CHPI006',\n",
    "           'CHPI007', 'CHPI008', 'CHPI009']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Raw  |  sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif, n_channels x n_times : 381 x 147300 (491.0 sec), ~435.5 MB, data loaded>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.drop_channels(to_drop)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Info | 27 non-empty fields\n",
       "    acq_pars : str | 21833 items\n",
       "    bads : list | 0 items\n",
       "    ch_names : list | MEG0113, MEG0112, MEG0111, MEG0122, MEG0123, MEG0121, ...\n",
       "    chs : list | 381 items (GRAD: 204, MAG: 102, EEG: 70, EOG: 2, ECG: 1, MISC: 1, STIM: 1)\n",
       "    comps : list | 0 items\n",
       "    custom_ref_applied : NamedInt | 0 (FIFFV_MNE_CUSTOM_REF_OFF)\n",
       "    description : str | 36 items\n",
       "    dev_head_t : Transform | 3 items\n",
       "    dig : list | 137 items (3 Cardinal, 5 HPI, 75 EEG, 54 Extra)\n",
       "    events : list | 1 items\n",
       "    experimenter : str | 3 items\n",
       "    file_id : dict | 4 items\n",
       "    highpass : float | 0.0 Hz\n",
       "    hpi_meas : list | 1 items\n",
       "    hpi_results : list | 1 items\n",
       "    hpi_subsystem : dict | 2 items\n",
       "    line_freq : float | 50.0\n",
       "    lowpass : float | 150.0 Hz\n",
       "    meas_date : tuple | 1941-03-22 11:04:14 GMT\n",
       "    meas_id : dict | 4 items\n",
       "    nchan : int | 381\n",
       "    proc_history : list | 1 items\n",
       "    proj_id : ndarray | 1 items\n",
       "    proj_name : str | 11 items\n",
       "    projs : list | 0 items\n",
       "    sfreq : float | 300.0 Hz\n",
       "    subject_info : dict | 2 items\n",
       "    acq_stim : NoneType\n",
       "    ctf_head_t : NoneType\n",
       "    dev_ctf_t : NoneType\n",
       "    device_info : NoneType\n",
       "    gantry_angle : NoneType\n",
       "    helium_info : NoneType\n",
       "    kit_system_id : NoneType\n",
       "    utc_offset : NoneType\n",
       "    xplotter_layout : NoneType\n",
       ">"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing raw data\n",
    "\n",
    "See https://mne.tools/0.16/auto_tutorials/plot_visualize_raw.html\n",
    "for more details.\n",
    "\n",
    "Let's look at how to:\n",
    "- browse data\n",
    "- turn On/Off the PCA/SSP projections\n",
    "- mark bad segments to obtained annotations\n",
    "- group channel by types\n",
    "- group channel by location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "\n",
    "raw.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>Do you see any bad channel?</li>\n",
    "    <li>Do you see any bad segment of data?</li>\n",
    "    <li>Do you see any more then EOG blinks?</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Annotations  |  1 segment : BAD_FLUX_JUMP (1), orig_time : 1941-03-22 11:04:14.988669>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.annotations.save('annot.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "onset,duration,description\r\n",
      "1941-03-22 11:08:22.697562,0.5860655737704974,BAD_FLUX_JUMP\r\n"
     ]
    }
   ],
   "source": [
    "!cat annot.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.filter?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>Filter the raw data between 0Hz and 40Hz.</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Raw  |  sub-01_ses-meg_task-facerecognition_run-01_proc-sss_meg.fif, n_channels x n_times : 381 x 147300 (491.0 sec), ~435.5 MB, data loaded>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.filter(0, 40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.filter?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>Plot the 10 first seconds of stimulation channel just using matplotlib.</li>\n",
    "    </ul>\n",
    "</div>\n",
    "\n",
    "Tips:\n",
    "\n",
    "- Pick the stim channel using `mne.pick_types`\n",
    "- Get the data for this channel\n",
    "- Plot it using `plt.plot`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.385637307611377e-10"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# d = raw.get_data(picks=['EEG001', 'EEG002'])\n",
    "# d = raw.get_data(picks=('eeg', 'mag'))\n",
    "d = raw.get_data(picks=('grad',))\n",
    "np.max(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 15000)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#### TODO\n",
    "start = 0\n",
    "stop = int(50 * raw.info['sfreq'])\n",
    "data = raw.get_data('STI101', start=start, stop=stop)\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(15000,)"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.times[start:stop].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0xa569bd950>]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plt.plot(raw.times[start:stop], data.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define and read epochs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First extract events:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "259 events found\n",
      "Event IDs: [   5    6    7   13   14   15   17   18   19  256  261  262  263  269\n",
      "  270  271  273  274  275 4096 4101 4102 4103 4109 4110 4111 4113 4114\n",
      " 4115 4352]\n"
     ]
    }
   ],
   "source": [
    "events = mne.find_events(raw, stim_channel='STI101', verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>What is the type of the variable events?</li>\n",
    "    <li>What is the meaning of the 3 columnes of events?</li>\n",
    "    <li>How many events of type 5 do you see?</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# events(:, 3) == 5 # matlab\n",
    "np.sum(events[:, 2] == 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There was a time offset of 34.5ms in the stimulus presentation. We need to correct events accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "delay = int(round(0.0345 * raw.info['sfreq']))\n",
    "events[:, 0] = events[:, 0] + delay"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's visualize the paradigm:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "events = events[events[:, 2] < 20] # take only events with code less than 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = mne.viz.plot_events(events, raw.info['sfreq']);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For event trigger and conditions we use a Python dictionary with keys that contain \"/\" for grouping sub-conditions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "event_id = {\n",
    "    'face/famous/first': 5,\n",
    "    'face/famous/immediate': 6,\n",
    "    'face/famous/long': 7,\n",
    "    'face/unfamiliar/first': 13,\n",
    "    'face/unfamiliar/immediate': 14,\n",
    "    'face/unfamiliar/long': 15,\n",
    "    'scrambled/first': 17,\n",
    "    'scrambled/immediate': 18,\n",
    "    'scrambled/long': 19,\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = mne.viz.plot_events(events, sfreq=raw.info['sfreq'],\n",
    "                          event_id=event_id);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot(event_id=event_id, events=events);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define epochs parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmin = -0.5  # start of each epoch (500ms before the trigger)\n",
    "tmax = 2.0  # end of each epoch (2000ms after the trigger)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the baseline period:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "baseline = (-0.2, 0)  # means from 200ms before to stim onset (t = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define peak-to-peak (amplitude range) rejection parameters for gradiometers, magnetometers and EOG:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "reject = dict(grad=4000e-13, mag=4e-12, eog=150e-6)  # this can be highly data dependent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "    <b>REMARK</b>:\n",
    "     <ul>\n",
    "    <li>The <a href=\"https://autoreject.github.io/\">autoreject</a> project aims to solve this problem of reject parameter setting. See the <a href=\"https://www.sciencedirect.com/science/article/pii/S1053811917305013\">paper</a>.</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we are picky again, this time with EOG\n",
    "picks = mne.pick_types(raw.info, meg=True, eeg=True, eog=True,\n",
    "                       stim=False, exclude='bads')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,\n",
       "        13,  14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,\n",
       "        26,  27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,\n",
       "        39,  40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,\n",
       "        52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,\n",
       "        65,  66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,\n",
       "        78,  79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,\n",
       "        91,  92,  93,  94,  95,  96,  97,  98,  99, 100, 101, 102, 103,\n",
       "       104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,\n",
       "       117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,\n",
       "       130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,\n",
       "       143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155,\n",
       "       156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168,\n",
       "       169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n",
       "       182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194,\n",
       "       195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207,\n",
       "       208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220,\n",
       "       221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233,\n",
       "       234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246,\n",
       "       247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259,\n",
       "       260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,\n",
       "       273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285,\n",
       "       286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298,\n",
       "       299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311,\n",
       "       312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324,\n",
       "       325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337,\n",
       "       338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350,\n",
       "       351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,\n",
       "       364, 365, 366, 367, 370, 371, 372, 373, 374, 375, 376, 377, 378,\n",
       "       379])"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "picks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract epochs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,\n",
    "                    picks=picks, baseline=baseline,\n",
    "                    reject=reject)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Epochs  |   146 events (good & bad), -0.5 - 2 sec, baseline [-0.2, 0], ~7.4 MB, data not loaded,\n",
      " 'face/famous/first': 25\n",
      " 'face/famous/immediate': 10\n",
      " 'face/famous/long': 14\n",
      " 'face/unfamiliar/first': 25\n",
      " 'face/unfamiliar/immediate': 12\n",
      " 'face/unfamiliar/long': 10\n",
      " 'scrambled/first': 25\n",
      " 'scrambled/immediate': 14\n",
      " 'scrambled/long': 11>\n"
     ]
    }
   ],
   "source": [
    "print(epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   79 events (all good), -0.5 - 2 sec, baseline [-0.2, 0], ~7.4 MB, data not loaded,\n",
       " 'face/famous/first': 13\n",
       " 'face/famous/immediate': 3\n",
       " 'face/famous/long': 6\n",
       " 'face/unfamiliar/first': 17\n",
       " 'face/unfamiliar/immediate': 4\n",
       " 'face/unfamiliar/long': 6\n",
       " 'scrambled/first': 15\n",
       " 'scrambled/immediate': 9\n",
       " 'scrambled/long': 6>"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs.drop_bad()  # remove bad epochs based on reject"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   79 events (all good), -0.5 - 2 sec, baseline [-0.2, 0], ~178.5 MB, data loaded,\n",
       " 'face/famous/first': 13\n",
       " 'face/famous/immediate': 3\n",
       " 'face/famous/long': 6\n",
       " 'face/unfamiliar/first': 17\n",
       " 'face/unfamiliar/immediate': 4\n",
       " 'face/unfamiliar/long': 6\n",
       " 'scrambled/first': 15\n",
       " 'scrambled/immediate': 9\n",
       " 'scrambled/long': 6>"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs.load_data()  # load data in memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explore the epochs namespace\n",
    "\n",
    "Hit ``epochs.<TAB>``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# epochs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See how epochs were dropped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs.plot_drop_log();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "['EOG062']\n",
      "['EOG062']\n",
      "[]\n",
      "['EOG062']\n",
      "[]\n",
      "[]\n",
      "['EOG062']\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "for drop_log in epochs.drop_log[:20]:\n",
    "    print(drop_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs.copy().drop(10, reason=\"I don't like this one\").plot_drop_log();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[],\n",
       " [],\n",
       " [],\n",
       " [],\n",
       " [],\n",
       " [],\n",
       " [],\n",
       " ['EOG062'],\n",
       " ['EOG062'],\n",
       " [],\n",
       " ['EOG062'],\n",
       " [],\n",
       " [],\n",
       " ['EOG062'],\n",
       " [\"I don't like this one\"],\n",
       " [],\n",
       " [],\n",
       " [],\n",
       " [],\n",
       " []]"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs.copy().drop(10, reason=\"I don't like this one\").drop_log[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Info | 27 non-empty fields\n",
       "    acq_pars : str | 21833 items\n",
       "    bads : list | 0 items\n",
       "    ch_names : list | MEG0113, MEG0112, MEG0111, MEG0122, MEG0123, MEG0121, ...\n",
       "    chs : list | 381 items (GRAD: 204, MAG: 102, EEG: 70, EOG: 2, ECG: 1, MISC: 1, STIM: 1)\n",
       "    comps : list | 0 items\n",
       "    custom_ref_applied : NamedInt | 0 (FIFFV_MNE_CUSTOM_REF_OFF)\n",
       "    description : str | 36 items\n",
       "    dev_head_t : Transform | 3 items\n",
       "    dig : list | 137 items (3 Cardinal, 5 HPI, 75 EEG, 54 Extra)\n",
       "    events : list | 1 items\n",
       "    experimenter : str | 3 items\n",
       "    file_id : dict | 4 items\n",
       "    highpass : float | 0.0 Hz\n",
       "    hpi_meas : list | 1 items\n",
       "    hpi_results : list | 1 items\n",
       "    hpi_subsystem : dict | 2 items\n",
       "    line_freq : float | 50.0\n",
       "    lowpass : float | 40.0 Hz\n",
       "    meas_date : tuple | 1941-03-22 11:04:14 GMT\n",
       "    meas_id : dict | 4 items\n",
       "    nchan : int | 381\n",
       "    proc_history : list | 1 items\n",
       "    proj_id : ndarray | 1 items\n",
       "    proj_name : str | 11 items\n",
       "    projs : list | 0 items\n",
       "    sfreq : float | 300.0 Hz\n",
       "    subject_info : dict | 2 items\n",
       "    acq_stim : NoneType\n",
       "    ctf_head_t : NoneType\n",
       "    dev_ctf_t : NoneType\n",
       "    device_info : NoneType\n",
       "    gantry_angle : NoneType\n",
       "    helium_info : NoneType\n",
       "    kit_system_id : NoneType\n",
       "    utc_offset : NoneType\n",
       "    xplotter_layout : NoneType\n",
       ">"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw.filter(0, 80)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(146, 3)"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(79, 3)"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs.events.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True],\n",
       "       [ True,  True,  True]])"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events[epochs.selection] == epochs.events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wait a second, did we just loose half of our epochs due to EOG???\n",
    "\n",
    "We can probably do better. Let's use the PCA-based signal space projection (SSP) to regress out spatial patterns related to EOG and other offenders, ie., ECG.\n",
    "\n",
    "Here is the workflow, we'll first detect EOG artifacts and visualize their impact. Then we'll compute related spatial patterns to mitigate these artifacts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Figure size 800x420 with 7 Axes>,\n",
       " <Figure size 800x420 with 7 Axes>,\n",
       " <Figure size 800x420 with 7 Axes>]"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can use a convenience function\n",
    "eog_epochs = mne.preprocessing.create_eog_epochs(raw.copy().filter(1, None))\n",
    "eog_epochs.average().plot_joint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot(events=eog_epochs.events);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare the y axes to the ERF/ERPs we just saw. We face important degrees of contamination!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "projs_eog, _ = mne.preprocessing.compute_proj_eog(\n",
    "    raw, n_mag=3, n_grad=3, n_eeg=3, average=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Projection  |  EOG-planar--0.200-0.200-PCA-01, active : False, n_channels : 204>,\n",
       " <Projection  |  EOG-planar--0.200-0.200-PCA-02, active : False, n_channels : 204>,\n",
       " <Projection  |  EOG-planar--0.200-0.200-PCA-03, active : False, n_channels : 204>,\n",
       " <Projection  |  EOG-axial--0.200-0.200-PCA-01, active : False, n_channels : 102>,\n",
       " <Projection  |  EOG-axial--0.200-0.200-PCA-02, active : False, n_channels : 102>,\n",
       " <Projection  |  EOG-axial--0.200-0.200-PCA-03, active : False, n_channels : 102>,\n",
       " <Projection  |  EOG-eeg--0.200-0.200-PCA-01, active : False, n_channels : 70>,\n",
       " <Projection  |  EOG-eeg--0.200-0.200-PCA-02, active : False, n_channels : 70>,\n",
       " <Projection  |  EOG-eeg--0.200-0.200-PCA-03, active : False, n_channels : 70>]"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "projs_eog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "layouts = [mne.find_layout(raw.info, ch_type=ch) for ch in (\"eeg\", \"mag\", \"grad\")]\n",
    "mne.viz.plot_projs_topomap(projs_eog, layout=layouts);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the important question is how many components one should keep? Pro-tip: some of them don't look like clear artifact patterns. The good news is that we don't need to decide __*right*__ now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Figure size 800x420 with 7 Axes>,\n",
       " <Figure size 800x420 with 7 Axes>,\n",
       " <Figure size 800x420 with 7 Axes>]"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# same business, same issue for ECG\n",
    "ecg_epochs = mne.preprocessing.create_ecg_epochs(raw.copy().filter(1, None))\n",
    "ecg_epochs.average().plot_joint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also face important insults from the cardiac signal... we'll project that out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [],
   "source": [
    "projs_ecg, _ = mne.preprocessing.compute_proj_ecg(\n",
    "    raw, n_mag=3, n_grad=3, n_eeg=3, average=True)\n",
    "mne.viz.plot_projs_topomap(projs_ecg, layout=layouts);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   146 events (good & bad), -0.5 - 2 sec, baseline [-0.2, 0], ~7.6 MB, data not loaded,\n",
       " 'face/famous/first': 25\n",
       " 'face/famous/immediate': 10\n",
       " 'face/famous/long': 14\n",
       " 'face/unfamiliar/first': 25\n",
       " 'face/unfamiliar/immediate': 12\n",
       " 'face/unfamiliar/long': 10\n",
       " 'scrambled/first': 25\n",
       " 'scrambled/immediate': 14\n",
       " 'scrambled/long': 11>"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# now let's see how that would theoretically improve data preservation\n",
    "reject2 = dict(mag=reject['mag'], grad=reject['grad']) \n",
    "\n",
    "epochs_clean = mne.Epochs(raw, events, event_id, tmin, tmax, proj=False,\n",
    "                          picks=picks, baseline=baseline,\n",
    "                          preload=False,\n",
    "                          reject=reject2)\n",
    "\n",
    "epochs_clean.add_proj(projs_eog + projs_ecg)\n",
    "#epochs_clean.copy().apply_proj().average().plot(spatial_colors=True);  # apply projs on a copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs_clean.copy().average().plot(proj='interactive', spatial_colors=True);  # apply projs on a copy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "now we keep all trials, probably we also removed some good signals.\n",
    "we will postpone the selection of SSP vectors to later study the impact on\n",
    "source localization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "    <b>REMARK</b>:\n",
    "     <ul>\n",
    "    <li>MNE keeps SSP projections inside the info and allows to apply them later.</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's overwrite\n",
    "epochs = epochs_clean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>Use ICA instead of SSP to remove artifacts</li>\n",
    "    <li>What are potential benefits or disadvantages?</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization Epochs\n",
    "\n",
    "See [this page](https://mne.tools/stable/auto_tutorials/epochs/plot_visualize_epochs.html) for options on how to visualize epochs.\n",
    "\n",
    "Here is just an illustration to make a so-called ERP/ERF image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.plot_psd(fmax=40);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs.plot_image(picks='EEG065', sigma=1.);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.close('all')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The epochs object is your MNE swiss army knife for processing segmented data!\n",
    "\n",
    "- specialized methods for diagnostic plotting of data\n",
    "- averaging\n",
    "- saving\n",
    "- manipulating data, e.g., rearranging or deleting single trials, resampling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>How could you get the epochs corresponding to face?</li>\n",
    "    <li>How could you get the epochs corresponding to a familiar face?</li>\n",
    "    <li>How could you get the epochs corresponding to a scrambled face?</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'face/famous/first': 5,\n",
       " 'face/famous/immediate': 6,\n",
       " 'face/famous/long': 7,\n",
       " 'face/unfamiliar/first': 13,\n",
       " 'face/unfamiliar/immediate': 14,\n",
       " 'face/unfamiliar/long': 15,\n",
       " 'scrambled/first': 17,\n",
       " 'scrambled/immediate': 18,\n",
       " 'scrambled/long': 19}"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs.event_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   97 events (all good), -0.5 - 2 sec, baseline [-0.2, 0], ~7.6 MB, data not loaded,\n",
       " 'face/unfamiliar/first': 25\n",
       " 'face/unfamiliar/immediate': 12\n",
       " 'face/unfamiliar/long': 10\n",
       " 'scrambled/first': 25\n",
       " 'scrambled/immediate': 14\n",
       " 'scrambled/long': 11>"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs[['unfamiliar', 'scrambled']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## basic IO "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The standard scenario is saving the epochs into .fif file together with all the header data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/claire/work/data/ds000117-practical/derivatives/meg_derivatives/sub-01/ses-meg/meg/sub-01_ses-meg_task-facerecognition_run-01_proc-sss-epo.fif'"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs_fname = raw_fname.replace('_meg.fif', '-epo.fif')\n",
    "epochs_fname"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting existing file.\n",
      "Loading data for 1 events and 751 original time points ...\n",
      "Loading data for 145 events and 751 original time points ...\n"
     ]
    }
   ],
   "source": [
    "epochs.save(epochs_fname, overwrite=True)  # note that epochs are save in files ending with -epo.fif"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(145, 378, 751)"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = epochs.get_data()\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scipy also supports reading and writing of matlab files. You can save your single trials with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(145, 378, 751)\n"
     ]
    }
   ],
   "source": [
    "from scipy import io\n",
    "epochs_data = epochs.get_data()\n",
    "print(epochs_data.shape)\n",
    "io.savemat('epochs_data.mat', dict(epochs_data=epochs_data),\n",
    "           oned_as='row')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Average the epochs to get ERF/ERP and plot it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Evoked  |  '0.17 * face/famous/first + 0.07 * face/famous/immediate + 0.10 * face/famous/long + 0.17 * face/unfamiliar/first + 0.08 * face/unfamiliar/immediate + 0.07 * face/unfamiliar/long + 0.17 * scrambled/first + 0.10 * scrambled/immediate + 0.08 * scrambled/long' (average, N=145), [-0.5, 2] sec, 376 ch, ~9.6 MB>"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# refresh evoked\n",
    "evoked = epochs.average()\n",
    "evoked.del_proj()  # delete previous proj\n",
    "# take first for each sensor type\n",
    "evoked.add_proj(projs_eog[::3] + projs_ecg[::3])\n",
    "evoked.apply_proj()  # apply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Evoked  |  '0.17 * face/famous/first + 0.07 * face/famous/immediate + 0.10 * face/famous/long + 0.17 * face/unfamiliar/first + 0.08 * face/unfamiliar/immediate + 0.07 * face/unfamiliar/long + 0.17 * scrambled/first + 0.10 * scrambled/immediate + 0.08 * scrambled/long' (average, N=145), [-0.5, 2] sec, 376 ch, ~9.6 MB>\n"
     ]
    }
   ],
   "source": [
    "#evoked = epochs.average()\n",
    "print(evoked)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.close('all')\n",
    "evoked.plot(proj=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also show sensor position as line color:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked.plot(spatial_colors=True, proj=True);  # note the legend"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times = [0.0, 0.1, 0.18]\n",
    "evoked.plot_topomap(ch_type='mag', times=times, proj=True);\n",
    "evoked.plot_topomap(ch_type='grad', times=times, proj=True);\n",
    "evoked.plot_topomap(ch_type='eeg', times=times, proj=True);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "# pure topography plots called topomap in the MNE jargon\n",
    "for ch_type in ('mag', 'grad', 'eeg'):\n",
    "    evoked.plot_topomap(times=np.linspace(0.05, 0.45, 8),\n",
    "                        ch_type=ch_type, proj=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>Exercise</b>:\n",
    "     <ul>\n",
    "    <li>How does SSP impact the evoked responses? Use proj=\"interactive\" to explore</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For example, let's say that we want to keep the first projs for now\n",
    "\n",
    "# refresh evoked\n",
    "evoked = epochs.average()\n",
    "evoked.del_proj()  # delete previous proj\n",
    "# take first for each sensor type\n",
    "evoked.add_proj(projs_eog[::3] + projs_ecg[::3])\n",
    "evoked.apply_proj()  # apply"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Topoplot and time series can also be shown in one single plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked.plot_joint(times=[0.17]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing and indexing epochs by condition\n",
    "\n",
    "Epochs can be indexed by integers or slices to select a subset of epochs but also with strings to select by conditions `epochs[condition]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   1 events (all good), -0.5 - 2 sec, baseline [-0.2, 0], ~7.6 MB, data not loaded,\n",
       " 'face/unfamiliar/first': 1>"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs[0]  # first epoch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   10 events (all good), -0.5 - 2 sec, baseline [-0.2, 0], ~7.6 MB, data not loaded,\n",
       " 'face/famous/first': 3\n",
       " 'face/unfamiliar/first': 4\n",
       " 'face/unfamiliar/immediate': 1\n",
       " 'face/unfamiliar/long': 1\n",
       " 'scrambled/first': 1>"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs[:10]  # first 10 epochs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Epochs  |   95 events (all good), -0.5 - 2 sec, baseline [-0.2, 0], ~7.6 MB, data not loaded,\n",
       " 'face/famous/first': 24\n",
       " 'face/famous/immediate': 10\n",
       " 'face/famous/long': 14\n",
       " 'face/unfamiliar/first': 25\n",
       " 'face/unfamiliar/immediate': 12\n",
       " 'face/unfamiliar/long': 10>"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "epochs['face']  # epochs for a face"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In event_id, \"/\" selects conditions in a hierarchical way, e.g. here, \"face\" vs. \"scrambled\", \"famous\" vs. \"unfamiliar\", and MNE can select them individually"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs['face'].average().\\\n",
    "    pick_types(meg='grad').crop(-0.1, 0.25).plot(spatial_colors=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply this to visualize all the conditions in `event_id`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.close('all')\n",
    "for condition in ['face', 'scrambled']:\n",
    "    epochs[condition].average().plot_topomap(times=[0.1, 0.15], title=condition);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write evoked data to disk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/claire/work/data/ds000117-practical/derivatives/meg_derivatives/sub-01/ses-meg/meg/sub-01_ses-meg_task-facerecognition_run-01_proc-sss-ave.fif'"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evoked_fname = raw_fname.replace('_meg.fif', '-ave.fif')\n",
    "evoked_fname"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked.save(evoked_fname)  # note that the file for evoked ends with -ave.fif"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or to write multiple conditions in 1 file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "evokeds_list = [epochs[k].average() for k in event_id]  # get evokeds\n",
    "mne.write_evokeds(evoked_fname, evokeds_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading evoked from disk\n",
    "\n",
    "It is also possible to read evoked data stored in a fif file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "evokeds_list = mne.read_evokeds(evoked_fname, baseline=(None, 0), proj=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or give the explicit name of the averaged condition:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked1 = mne.read_evokeds(evoked_fname, condition=\"face/famous/first\",\n",
    "                           baseline=(None, 0), proj=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Remark:** Did you notice that you can apply some preprocessing on reading the evokeds from disk?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute a contrast:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked_face = epochs['face'].average()\n",
    "evoked_scrambled = epochs['scrambled'].average()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "contrast = mne.combine_evoked([evoked_face, evoked_scrambled], [0.5, -0.5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this combines evokeds taking into account the number of averaged epochs (to scale the noise variance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24\n",
      "131.03448275862067\n"
     ]
    }
   ],
   "source": [
    "print(evoked1.nave)  # average of 12 epochs\n",
    "print(contrast.nave)  # average of 116 epochs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Evoked  |  '0.500 * 0.25 * face/famous/first + 0.11 * face/famous/immediate + 0.15 * face/famous/long + 0.26 * face/unfamiliar/first + 0.13 * face/unfamiliar/immediate + 0.11 * face/unfamiliar/long + -0.500 * 0.50 * scrambled/first + 0.28 * scrambled/immediate + 0.22 * scrambled/long' (average, N=131.03448275862067), [-0.5, 2] sec, 376 ch, ~9.8 MB>\n"
     ]
    }
   ],
   "source": [
    "print(contrast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = contrast.copy().pick('grad').crop(-0.1, 0.3).plot_joint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Evoked  |  '0.25 * face/famous/first + 0.11 * face/famous/immediate + 0.15 * face/famous/long + 0.26 * face/unfamiliar/first + 0.13 * face/unfamiliar/immediate + 0.11 * face/unfamiliar/long' (average, N=95), [-0.5, 2] sec, 376 ch, ~9.8 MB>"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evoked_face"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Evoked  |  '0.50 * scrambled/first + 0.28 * scrambled/immediate + 0.22 * scrambled/long' (average, N=50), [-0.5, 2] sec, 376 ch, ~9.8 MB>"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evoked_scrambled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save your figure as pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "import numpy as np\n",
    "contrast.plot_topomap(times=np.linspace(0.05, 0.15, 5), ch_type='mag')\n",
    "plt.savefig('toto.pdf')\n",
    "!open toto.pdf  # works only on a mac"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    <b>EXERCISE</b>:\n",
    "     <ul>\n",
    "      <li>Compute the evoked data for 'famous', 'unfamiliar', 'scrambled' faces</li>\n",
    "      <li>Crop the data between -0.2s and 0.4s</li>     \n",
    "      <li>Plot the channel EEG065 in all 3 conditions using mne.viz.plot_compare_evokeds function</li>\n",
    "    </ul>\n",
    "</div>\n",
    "\n",
    "See: https://mne.tools/stable/generated/mne.viz.plot_compare_evokeds.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked_famous = epochs['famous'].average().crop(-0.1, 0.4)\n",
    "evoked_scrambled = epochs['scrambled'].average().crop(-0.1, 0.4)\n",
    "evoked_unfamiliar = epochs['unfamiliar'].average().crop(-0.1, 0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.close('all')\n",
    "mne.viz.plot_evoked_topo([evoked_famous, evoked_scrambled, evoked_unfamiliar]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evokeds = {k:epochs[k].average().crop(-0.1, 0.4)\n",
    "           for k in ['famous', 'unfamiliar', 'scrambled']}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.close('all')\n",
    "mne.viz.plot_compare_evokeds(evokeds, picks='EEG065');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ADVANCED: Customize your plots\n",
    "\n",
    "Want to have every text in blue?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib as mpl\n",
    "fig = evoked1.plot(show=False)  # butterfly plots\n",
    "fig.subplots_adjust(hspace=1.0)\n",
    "for text in fig.findobj(mpl.text.Text):\n",
    "    text.set_fontsize(18)\n",
    "    text.set_color('blue')\n",
    "for ax in fig.get_axes():\n",
    "    ax.axvline(0., color='red', linestyle='--')\n",
    "plt.tight_layout()\n",
    "fig.savefig('plot_erf.pdf');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}