Skip to content

jmarkow/intan_frontend

Repository files navigation

Welcome (i.e. what does this do?)

This package is designed to work with data generated the by the Intan RHA/RHD series acquisition boards in MATLAB. The typical usage scenario is to run this script in MATLAB as a daemon to process files generated by the RHA or RHD acquisition systems. As files are generated, the software automatically extracts "relevant" data and discards "irrelevant" data. In Tim Gardner's lab, we use the software to only extract electrophysiology recorded during zebra finch vocalizations. Some other uses include extracting data aligned to stimulus playback.

The input is RHA/RHD acquisition board-generated data, and the output is automatically organized files read to be analyzed in MATLAB. It should be noted that using this toolbox requires a passing familiarity with MATLAB, as it is command window oriented (i.e. no pretty GUI).

===

Table of Contents

  1. Requirements
  2. Data types
  3. Directories
  4. Conventions
  5. Data extraction
  6. Extracted data structure
  7. Filename Options
  8. Script Options

###Requirements

This has been tested using MATLAB 2010A and later on Windows and Mac (Linux should be fine). You must have the markolab and zftftb toolboxes in your MATLAB path. The only MATLAB Toolbox required is the Signal Processing toolbox.

###Data types

So far the two data types that are guaranteed to work are files generated by the Intan RHA/RHD acquisition systems, which generated *.int and *.rnd files, respectively. The software can be extended to work with other file types, and Open Ephys is already in the works.

###Directories

First you need to create the following five directories:

/WHERE_I_STORE_DATA/staging/
/WHERE_I_STORE_DATA/staging/processed/
/WHERE_I_STORE_DATA/staging/unprocessed/
/WHERE_I_STORE_DATA/staging/unorganized/
/WHERE_I_STORE_DATA/intan_data/

Note that these can all be automatically generated using the script intan_frontend_mkdirs.m

Directory Notes
staging Contains all subdirectories used for processing "raw" Intan data
staging/processed/ Contains raw data files already processed
staging/unprocessed/ Data goes here to be processed
staging/unorganized/ Data is dumped here if it can't be parsed
intan_data/ Base directory for processed data

All of these directories can be customized (see script options).

###Conventions

By default, the software will attempt to parse metadata about your recording from the filename itself. This means that proper data extraction requires appropriate naming conventions. The only two required specifications are the ANIMAL ID and RECORDING ID. More precisely:

[ANIMALID]_[RECORDINGID]_[OPTION1]_[OPTION2]_[TIMESTAMP]

Underscore _ is the default delimiter, but this can be changed in the script options. The timestamp is automatically appended by the Intan acquisition software, so you are only responsible for the other options. An example base filename (i.e. without the timestamp) is:

lb119_LeftHVC_mic1adc_playback3adc

This means that the animal ID is lb119 the recording ID is LeftHVC and the final two options specify where task-relevant data can be found (more on these [here] (#filename-options)). The extracted data will be placed in:

intan_data/lb119/LeftHVC/YYYY-MM-DD

Where YYYY-MM-DD is the datestamp for the data (e.g. 2014-02-13).

###Data Extraction

When "relevant" data is found, it is extracted into the /intan_data/ directory with the following structure by default:

/ANIMAL_ID/RECORDING_ID/YYYY-MM-DD

Sub-directories of this directory contain different data types:

Sub-dir Data type
gif Spectrograms of audio
mat MATLAB files that containing all extracted data
wav *.wav files of audio

If the task is based on TTL signals or playback this will be appended to the basic sub-directory names for clarity, e.g. gif_playback would contain spectrograms of stimuli played to the animal, or mat_ttl would contain data aligned to TTL signals.

###Extracted Data Structure

Data contained in the gif and wav directories should be self-explanatory. The MATLAB files in the mat directory contain the following variables:

Variable Data type Description
ttl structure (fields: fs, data, time) Contains TTL trace
ephys structure (fields: fs, data, time, labels, ports) Contains ephys data
audio structure (fields: fs, data, time) Contains audio data
playback structure (fields: fs, data, time) Contains playback data
file_datenum MATLAB date number File first write datestamp

###Filename Options

Many relevant options can be specified in the base filename itself. If they are not specified, they can be passed using the script option parse_options (see script options for more details). Recall the format:

[ANIMALID]_[RECORDINGID]_[OPTION1]_[OPTION2]_[OPTIONX]

These options can be broken down into specifying the data type, source (RHD only), channel, and port (RHD only). The data type can be:

Type Notes
mic Audio source
ttl TTL source
playback playback source

This is followed by the channel then the source. Note that the channel can contain ranges indicated using a hyphen (e.g. 4-8 expands to 4,5,6,7,8) and multiple channels can be specified using any symbol other than the field_delimiter or bird_delimiter (e.g. 4-8+10 expands to 4,5,6,7,8,10 and 4-8+12-15+17 to 4,5,6,7,8,12,13,14,15,17). The data source can be (relevant only for RHD system):

Source Text Notes
ADC adc Acquisition board ADC
Digital input digin Acquisition board digital input
Digital output digout Acquisition board digital output
Ephys channel m Headstage ephys amplifier
AUX aux Headstage auxiliary input

So to indicate that you're recording audio data on ADC-0 and playback data on ADC-1 with animal LDR4 with recording ID NCM:

LD4_NCM_mic0adc_playback1adc

By default, audio and playback data will be saved in the MATLAB files, while spectrograms and wav files will only contain the audio data from ADC-0 (the script can generate spectrograms and wav files for the playback data by specifying script options).

Also, multiple animals can be specified using the bird_delimiter (see script options). For instance:

LD4_NCM_mic0adc_playback1adc&rm7_HVC_mic2adc_ttl4digin

This base filename would specify that animal ID 1 is LD4 recording ID 2 is NCM, animal ID 2 is rm7 and recording ID 2 is HVC. The various options specify where audio, playback, or TTL signals reside.

###Script Options

There are lots and lots of options to specify how things run "under-the-hood", e.g. the spectrogram parameters, sensitivity of vocalization detection, what directories to use, etc. All options are specified using parameter/value pairs (examples are given below the table). Here's the full list, organized thematically:

Parameter Description Default
song_band Lower and upper edge of song detection band [2e3 6e3]
song_len Song detection fft window (s) .005
song_overlap Song detection fft overlap (s) 0
song_ratio Song detection in:out power ratio threshold 2
song_pow Song detection power threshold .3
song_duration Song detection smoothing parameter (seconds) .8
song_thresh Song detection final threshold .2
folder_format Extracted data folder format (MATLAB datestring format) yyyy-mm-dd
delimiter Delimiter used to parse filename options _
bird_delimiter Delimiter used to parse different animals &
auto_delete_int Number of days to wait before automatically deleting processed "raw" data inf
disp_band Spectrogram display frequency band [1 10e3]
filtering High-pass audio signal for display and song detection (corner freq. in Hz, leave empty to turn this feature off) 300
audio_pad Amount of data to extract before and after song detection (seconds) 7
ttl_extract Extract data segments based on TTL high 0
ttl_skip Skip song and playback detection if TTL high 0
playback_extract Extract data segments based on playback 0
playback_skip Skip song detection if playback detected 0
playback_rmswin Computed RMS using specified window (seconds) .025
playback_thresh Playback RMS detection threshold .01
sleep_window Time window to separately collect sleep data (24 hr format) [22 7]
birdid Manually specify the animal ID (if empty auto-detect)
recid Manually specify the recording ID (if empty auto-detect)
parse_options Specify additional parse options using the file options syntax (use delimiter for multiple options)
sleep_fileinterval How often to start saving sleep data (minutes) 10
sleep_segment How much data to keep per file interval (seconds) 5
email_monitor E-mail user if file hasn't been created in N minutes (0 turns this feature off) 0
email_noisecut E-mail user if 60 Hz power exceeds this threshold (0 turns this feature off) 0
email_noiselen E-mail user of 60 Hz noise threshold exceeded for N seconds 0