Bark is a standard for electrophysiology data.
Version: 0.2
- minimal specification and implementation
- simple file formats
- small, chainable utilities
By emphasizing filesystem directories, plain text files and a common binary array format, Bark makes it easy to use both large external projects and simple command-line utilities.
Bark's small specification and Python implementation are easy to use in custom tools.
These tools can be chained together using GNU Make to build data pipelines.
Bark takes the architecture of ARF and replaces HDF5 with common data storage formats, the advantages of this approach are:
- Use standard Unix tools to explore your data (cd, ls, grep, find, mv)
- Build robust data processing pipelines with shell scripting or make.
- Edit metadata with a standard text editor
- Leverage any third party tools that use Bark's common data formats.
- Raw binary tools: Aplot, Neuroscope, Plexon Offline Sorter, Wave_clus, spyking circus, phy, sox.
- CSV tools: R, Pandas (Python), Excel, csvkit and Unix tools like sed and awk.
- Include non-standard data such as images or video in Bark entries.
Bark trees are made from the following elements:
- SampledData stored as raw binary arrays. Metadata is stored in another file with ".meta.yaml" appended to the dataset's filename.
- EventData stored in CSV files. As above, metadata is stored in a "X.meta.yaml" file.
- Entries (often trials) are directories containing Datasets that share a
common time base. These directories contain a
meta.yamlfile and any number of Datasets. - Root is a directory containing a collection of related Entries. Often a Root is used to contain all data from an experimental session.
This repository contains:
- The specification for bark (in specification.md)
- A python interface for reading and writing Bark files
- Tools to accomplish common Bark tasks
The python interface requires Python 3.5+. Installation with Conda is recommended.
git clone https://github.com/kylerbrown/bark
cd bark
git clone https://github.com/kylerbrown/resin
cd resin
pip install .
cd ..
pip install -r requirements.txt
pip install .
# optional tests
pytest -v
You'll also probably want to install Neuroscope. Sox is also useful.
Every command has help accessible with the flag -h (e.g. bark-entry -h).
bark-entry-- create entry directories for datasetsbark-attribute-- create or modify an attribute of a bark entry or datasetbark-column-attribute-- create or modify an attribute of a bark dataset columnbark-clean-orphan-metas-- remove orphan.meta.yamlfiles without associated data filesdat-select-- extract a subset of channels from a sampled datasetdat-join-- combine the channels of two or more sampled datasetsdat-split-- extract a subset of samples from a sampled datasetdat-cat-- concatenate sampled datasets, adding more samplesdat-filter-- apply zero-phase Butterworth or Bessel filters to a sampled datasetdat-decimate-- down-sample a sampled dataset by an integer factor, you want to low-pass filter your data first.dat-diff-- subtract one sampled dataset channel from anotherdat-ref-- for each channel: subtract the mean of all other channels, scaled by a coefficient such that the total power is minimizeddat-artifact-- removes sections of a sampled dataset that exceed a thresholddat-enrich-- concatenates subsets of a sampled dataset based on events in an events datasetdat-spike-detect-- detects spike events in the channels of a sampled datasetdat-envelope-classify-- classifies acoustic events, such as stimuli, by amplitude envelopedat-segment-- segments a sampled dataset based on a band of spectral power, as described in Koumura & Okanoya
There are many external tools for processing CSV files, including pandas and csvkit.
bark-scope-- opens a sampled data file in neuroscope. (Requires an installation of neuroscope)
Note for MacOS users: run this command in the terminal:
$ ln -s /Applications/neuroscope.app/Contents/MacOS/neuroscope /usr/local/bin/neuroscopebark-label-view-- Annotate or review events in relation to a sampled dataset, such as birdsong syllable labels on a microphone recording.
bark-db-- adds the metadata from a Bark tree to a databasebark-convert-rhd-- converts Intan .rhd files to datasets in a Bark entrybark-convert-openephys-- converts a folder of Open-Ephys .kwd files to datasets in a Bark entrybark-convert-arf-- converts an ARF file to entries in a Bark Rootcsv-from-waveclus-- converts a wave_clus spike time file to a CSVcsv-from-textgrid-- converts a praat TextGrid file to a CSVcsv-from-lbl-- converts an aplot lbl file to a CSVcsv-from-plexon-csv-- converts a Plexon OFS waveform CSV to a bark CSVdat-to-wave-clus-- convert a sampled dataset to a wave_clus compatible Matlab filedat-to-audio-- convert a sampled dataset to an audio file. Uses SOX under the hood, and so it can convert to any file type SOX supports.
bark-for-each-- apply a command to a list of Entries.
More tools with less generality can be found in the bark-extra repository.
import bark
root = bark.read_root("black5")
root.entries.keys()
# dict_keys(['2016-01-18', '2016-01-19', '2016-01-17', '2016-01-20', '2016-01-21'])
entry = root['2016-01-18']
entry.attrs
# {'bird': 'black5',
# 'experiment': 'hvc_syrinx_screwdrive',
# 'experimenter': 'kjbrown',
# 'timestamp': '2017-02-27T11:03:21.095541-06:00',
# 'uuid': 'a53d24af-ac13-4eb3-b5f4-0600a14bb7b0'}
entry.datasets.keys()
# dict_keys(['enr_emg.dat', 'enr_mic.dat', 'enr_emg_times.csv', 'enr_hvc.dat', 'raw.label', 'enr_hvc_times.csv', 'enr.label'])
hvc = entry['enr_hvc.dat']
hvc.data.shape
# (7604129, 3)The Stream object in the bark.stream module exposes a powerful data pipeline design system for sampled data.
Example usage:
Some links to get started with Make:
- http://kbroman.org/minimal_make/
- https://bost.ocks.org/mike/make/
- https://swcarpentry.github.io/make-novice/
- NEO https://github.com/NeuralEnsemble/python-neo
- NWB http://www.nwb.org/
- NIX https://github.com/G-Node/nix
- neuroshare (http://neuroshare.org) is a set of routines for reading and writing data in various proprietary and open formats.
Dan Meliza created ARF. Bark was was written by Kyler Brown so he could finish his damn thesis in 2017. Graham Fetterman also made significant contributions.