Agenda.txt

// RELEASE NOTES
// ==========================

20170814
--------
+ Fix new bug configuring only nidq stream.

20170808
--------
+ Fix bug in TTL trigger.
+ Begin hidden multistream work.

20170724
--------
+ Improve look and abort function of run startup dialog.
+ Fix bug in Timmed trigger when gate is closed-reopened.
+ Fix bug setting audio hipass filter from MATLAB.
+ Add MATLAB SetMetaData function.

20170501
--------
+ Audio output for either stream.
+ Smoother quitting of runs upon Imec errors.
+ Menu items to Explore run dir, app dir.
+ Force gets 'Explore ImecProbeData' button.
+ Force '11 -> 9' button sets probe option.
+ FileViewer gets channel submenus.
+ Add MATLAB GettingStarted.txt.
+ Improve 'Metadata.html' discussion.

20170315
--------
+ Fix FileViewer bug drawing nidq aux channels.
+ Current <G T> file labels added to Graphs window.
+ Better graph label legibility.
+ Better BinMax (spike drawing) signal-to-noise.
+ Include 'Metadata.html' explaining tags.
+ Include MATLAB code for parsing output files.
+ Remember screen layout.

20161201
--------
+ Prettier dialogs that resize:
+    Config, IM-Force, IMROTable, ShankMaps, ChanMaps,
+    CmdServer, RemoteServer, Help,
+    AO, FVWOptions, BIST, PAR2.
+ Add shank viewer help text.
+ Graphs get save and AO indicators.
+ Right-click graphs to edit sort order.
+ Right-click graphs to edit saved channels.
+ Right-click graphs to colorize TTL events.
+ Sort graphs along shank: forward or reverse order.
+ User notes field on Config::Save tab.
+ User notes in metadata and FileViewer.
+ FileViewer Zoom in/out key commands.
+ FileViewer Zoom by ctrl-click & drag.
+ FileViewer Set Export Range now shift-click & drag.
+ FileViewer gets option to manually update graphs.
+ Extend TTL triggering to imec and nidq digital lines.
+ Improved graph time axis accuracy.

20161101
--------
+ Increase FVW YScale to 999.
+ Gray config items if stream disabled.
+ Improve ShankMap editors.
+ Add ShankMap viewers.

20160806
--------
+ Fix enabled state of "Edit On/Off" R-click item.
+ Fix filters to obey order: Flt:<T>:<S>:BinMax.

20160805
--------
+ More predictable file starts in Timed trigger mode.
+ Rename graphing option "-DC" to "-<T>".
+ Add -<T> option to online NI stream.
+ Add default ShankMap backing spatial channel averaging.
+ Add -<S> option to online streams.
+ Add -<S> option to offline viewers.
+ Split Config SNS tab into Map and Save tabs.
+ Add ShankMap file support.
+ Add ShankMap editors.
+ Can order ChanMap according to ShankMap.
+ Imec internal refs: UI to turn off option 3 chans.
+ Right-click graphs to edit imro table or on/off.
+ Make sure R-click changes update ShankMap.

20160703
--------
+ Imec v4.3.
+ Fix spelling error.
+ Update doc texts.
+ Better ChanMap editor.
+ FVW can hide all.
+ FVW can edit ChanMap.
+ FVW better channel selection.
+ FVW fix export chanMap.
+ FVW fix export time span.

20160701
--------
+ Clean up Imec startup.
+ Fix AO initialization.
+ Correct MATLAB docs.
+ Better default server addresses.
+ Meta file written before and after.
+ Meta gets firstCt time stamp.
+ File viewer opens any stream type.
+ Better FVW UI for graph and time ranges.
+ Faster FVW param setters and dragging.
+ FVW linking.
+ FVW help text.

20160601
--------
+ MATLAB: GetFileStartCount.
+ Fix TTL trigger bug.

20160515
--------
+ Imec v4.2.
+ Add option to disable LEDs.
+ Imec diagnostics (BIST) dialog.
+ Update Imec start sequence.
+ More AO filter options.
+ Fix AO deadlock.
+ Render digital channels.
+ Nicer default values.

20160511
--------
+ Update help text.

20160505
--------
+ Add BinMax checkbox.
+ Fix AO dialog issue.
+ Split Imec data files.
+ Gate start waits for sample arrival.
+ Improve stream sync in all triggers.
+ Add recording time display.

20160502
--------
+ Retire DataFile::badData.
+ Fix bug in TTL trigger type.
+ Better handling of "?" button.
+ Add remote sort order query.

20160501
--------
+ ADC/gain handling for broken EEPROMs.
+ More accurate time displays.

20160408
--------
+ Imec support at v4.1.
+ (ADC calib, optional gain calib, sync input).
+ Imec setup panel gets force identity button.

20160404
--------
+ Reenable spike and TTL triggers.
+ Recast trigger pause as gate override.
+ MATLAB: SetTrgEnable -> SetRecordingEnable.

20160401
--------
+ Imec support at v3.5 (ADC calibration).
+ Detect if USB NI device connected.
+ Multiple SpikeGLX can run but one owns NI.
+ Errors bring console window to top.
+ Config dialog gets disk space calculator.

20160305
--------
+ Fix timing issue with USB NI-DAQ.
+ Disable Imec ref disconnection.
+ Flesh out SN field of imro table.
+ Add trigger stream selector.
+ Prelim work on spike trigger, still disabled.

20160304
--------
+ Imec support at v3.4 (faster startup, improved reference signal routing).
+ Fix run startup bug.
+ Fix DC subtraction bug.
+ Fix NI digital channel issue.
+ Message if set bank out of range.
+ Right-click to edit Imec settings.

20160302
--------
+ Correctly set Bank 0.
+ Start including SpikeGLX-MATLAB-SDK in release.

20160301
--------
+ Imec support.
+ Start releasing SpikeGLX.exe and SpikeGLX_NISIM.exe.
+ SpikeGLX folder requires "libNeuropix_basestation_api.dll".
+ Configuration dialog gets {Devices, IM Setup} Tabs.
+ Updated run pause/resume options moved from Triggers Tab to Gates Tab.
+ Data are processed in two parallel "streams": Imec stream, nidq stream.
+ Each stream generates own datafiles {.imec.bin, .imec.meta} {.nidq.bin, .nidq.meta}.
+ Some renaming of metadata items to distinguish stream type.
+ Each stream gets own channel map and own save-channel string.
+ Each stream gets own Graphs window viewer pane.
+ All graphing uses pages of stacked traces.
+ New Graphs window Run Toolbar layout.
+ NI viewer gets low-pass filter option.
+ MATLAB API modified to fetch from either imec or nidq stream.
+ Online help text rewritten.

Temporarily disabled:
- Spike trigger option.
- TTL trigger option.
- AO for Imec channels.
- Offline file viewer for Imec files.
- Graphs window save-channel checkboxes.

20160101
--------
- Introduces "multi-trace" MGraph class set and a revised FVW to use, test
and develop that. MGraph is a QGLWidget (or QOpenGLWidget) that draws
unlimited numbers of traces in a vertical stack arrangement. MGraph can
be child of any QWidget, or can be used as the viewport of an MScroll
(subclasses QAbstractScrollArea) as we are in FVW.
- Fix bug: SpikeGLX hangs on remote setTrgEnable command.
- Fix bug: Remote setRunName doesn't handle _g_t tags correctly.
- Data files now get double extension '.nidq.bin', '.nidq.meta'.
- Offline file viewer enabled.

20151231
--------
Function set very close to original SpikeGL. Major UI difference is disabled
fileViewerWindow (FVW) because the model of one GLGraph per channel creates
too many graphics objects. This version is effectively same as "prerelease"
versions distributed around Janelia and elsewhere.


// QT VERSIONS
// ==========================

SpikeGLX builds/runs successfully against following versions, with
noted issues. As of December 2015 all prereleases to Janelia and to
Imec partner labs ship with 53MinGW to support XP.

- 5.3.2 MinGW482 32bit
    OpenGL:     QGLWidget
    HTML:       Renders pandoc.css properly
    Targets:    WinOS XP, 7, 8, 10
    Remarks:    - Low graph count (poor sharing).
                - Slow creation/resizing with QGLWidget requires GraphPool.
                - FileViewerWindow::initCloseLbl() issues following benign
                    warning as explained there:
                    "void QWindow::setTransientParent(QWindow*)
                     QWidgetWindow(0x431c048, name = "QWidgetClassWindow")
                     must be a top level window."

- 5.4.2 MinGW491 32bit
    OpenGL:     QGLWidget or QOpenGLWidget
    HTML:       Renders pandoc.css properly
    Targets:    WinOS XP*, 7, 8, 10
    Remarks:    - *QOpenGLWidget very fast but steps on XP Aero theme.
                - Works in XP if disallow Aero theme in Compatibility Settings.

- 5.5.1 MinGW492 32bit
    OpenGL:     QGLWidget or QOpenGLWidget
    HTML:       pandoc.css not rendering <pre><code> blocks properly**
    Targets:    WinOS XP*, 7, 8, 10
    Remarks:    - HTML CSS support degraded between version 5.4 and 5.5.
                - *QOpenGLWidget very fast but steps on XP Aero theme.
                - Works in XP if disallow Aero theme in Compatibility Settings.

- 5.6.0 Beta {MinGW492 32bit, MSVC2014 32bit, MSVC2014 64bit}
    OpenGL:     QGLWidget or QOpenGLWidget
    HTML:       pandoc.css not rendering <pre><code> blocks properly**
    Targets:    WinOS 7, 8, 10
    Remarks:    - HTML CSS support degraded between version 5.4 and 5.5.
                - XP not supported.
                - Get benign startup warning issuing before main():
                    "qt.qpa.gl: Unable to disable rotation."
                - Get warning on closing FVW (not yet resolved):
                    "External WM_DESTROY received for
                    QWidgetWindow(0x4b954c40, name="ExportDialogWindow"),
                    parent:  QWindow(0x0), transient parent:
                    QWidgetWindow(0x4b9549a0, name="FileViewerWindowClassWindow")"
                - 12/28/2015: Not yet built 64-bit against NI-DAQ.

// **
// BK: Suspicious change here:
// C:\Qt\5.3\Src\qtwebkit\Source\WebCore\rendering\...
// ...RenderBlockLineLayout.cpp::CheckMidpoints() ; line 426


// ISSUES
// ==========================

- NIDAQ digital input issue: Persisting from (at least) DAQmx 9.7.0
through latest 15.0.1: If acquire from 1 digital line CPU load is
very low, like 3% on my bench. If set 2 DI lines, jumps up to 6-7%.
By 4 DI lines I'm up to 30% CPU usage. These high loads occur only
if also acquiring from at least 1 AI channel. However, adding more
AI channels does not increase CPU usage at all. High load only if
using external clock. On Brian's bench (2X RAM) load is low up to
4 DI lines, but rises strongly as add DI lines. May be related to
mem usage in DAQmxReadDigitalLines which returns a byte per line
Reasons I use this are discussed @ CniAcqDmx::run().

- I use pandoc to translate markdown to html help text. I use:
    > pandoc --self-contained -s- c pandoc.css ...
which directly encodes css and img src data into tags like this:
    <link href="data:text/css;charset=utf-8,...thedata..."
The problem is that the html parser used in QTextBrowser does not
understand refs that directly encode data without a filename, so when
running under the Creator environment the Application Output pane logs:
    "QFSFileEngine::open: No file name specified"
for each embedded encoding.


// LOOSE ENDS
// ==========================

- Remote queries about dataFile objects are ill-advised or
(better?) forbidden.


// DOCUMENTATION
// ==========================

- Revisit all BK: notes.

- More structure docs.

- Review documentation of file formats.

- All docs must reflect updated system requirements:
    - RAM 4 GB minimum, 8+ GB preferred.
    - HD sustained write > 200 MB/s (SSD SATA 3 recommended).

- Notes on sustained write speeds:
    - Seagate barracuda 7200.11 (1.5TB) 120MB/s
    - WD 1600 AAJS 93.5 MB/s
    - SanDisk Extreme Pro SSD (240BG) 520MB/s


// UI IMPROVEMENT
// ==========================

-GW TODO:
    - Imec !filter currently adds (AP+LF)...but superposition better.

- IMEC TODO:
    - Imec can be run alone using software or hardware start, or get
        hardware start from nidaq (need to break D0 out from Whisper!).

- Perhaps add more windows {ShankView} to layout?

- Save/load parameter sets by name (also remotely). Perhaps a single
file or package? Perhaps a master catalog file pointing to others?

- Consider not just a command set for software but a script capability
to better handle timing among features/functions.


// NEW FEATURES
// ==========================

- Lee group asks for specified channel to act as reference for all others.

- Brian asks for FVW to output measurements like FFT or RMS on each channel.
Perhaps add an Analysis menu.

- GateCtl gets more modes?

- GateCtl should provide option to put acq hardware resets on gate
lo-hi and hi-lo edges. This is to counter analog intan saturation.

- Should allow configuring user digital-out on threshold crosses, like
evt saving modes with {aiChan, doChan, thresh, W, duration, indef}.

- Joao Couto suggests it could be nice to have offline viewer play back
recorded file at true speed, and, to allow remote interface to fetch data
for development simulations.

- Jeremy Cohen suggest two interesting trigger modes:
(1) Rate trigger (to detect activity periods w/ chronic implants) wherein
recording starts (perievtwindow) iff x-percent of given channel subset is
firing with rate > R.

(2) Ripple mode (sharp waves) wherein recording starts (perievtwindow)
iff x-percent of given channel subset is firing (not necessarily at a high
rate, but...) within a time window of width W.

- Consider HDF5 file format.


// MISC NOTES
// ==========================

- Compiling with C++11:
    - Try CONFIG += c++11
    - Try QMAKE_CXXFLAGS += -std=c++11

- Pixmap (xpm) editing notes:
(1) Copy existing .xpm to linux.
(2) Open .xpm in Inkscape, edit/save as .svg.
(3) Open .svg in GIMP, save as xpm.
(4) Open .xpm in kwrite, change array type to 'static const char * const'.

- Typical/high aux channel usage might be 6-12 analog in, 16 digital in.

- Use CniAcqDmx to monitor load. If nWhole count increasing beyond expected,
save restore states of {graph-pause, AO-enable}. This may help on systems
with fewer cores.

- Thread count is main load and best hardware is probably quad+ core.
As always, higher clock rate is better. Next issue is spooling data to
same disk as main OS activities. Better to have dedicated data drive.

- Nice test for cursor in window frame:
//    if( mywindow->frameGeometry().contains( QCursor::pos() )
//        && !mywindow->geometry().contains( QCursor::pos() ) ) {
//
//        // do something
//    }