IMOS Toolbox v2.6.6 - 1 May 2020

Release 2.6.6 is the Q3 release of the IMOS Toolbox.

This release contains mostly the first part of the Time Series Spike QC Functionality.

Changes since v2.5.5:

• Quality Control:

  • imosTimeSeriesSpikeQC ccb3435 to 93328a9

These commits introduce the general QC test and three different types of spike detection:

1. OTSU threshold
2. OTSU threshold with a SavGol Filter.
3. Hampel method.

The methods can only be applied to a set of variables at the moment in non-burst data. Time series with bogus time samples or burst data is still in the works. For more details, see the help on these functions. A soon to be merged section in the wiki will be available too.

• Misc Internal Improvements:

  • Renaming of some functions and folders e8cc77b
  • Preparation for burst-data processing in despiking tests:
  • Improvements in timeSamplingInfo for bogus time sampling schemes. c2ebbdd
  • Refactoring of combineStructFields and extra tests. f0dda8e

These changes are small fixes in name conventions, some preparation for some time sampling processing, and detection of bogus data in time variables required for burst-data despiking tests.

• Version Bump & binary updates 19f6d22

IMOS Toolbox v2.6.5 - 28 Jan 2019

Release 2.6.5 is the Q2 release of the IMOS Toolbox.

Changes since v2.5.4:

Metadata:

• Now we use time_deployment_[start/end] instead of time_coverage_[start/end] global attribute to name the netcdf files. 7a4f33a

Data Ingestion:

• Staroddi Parser - New Parser to handle all Star-oddi instruments. Support is now provided to both old and new versions, in a transparent way to the user.
  2e065d4 , 0b1b08a, 0f95a62, f47bc47, 096adc7

• Voltage names - report correct names/metadata for voltages. 82a71bd

• Nortek Signature parser - Refactoring to allow different Id's and different sampling scheme (ping interval == burst interval == measurement average interval). 05f78a0

• GenericParser Machinery - A new machinery to write and update parsers. 3c040d5

• AquatrackaPP - Allow multiple integer variable names in aquatrackapp.
  ee03db1

Tests:

• Batch Testing scripts. Now you can run the entire test suite by just executing the runalltests.bat or (runalltests.sh). You need all the instrument test files, however (by using get_testfiles.py). 688a1e6

• Enhanced test coverage for Staroddi Parser. 0b1b08a

• Enhanced test coverage in windows. 0b1b08a

• Allow external users to obtain the test files without setting up AWS credentials.
  bd2a440

• Modernize older tests and remove all instrument files from the repository. f52575d

• Basic Nortek Signature tests. bd7e0d3

• Tests for some utility functions. 4341749

Misc:

• New general auxiliary functions :

  • batchTesting run all tests and report failed ones in a better format. e23a7b5
  • rdir recursive list of files/folders. 88edb13
  • toolboxRootPath return the imosToolbox root path. 88edb13
  • setToolboxPaths add all toolbox folders to the matlab path. e23a7b5
  • toCelsius - convert to Celsius from Fahr or Kelvin. c286607

• Other auxiliar functions:

  • convertVariables - Convert toolbox data structures to imos data structures. 1dec95b
  • getIMOSType - wrapper to mtype = str2func(netcdf3ToMatlabType(imosParameters(imos_parameter_str, 'type')));. 663d498
  • isnumbered - detect if a string follows the _2,_3,_4,... moniker. 663d498
  • loadTimeSeriesSampleTemplate - load the default timeseries structures (TIME,LAT,LON,NOMINAL_DEPTH). 1dec95b
  • resolveString - add a underscore to a string if already present in a list state. 663d498
  • resolveIMOSName wrapper to resolveString to resolve the name to a valid IMOS variable name. 663d498

• Version Bump & binary updates 2d9ddd2265ade2310df94ac38107237dc9dd40bd

• The total number of test files increased to 591.

IMOS Toolbox v2.6.4 - 13 Nov 2019 - quickfix release

Release v2.6.4.

This is a quick-fix release. Please disregard the release tag 2.6.3, since that included wrongly copied win64 binaries.

Changes since 2.6.2/2.6.3:

• Include a version bump in all binaries/files. f8ce94b
• Fix build.py script to fully work on windows. 61ae62b
• Fix the DepthProfile view of Raw data and QC data (also fixed the error msg). 8003a46

IMOS Toolbox v2.6.3 - 12 Nov 2019 - quickfix release

This release is incomplete.

Please don't use this release, since the windows binaries were wrongly updated.

Release 2.6.3.

This is a quick-fix release.

Changes since 2.6.2:

• Include a version bump in the Linux binaries but not on Windows binaries. daab461
• Fix build.py script to fully work on windows. 61ae62b
• Fix the DepthProfile view of Raw data and QC data (also fixed the error msg). 8003a46

IMOS Toolbox v2.6.2 - 31 Oct 2019 - quickfix release

Release 2.6.2.

This is a quick-fix release. Release 2.6.1 didn't include a version bump.

Changes since 2.6.1:

• Version of the toolbox is bumped correctly.

IMOS Toolbox v2.6.1 - 23 Oct 2019

Release 2.6.1

This is a new release of the IMOS Toolbox to mark update requirements regarding the Matlab version and Runtime libraries.

The toolbox minimum requirements are now R2018b and the Matlab Component Runtime v95. Please see installation instructions on the wiki page.

Changes since v2.5.42:

General:

• Stable releases are now only supported for R2018b and MCR v95 a66be27
• Stand-alone applications are provided for MCR v95 only 40287a2

Data Ingestion:

• Fixed SBE/WQM reader to ignore primary variables not recorded. f48dba4
• Added Fluorescence/PAR from SBE19Plus loggers. Currently, the fluorescence name from Aquatracka UV is being redirected to Aquatracka III namespace with a special comment on it. a3b2ffd
• Handle infinitySD instruments configured in high-frequency samplings, that caused the manufacturer software to output repeated time values (dt<=microseconds). 3f7617d
• Remove old names of CPHL (CHLF/CHLU), enumerate CPHL in case of multiple fields, CPHL comments revisited 0f8d13b

Misc:

• new test files are now available and outside of the repository: use get_testfiles.py to obtain all of them. a66be27
• new utility functions for schema validation: see createTree and treeDiff functions. 75aebf5
• new utility functions to work with cells and structs: see CellUtils and StructUtils folders. 1117fca
• Repository normalization for developers in both windows and Linux (line-endings adjustments). 2d6ba23

IMOS Toolbox v2.5.42 - 21 Jan 2019

Changes since v2.5.41:

• Improved performance and fix parsing for full directional spectrum data from .wds file. Fix case for when full directional spectrum data from .wds file is missing data for some time stamps. ddb67bf
• Fix typo and other bugs in signatureParse to support .mat datasets in ENU coordinates. c566bbc
• Increment toolbox version to 2.5.42 and update binaries accordingly. ac369f5

IMOS Toolbox v2.5.41 - 14 Dec 2018

Changes from 2.5.40:

• Clean up adcpBinMappingPP code. 52768ed
• adcpBinMappingPP can now handle datasets originally collected in beam coordinates. d5fc895
• adcpNortekBeam2EnuPP can now handle datasets originally collected in beam coordinates. 1f778a8
• Fix bug in diagramMooring2DVarAgainstOther where iYAxisVar is assumed to be the same accross considered instruments. 179c17b
• Set maximum valid values for turbidity and chlorophyll-a to respectively 4 and 5. This will be taken into account in the imosGlobalRangeQC test. fe333fc
• Fix problematic use of gca in cbCLimRange callback in setTimeSerieColorbarContextMenu for r2017a onwards. Axes handle is now passed on to setTimeSerieColorbarContextMenu. a1e0eed
• Fix problematic use of colormap when not specifying the targeted axes for r2017a and newer. 7f7865e
• Give up on the use of option peer in colorbar() for version >= r2014b as recommended by Matlab documentation. c9afa71
• Rename setTimeSerieColorbarContextMenu to setTimeSeriesColorbarContextMenu. 8afbf24
• Speed of sound used by ADCPs to calculate distances is now stored in SSPD variable for all ADCPs. 6703bc6
• Fix incorrect structure assignment, fix incorrect structure access. 528c3a5
• Increment version to 2.5.41 and update binaries accordingly. dbd5869

IMOS Toolbox v2.5.40 - 23 Oct 2018

Changes from 2.5.39:

• Fix bug on magneticDeclinationPP so that it updates the reference_datum variable attributes accordingly. 3426e67
• Add full stop at the end of salinity comment for salinityPP. eaa14ce
• Tidy up a tiny bit of code in adcpNortekVelocityEnu2BeamPP. cd3674f
• Fix warning message with error on console when exporting NetCDF files. d7ee298
• Updated labels on colorbar context menu to reflect new default settings. db6d402
• Add support to Nortek ADCPs collected in Beam coordinates (except Signature). magneticDeclinationPP had to be moved in the chain to after coordinate trnasformations / bin mapping occur. ffdf9ac
• Add velocityMagDirPP to the default chain of pre-processing routines. velocityMagDirPP derives CDIR and CSPD from UCUR and VCUR (useful for Nortek datasets). fe85b2a
• When exporting raw files, original file extension is retained in the file name. 2f29508
• Increment toolbox version to 2.5.40 and update binaries accordingly. 66fdef9

IMOS Toolbox v2.5.39 - 12 Jul 2018

Changes from v2.5.38:

• Fix instrument_sample_interval and instrument_burst_duration global attributes in Wetlabs readWQMraw parser for the case when only one sample is output for a burst. fe0d446
• Update readWQMraw.m to handle WQMHost v1.59 type header. Added some other serial number captures. Take advantage of the 'File Format' string when available to support other data columns layout than the default one. Updated comments on 'State' numbers. Added unit test to test for successful reading of files from a range of different firmware versions. 3ed93eb
• Update ABSIC and CMAG variables valid_min/max and units to count. 348613e
• NOBS parameter should be of type int to handle greater values than 127. b698343
• SPCT parameter should be of type float so that NaN/Fillvalues are handled well. Its valid_min/max attributes have been set too. e01b7b1
• Added information about ADCPs depths cell position in variable attributes comment. 04a17ef
• Update tickbox string of extra instrument graphing option to make it clear only good data is plotted. 68a71e3
• Added tickbox in main window GUI to enable/disable the display of flags on top of data. e0bc98f
• Add checkbox to mainWindow to hide in the plots or not the data already flagged as bad. 371319f
• Default color ranges for 2D data have been set to full range when necessary, in order to attract the user attention on potential outliers. 0d44466
• Increment version toolbox to 2.5.39 and update binaries accordingly. edd49ad