Weights

This page has all information on Weights: Saving, Loading, etc

Compression

Just add a .gz to the filename when saving weights and they will automatically be compressed! Likewise, when opening a weights file, if it has the .gz extension, it will automatically be uncompressed upon loading.

Note that if you're packing weights into a bindata.go using go-bindata (e.g., https://github.com/shuLhan/go-bindata), it automatically compresses the weights itself, and our Open method does not recognize the compression when loading this way, so use a regular .wts file for that case. You can just do gzcat weights.wts.gz > weights.wts to create an uncompressed copy of the compressed file.

Standard Formats

There are two major standards for backprop-style neural networks, NNEF and ONNX: https://www.khronos.org/blog/nnef-and-onnx-similarities-and-differences. The NNEF is probably the more relevant given its more official, broader status, and its data format is defined here: https://www.khronos.org/registry/NNEF/specs/1.0/nnef-1.0.html#storing-data It is a binary format with multiple files within a directory structure, typically packed into a tar file. This is great for maximum flexibility but not that practical for "everyday use" in our models -- don't want to deal with tar etc. Having a binary format is a good idea, but for smaller networks, being able to edit the values etc in text is important.

We weren't able to find a weight file standard for the more biologically-oriented frameworks, NeuroML and PyNN -- see Standards

Thus, we will initially support a JSON-based text format, and then add support for binary NNEF later.

JSON Text format

Note: this format changed as of 8/25/2019 -- prior weight files are not valid.

Basically, it is a straightforward format with Network then nested []Layer with nested []Neuron etc.

NNEF Binary format

TODO