Skip to content

Commit

Permalink
Merge pull request #112 from pakozm/devel
Browse files Browse the repository at this point in the history 
Devel, new version incomming, v0.4.0
  • Loading branch information
@pakozm
pakozm committed Nov 18, 2014
2 parents a29b92d + 3009d37 commit 38869fb
Show file tree
Hide file tree
Showing 231 changed files with 8,765 additions and 7,260 deletions.
6 changes: 6 additions & 0 deletions CHANGELIST.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ v0.4.0

### Unstable changes

- Added `matrix.op.repmat` function.
- Added `matrix.ext.iterate` iterator.
- Added statistical distributions in `stats.dist`.
- Added `matrix.ext.convolution` and `matrix.ext.real_fftwh`.
- Added `Matrix<T>::convolution` method. It is in experimental stage, please,
Expand All @@ -28,6 +30,9 @@ v0.4.0

### API Changes

- Removed major order differentiation in `matrix`.
- `tokens.matrix` **automatically** wraps `matrix` instances, from Lua to C++.
- `matrix` **automatically** unwraps `tokens.matrix` instances, from C++ to Lua.
- Added new methods to `AprilMath::Limits` class.
- Added `metrics.roc` for ROC computation.
- Added new `class` behavior taken from
Expand Down Expand Up @@ -62,6 +67,7 @@ v0.4.0

### C/C++

- Added `LuaTable` class to allow access of Lua tables from C++ code.
- Added TAR support in C/C++, allowing to use streams as the standard I/O
objects in APRIL-ANN.
- Added `basics` namespace which stores almost all C/C++ code in `basics`
Expand Down
4 changes: 1 addition & 3 deletions EXAMPLES/xor.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,7 @@ trainer:randomize_weights{
sup = 0.1 }

trainer:set_option("learning_rate", 8.0)
trainer:set_option("momentum", 0.5)
trainer:set_option("weight_decay", 1e-05)
trainer:set_layerwise_option("b.*", "weight_decay", 0.0)
trainer:set_option("momentum", 0.8)

local m_xor = matrix.fromString[[
4 3
Expand Down
Loading

1 comment on commit 38869fb

@pakozm
Copy link
Member Author

@pakozm pakozm commented on 38869fb Nov 18, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request suposes big important changes, becoming definitively the new version of APRIL-ANN. Take into account that changes in the order of computations has introduced minor differences in optimizers of ANNs. In CNNs this difference is more important.

The most important changes are:

  • Matrix classes doesn't have a major order flag. They are created always following row-major order, and it is possible to change this order by transposing dimensions of the original matrix. Math operations has been re-implemented asking for the order of the data, and working properly depending in that.
  • Tokens are automatically unwrapped when they are pushed from C++ to Lua, and automatically wrapped when they are taken in C++ bindings. For this behavior, use the name AuxToken when using macros LUABIND_GET_* or LUABIND_RETURN.
  • A LuaTable class has been added to allow a direct communication between C++ and Lua without replicating in C++ the data stored in Lua. It is useful for options tables, or to get dictionaries as weight matrix dictionaries of ANNs in trainable package.

Unstable changes

  • Added matrix.op.repmat function.
  • Added matrix.ext.iterate iterator.
  • Added statistical distributions in stats.dist.
  • Added matrix.ext.convolution and matrix.ext.real_fftwh.
  • Added Matrix<T>::convolution method. It is in experimental stage, please,
    be advice to use them by your own risk.
  • Because of the changes in Image, several classes in package imaging has
    been reviewed, consistent tests are necessary to merge with master branch.

API Changes

  • Removed major order differentiation in matrix.
  • tokens.matrix automatically wraps matrix instances, from Lua to C++.
  • matrix automatically unwraps tokens.matrix instances, from C++ to Lua.
  • Added new methods to AprilMath::Limits class.
  • Added metrics.roc for ROC computation.
  • Added new class behavior taken from
    Lua OOP-iter, in order to introduce
    more modularity in APRIL-ANN.
  • New april_doc and april_set_doc API, works with values instead of strings.
  • Added table.values, table.ivalues, table.keys and table.ikeys
    iterators.
  • matrix.dict could store sparse and dense float matrices.
  • Added matrix.cholesky(...) method.

Bugs removed

  • Solved bug at matrix:max() and matrix:min() methods.
  • Removed memory leak at SelectANNComponent::doBackprop() method.
  • Solved bug at CopyANNComponent::doBackprop() method, incorrect behavior for
    multi-dimensional matrices.
  • Solved bug at ZCAWhiteningANNComponent::doBackprop() method was wrong.
  • Solved bug in stats.boot() function, it wasn't correctly updated to new
    class functions.
  • Solve bug in trainable.supervised_trainer, problem with smooth_gradients
    flag.
  • Solved bug in trainable.lua train_holdout:execute() method, the pocket
    algorithm wasn't work with negative loss functions.
  • Solved bug at class.lua, inifite loop when calling class.is_a().
  • Solved bug at autoencoder training.
  • Solved bug in ROC computation.
  • CUDA is working, time performance of convolutions needs a review.
  • Solved bug in constString extract numeric methods which returns false when
    the extraction procedure ends up to the character after the last valid number.
  • Solved bug at Matrix<T>::div and SparseMatrix<T>::div methods.

C/C++

  • Added LuaTable class to allow access of Lua tables from C++ code.
  • Added TAR support in C/C++, allowing to use streams as the standard I/O
    objects in APRIL-ANN.
  • Added basics namespace which stores almost all C/C++ code in basics
    packages.
  • Simplified read/write of matrices using the new StreamInterface objects.
  • Added new StreamInterface for input/output operations, with specializations
    for C files, C strings, Lua strings, GZIP files, ZIP files. Other
    specializations are easier to implement in the future.
  • Added statistical distribution classes.
  • Removed C stack trace in ERROR_PRINT and ERROR_EXIT when compiling without
    debug.
  • Image class code has been reviewed to be more coherent with current Matrix
    class implementation.

Please sign in to comment.