April-ANN v0.3.1-alpha (pre-release of v0.3.1-beta)
Pre-release
Pre-release
API Changes
- Added
matrix:linspace(a,b)
andmatrix:linspace(a,b,base)
. matrix:transpose()
returns a matrix which references the original. Any
modification to the tranposed matrix, will be reflected at the original.matrix:cmul
method is now in-place, so, if you don't want to modify the
original matrix, you need to dom:clone():cmul(...)
instead of
m:cmul(...)
.update
property is added toann.optimizers.sgd
object in order to compute
the momentum.- Optimizers
execute
method receives a functions which computes the loss, the
gradient (mandatory). Any other data is optional. - Removed dropout code from activation functions.
- Deleted options from ANN components.
- Added methods
unroll
andget
toann.components.stack
class. - Added
inf
andsup
limits to Hard-Tanh activation function. - Added
random:to_lua_string()
method. - Moved
ann.loss.__base__
toann.loss
. - Moved
ann.components.actf.__base__
toann.components.actf
.
New features
- Added
matrix.dict
, a hash set dictionary in C++ binded to Lua, which allows
to execute basic math operations and reductions over the whole set of
contained matrices. It major purpose is to represent a set of connection
weights or gradients in ANNs. - Added
dataset.token.filter
, which allows ANN components as filters. - Added
trainable.train_holdout_validation
class, which replace
trainable.supervised_trainer:train_holdout_validation
method. - Added
trainable.train_wo_validation
class, which replace
trainable.supervised_trainer:train_wo_validation
method. - Added
trainable.dataset_pair_iterator
and
trainable.dataset_multiple_iterator
, useful to iterate over datasets
following different traversal schemes: sequential, shuffled, shuffled with
replacement, shuffled with distribution. - Added method
precompute_output_size
in ANN components. - Added
ann.optimizer.cg
, Conjugate Gradient algorithm. - Added
ann.optimizer.rprop
, Resilient Prop algorithm. - Added
batch_fmeasure_micro_avg
andbatch_fmeasure_macro_avg
for
multi-class FMeasure computation. - Renamed loss function
local_fmeasure
asbatch_fmeasure
, and improved to
work with multi-class models. - Added
ann.loss.zero_one
loss function. - Added
DEPENDENCIES-INSTALLER.sh
. - Added syntactic sugar for
matrix:slice(...)
method:m("1:2","3:4")
or
m({1,2},{3,4})
, like in Matlab or Octave. - Added
matrix.svd
andmatrix.diagonalize
. - Added
stats.pca
,stats.mean_centered
,stats.pca_whitening
.
Bugs removed
- Memory leak due to the GPUMirroredMemoryBlock pool was solved.
- Solved bug at
stats.correlation.pearson
. - Solved bug at
trainable
when usinguse_dataset
,train_dataset
,
validate_dataset
,grad_check_dataset
methods without abunch_size
parameter, and with a trainer which has not abunch_size
defined at
construction. - Stabilization of log-logistic activation function.
- Stabilization of training with cross-entropy and multi-class-entropy.
- Solved bug when reading using
matrix.fromTabFilename
. The loader failed
when the file had empty lines. - Solved bug at
Matrix<T>::select(...)
C++ method. The matrix offset wasn't be
added to the resulting matrix offset. - Solved bug at
SlidingWindow::setAtWindow(...)
C++ method. The matrix offset
wasn't be added to the computed window position. - Solved bug at
buffered_memory.h
. Such bug introduces an early stop when
reading matrices, ignoring the last lines of files. - Solved problem with
rlcompleter
, which didn't work properly when loaded as a
Lua module. - Modified
configure.sh
to inform about any error during Lua building. - Loadable modules are working on MacOs X.
C/C++ code changes
- Added
Matrix<T>::random_acces_iterator
, which reduces the access overhead
for random access of aMatrix
object. It retains the memory pointer forcing
an update between host and device (GPU) memory. - Generalized
GPUMirroredBlockBase
to allow the reinterpretation of the
underlying memory pointer using different types (reinterpret_cast). - Added
MatrixSet
class template, which stores a dictionary of STRING->MATRIX,
useful for ANNs and gradient descent purposes. - Added
StochasticANNComponent
which is base class for stochastic components. - Simplified coupling between ANN components, ANN loss functions, introducing
automatic binding between MatrixFloat and TokenMatrixFloat. - ANN Components has a pointer to a
MatrixFloat
instead ofANN::Connections
. ANN::Connections
is a static class with helper functions, and it is binded
asann.connections
.- Old-weights property is removed from ANN connections.
- Added automatic conversion between DataSetFloat and DataSetToken in
dataset.token.filter
anddataset.token.union
. - Added
FunctionInterface
class, in Lua asfunctions
class, superclass of
ANN components.