Eigen3 for Linear Algebra

What is Eigen?

Eigen 3 is a lightweight C++ template library for vector and matrix math, a.k.a. linear algebra. For those coming in from the Python world, think of it as the numpy for C++. This would be useful for any kind of production-level scientific computing required within Peloton.

Why Eigen?

There are a number of linear algebra libraries available for C++. Of all of them, Eigen and Armadillo are the most popular with a good community and active development. In fact both these libraries are also used in popular C++ ML libraries(Shogun uses Eigen and mlpack uses Armadillo.) Below is a comparison between the two in an attempt to summarize why we chose Eigen finally.

Non-exaustive comparison between Eigen & Armadillo

Eigen

• Pros:
  • Support row-major storage.
  • Super easy to install & use: Just download the header files and put them to the desired position.
  • Various matrix type support, fast & optimized matrix/vector processing:
    • dynamic/fixed size matrix, dense/sparse matrix
    • various numeric types, matrix decompositions
    • make the best use of parallel computing and cache
  • Reliable and easy to implement new algorithms.
    • tested through a thorough testing suite
    • APIs are clean and expressive
  • Good compiler support.
  • Documentation is pretty detailed.
• Cons:
  • Matrices/Vectors in general don't support element-wise operations, but arrays do. Thus, extra conversion between arrays and matrices/vectors is somewhat inevitable.
  • Arithmetic operations for matrices/vectors are limited. For example, there is no built-in stddev function.
  • Reshape/Slicing functionality is limited, need Eigen::Map class to mimic the slicing feature.

Armadillo

• Pros:
  • Extensive associated functions for matrix/vector operations.
  • High level APIs similar to Matlab. Provide quick conversion of research code into production environments.
  • Good support for machine learning: ML-related helper functions are provided.
• Cons
  • Only support column-major storage. According to mlpack, this is counter to most standard machine learning texts & data processing.
  • Documentation is somehow simple.

Reasons for migrating to Eigen

• We have lots of row operations (slicing, vectorizing, etc.), thus using a row-major storage is good for our performance.
• Constructing ML algorithms in column major (using Armadillo) is unintuitive. If we have to use Armadillo (and mimic the row-major feature), we would need to perform extra transpose operations at multiple steps.
• Eigen has good documentation over APIs, giving us a strong support when we implement algorithms on top of Eigen.
• Eigen is optimized for full utilization of the computing resource on the machine.