Backward mode differentiation

[AraneusRota]

• CPS with manual continuation passing where all continuation build the forward pass and the code after the continuation are operations for the backward pass
• functional CPS (no side-effects)
• CPS where the user does not need to use continuation explicitly by using Monads and for-comprehensions
• chainable monad implementation (functions can be chained)
• Monads without for comprehensions by using macros
• Tape (save all instructions which should be performand in the backward pass on a "tape")
• functional tape (no side-effects)
• CHAD-paper implementation (sum, mul, if, loop, val), no higher order differentiation

Machine learning tasks use backpropagation to learn with neural networks therefore backward mode differentiation must be supported.

Motivation:
Usually a neural network has many inputs but only a few outputs

Reverse accumulation is more efficient than forward accumulation for functions f : ℝ^n → ℝ^m with m ≪ n as only m sweeps are necessary, compared to n sweeps for forward accumulation.
https://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation

Multiple implementation styles are tried out (see checklist on top). Dual numbers aren't an own category anymore because they are probably used in both styles because you often need access to the normal evaluation result and the derivative of a term which can be achieved by using dual numbers.