Memory issues for large inputs in loss

[TimRoith]

In bc69cd0 the multirun strategy via $M\times N\times d$ particles was introduced. In the typical use case we assume a function

$$ \begin{align*} \tilde{f} :\mathbb{R}^{k\times d} &\to \mathbb{R}^{ k \times d}\\ x&\mapsto (f(x_{1,:}) ,\ldots, f(x_{k, :})) \end{align*} $$

where $f$ denotes the given objective. In order to apply this function on $x\in\mathbb{R}^{M\times N\times d}$ we would transform $x$ via reshaping to $\tilde{x}\in\mathbb{R}^{(M\cdot N) \times d}$ and use this as the input. However, this amounts to applying $f$ to $M\cdot N$ vectors simultaneously. This is handy and quick for loss functions that are easy to evaluate, but problematic for memory-heavy loss functions. There we would need a batching strategy so that the loss is successively applied to sub-batches of $\tilde{x}$.