More options for which (intermediate) results to store

[dweindl]

Some users want the complete record of all evaluated points, others are interested only in the best point and nothing else. Again others may want to get only the local optima, or only the N best points ever encountered.

If objective evaluations are expensive, we may want to write intermediary results to disk more frequently. If they are very fast, this may quickly become a bottleneck.

Make that configurable.

[dweindl]

What should be supported and demonstrated:

• (1) Tracking the N best parameter vectors, independent of whether they occurred during local search or elsewhere. Relevant to generate ensembles. (Enable tracking of k-best or within-threshold-from-best parameters #22)
• (2) Tracking all parameter vectors according to some likelihood ratio threshold, independent of whether they occurred during local search or elsewhere. Relevant to generate ensembles. (Enable tracking of k-best or within-threshold-from-best parameters #22)
• (3) Just tracking the global best value and respective (monotonic) history -> wrap as pypesto.Optimizer, use pypesto HistoryOptions. Not recommended but requested. Basically (1) with N=1.
• (4) Tracking all function evaluations. Slow and memory intensive, probably only relevant for debugging and demonstration of the algorithm. Basically (1) with N=inf. (Initial solution for tracking function evaluations #20)

This will probably be hard to achieve with the current pypesto.History framework due to parallelization (multiple sacess workers, and potentially parallel objective evaluations inside eSS). We would have to create individual histories and merge them later.
Another issue is that pypesto.History currently expects some x0 which doesn't exist in the scatter search case.