Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul handling of callbacks #135

Open
schmoelder opened this issue May 2, 2024 · 1 comment
Open

Overhaul handling of callbacks #135

schmoelder opened this issue May 2, 2024 · 1 comment

Comments

@schmoelder
Copy link
Contributor

schmoelder commented May 2, 2024

__eval__

Currently, the Callback class is the only one of the optimization problem evaluation function classes where the __eval__ function not only takes the current "request" but also the evaluation object as arguments. We could unify this by always passing the evaluation object to all of the __eval__ methods and simply not pass it on internally.
The Callback's method would then have to return an empty numpy array (for consistency).

Caching

To ensure the callback function is always called s.t. the output (e.g. figures) is always regenerated on every iteration, we do not want to cache the output of callbacks (which don't return anything anyways).
Since force=True would also force the re-evaluation of intermediate steps, we currently simply check the type of the step and continue if it's a Callback. However, a unified behavior would be preferable; e.g. do not cache if nothing is returned.

callback_dirs

The handling of directories for callbacks should be improved. It is overly complicated and error-prone.

Currently, a directory can be specified when registering a callback function in the optimization problem (optimization_problem.add_callback(..., callbacks_dir='./some_path')).

A callbacks_dir is also set up when calling the OptimizerBase.optimize method, which puts the directory within the results directory (which is usually what we want). This is also only used if no callbacks_dir has been specified when registering the method.

The output of a callback is directly stored in that directory. However, if keep_progress=True, the output is moved into a new subdirectory after each generation in the cleanup method of the Callback object.
To register the current_iteration, as well as the temporary callbacks_dir, "private" attributes of the Callback are used.

*args, **kwargs

Currently, each of the evaluation function classes allows the specification of additional kwargs on registration. However, it might be beneficial being able to override those values (e.g. the callbacks_dir)

@schmoelder
Copy link
Contributor Author

See also #133

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant