Adaptive Gradient Descent Method for e.g., BCG

[pokutta]

Implementation of the two adaptive gradient descent methods from https://arxiv.org/abs/2308.02261 for usage with e.g., BCG and other methods that use gradient descent as subroutine.

Current version is the unconstrained one that should be sufficient for our applications

[pokutta]

includes algorithm, example, and test file

[pokutta]

please also check that the example is contained in the example test -> not sure what triggers it and where it needs to be registered

[pokutta]

some tests are failing but they seem unrelated to the adaptive gradients but happening in BPCG with many "useless steps" - similar to what we have experienced in the comp context @matbesancon @dhendryc . Did we have recent changes to BPCG that might have created this?

[dhendryc]

some tests are failing but they seem unrelated to the adaptive gradients but happening in BPCG with many "useless steps" - similar to what we have experienced in the comp context @matbesancon @dhendryc . Did we have recent changes to BPCG that might have created this?

This should hopefully fix it.
#550

[pokutta]

Finally solution for CTRL-C #159 via the best intern!

[matbesancon]

the try catch block is a solution but we should be careful with it, top of my head it can have a significant performance impact

[matbesancon]

I replaced the ad-hoc prox with the function defined in ProximalCore, it's lightweight enough and it means we can have all the proxes in the world through the ProximalOperators added in the extension.

The only ones I kept are identity and the simplex prox for BCG

[pokutta]

the try catch block is a solution but we should be careful with it, top of my head it can have a significant performance impact

yes indeed - we were thinking of building a macro wrapper that only sets a flag "interrupted" that we catch in the while statement and also have a flag that fully deactivates it. we want to push this also to downstream codes as boscia similar to scip and gurobi. @dhendryc @matbesancon

[pokutta]

i am getting lots of precompile warnings and errors now:

Info Given FrankWolfe was explicitly requested, output will be shown live
WARNING: Method definition prox!(Any, Any, Any, Any) in module FrankWolfe at /Users/x_bas/Documents/Mathematik/projects/2022_branchWolfe/FrankWolfe.jl/src/gradient_descent.jl:412 overwritten at /Users/x_bas/Documents/Mathematik/projects/2022_branchWolfe/FrankWolfe.jl/src/gradient_descent.jl:415.
ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
? FrankWolfe
[ Info: Precompiling FrankWolfe [f55ce6ea-fdc5-4628-88c5-0087fe54bd30] (cache misses: include_dependency fsize change (2), incompatible header (8), mismatched flags (2))
WARNING: Method definition prox!(Any, Any, Any, Any) in module FrankWolfe at /Users/x_bas/Documents/Mathematik/projects/2022_branchWolfe/FrankWolfe.jl/src/gradient_descent.jl:412 overwritten at /Users/x_bas/Documents/Mathematik/projects/2022_branchWolfe/FrankWolfe.jl/src/gradient_descent.jl:415.
ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.
┌ Info: Skipping precompilation due to precompilable error. Importing FrankWolfe [f55ce6ea-fdc5-4628-88c5-0087fe54bd30].
└ exception = Error when precompiling module, potentially caused by a precompile(false) declaration in the module.

@matbesancon

[matbesancon]

Last thing needed is removing the tests on the prox operators since these are now in ProximalOperators.jl and replace the call to the prox functions

[sebastiendesignolle]

Looks good to me! I'm made some minor modifs, nothing important.

[matbesancon]

Thanks! Was there a reason for d6c9335
? Otherwise I'd just revert it, the spaces don't seem necessary?

[sebastiendesignolle]

The goal is to have the tests messages aligned, but I acknowledge that's mostly a symptom of my neurosis...