Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ideas to make ProximalProjection more robust #802

Open
f0uriest opened this issue Dec 12, 2023 · 3 comments
Open

Ideas to make ProximalProjection more robust #802

f0uriest opened this issue Dec 12, 2023 · 3 comments
Assignees
Labels
enhancement General label for enhancement. Please also tag with "Speed", "Interface", "Functionality", etc robustness Make the code more robust speed New feature or request to make the code faster

Comments

@f0uriest
Copy link
Member

Right now we still have cases where the proximal optimizer keeps trying smaller and smaller steps until eventually it quits saying Warning: A bad approximation caused failure to predict improvement.

This seems to be due to inconsistent linearizations or a possibly instability, where small changes in the boundary (the x variable the optimizer sees) lead to large changes in the volume (the f values the optimizer sees).

Some possible ideas to make this better:

  • Have some "patience" criteria, where if the perturb/solve stuff doesn't find an acceptable step in N steps, we go back and do a full continuation solve.
  • Have some check on the step size in both boundary and volume and reject steps if the ratio is too large.
  • Use the 2nd order perturbations to do the above, where we reject a step if the 2nd order correction is larger than some fraction of the first order step (similar to geodesic acceleration).
  • If several attempts don't find an acceptable step, return a smaller function value to force it to accept a "bad" step in the hopes that linearizing around the new point will be better.
@f0uriest f0uriest self-assigned this Apr 5, 2024
@dpanici
Copy link
Collaborator

dpanici commented Apr 29, 2024

Case for a precise-QA-like equilibrium in free boundary, each step in the proximal just gives a bad approximation, files are here: https://drive.google.com/drive/folders/19-F7cVb1yxks42eCI1Nx7DlxjCkyxaaf?usp=drive_link

@dpanici dpanici added the enhancement General label for enhancement. Please also tag with "Speed", "Interface", "Functionality", etc label Jun 25, 2024
f0uriest added a commit that referenced this issue Jul 18, 2024
Adds the option to use QR decomposition for solving least squares trust
region problem. This appears to be significantly faster than SVD with
negligible loss in accuracy (at least on our tests). This also makes QR
the new default instead of SVD.

There are a few other places that we use SVD that could possibly also be
replaced by QR:
- In `perturb.py`, though probably not worth it here, since initial SVD
and initial QR take about the same time, but SVD can reuse the
factorization for 2nd and higher order perturbations.
- In `ProximalProjection._jvp_f` - I tried replacing this but it seems
here we need the extra accuracy. Could maybe get it to work if we also
implement some of the ideas from #802 but that can wait for another PR.

Resolves #708
@dpanici dpanici added robustness Make the code more robust speed New feature or request to make the code faster labels Aug 20, 2024
@dpanici
Copy link
Collaborator

dpanici commented Aug 20, 2024

@dpanici check thru notebooks and link here along with current commit that it occurs at

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement General label for enhancement. Please also tag with "Speed", "Interface", "Functionality", etc robustness Make the code more robust speed New feature or request to make the code faster
Projects
None yet
Development

No branches or pull requests

2 participants