MLMG fails on high resolution

[Aidyn-A]

Whenever I try to run premixed flames on high resolution, I get "MLMG failed".
Particularly, FlameSheet problem on 128x128 grid fails at the initialization step, while on 32x32 grid it runs okay.

[esclapez]

Hi Aidyn, thank you for reaching out,

We are aware that this problem arises and it is probably related to the issue #48 still unresolved. To make sure it is indeed the same problem, could you provide a couple of precision:

• which branch of PeleLM are you using?
• when you use a 128x128 base grid, are you still using amr.max_level = 2 ?
• there are several MLMGs in PeleLM algorithm, could you turn on the verbose on those in order to identify which one is failing and whether the residual of the MLMG are stalled or diverging ? To do so, you can turn on the following in the input file: diffuse.v = 4 (diffusion), proj.v = 4 (nodal projection), mac.v = 4 (mac projection). Note that the code will be much more talkative with these options.

Thank you !

[Aidyn-A]

Hi Lucas,

Thanks for your response.

I am using the latest Tutorials branch in PeleProduction.
The command "git rev-parse --short HEAD" reports: 75a8c0b
I ran the same case with flags mentioned and still get the same thing.
Additionally, I ran a case with a single-level 512x512 grid and MLMG failed again.
Here are the log files:
log1Level.txt
log3Levels.txt

Thank you,
Aidyn

[esclapez]

Hi Aidyn,

Thank you for the clarifications.
So it turns out that the input specifier controlling the verbose of the initialVelocityProjection (where your run fails) is none of the one I've provided above but nodal_proj.v. If you set it to 2, nodal_proj.v = 2 (the value 4 for the verbose mentioned earlier is probably an overkill), you will obtain the following during the initialVelocityProjection (with a single 512x512 grid):

Projection::initialVelocityProject(): iteration 0
Nodal Projection:
 >> Before projection:
  * On lev 0 max(abs(rhs)) = 2666.243975

MLMG: # of AMR levels: 1
      # of MG levels on the coarsest AMR level: 9
MLMG: Initial rhs               = 2666.243975
MLMG: Initial residual (resid0) = 2666.243975
MLMG: Iteration   1 Fine resid/bnorm = 0.03858916872
MLMG: Iteration   2 Fine resid/bnorm = 0.001142880258
MLMG: Iteration   3 Fine resid/bnorm = 3.300053779e-05
MLMG: Iteration   4 Fine resid/bnorm = 9.433906375e-07
MLMG: Iteration   5 Fine resid/bnorm = 2.665697369e-08
MLMG: Iteration   6 Fine resid/bnorm = 7.40910596e-10
MLMG: Iteration   7 Fine resid/bnorm = 2.071981144e-11
MLMG: Iteration   8 Fine resid/bnorm = 2.66772528e-12
MLMG: Iteration   9 Fine resid/bnorm = 2.568558082e-12
MLMG: Iteration  10 Fine resid/bnorm = 2.713587827e-12
MLMG: Iteration  11 Fine resid/bnorm = 2.490776046e-12
MLMG: Iteration  12 Fine resid/bnorm = 2.41198728e-12
MLMG: Iteration  13 Fine resid/bnorm = 2.527429436e-12
MLMG: Iteration  14 Fine resid/bnorm = 2.431036667e-12
MLMG: Iteration  15 Fine resid/bnorm = 2.479456555e-12
MLMG: Iteration  16 Fine resid/bnorm = 2.28960372e-12
MLMG: Iteration  17 Fine resid/bnorm = 2.541484652e-12
MLMG: Iteration  18 Fine resid/bnorm = 2.522691579e-12
MLMG: Iteration  19 Fine resid/bnorm = 2.508988366e-12
...

until the maximum number of iterations is reached (100). This indicates that the MultiLevel MultiGrid (MLMG) linear solver is stalled right above the tolerance specified in the input file by proj.proj_tol = 1.0e-12. With such a fine grid and to accommodate the initial solution, you can relax the tolerance to 1.0e-11 and it should run successfully. Note that there is a bit of leeway in the linear solver tolerances (anywhere between 1e-12 and 1e-10 is reasonable), but if the MLMG is continuously failing with tolerances >1e-9, it might be an indication that something is wrong with the simulation setup.

As a side note, if you are trying to go through the TripleFlame tutorial, there is currently a bug that makes this case crash unexpectedly. PR #128 will fix the problem and we will update the PeleProduction once the fix is merged into the development branch.

Lucas

[Aidyn-A]

All right Lucas,

Thanks for detail clarification.
I really appreciate it.

Aidyn