Solver recommendation for large advection-diffusion (500k cells)

[FojoKojo]

Hi,
I'm at the New University of Lisbon (NOVA), working on a collaborative project with Dalhousie University and the Canadian DFO.
Background: I've successfully used CVODE for a diagenetic model (stiff biogeochemistry + vertical advection-diffusion in sediment layers). It works brilliantly - thank you to the development team!
New problem: I'm extending a related application that simulates dissolved substance transport (e.g. ammonia) from fish farms. The spatial domain is large:

100-120 × 100-120 XY cells
20-30 vertical (Z) layers
Up to ~500,000 cells total

The physics is advection-diffusion between cells - not stiff, just large. Currently using explicit Euler/RK but hitting stability limits with diffusion.
Question: Which SUNDIALS solver would you recommend for this scale?

Memory efficiency is a key concern given the matrix size.
Thanks for any guidance,
João G. Ferreira

[drreynolds]

Hi João,

Others might have input as well, but I can think of two existing methods that might help, along with one that we intend to release in the next few months.

1. You might be able to use an implicit-explicit additive Runge Kutta method from the ARKODE "ARKStep" solver, where the diffusion is treated implicitly and the advection explicitly. If the diffusion is not stiff, then the implicit problems could potentially be solved using either (a) an un-preconditioned iterative linear solver, (b) a simple Jacobi-preconditioned iterative linear solver, or (c) the Anderson-accelerated fixed-point nonlinear solver. For both (a) and (b), if the Jacobian of the diffusion operator would be symmetric then I recommend using PCG for the iterative linear solver.
2. You could try using CVODE for this overall problem, but where you use the fixed-point implicit solver instead of Newton.

Let us know if either of those work for you. When we release some new "extended STS" methods later this Spring, it would be fantastic if you could try those out as well.

[Steven-Roberts]

You might find these advection-diffusion examples helpful:
https://github.com/llnl/sundials/blob/main/examples/cvode/serial/cvAdvDiff_bnd.c
https://github.com/llnl/sundials/blob/main/examples/arkode/CXX_serial/ark_advection_diffusion_reaction.cpp
https://github.com/llnl/sundials/blob/main/examples/arkode/C_serial/ark_advection_diffusion_reaction_splitting.c

To address you concern about memory, you'll want to use sparse or banded matrices or go matrix free (see ARKodeSetJacTimes). Paired with an appropriate linear solver like Dan discussed should keep the memory to 10-100 megabytes.

[FojoKojo]

Dear Daniel, Steven,
Many thanks to both for your help. I'll see how things work with ARKODE and see how things go. Will report. And I will certainly try the new methods when they are available. Kind regards, Joao