Fokker-Planck Collisions for Vlasov Solver #592
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…on coefficient for FPO update from arbitrary potentials in 1x3v to 3x3v.
…rojects H and G onto phase basis and H, G, dH/dv, dG/dv, and d2G/dv2 onto phase surface basis at each velocity space boundary. Currently no cross derivatives in second derivative of G.
…s work as expected, but we'd like to switch to a nodal expansion for continuity across cell boundaries in surface projections
…ity at cell boundaries for surface expansions should make our lives easier when calculating diffusion tensor cross terms at domain boundaries.
…for accuracy for H, dH/dv, G, d2G/dv2, and eval_on_nodes routine for cell boundary continuity for dG/dv. Makes the recovery process a lot simpler for the diffusion coefficient.
… as expected. Updated to use Mana's domain stencil decomposition so domain surface and volume routines are all handled the same way. Need to add a unit test for these calculations
…pdated gen_stencil_advance in hyper_dg.c to match domain stencil method used for diffusion coefficient, etc.
…ing as expected. Using hyper_dg_advance for drag term and hyper_dg_gen_stencil for diffusion term. Removed the the gamma factor from the potential projection so we aren't accounting for it twice. Calculating primitive moments as part of the FPO update to feed into potential calculation; that infrastructure is pulled straight from the LBO.
…er than the LBO infrastructure that includes the conservation corrections for LBO. Had to generate p2 kernels.
…ter rather than the LBO infrastructure that includes the conservation corrections for LBO. Had to generate p2 kernels." This reverts commit d6250a1.
…rvative scheme. Might have broken the FPO regression tests in the merge. Need to check, but committing these regardless as a starting point
…we might be comparing junk offsets and we get valgrind errors.
… part of various names since we will always use the "primitive" moments for the potential calculation. Using new LTE moments object to compute n, u_drift, and T/m for use in the maxwellian potentials object. Standardizing the regression tests to just have a bunch of ~unity factors for now. Will hopefully change back as we figure out things out.
…g. At first it seemed like a resolution thing with the robustness of the FPO but now I think it's that there's some factors missing in the potential calculation or something because I don't think the FPO works when vt =/= 1.0
… need to make the maxwellian potential projection and drag and diffusion coefficient calculation GPU-ified, but want to start this now to start debugging build issues which may occur from final trying to port hyper_dg_gen_stencil to GPUs (and also the way we set the kernels for the FPO diffusion)
…han hybrid, causing me great pain. Regenerated these kernels to use hybrid, confirming that the p1 FPO does in fact conserve energy
…nd regenerated the kernels to match
…ng write_diagnostics = true to the collisions object initialization in an input file
…he FPO branch... the conflicts were related to a slight refactoring of hyper_dg that moved around where the cu updaters are defined and how they're utilized (directly calling the cu methods if use_gpu is true, instead of having to include if statements in updater methods). Copied this syntax for the gen_stencil method. Also updated the vlasov.c write method for the FPO potentials to utilize metadata.
…and deleting old FPO regression tests. Also fixing unit test compilation.
…huge that nvcc breaks
…here the Nvidia compiler is just straight up giving up on the size of the Gkeyll system at this point. Basically we get errors that nvcc can no longer handle the optimizations it does to create temporary variables and put things in registers. This development is unfortunate, as it has nothing to do with FPO or main per se. Both independently compile with GPUs, but their combination does not. Further, @jmrodman and I aggressively refactored parts of the FPO over the last 72 hours to make the compiler have an easier time and now the FPO is not even the worst culprit in the Gkeyll infrastructure for long lines and non-trivial compiler optimizations. Unfortunately the error persists. As a truly desperate measure to try to solve this issue, I have gone through the code base and looked for where we have been doing things historically the GPU compiler struggles to optimize, such as other kernels which have long lines but the GPU was fine with, in the hope that the problem is not the FPO, but again, the size of Gkeyll at this point and thus try to make the whole Gkeyll system easier to compile. 1. Getting rid of remaining 3x2v_p2 gyrokinetic kernels that are not being utilized. 2. Getting rid of the vlasov_gen_geo kernels which are being superseded by canonical PB 3. Redoing the Vlasov and SR Vlasov volume kernels to eliminate some temporary variables and perform the volume updates *sequentially*, so that in 2x3v and 3x3v, we do not try to do one final super update, but a sequence of updates (which breaks up lines that were >20k large into 3 7k long lines). May God have mercy on me and grant me salvation from my sins.
Collaborator
|
Posting a picture of the GPU bug here for our records |
…is struct was removed.
…t thought things were moved around and a couple of bug fixes that were attempted in here as part of "solving the kernel size" issue. But it compiles and runs. So let's try GPUs
…onfiguration space for the FPO (so that we can switch the FPO to be a per-quadrature-point in configuration space update to make it a generic 3V update with the potential for Rosenbluth solves), we delete the p=2 FPO kernels, retaining the Serendipity p=1 kernels in 1x3v and 2x3v (that are still p=2 in velocity space with the hybrid basis).
…issue in Dxy (thanks Atmik).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds Fokker-Planck collisions to the Vlasov solver. More details about the implementation are in the design review: here.
Summary of Changes
hyper_dg_advancehyper_dg_gen_stencil_advanceThese changes interface with the Vlasov solver through
apps/vm_species_fpo.candstruct vm_fpo_collisionsinapps/gkyl_vlasov_priv.h. However, one major point of note is that changes were made tozero/hyper_dg.candzero/hyper_dg_cu.cu. These changes are exclusively to the functiongkyl_hyper_dg_gen_stencil_advanceand its CUDA counterpart, NOTgkyl_hyper_dg_advance.Notes
These pieces are implemented for both CPUs and GPUs for:
Higher dimensionality should also work, kernels just get very large.
Kernels generated for:
New unit tests:
unit/ctest_fpo_vlasov_coeff.cunit/ctest_fpo_vlasov_coeff_correct.cNew regression tests:
regression/rt_fpo_vlasov_relax_1x3v_p1.cregression/rt_fpo_vlasov_relax_1x3v_p2.cChecklist