FW-CADIS Weight Window Generation with Random Ray

[jtramm]

Overview

This PR introduces global weight window generation with the FW-CADIS algorithm. FW-CADIS uses an adjoint flux distribution generated from a random ray transport solve as the input for weight window generation. Early testing of the new implementation reveals about an order of magnitude improvement in numerical performance as compared to MAGIC.

While this is listed as a 1300+ line PR, in reality the technical changes related to FW-CADIS are only about 10-20 lines, thanks to 1) the highly reusable interface in OpenMC that was developed for weight window generation with MAGIC, and 2) adjoint transport with random ray was previously added in #3191. The rest of the lines are added documentation and tests.

Interface

Generating weight windows with FW-CADIS and random ray uses the same existing interface as for MAGIC. The only major change is that you need to specify the method when defining the WeightWindowGenerator object. An example of adding generation of weight windows to an existing random ray input deck is as follows:

# Define weight window spatial mesh
ww_mesh = openmc.RegularMesh()
ww_mesh.dimension = (10, 10, 10)
ww_mesh.lower_left = (0.0, 0.0, 0.0)
ww_mesh.upper_right = (100.0, 100.0, 100.0)

# Create weight window object and adjust parameters
wwg = openmc.WeightWindowGenerator(method='fw_cadis', mesh=ww_mesh, max_realizations=settings.batches)

# Add generator to openmc.settings object
settings.weight_window_generators = wwg

The ultimate goal is to be able to take an existing continuous energy Monte Carlo input deck and just add a 5-10 lines to the input deck with the rest happening behind the scenes. Currently, however, a few steps are required to generate multigroup cross section data and to setup the random ray solver. More details on the current interface and FW-CADIS workflow are available in the updated docs.

Updated Docs

As documentation for MAGIC weight window generation has not yet been added to the OpenMC docs, this PR also adds in variance reduction sections to the methods docs and user guide. The MAGIC sections are fairly short, but can easily be extended in a future PR. Only generation with the WeightWindowGenerator class is discussed. The more commonly used library method for generating weight windows is not presented, but would make for a nice future addition. The FW-CADIS sections are a little longer and detail the intended use case and workflow.

I also added in some concrete examples for generating multigroup cross section data for input to the random ray solver, as this is a key part of the overall FW-CADIS workflow currently. These examples were added to a new section of the random ray user guide.

Limitations

The main limitation is that similar to tally meshes, weight window meshes cannot subdivide cells or else will result in undefined behavior. It is ok for multiple cells to be within the same tally mesh element, but the reverse cannot be true. Longer term, this restriction will be lifted when cell-under-voxel subdivision is added for random ray (currently tracked by issue #2832).

For now, you can either restrict the weight window mesh to be conformant to existing lattice or cell boundaries in the problem, or you can (in theory) make a top level lattice with each element being filled with the full simulation universe appropriately untranslated.

CI Testing Improvements

I added a new python testing harness class (WeightWindowPyAPITestHarness) for testing weight windows. The .dat result files are stringified versions of the weight_windows.h5 files, containing a few lines of basic information of the mesh and then a printout of all lower and upper bound values, which seems sufficient to evaluate if the weight windows were generated correctly. This harness is used for the new FW-CADIS test, and may be useful as more weight window features are added in the future. Additionally, it may be nice to update the MAGIC test in the future to use the new harness to simplify things.

MG Weight Window Bug Fix

This PR also fixed a small bug when using weight windows in multigroup transport mode. Previously, weight windows were applied before sampling a reaction, whereas this PR fixes it so that it matches the logic in CE mode, where weight windows are applied after sampling a reaction. Without this, weight windows do not behave correctly in multigroup mode, resulting in significant bias due to a lack of conservation of weight (which showed up on testing with the Kobayashi dog leg benchmark).

Validation and Results

To validate the new feature, I utilized the same fixed source 3D cask shielding problem as from the Raffuzzi et al. paper on random ray FW-CADIS. The side view of the problem looks as below:

The problem is 6.5 x 3.5m x 3.5m, featuring an encased cask source sitting in a room, with a person-sized water target in an adjacent room shielded by a 2m thick concrete wall. As originally described in Raffuzzi et al., the thick shield results in analog Monte Carlo being unable to result in any scores to the target region even after billions of histories. To ensure that no bias was being induced by use of weight windows, I altered the density of the concrete down to 0.95 g/cc so as to allow for some particles to make it through the shield and for a reasonable analog reference solution to be generated.

I then compared the reference CE solution against the Monte Carlo MAGIC method, the random ray MAGIC method (which uses the forward flux as a source for weight window generation), and the random ray FW-CADIS method. I recorded the simulation runtimes (which vary significantly due to differences in the number of secondary particles the weight windows generate), tally results, and tally uncertainties. I then computed the FOM as 1/(time * sigma^2), so as to appropriately weight changes in runtime with changes in uncertainty.

Ideally I would have a tighter convergence on the reference solution and weight window cases to really check for any biases, but these results at least show that any biases would likely be pretty small (~1%), and more likely we're just looking at noise. I will also note that the above data excludes the cost of weight window generation itself. If we were to include these costs, it would definitely water down the MC MAGIC FOM, given that weight window generation costs with Monte Carlo are sizeable. However, the random ray FW-CADIS generation costs are negligible, being just a few seconds. As such, this data may be understating a little the difference in FOM between MAGIC and FW-CADIS, though perhaps there are also more optimal parameters that could be selected for MAGIC.

I also did some preliminary experimentation with weight window parameters and multigroup XS discretizations. Generally I found that the CASMO two group structure and linear sources were best for the random ray solver, along with a very coarse 20cmx20cmx20cm single-group weight window mesh. That said, I did not experiment with many of the options for weight windows in OpenMC, and there may be more optimal parameters for both the traditional MAGIC technique and FW-CADIS that may improve FOMs further.

Additionally, it is worth noting that the FOMs can likely be extended towards infinity compared to analog CE MC by simply increasing the density of concrete back up to a more typical value (e.g., 2.3 g/cc), as no particles will reach the target when using only analog MC.

Closes #2837.

Checklist

• I have performed a self-review of my own code
• I have run clang-format (version 15) on any C++ source files (if applicable)
• I have followed the style guidelines for Python source files (if applicable)
• I have made corresponding changes to the documentation (if applicable)
• I have added tests that prove my fix is effective or that my feature works (if applicable)

[yardasol]

Great work on this @jtramm!! I'll take a look at this tomorrow!

[yardasol]

Great job @jtramm! You might want to take a look at this commit in a branch on my fork, seems relevant to MGXS generation (but potentially unneeded)

[jtramm]

Thanks so much for the review/feedback @yardasol!

You might want to take a look at this commit in a branch on my fork, seems relevant to MGXS generation (but potentially unneeded)

Thanks for sharing this commit. OpenMC is currently capable of generating the MGXS needed for random ray, as shown in the examples I have added to the docs in this PR. However, in a similar vein as your commit, I think it would be great to add in an attribute to the openmc.mgxs.Library class such that a user can signify that they just want to generate whatever MGXS data is needed for use with the random ray solver, without having to manually specify all the details. When this flag is given when creating the openmc.mgxs.Library object, all the boilerplate from my examples, e.g.,:

mgxs_lib.mgxs_types = ['total', 'absorption', 'nu-fission', 'fission',
                    'nu-scatter matrix', 'multiplicity matrix', 'chi']

could be done away with, hopefully simplifying the MGXS generation process for the user. We could also add in/modify some default behaviors for the class to make things easy for getting started. In the ideal case, the user would just specify:

mgxs_lib = openmc.mgxs.Library(geometry, random_ray=True)

and be initialized with reasonable defaults (including a reasonable group structure) for random ray. It would also be great if the handling of domain names were streamlined.

[yardasol]

Nice job @jtramm. I think it might be nice to have some more technical detail about weight windows, as they aren't that complicated, but that does not have to be done in this PR :)

[paulromano]

Thanks for this PR @jtramm! The new documentation looks great, and it's impressive how few lines of code were necessary to handle the FW-CADIS method. In addition to the line comments below, the only other question I have is whether we should rename that update_magic method since it handles both MAGIC and FW-CADIS.

[jtramm]

Thanks for the review @promano -- it is very much appreciated!

... the only other question I have is whether we should rename that update_magic method since it handles both MAGIC and FW-CADIS.

I similarly was considering that! However, the update_magic function is part of the CAPI and is used in a lot of the example notebooks @pshriwise has. I figured it would be better to avoid changing the existing API if possible, though perhaps it's worthwhile? I could also in theory maintain the old API, but just make the update_magic() function a wrapper around the more generically named function? I'm open to ideas.

[jtramm]

In looking at the update_magic question more closely -- I was mistaken w.r.t the CAPI, there is a wrapper already around this so should be easy to change the name. I'll make that change.