Merge pull request #175 from jpmorgan98/main

Hotfixes for issues

Author: jpmorgan98

docs/paper.bib

@@ -65,12 +65,29 @@ @inproceedings{mcdc:qmcabs
 
 @inproceedings{mcdc:variansyah_physor22_pct,
     Booktitle = {International Conference on Physics of Reactors},
-    title = {An effective initial particle sampling technique for {M}onte {C}arlo reactor transient simulations},
+    title = {Performance of Population Control Techniques in {M}onte {C}arlo Reactor Criticality Simulation},
     year = {2022},
     author = {Ilham Variansyah and Ryan G. McClarren},
     address = {Pittsburgh, Pennsylvania, USA},
 }
 
+@inproceedings{variansyah_mc23_ic,
+    Booktitle = {International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering},
+    title = {An effective initial particle sampling technique for {M}onte {C}arlo reactor transient simulations},
+    year = {2023},
+    author = {Ilham Variansyah and Ryan G. McClarren},
+    address = {Niagara Falls, Ontario, Canada},
+    doi = {10.48550/arXiv.2305.07646}
+}
+
+@inproceedings{variansyah_mc23_moving_object,
+    Booktitle = {International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering},
+    title = {High-fidelity treatment for object movement in time-dependent {M}onte {C}arlo transport simulations},
+    year = {2023},
+    author = {Ilham Variansyah and Ryan G. McClarren},
+    address = {Niagara Falls, Ontario, Canada},
+    doi = {10.48550/arXiv.2305.07641}
+}
 
 @article{mcdc:variansyah_nse22_pct,
     author  = {Ilham Variansyah and Ryan G McClarren}, 
@@ -82,18 +99,32 @@ @article{mcdc:variansyah_nse22_pct
     doi     = {10.1080/00295639.2022.2091906}
 }
 
+@article{mcdc:clements_variance_2024,
+	title = {A variance deconvolution estimator for efficient uncertainty quantification in {Monte} {Carlo} radiation transport applications},
+	volume = {319},
+	issn = {0022-4073},
+	url = {https://www.sciencedirect.com/science/article/pii/S0022407324000657},
+	doi = {https://doi.org/10.1016/j.jqsrt.2024.108958},
+	abstract = {Monte Carlo simulations are at the heart of many high-fidelity simulations and analyses for radiation transport systems. As is the case with any complex computational model, it is important to propagate sources of input uncertainty and characterize how they affect model output. Unfortunately, uncertainty quantification (UQ) is made difficult by the stochastic variability that Monte Carlo transport solvers introduce. The standard method to avoid corrupting the UQ statistics with the transport solver noise is to increase the number of particle histories, resulting in very high computational costs. In this contribution, we propose and analyze a sampling estimator based on the law of total variance to compute UQ variance even in the presence of residual noise from Monte Carlo transport calculations. We rigorously derive the statistical properties of the new variance estimator, compare its performance to that of the standard method, and demonstrate its use on neutral particle transport model problems involving both attenuation and scattering physics. We illustrate, both analytically and numerically, the estimator’s statistical performance as a function of available computational budget and the distribution of that budget between UQ samples and particle histories. We show analytically and corroborate numerically that the new estimator is unbiased, unlike the standard approach, and is more accurate and precise than the standard estimator for the same computational budget.},
+	journal = {Journal of Quantitative Spectroscopy and Radiative Transfer},
+	author = {Clements, Kayla B. and Geraci, Gianluca and Olson, Aaron J. and Palmer, Todd S.},
+	year = {2024},
+	keywords = {Monte Carlo radiation transport, Stochastic solvers, Uncertainty quantification},
+	pages = {108958},
+}
+
 @inproceedings{variansyah_mc23_mcdc,
     Booktitle = {International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering},
     title = {Development of {MC/DC}: a performant, scalable, and portable {P}ython-based {M}onte {C}arlo neutron transport code},
     year = {2023},
-    author = {Ilham Variansyah and Joanna Piper Morgan and Kyle E. Niemeyer and Ryan G. McClarren},
+    author = {Ilham Variansyah and Joanna Piper Morgan and Jordan Northrop and Kyle E. Niemeyer and Ryan G. McClarren},
     address = {Niagara Falls, Ontario, Canada},
     doi = {10.48550/arXiv.2305.07636}
 }
 
 @article{brax2023,
     author = {Cuneo, Braxton and Bailey, Mike},
-    title = {Divergence Reduction in Monte Carlo Neutron Transport with On-GPU Asynchronous Scheduling},
+    title = {Divergence Reduction in {M}onte {C}arlo Neutron Transport with On-{GPU} Asynchronous Scheduling},
     year = {2023},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
@@ -116,3 +147,95 @@ @book{lewis_computational_1984
 }
 
 
+#shift
+@article{shift,
+	title = {Continuous-energy {Monte} {Carlo} neutron transport on {GPUs} in the {Shift} code},
+	volume = {128},
+	issn = {03064549},
+	url = {https://linkinghub.elsevier.com/retrieve/pii/S0306454919300167},
+	doi = {10.1016/j.anucene.2019.01.012},
+	abstract = {A continuous-energy Monte Carlo neutron transport solver executing on GPUs has been developed within the Shift code. Several algorithmic approaches are considered, including both history-based and event-based implementations. Unlike in previous work involving multigroup Monte Carlo transport, it is demonstrated that event-based algorithms significantly outperform a historybased approach for continuous-energy transport as a result of increased device occupancy and reduced thread divergence. Numerical results are presented for detailed full-core models of a small modular reactor (SMR), including a model containing depleted fuel materials. These results demonstrate the substantial gains in performance that are possible with the latest-generation of GPUs. On the depleted SMR core configuration, an NVIDIA P100 GPU with 56 streaming multiprocessors provides performance equivalent to 90 CPU cores, and the latest V100 GPU with 80 multiprocessors offers the performance of more than 150 CPU cores.},
+	language = {en},
+	urldate = {2023-11-03},
+	journal = {Annals of Nuclear Energy},
+	author = {Hamilton, Steven P. and Evans, Thomas M.},
+	month = jun,
+	year = {2019},
+	pages = {236--247},
+	file = {1-s2.0-S0306454919300167-am.pdf:/Users/jonesy/Documents/PapersLibrary/1-s2.0-S0306454919300167-am.pdf:application/pdf},
+}
+
+
+#mcatk
+@article{mcatk,
+	title = {Monte {Carlo} {Application} {ToolKit} ({MCATK})},
+	volume = {82},
+	issn = {0306-4549},
+	url = {https://www.sciencedirect.com/science/article/pii/S0306454914004472},
+	doi = {https://doi.org/10.1016/j.anucene.2014.08.047},
+	abstract = {The Monte Carlo Application ToolKit (MCATK) is a component-based software library designed to build specialized applications and to provide new functionality for existing general purpose Monte Carlo radiation transport codes. We will describe MCATK and its capabilities along with presenting some verification and validations results.},
+	journal = {Annals of Nuclear Energy},
+	author = {Adams, Terry and Nolen, Steve and Sweezy, Jeremy and Zukaitis, Anthony and Campbell, Joann and Goorley, Tim and Greene, Simon and Aulwes, Rob},
+	year = {2015},
+	keywords = {Agile development, Component software, Monte Carlo particle transport, Parallel computing, Population control, Time-dependent},
+	pages = {41--47},
+	annote = {Joint International Conference on Supercomputing in Nuclear Applications and Monte Carlo 2013, SNA + MC 2013. Pluri- and Trans-disciplinarity, Towards New Modeling and Numerical Simulation Paradigms},
+}
+
+
+# mcnp
+@techreport{mcnp,
+  address = {Los Alamos, NM, USA},
+  author = {Rising, Michael Evan and Armstrong, Jerawan Chudoung and Bolding, Simon R. and Brown, Forrest Brooks and Bull, Jeffrey S. and Burke, Timothy Patrick and Clark, Alexander Rich and Dixon, David A. and Forster, III, Robert Arthur and Giron, Jesse Frank and Grieve, Tristan Sumner and Hughes, III, Henry Grady and Josey, Colin James and Kulesza, Joel Aaron and Martz, Roger Lee and McCartney, Austin P. and McKinney, Gregg Walter and Mosher, Scott William and Pearson, Eric John and Solomon, Jr., Clell Jeffrey and Swaminarayan, Sriram and Sweezy, Jeremy Ed and Wilson, Stephen Christian and Zukaitis, Anthony J.},
+  doi = {10.2172/1909545},
+  institution = {Los Alamos National Laboratory},
+  month = {January},
+  number = {LA-UR-22-33103, Rev.~1},
+  title = {{MCNP\textsuperscript{\textregistered} Code Version 6.3.0 Release Notes}},
+  url = {https://www.osti.gov/biblio/1909545},
+  year = {2023}
+}
+
+# openmc
+@article{openmc,
+    author = {Romano, Paul K. and Forget, Benoit},
+    address = {United States},
+    copyright = {2012 Elsevier Ltd},
+    issn = {0306-4549},
+    journal = {Annals of nuclear energy},
+    keywords = {Code (cryptography) ; Computational science ; Computer science ; Criticality ; High performance computing ; Monte Carlo ; Monte carlo code ; Monte Carlo method ; Neutron transport ; Nuclear Science \& Technology ; Open source ; Particle transport ; Software design ; Supercomputer ; Theoretical computer science},
+    language = {eng},
+    number = {C},
+    organization = {UT-Battelle LLC/ORNL, Oak Ridge, TN (Unted States)},
+    pages = {274-281},
+    publisher = {Elsevier Ltd},
+    title = {The {OpenMC} {M}onte {C}arlo particle transport code},
+    volume = {51},
+    year = {2013},
+}
+
+
+# geant4
+@article{geant4,
+    title = {Geant4—a simulation toolkit},
+    journal = {Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment},
+    volume = {506},
+    number = {3},
+    pages = {250-303},
+    year = {2003},
+    issn = {0168-9002},
+    doi = {10.1016/S0168-9002(03)01368-8},
+    url = {https://www.sciencedirect.com/science/article/pii/S0168900203013688},
+    author = {S. Agostinelli and J. Allison and K. Amako and J. Apostolakis and H. Araujo and P. Arce and M. Asai and D. Axen and S. Banerjee and G. Barrand and F. Behner and L. Bellagamba and J. Boudreau and L. Broglia and A. Brunengo and H. Burkhardt and S. Chauvie and J. Chuma and R. Chytracek and G. Cooperman and G. Cosmo and P. Degtyarenko and A. Dell'Acqua and G. Depaola and D. Dietrich and R. Enami and A. Feliciello and C. Ferguson and H. Fesefeldt and G. Folger and F. Foppiano and A. Forti and S. Garelli and S. Giani and R. Giannitrapani and D. Gibin and J.J. {Gómez Cadenas} and I. González and G. {Gracia Abril} and G. Greeniaus and W. Greiner and V. Grichine and A. Grossheim and S. Guatelli and P. Gumplinger and R. Hamatsu and K. Hashimoto and H. Hasui and A. Heikkinen and A. Howard and V. Ivanchenko and A. Johnson and F.W. Jones and J. Kallenbach and N. Kanaya and M. Kawabata and Y. Kawabata and M. Kawaguti and S. Kelner and P. Kent and A. Kimura and T. Kodama and R. Kokoulin and M. Kossov and H. Kurashige and E. Lamanna and T. Lampén and V. Lara and V. Lefebure and F. Lei and M. Liendl and W. Lockman and F. Longo and S. Magni and M. Maire and E. Medernach and K. Minamimoto and P. {Mora de Freitas} and Y. Morita and K. Murakami and M. Nagamatu and R. Nartallo and P. Nieminen and T. Nishimura and K. Ohtsubo and M. Okamura and S. O'Neale and Y. Oohata and K. Paech and J. Perl and A. Pfeiffer and M.G. Pia and F. Ranjard and A. Rybin and S. Sadilov and E. {Di Salvo} and G. Santin and T. Sasaki and N. Savvas and Y. Sawada and S. Scherer and S. Sei and V. Sirotenko and D. Smith and N. Starkov and H. Stoecker and J. Sulkimo and M. Takahata and S. Tanaka and E. Tcherniaev and E. {Safai Tehrani} and M. Tropeano and P. Truscott and H. Uno and L. Urban and P. Urban and M. Verderi and A. Walkden and W. Wander and H. Weber and J.P. Wellisch and T. Wenaus and D.C. Williams and D. Wright and T. Yamada and H. Yoshida and D. Zschiesche},
+}
+
+@confrence{mcdc:cuneo2024alternative,
+      title={An Alternative to Stride-Based RNG for {M}onte {C}arlo Transport}, 
+      author={Braxton S. Cuneo and Ilham Variansyah},
+      year={2024},
+      doi = {10.48550/arXiv.2403.06362},
+      journal = {Submitted to Transactions of the American Nuclear, Annual meeting 2024},
+      eprint={2403.06362},
+      archivePrefix={arXiv},
+      primaryClass={physics.comp-ph}
+}

docs/paper.md

@@ -109,16 +109,18 @@ It uses the Numba compiler for Python to compile compute kernels to a desired ha
 These acceleration and abstraction techniques allow `MC/DC` developers to remain in a pure Python development environment without needing to support compiled or domain-specific languages.
 This has allowed `MC/DC` to grow from its initialization less than two years ago into a codebase that supports full performant neutron transport and investigation of novel transport algorithms, with development mostly from relative novices.
 
-Many of the traditionally developed neutron-transport codes are export-controlled (i.e., are not open source and difficult to access) and notoriously difficult to install, use, and develop in.
-Because `MC/DC` is an open-source and easily installable Python package (with a `pip`-installable distribution), it is ideal for use in an academic environment for both research and education.
-This is further assisted by the test suite we have developed for unit, regression, verification, and performance tests, which are mostly run using continuous integration via GitHub Actions.
+Many traditionally developed neutron-transport codes are export-controlled (e.g. `MCNP` [@mcnp], `Shift` [@shift], and `MCATK` [@mcatk]) and some are known to be difficult to install, use, and develop in.
+`MC/DC` is open-source, and thus, similar to other open-source Monte Carlo neutron-transport codes (e.g., `OpenMC` [@openmc]), it promotes knowledge sharing, collaboration, and inclusive, community-driven development.
+What makes `MC/DC` unique is that its code base is exclusively written in Python, making it a good method exploration tool and an excellent entry point for students.
+Furthermore, `MC/DC` is wrapped as a Python package that can be conveniently installed via the `pip` distribution, and its development is assisted by a suite of unit, regression, verification, and performance tests, which are mostly run using continuous integration via GitHub Actions.
+This all together makes `MC/DC` ideal for use in an academic environment for both research and education.
 
-`MC/DC` has support for continuous energy and multi-group treatments of the neutron distribution in energy.
+`MC/DC` has support for continuous and multi-group energy neutron transport physics with constructive solid geometry modeling.
 It can solve k-eigenvalue problems (used to determine neutron population growth rates in reactors) as well as fully dynamic simulations.
-It has a novel continuous geometry movement function that models transient elements (e.g., control rods or pulsed neutron experiments) more accurately than the step functions used by other codes.
 It also supports some simple domain decomposition, with more complex algorithms currently being implemented.
+In an initial code-to-code performance comparison, `MC/DC` was found to run about 2.5 times slower than the Shift Monte Carlo code for a simple problem and showed similar scaling on some systems [@mcdc:variansyah_mc23_mcdc].
 
-`MC/DC`-enabled explorations into dynamic neutron transport algorithms have been successful, including quasi-Monte Carlo techniques [@mcdc:variansyah_physor22_pct], hybrid iterative techniques for k-eigenvalue simulations [@mcdc:qmc; @mcdc:qmcabs], transient population control techniques [@mcdc:variansyah_nse22_pct], hash-based random number generation, uncertainty and global sensitivity analysis [@mcdc:clements_mc23], residual Monte Carlo methods, and machine learning techniques for dynamic node scheduling, among others.
+`MC/DC`-enabled explorations into dynamic neutron transport algorithms have been successful, including quasi-Monte Carlo techniques [@mcdc:qmc], hybrid iterative techniques for k-eigenvalue simulations [@mcdc:qmcabs], population control techniques [@mcdc:variansyah_nse22_pct; @mcdc:variansyah_physor22_pct], continuous geometry movement techniques that model transient elements [@mcdc:variansyah_mc23_moving_object] (e.g., control rods or pulsed neutron experiments) more accurately than step functions typically used by other codes, initial condition sampling technique for typical reactor transients [@mcdc:variansyah_mc23_ic], hash-based random number generation [@mcdc:cuneo2024alternative], uncertainty and global sensitivity analysis [@mcdc:clements_mc23; @mcdc:clements_variance_2024], residual Monte Carlo methods, and machine learning techniques for dynamic node scheduling, among others.
 
 # Future Work
 
@@ -127,7 +129,7 @@ We currently have operability on Nvidia GPUs (supported via Numba), and work is
 On GPUs, `MC/DC` will use the `harmonize` asynchronous GPU scheduler to increase performance [@brax2023].
 `harmonize` works by batching jobs during execution such that similar operations get executed simultaneously, reducing the divergence between parallel threads running on the GPU.
 
-We will continue to explore novel methods for dynamic neutron transport and will keep pushing to make `MC/DC` not only a proven platform for rapidly exploring neutron-transport methods, but also a fully fledged simulation code for academic and industrial use.
+We will continue to explore novel methods for dynamic neutron transport and will keep pushing to make `MC/DC` not only a proven platform for rapidly exploring neutron-transport methods, but also a fully-fledged simulation code for academic and industrial use.
 
 # Acknowledgements
 

docs/source/user.rst

@@ -226,13 +226,14 @@ Using MPI
 MC/DC can be executed using MPI with or without Numba acceleration.
 If ``numba-mode`` is enabled the ``jit`` compilation, which is executed on all threads, can take between 30s-2min.
 For smaller problems, Numba compilation time could exceed runtime, and pure python mode could be preferable.
-Below, ``--mode`` can equal python or numba.
+Below, ``--mode`` can equal python or numba. MC/DC gets MPI functionality via `mpi4py <https://mpi4py.readthedocs.io/en/stable/>`_. 
+As an example, to run on 36 processes in Numba mode with `SLURM <https://slurm.schedmd.com/documentation.html>`_:
 
 .. code-block:: python3
 
-    srun python input.py --mode=<python/numba>
-
+    srun -n 36 python input.py --mode=<python/numba>
 
+For systems that do not use SLURM (i.e., a local system) try ``mpiexec`` or ``mpirun`` in its stead.
 
 Postprocessing Results
 ----------------------
@@ -291,7 +292,7 @@ A tool like ``matplotlib`` will work great for plotting results.
 For more complex simulations, open source professional visualization software like
 `Paraview <https://www.paraview.org/>`_  or `Visit <https://sd.llnl.gov/simulation/computer-codes/visit>`_ are available.
 
-As the problem we ran above is pretty simple and has no scattering or fission, we have an analytic solution we can import:
+As the problem we ran above is pretty simple and has no scattering or fission, we have an `analytic solution we can import <https://github.com/CEMeNT-PSAAP/MCDC/blob/main/examples/fixed_source/slab_absorbium/reference.py>`_:
 
 .. code-block:: python3
 
@@ -300,7 +301,7 @@ As the problem we ran above is pretty simple and has no scattering or fission, w
 In the script below, we plot the space-averaged flux and space-averaged current, including their statistical noise.
 We also use the space-averaged flux and current to compute a new quantity, the space-averaged angular flux, and
 plot it over space and angle in a heat map.
-Remember that when reporting results from a Monte Carlo solver, you should **always include the statical error!**
+Remember that when reporting results from a Monte Carlo solver, you should **always include the statistical error!**
 
 
 .. code-block:: python3

examples/c5g7/3d/TDX/input.py

@@ -299,6 +299,10 @@ def set_mat(mat):
 
 # Setting
 mcdc.setting(N_particle=1e2, active_bank_buff=1000)
+# small particle count, a more realistic number is 1e7 or higher which
+# will require some acceleration to run at those high particle counts
+# This is a very hard problem to solve and may also require multiple
+# nodes depending on your systems
 
 # Run
 mcdc.run()