Merge pull request #63 from myerspat/paper-fixes

CHF benchmark using synthetic data, paper finalized fixes too
aims-umich · Aug 23, 2024 · efc4672 · efc4672
2 parents 7c9de7a + ea21341
commit efc4672
Show file tree

Hide file tree

Showing 61 changed files with 21,272 additions and 7,554 deletions.
diff --git a/.github/workflows/release_CI.yml b/.github/workflows/release_CI.yml
@@ -0,0 +1,30 @@
+name: CI for Release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11"]
+
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v3
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install flake8 pytest
+        pip install pyMAISE
+    - name: Test with pytest
+      run: |
+        cd tests & pytest
diff --git a/README.md b/README.md
@@ -41,3 +41,4 @@ You can find the pyMAISE benchmarks [here](https://pymaise.readthedocs.io/en/lat
 - [BWR Micro Core](https://nbviewer.org/github/myerspat/pyMAISE/blob/develop/docs/source/benchmarks/bwr.ipynb)
 - [HTGR Micro-Core Quadrant Power](https://nbviewer.org/github/myerspat/pyMAISE/blob/develop/docs/source/benchmarks/HTGR_microreactor.ipynb)
 - [NEACRP C1 Rod Ejection Accident](https://nbviewer.org/github/myerspat/pyMAISE/blob/develop/docs/source/benchmarks/rod_ejection.ipynb)
+- [Critical Heat Flux (CHF) Prediction](https://nbviewer.org/github/myerspat/pyMAISE/blob/develop/docs/source/benchmarks/chf.ipynb)
diff --git a/docs/source/benchmarks.rst b/docs/source/benchmarks.rst
@@ -6,6 +6,8 @@ Benchmark Jupyter Notebooks
 
 Welcome to the pyMAISE benchmarked Jupyter Notebooks! These notebooks include the machine learning benchmarks for reactor control, reactor physics, fuel performance, and heat conduction data sets. Follow the :doc:`benchmarks/mit_reactor` notebook for an introductory tutorial with pyMAISE. The other notebooks focus on the models and methods instead of applying pyMAISE.
 
+**Regression**
+
 .. toctree::
    :maxdepth: 1
 
@@ -16,6 +18,9 @@ Welcome to the pyMAISE benchmarked Jupyter Notebooks! These notebooks include th
    benchmarks/bwr.ipynb
    benchmarks/HTGR_microreactor.ipynb
    benchmarks/rod_ejection.ipynb
+   benchmarks/chf.ipynb
+
+**Classification**
 
 -----------------------
 Creating Your Benchmark
@@ -29,7 +34,10 @@ pyMAISE aims to be a medium for AI/ML researchers to benchmark their data sets a
     2. Add a load function to ``pyMAISE/datasets/_handler.py`` and include a description of the data. This load function should return ``xarray.DataArray``.
     3. Create and run a Jupyter notebook for the benchmark in ``docs/source/benchmarks/``.
     4. Add the relative path to the notebook to ``docs/source/benchmarks.rst`` under the ``toctree``.
-    5. If a published paper exists for the data set, add the BibTeX citation to ``docs/source/data_refs.bib``.
+    5. Add a blurb about the data to the user guid under the :ref:`preprocessing <preprocessing>` section in ``docs/souce/user_guide.rst``.
+    6. Add the load function to the ``toctree`` in ``docs/source/pymaise_api.rst`` under :ref:`Data Sets <datasets_api>`.
+    7. Add the nbviwer link to ``README.md``. Ensure this link is for the develop branch.
+    8. If a published paper exists for the data set, add the BibTeX citation to ``docs/source/data_refs.bib``.
 3. Once these steps are completed, you can push the benchmark, ensuring to adhere to the workflow outlined in the :ref:`dev_guide`, and create a `pull request <https://github.com/myerspat/pyMAISE/pulls>`_.
 
 A reviewer will ensure the validity of the benchmark and data. They will offer feedback and possible revisions for you. Thank you for contributing!
diff --git a/docs/source/benchmarks/HTGR_microreactor.ipynb b/docs/source/benchmarks/HTGR_microreactor.ipynb
diff --git a/docs/source/benchmarks/bwr.ipynb b/docs/source/benchmarks/bwr.ipynb
diff --git a/docs/source/benchmarks/chf.ipynb b/docs/source/benchmarks/chf.ipynb
diff --git a/docs/source/benchmarks/fuel_performance.ipynb b/docs/source/benchmarks/fuel_performance.ipynb
diff --git a/docs/source/benchmarks/heat_conduction.ipynb b/docs/source/benchmarks/heat_conduction.ipynb
diff --git a/docs/source/benchmarks/mit_reactor.ipynb b/docs/source/benchmarks/mit_reactor.ipynb
diff --git a/docs/source/benchmarks/reactor_physics.ipynb b/docs/source/benchmarks/reactor_physics.ipynb
diff --git a/docs/source/benchmarks/rod_ejection.ipynb b/docs/source/benchmarks/rod_ejection.ipynb
diff --git a/docs/source/benchmarks/supporting/HTGR_microreactor_network_0.png b/docs/source/benchmarks/supporting/HTGR_microreactor_network_0.png
diff --git a/docs/source/benchmarks/supporting/HTGR_microreactor_network_1.png b/docs/source/benchmarks/supporting/HTGR_microreactor_network_1.png
diff --git a/docs/source/benchmarks/supporting/bwr_network.png b/docs/source/benchmarks/supporting/bwr_network.png
diff --git a/docs/source/benchmarks/supporting/chf_network.png b/docs/source/benchmarks/supporting/chf_network.png
diff --git a/docs/source/benchmarks/supporting/fuel_performance.png b/docs/source/benchmarks/supporting/fuel_performance.png
diff --git a/docs/source/benchmarks/supporting/heat_conduction_network.png b/docs/source/benchmarks/supporting/heat_conduction_network.png
diff --git a/docs/source/benchmarks/supporting/mit_reactor_network.png b/docs/source/benchmarks/supporting/mit_reactor_network.png
diff --git a/docs/source/benchmarks/supporting/reactor_physics_network.png b/docs/source/benchmarks/supporting/reactor_physics_network.png
diff --git a/docs/source/benchmarks/supporting/rod_ejection_network.png b/docs/source/benchmarks/supporting/rod_ejection_network.png
diff --git a/docs/source/data_refs.bib b/docs/source/data_refs.bib
@@ -298,3 +298,11 @@ @article{PRICE2022111776
                         the found optima were verified with the original model
                         used to train surrogates.},
 }
+@article{CHF_Benchmark,
+            title = {Benchmark on Artificial Intelligence and Machine Learning for Scientific Computing in Nuclear Engineering},
+            subtitle = {Phase 1: Critical Heat Flux Exercise Specifications},
+            journal = {NEA Working Papers},
+            author = {Jean-Marie Le Corre and Gregory Delipei and Xingang Zhao},
+            year = {2024},
+            url = {https://www.oecd-nea.org/jcms/pl_89619/benchmark-on-artificial-intelligence-and-machine-learning-for-scientific-computing-in-nuclear-engineering-phase-1-critical-heat-flux-exercise-specifications?details=true},
+}
diff --git a/docs/source/installation.rst b/docs/source/installation.rst
@@ -25,6 +25,9 @@ pyMAISE will also install all needed dependencies.
    -  `TensorFlow <https://tensorflow.org>`_
    -  `SciKeras <https://adriangb.com/scikeras/stable/>`_
    -  `Matplotlib <https://matplotlib.org/stable/>`_
+   -  `tqdm <https://tqdm.github.io/>`_
+   -  `pydot <https://github.com/pydot/pydot>`_
+   -  `graphviz <https://graphviz.org/>`_
 
 .. admonition:: Optional
    :class: note
@@ -63,7 +66,14 @@ running or creating pyMAISE benchmarks, install the additional dependencies:
 *************
 
 For the latest features in development or access to benchmark, install
-pyMAISE from source. Clone the repository using ``git`` and running:
+pyMAISE from source. You can install from source without cloning the environment
+by running:
+
+.. code:: sh
+
+   pip install git+https://github.com/myerspat/pyMAISE.git
+
+Alternatively, you can clone the repository using ``git``:
 
 .. code:: sh
 

diff --git a/docs/source/pymaise_api.rst b/docs/source/pymaise_api.rst
@@ -77,6 +77,7 @@ From pyMAISE, we cannot directly access the data set load functions.
    pyMAISE.datasets.load_BWR
    pyMAISE.datasets.load_HTGR
    pyMAISE.datasets.load_rea
+   pyMAISE.datasets.load_chf
    pyMAISE.datasets.load_loca
 
 .. _preprocessing_api:

diff --git a/docs/source/user_guide.rst b/docs/source/user_guide.rst
@@ -68,7 +68,8 @@ pyMAISE offers several data sets for building and testing ML models. Each of the
 - :doc:`benchmarks/heat_conduction`: effect of heat conduction parameters on fuel rod centerline temperature,
 - :doc:`benchmarks/bwr`: effect of BWR core parameters on :math:`k` and peaking factors,
 - :doc:`benchmarks/HTGR_microreactor`: effect of control drum angle on neutron flux,
-- :doc:`benchmarks/rod_ejection`: effect of reactor kinetics parameters on max power, burst width, max fuel centerline temperature, and average coolant temperature.
+- :doc:`benchmarks/rod_ejection`: effect of reactor kinetics parameters on max power, burst width, max fuel centerline temperature, and average coolant temperature,
+- :doc:`benchmarks/chf`: effect of experimental parameters on critical heat flux (CHF).
 
 Each data set has a load function in the :mod:`pyMAISE.datasets` module. For details, refer to the :doc:`pymaise_api`.
 
@@ -159,19 +160,9 @@ For neural networks, we define both the hyperparameters that remain constant dur
        "models": ["FNN"],
        "FNN": {
            "structural_params": {
-               "Dense_input": {
-                   "units": mai.Int(min_value=50, max_value=400),
-                   "input_dim": xtrain.shape[-1],
-                   "activation": "relu",
-                   "kernel_initializer": "normal",
-                   "sublayer": mai.Choice(["Dropout", "None"]),
-                   "Dropout": {
-                       "rate": mai.Float(min_value=0.4, max_value=0.6),
-                   },
-               },
                "Dense_hidden": {
                    "num_layers": mai.Int(min_value=0, max_value=3),
-                   "units": mai.Int(min_value=25, max_value=250),
+                   "units": mai.Int(min_value=25, max_value=400),
                    "activation": "relu",
                    "kernel_initializer": "normal",
                    "sublayer": mai.Choice(["Dropout_hidden", "None"]),
@@ -333,12 +324,13 @@ Performance Metrics
 
 The :meth:`pyMAISE.PostProcessor.metrics` function evaluates performance metrics for the training and testing predictions of each model. :meth:`pyMAISE.PostProcessor.metrics` by default evaluates
 
-- r-squared: :math:`\text{R}^2 = 1 - \frac{\sum_{i = 1}^{n}(y_i - \hat{y_i})^2}{\sum_{i = 1}^{n}(y_i - \bar{y_i})^2}`,
-- mean absolute error: :math:`\text{MAE} = \frac{1}{n}\sum_{i = 1}^{n}|y_i - \hat{y_i}|`,
-- mean squared error: :math:`\text{MSE} = \frac{1}{n}\sum_{i = 1}^n(y_i - \hat{y_i})^2`,
-- root mean squared error: :math:`\text{RMSE} = \sqrt{\frac{1}{n}\sum_{i = 1}^n(y_i - \hat{y_i})^2}`,
+- r-squared: :math:`\text{R}^2 = 1 - \frac{\sum_{i = 1}^{n}(y_i - \hat{y_i})^2}{\sum_{i = 1}^{n}(y_i - \bar{y}_i)^2}`,
+- mean absolute error: :math:`\text{MAE} = \frac{1}{n}\sum_{i = 1}^{n}|y_i - \hat{y}_i|`,
+- mean absolute percentage error :math:`\text{MAPE} = \frac{100}{n}\sum_{i = 1}^{n}\frac{|y_i - \hat{y}_i|}{\text{max}(\epsilon, |y_i|)}`,
+- root mean squared error: :math:`\text{RMSE} = \sqrt{\frac{1}{n}\sum_{i = 1}^n(y_i - \hat{y}_i)^2}`,
+- root mean square percentage error: :math:`\text{RMSPE} = 100\sqrt{\frac{1}{n}\sum_{i = 1}^n\Big(\frac{y_i - \hat{y}_i}{\text{max}(\epsilon, |y_i|)}\Big)^2}`
 
-for regression problems where :math:`y` is the actual outcome, :math:`\hat{y}` is the model predicted outcome, :math:`\bar{y}` is the average outcome, and :math:`n` is the number of observations. For classification problems the defaults are
+for regression problems where :math:`y` is the actual outcome, :math:`\hat{y}` is the model predicted outcome, :math:`\bar{y}` is the average outcome, :math:`\epsilon` is an arbitrarily small positive number to avoid undefined values, and :math:`n` is the number of observations. For classification problems the defaults are
 
 - accuracy: :math:`\text{Accuracy} = \frac{\text{Number of correct predictions}}{\text{Total number of predictions}}`,
 - recall: :math:`\text{Recall} = \frac{\text{True positives}}{\text{True positives} + \text{False negatives}}`,
@@ -350,7 +342,7 @@ Additionally, we can supply our own metrics to the ``metrics`` as callables. We
 Performance Visualized
 ^^^^^^^^^^^^^^^^^^^^^^
 
-To visualize the performance of each of these models we can use :meth:`pyMAISE.PostProcessor.diagonal_validation_plot`, :meth:`pyMAISE.PostProcessor.validation_plot`, and :meth:`pyMAISE.PostProcessor.nn_learning_plot`. The first two methods provide a comparison of the predicted outcomes versus the actual and :meth:`pyMAISE.PostProcessor.nn_learning_plot` provides a neural network learning curve for comparing training and validation performance.
+To visualize the performance of each of these models we can use :meth:`pyMAISE.PostProcessor.diagonal_validation_plot`, :meth:`pyMAISE.PostProcessor.validation_plot`, and :meth:`pyMAISE.PostProcessor.nn_learning_plot`. The first two methods provide a comparison of the predicted outcomes versus the actual and :meth:`pyMAISE.PostProcessor.nn_learning_plot` provides a neural network learning curve for comparing training and validation performance. You can plot neural network structures with :meth:`pyMAISE.PostProcessor.nn_network_plot`.
 
 For classification problems we can create a confusion matrix using :meth:`pyMAISE.PostProcessor.confusion_matrix`.
 
@@ -361,7 +353,8 @@ Finally, the :class:`pyMAISE.PostProcessor` is equipped with several additional
 
 - :meth:`pyMAISE.PostProcessor.get_params`: get the parameter configurations from a specific model,
 - :meth:`pyMAISE.PostProcessor.get_model`: get the model wrapper,
-- :meth:`pyMAISE.PostProcessor.get_predictions`: get the training and testing predictions from a specific model.
+- :meth:`pyMAISE.PostProcessor.get_predictions`: get the training and testing predictions from a specific model,
+- :meth:`pyMAISE.PostProcessor.print_model`: print a models tuned hyperparameters.
 
 ---------------
 pyMAISE Testing

diff --git a/pyMAISE/__init__.py b/pyMAISE/__init__.py
@@ -1,7 +1,18 @@
+# Determine if display is terminal or notebook
+try:
+    import IPython
+
+    IS_NOTEBOOK = "Terminal" not in IPython.get_ipython().__class__.__name__
+
+except (NameError, ImportError):
+    IS_NOTEBOOK = False
+
 from pyMAISE.postprocessor import PostProcessor
 from pyMAISE.settings import ProblemType, init
 from pyMAISE.tuner import Tuner
-from pyMAISE.utils import Boolean, Choice, Fixed, Float, Int
+from pyMAISE.utils import Boolean, Choice, Fixed, Float, Int, _try_clear
+
+_try_clear()
 
 # This should always be the last line of this file
 __version__ = "1.0.0b0"