From d169d74c7b6aed4fa4fc0c4d14f4850991913ea3 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Mon, 17 Dec 2018 18:46:23 -0500 Subject: [PATCH 01/27] Add draft PLEP on plasma simulation to PLEP indices --- PLEP-0000.rst | 3 ++- README.rst | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/PLEP-0000.rst b/PLEP-0000.rst index 0d87c3a..3726880 100644 --- a/PLEP-0000.rst +++ b/PLEP-0000.rst @@ -69,7 +69,8 @@ PLEPs in preparation or under consideration +--------+---------------------------------------------------------------------------+---------------------------+---------------+ | number | title | author(s) | type | +========+===========================================================================+===========================+===============+ -| | | | | +| 7 | `A Next Generation Plasma Simulator | Nicholas A. Murphy | | +| | `__ | | | +--------+---------------------------------------------------------------------------+---------------------------+---------------+ Tentatively Reserved PLEPs diff --git a/README.rst b/README.rst index 0763455..82df3c8 100644 --- a/README.rst +++ b/README.rst @@ -35,3 +35,6 @@ convenience, an abbreviated index of PLEPs is provided below. | 6 | `A New General-Purpose Plasma Object <./PLEP-0006.rst>`__ | .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1460977.svg | | | | :target: http://doi.org/10.5281/zenodo.1460977 | +--------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| 7 | `A Next Generation Plasma Simulator <./PLEP-0007.rst>`__ | | +| | | | ++--------+----------------------------------------------------------------------------------+--------------------------------------------------------------------+ From 0387a99be778038e75b951fd15ba7bfdef392245 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Mon, 17 Dec 2018 18:47:14 -0500 Subject: [PATCH 02/27] Add first draft of plasma simulator PLEP This is just a start and I still need to add the entire description of the software architecture. --- PLEP-0007.rst | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 PLEP-0007.rst diff --git a/PLEP-0007.rst b/PLEP-0007.rst new file mode 100644 index 0000000..2985bae --- /dev/null +++ b/PLEP-0007.rst @@ -0,0 +1,208 @@ +============================================== +PLEP-0007 – A Next Generation Plasma Simulator +============================================== + ++-------------------+---------------------------------------------+ +| PLEP | number | ++===================+=============================================+ +| author(s) | Nicholas A. Murphy | ++-------------------+---------------------------------------------+ +| contact email | namurphy@cfa.harvard.edu | ++-------------------+---------------------------------------------+ +| date created | 2018-12-17 | ++-------------------+---------------------------------------------+ +| date last revised | 2018-12-17 | ++-------------------+---------------------------------------------+ +| type | standard | ++-------------------+---------------------------------------------+ +| status | discussion | ++-------------------+---------------------------------------------+ +| DOI | | +| | | ++-------------------+---------------------------------------------+ + +Abstract +======== + +This proposal describes the current status of simulation software in +the plasma physics community and describes a next generation general +purpose plasma simulation framework in the PlasmaPy ecosystem. This +plasma simulator should be developed with sufficient modularity and +flexibility to enable straightforward switching of both physical +models and numerical methods. + +Detailed Description +==================== + +Current Status of Plasma Simulation +----------------------------------- + +* **Codes are usually written in low level compiled languages such as + Fortran and C.** While these compiled languages offer excellent + performance, code development is significantly slowed down because + the languages are not interactive and variable types must be + declared. + +* **Access to codes is commonly restricted in some way.** Closed + source codes hinder scientific reproducibility. Open source codes + often have a lower density of bugs because more people are able to + review the code and make contributions. + +* **Codes often lack sufficient documentation.** Often it is + necessary to work closely with a core developer of a code in order + to set up a problem. + +* **Code compilation and installation is often difficult and + time-consuming.** This is particularly true for codes that depend on + external libraries or include multiple languages. Compilation often + requires detailed knowledge of makefiles which is a significant + barrier to entry for newcomers. Lack of documentation on how to + compile code exacerbates this problem. + +* **Codes frequently lack unit testing and continuous integration + testing frameworks.** These testing strategies help us improve the + reliability of the code, track down bugs as soon as they appear, and + greatly reduce the likelihood that we will introduce bugs as we + modify or maintain the code. Continuous integration testing + platforms such as Travis CI do not support direct testing of Fortran + code. + +* **Code is often diffucult to read.** Often code uses highly + abbreviated variable names that obfuscate what the variable + represents. + +* **Code is often difficult to maintain.** + +* **Benchmarking two algorithms or comparing physical models usually + requires setting up the same problem in multiple codes.** Usually + the same case must be set up by an experienced or expert user of + each code. Often this requires careful discussions to make sure + that things like normalizations and boundary conditions are able to + be matched. The data are often stored in different formats, so + +* **The code for the problem setup, physical model, and numerical + method are often intertwined with each other.** + +* **Simulation output is rarely stored in a standardized form.** Few + standards exist (with the exception of OpenPMD). The lack of a + standardized format makes developing general purpose plotting + software much more difficult. + +* **Research projects involving plasma simulations are almost always + difficult to reproduce.** + +Development Principles +---------------------- + +* **Optimize for both productivity and performance.** Plasma + simulation should be straightforward and not require expertise into + the details of the code. Optimize code only after it works, and + then only after it becomes apparent what the bottlenecks are. + +* **Prioritize readability, usability, and maintainability.** Code + should be straightforward to install. + +* **Prioritize documentation.** The documentation should be + sufficient for a student taking their first plasma physics class to + be able to use it without too much difficulty. + +* **Make the code as modular as possible.** + +* **Use the SOLID principles for software development.** + + - The *single responsibility principle*: There should never be more + than one reason for a class to change. + + - The *open-closed principle*: Software entities (classes, modules, + functions, etc.) should be open for extension but closed for + modification. + + - The *Liskov substitution principle*: Objects in a program should + be replaceable with instances of their subtypes without altering + the correctness of that program. + + - The *interface segregation principle*: Clients should not be + forced to depend upon interfaces that they do not use. + + - The *dependency inversion principle*: High level modules should + not depend upon low level modules. Abstractions should not depend + upon details. Details should depend upon abstractions. + + +Implementation +============== + +Choice of language +------------------ + +Most plasma simulation software is currently written in low level +languages such as Fortran and C. These languages offer excellent +performance. However, development tends to be slow because the +languages are not interactive and types must be declared. + +Pure Python code is slow because Python is an interpreted language. +Considerable speedup can be achieved using NumPy, though NumPy does +not provide the performance of compiled languages. Performance +comparable to compiled languages can be achieved using several +methods. Python can call code from Fortran and C, but with the +drawbacks that (1) the implementations in code are often clunkier; (2) +developers must understand and maintain code in more than one +language; and (3) difficulties with compilers sometimes arise. Cython +is a superset of the Python language. By adding type declarations, +Cython is able to generate C code from Python-like code which is then +compiled in order to give very good performance. In practice, we +found Cython to be difficult to implement in PlasmaPy. Numba provides +a just-in-time compiler to get compiled speeds at run-time. However, +Numba compiles different functions separately and cannot do global +optimizations. + +Julia is a new high-level open source language that synthesizes the +best features of Fortran, C, Python, R, MATLAB, and Lisp for +scientific computing. Julia uses a just-in-time compiler with type +inference and multiple dispatch to acheive performance comparable to C +and Fortran. Unlike C and Fortran, Julia can be run interactively +which greatly speeds up code development and allows prototyping in the +same language to be used for performance runs. Julia natively +supports parallelization and can call code from Fortran, C, and +Python. Julia proves that high performance can be acheived with a +dynamic interactive language. + +Abstract Interfaces +------------------- + +`Abstract base classes +`_ (ABCs) in Python +allow users to define what methods and attributes must be defined in a +subclass of that ABC. This functionality is used in PlasmaPy's +``Plasma`` class. An equivalent to ABCs has not yet been implemented +in Julia (see `Julia issue #6875 on GitHub +`_). An alternative +to ABCs would be to create a macro that checks that a particular class +or class instance has all of the required methods. + +Python interface +---------------- + +The implementation shall be written entirely in Julia, but shall have +a Python interface. The interface may either be included in the +PlasmaPy core package or as an affiliated package. + +Issues, Pull Requests, and Branches +=================================== + + + +Backward Compatibility +====================== + +Creation of this general purpose plasma simulator may necessitate +changes to base classes such as ``Plasma`` which are still under +development. + +Alternatives +============ + + + +Decision Rationale +================== From 9ce55a6eb506380d3557c2cd74facacce7c29dc4 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Tue, 18 Dec 2018 19:46:48 -0500 Subject: [PATCH 03/27] Update PLEP on plasma simulator I moved the discussion on alternative languages to the alternatives section, and put it into the form of a bullet point list. --- PLEP-0007.rst | 118 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 85 insertions(+), 33 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 2985bae..f75396a 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -11,7 +11,7 @@ PLEP-0007 – A Next Generation Plasma Simulator +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2018-12-17 | +| date last revised | 2018-12-18 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -31,12 +31,23 @@ plasma simulator should be developed with sufficient modularity and flexibility to enable straightforward switching of both physical models and numerical methods. -Detailed Description -==================== +*Due to the magnitude of this project, this PLEP will be implemented +only if sufficient funding for code development becomes available.* + +Background +========== + +Numerical simulation is a powerful research tool that allows us to +investigate plasma phenomena such as disruptions in tokamaks, solar +eruptions, and accretion into black holes. Numerical methods for +plasma simulation are continually advancing. Current Status of Plasma Simulation ----------------------------------- +* **Development of numerical methods for plasma simulations are very + advanced.** + * **Codes are usually written in low level compiled languages such as Fortran and C.** While these compiled languages offer excellent performance, code development is significantly slowed down because @@ -71,8 +82,6 @@ Current Status of Plasma Simulation abbreviated variable names that obfuscate what the variable represents. -* **Code is often difficult to maintain.** - * **Benchmarking two algorithms or comparing physical models usually requires setting up the same problem in multiple codes.** Usually the same case must be set up by an experienced or expert user of @@ -81,18 +90,27 @@ Current Status of Plasma Simulation be matched. The data are often stored in different formats, so * **The code for the problem setup, physical model, and numerical - method are often intertwined with each other.** + method are often intertwined with each other.** When these aspects * **Simulation output is rarely stored in a standardized form.** Few standards exist (with the exception of OpenPMD). The lack of a standardized format makes developing general purpose plotting software much more difficult. -* **Research projects involving plasma simulations are almost always - difficult to reproduce.** +* **Code is often difficult to maintain.** + +* **Research projects involving plasma simulations are difficult to + reproduce.** + +Detailed Description +==================== + + + + Development Principles ----------------------- +====================== * **Optimize for both productivity and performance.** Plasma simulation should be straightforward and not require expertise into @@ -128,34 +146,12 @@ Development Principles not depend upon low level modules. Abstractions should not depend upon details. Details should depend upon abstractions. - Implementation ============== Choice of language ------------------ -Most plasma simulation software is currently written in low level -languages such as Fortran and C. These languages offer excellent -performance. However, development tends to be slow because the -languages are not interactive and types must be declared. - -Pure Python code is slow because Python is an interpreted language. -Considerable speedup can be achieved using NumPy, though NumPy does -not provide the performance of compiled languages. Performance -comparable to compiled languages can be achieved using several -methods. Python can call code from Fortran and C, but with the -drawbacks that (1) the implementations in code are often clunkier; (2) -developers must understand and maintain code in more than one -language; and (3) difficulties with compilers sometimes arise. Cython -is a superset of the Python language. By adding type declarations, -Cython is able to generate C code from Python-like code which is then -compiled in order to give very good performance. In practice, we -found Cython to be difficult to implement in PlasmaPy. Numba provides -a just-in-time compiler to get compiled speeds at run-time. However, -Numba compiles different functions separately and cannot do global -optimizations. - Julia is a new high-level open source language that synthesizes the best features of Fortran, C, Python, R, MATLAB, and Lisp for scientific computing. Julia uses a just-in-time compiler with type @@ -190,7 +186,7 @@ PlasmaPy core package or as an affiliated package. Issues, Pull Requests, and Branches =================================== - +* Backward Compatibility ====================== @@ -202,7 +198,63 @@ development. Alternatives ============ - +Julia is not the only language that could be used for this project. +The main alternatives are listed below. The most significant +disadvantages are shown in bold. + +* **Fortran**, **C**, or **C++** + - Advantages + - Exceptional performance as a compiled language + - Many plasma physicists have considerable knowledge and + experience with these languages + - Can call code from Python + - Disadvantages + - **Development is greatly slowed because the language is not + interactive** + - Code in these languages is often several times the length of + equivalent code in Julia or Python + - Continuous integration testing platforms such as Travis CI do + not directly support testing of Fortran code + - Experience with Fortran is less likely to be helpful for plasma + physics students who find jobs outside of research and academia + +* **Python with NumPy** + - Advantages + - Fast development + - No need to compile code + - Disadvantage + - **Slow performance as an interpreted language** + - NumPy does not offer a sufficient speedup + +* **Python with Cython** + - Advantages + - Because Cython allows us to generate C code from Python-like + code, we are able to get compiled speeds + - Allows for a Python interface + - Disadvantages + - **Difficult to work with and maintain** + +* **Python with Numba** + - Advantages + - Uses a just-in-time compiler to get compiled speeds + - Often decorating a function with ``numba.jit`` is enough to get + compiled speeds + - Disadvantages + - Because Numba compiles one function at a time, it may be unable + to do global optimizations while compiling + - **Does not yet provide a full language solution to the + performance vs. productivity conundrum** + +* **LuaJIT** + - Advantages + - Uses a just-in-time compiler to get compiled speeds + - Offers exceptional performance, including for graphical + processing units (GPUs) + - Disadvantages + - **Less active community surrounding scientific LuaJIT** + - Fewer scientific libraries written in LuaJIT Decision Rationale ================== + +This PLEP has not been decided upon yet. From 161ed06cd867773e6cb12fbab934e1497f23fe65 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Wed, 2 Jan 2019 19:46:39 -0500 Subject: [PATCH 04/27] Update PLEP on creating a plasma simulator This is still a work in progress and I am still trying to figure out the best way to structure this PLEP. --- PLEP-0007.rst | 114 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 83 insertions(+), 31 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index f75396a..fd599fb 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -11,7 +11,7 @@ PLEP-0007 – A Next Generation Plasma Simulator +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2018-12-18 | +| date last revised | 2019-01-02 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -40,19 +40,21 @@ Background Numerical simulation is a powerful research tool that allows us to investigate plasma phenomena such as disruptions in tokamaks, solar eruptions, and accretion into black holes. Numerical methods for -plasma simulation are continually advancing. +plasma simulation are continually advancing. Current Status of Plasma Simulation ----------------------------------- -* **Development of numerical methods for plasma simulations are very - advanced.** +* **Numerical methods for plasma simulations are very advanced.** + Plasma scientists have developed highly sophisticated numerical + methods for simulating the behavior of laboratory, heliospheric, + space, and astrophysical plasmas. -* **Codes are usually written in low level compiled languages such as - Fortran and C.** While these compiled languages offer excellent - performance, code development is significantly slowed down because - the languages are not interactive and variable types must be - declared. +* **Plasma simulation codes are usually written in low level compiled + languages such as Fortran and C.** While these compiled languages + offer excellent performance, code development is significantly slowed + down because the languages are not interactive and variable types must + be declared. * **Access to codes is commonly restricted in some way.** Closed source codes hinder scientific reproducibility. Open source codes @@ -71,23 +73,21 @@ Current Status of Plasma Simulation compile code exacerbates this problem. * **Codes frequently lack unit testing and continuous integration - testing frameworks.** These testing strategies help us improve the + testing frameworks.** These testing strategies help us improve the reliability of the code, track down bugs as soon as they appear, and greatly reduce the likelihood that we will introduce bugs as we - modify or maintain the code. Continuous integration testing - platforms such as Travis CI do not support direct testing of Fortran - code. + modify or maintain the code. -* **Code is often diffucult to read.** Often code uses highly +* **Code is often difficult to read.** Codes commonly use highly abbreviated variable names that obfuscate what the variable - represents. + represents, especially when there are no comments that describe what * **Benchmarking two algorithms or comparing physical models usually requires setting up the same problem in multiple codes.** Usually the same case must be set up by an experienced or expert user of each code. Often this requires careful discussions to make sure that things like normalizations and boundary conditions are able to - be matched. The data are often stored in different formats, so + be matched. The data are often stored in different formats, so * **The code for the problem setup, physical model, and numerical method are often intertwined with each other.** When these aspects @@ -99,12 +99,47 @@ Current Status of Plasma Simulation * **Code is often difficult to maintain.** -* **Research projects involving plasma simulations are difficult to - reproduce.** +* **Research projects involving plasma simulations are usually very + difficult to reproduce.** + +Project Goals +============= + +We propose to develop a flexible open source framework for plasma +simulation. The choice of Julia will allow us to optimize both +performance and productivity. Julia's use of a JIT compiler allows +programs to achieve performance comparable to compiled languages like +Fortran and C. Productivity is enhanced because Julia is a dynamically +typed language that can be used interactively and because +parallelization is a built-in feature of the language. Flexibility will +be achieved by taking a modular approach and using well-defined abstract +interfaces. Detailed Description ==================== +Project Goals +------------- + +* **Allow users to perform high quality plasma simulations with the + minimal amount of effort and required knowledge.** + +* **Allow users to switch between physical models and numerical methods + with minimal effort.** + +* **Optimize for both productivity and performance.** + +* **Code should be readable, and self-documenting when possible.** + +* **Provide readable and maintainable code.** Code should be + self-documenting when possible. + +* **Make it hard for users to make mistakes.** + +* **Provide a package that can be installed in one or two lines.** + +* **Begin development using well-understood, published numerical + methods.** @@ -112,19 +147,38 @@ Detailed Description Development Principles ====================== -* **Optimize for both productivity and performance.** Plasma - simulation should be straightforward and not require expertise into - the details of the code. Optimize code only after it works, and - then only after it becomes apparent what the bottlenecks are. +* **Make the code as modular as possible.** + +* **Optimize for both productivity and performance.** Plasma simulation + should be straightforward. Running plasma simulations should not + require expertise into the details of the code. Code should be + optimized for performance only after it works, and then only after it + becomes apparent what the bottlenecks are. -* **Prioritize readability, usability, and maintainability.** Code - should be straightforward to install. +* **Prioritize usability, readability, and maintainability.** Code + should be straightforward to install. The packages resulting from + this project should all be installable by using Julia's built-in + package manager. + +* **The software should be easy to install.** The plasma simulation + package should be installable using Julia's built-in package manager. + Users shall not be required to run bash scripts or install libraries. * **Prioritize documentation.** The documentation should be sufficient for a student taking their first plasma physics class to be able to use it without too much difficulty. -* **Make the code as modular as possible.** +* **Prioritize flexible numerical methods.** There are a number of + numerical methods that have been developed for systems of equations + written in conservative form. + +* **Minimize the amount of work and knowledge required by the end + user.** + +* **Make it difficult for users to make silent mistakes.** + +* **Make it straightforward to perform convergence studies and to test + solutions using different numerical methods.** * **Use the SOLID principles for software development.** @@ -160,8 +214,8 @@ and Fortran. Unlike C and Fortran, Julia can be run interactively which greatly speeds up code development and allows prototyping in the same language to be used for performance runs. Julia natively supports parallelization and can call code from Fortran, C, and -Python. Julia proves that high performance can be acheived with a -dynamic interactive language. +Python. Julia proves that high performance can be achieved with a +dynamically typed interactive language. Abstract Interfaces ------------------- @@ -186,8 +240,6 @@ PlasmaPy core package or as an affiliated package. Issues, Pull Requests, and Branches =================================== -* - Backward Compatibility ====================== @@ -214,7 +266,7 @@ disadvantages are shown in bold. - Code in these languages is often several times the length of equivalent code in Julia or Python - Continuous integration testing platforms such as Travis CI do - not directly support testing of Fortran code + not support direct testing of Fortran code - Experience with Fortran is less likely to be helpful for plasma physics students who find jobs outside of research and academia @@ -232,7 +284,7 @@ disadvantages are shown in bold. code, we are able to get compiled speeds - Allows for a Python interface - Disadvantages - - **Difficult to work with and maintain** + - **Cython is difficult to work with and maintain** * **Python with Numba** - Advantages From 3674be089bff6f1cfc3de0653db745f9709f18a2 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 4 Jan 2019 18:42:29 -0500 Subject: [PATCH 05/27] Expand PLEP on plasma simulator --- PLEP-0007.rst | 206 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 175 insertions(+), 31 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index fd599fb..650b270 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -42,8 +42,34 @@ investigate plasma phenomena such as disruptions in tokamaks, solar eruptions, and accretion into black holes. Numerical methods for plasma simulation are continually advancing. -Current Status of Plasma Simulation ------------------------------------ + +Major problems include: + +* Verification and validation (how do we compare against reality?) + +Open source plasma simulation codes include: + +Current Status of Plasma Simulation Research +-------------------------------------------- + +Major successes include: + +* Highly sophisticated simulations of tokamaks + +* Reaching petascale computing + +* Turbulence studies + +* GRMHD, simulation of plasmas in extreme environments + +Sample major topics include: + +* Whole device simulation of tokamaks +* Multi-scale simulations of magnetic reconnection in the solar corona + (where PIC and fluid codes are important) + +Current Status of Plasma Simulation Software +-------------------------------------------- * **Numerical methods for plasma simulations are very advanced.** Plasma scientists have developed highly sophisticated numerical @@ -115,12 +141,6 @@ parallelization is a built-in feature of the language. Flexibility will be achieved by taking a modular approach and using well-defined abstract interfaces. -Detailed Description -==================== - -Project Goals -------------- - * **Allow users to perform high quality plasma simulations with the minimal amount of effort and required knowledge.** @@ -141,12 +161,157 @@ Project Goals * **Begin development using well-understood, published numerical methods.** +Anticipated User Experience +=========================== + +A requirement of this package is to allow users to specify the problem +setup, physical model, and numerical method as independently as +possible. This separation of responsibilities is necessary to allow +users to straightforwardly switch between different systems of equations +and computational algorithms. + +Users will first create a problem setup object or module that should +contain all of the information necessary to set up a simulation +independent of physical model and numerical method. + +* Coordinate system +* Physical domain +* Initial conditions +* Boundary conditions +* Time interval + +The choice of coordinate system will provide the dimensionality (i.e., +if the simulation is 1D, 2D, or 3D). The physical domain should be +capable of being multiply connected or with a more complicated geometry +(such as a stellarator). The initial and boundary conditions will +provide a list of the dependent variables. The initial conditions +should be able to be specified by + +* Choosing a pre-defined standard setup +* Specifying functions for different fields +* Passing in arrays of values +* Using helper tools such as a Grad-Shafranov solver + +The boundary conditions should be able to be specified by + +* Choosing pre-defined boundary conditions (e.g., periodic or no-slip + conducting wall boundaries) +* Specifying functions for different fields along different boundaries + +For fluid simulations, users will create a physical model object that +contains the system of equations. + + +.. Users will next define the system of equations or physical model to be + solved. It is at this point that users will choose the style of + simulation (including but not limited to fluid, particle-in-cell, and + hybrid approaches). The physical model will be checked to be consistent + with the initial and boundary conditions. + +.. Users will then define the numerical method and specify inputs for the + +.. Users define the problem setup. This is done independently of the + physical model (with the caveat that all fields have to be defined or + assumed to be zero, and also be physically realizable). The package + will check that all three are consistent with each other. + - Physical domain + - Coordinate system + - Dimensionality + - Size + - Allow for multiply connected geometries (like in MRX) + - Initial conditions + - Boundary conditions + - Have pre-set BCs for things like a conducting wall. BCs can be + messy to implement, like putting them into a +* Users define the physical model. + - Options for user input + - Pre-defined sets of equations with options to specify different + coefficients (like resistive MHD with uniform, Spitzer, anomalous, + or a user-defined function) + - List of strings containing the different equations + - If the equations are in conservative form (including with sources + and sinks) then + - We can have pre-defined sets of equations + - We can have pre-defined sets of equations as a string (including + unicode characters) following Dedalus approach + - This can be done best if numerical method can be automagically + generated + - Julia allows us to pass functions around as arguments (A function + can have a function as an argument, and a function can return a + function) + - We can sometimes use pre-set equations (like resistive MHD, with + uniform or temperature dependent or anomalous resistivity) + - Numerical method + * Post-processing + - Maybe we could create a function that automatically writes text that + describes the numerical method and such. + +Proposed Package Structure +========================== + +.. Mathematical functions + - Basis functions that are not defined in other packages + * Physics coefficients + - Resistivities + - Transport coefficients + - Plasma parameters + * Built-in grid tools + - Methods for creating a grid + - Should be able to define: + - Finite difference grids (including staggered grids) + - Finite volume grids (including staggered grids) + - Finite element and spectral element grids + - Including for multiply defined geometries + - Ways to specify grids for finite difference and finite volume + methods (including on staggered grids) + - Ways to specify grids +Choice of language +================== +Julia is a new high-level open source language that synthesizes the best +features of Fortran, C, Python, R, MATLAB, and Lisp for scientific +computing. Julia uses a just-in-time (JIT) compiler with type inference +and multiple dispatch to achieve performance comparable to C and +Fortran. Unlike C and Fortran, Julia can be run interactively and does +not require type declarations. These features greatly speed up code +development by allowing prototyping in the same language to be used for +performance runs. Julia natively supports parallelization, and has been +used to achieve petascale computing. **Julia proves that high +performance can be achieved with a dynamically typed interactive +language without sacrificing usability.** + +Julia can call code from Fortran and C, and can act as a wrapper for +codes written in these compiled languages. The main drawback of this +approach is that the resulting code would be harder to maintain because +developers would need to know two or three languages. A potential +drawback is that problems can arise in practice when code in one +language is called from a different language. Global optimizations +might also not be possible when mixing more than one language (though +Julia may become capable of optimizations across language boundaries in +the future. If possible, the package itself should be written entirely +in Julia and depend only on packages that can be installed using Julia's +built-in package manager. Users shall *not* be required to compile or +install any external libraries or use any shell scripts. + +The implementation shall be written entirely in Julia, but shall have a +Python interface. The interface may either be included in the PlasmaPy +core package or in an affiliated package. The ``Plasma`` class should +be able to handle the output of simulations performed using this plasma +simulation framework. Development Principles ====================== +* **Optimize for both productivity and performance.** With Julia, it is + possible to have the performance of a compiled language with the + +.. Plasma simulation + should be straightforward + +* **Develop the overall architecture under the point of view that we + have not decided on the numerical method or physical model yet.** + * **Make the code as modular as possible.** * **Optimize for both productivity and performance.** Plasma simulation @@ -200,22 +365,8 @@ Development Principles not depend upon low level modules. Abstractions should not depend upon details. Details should depend upon abstractions. -Implementation -============== - -Choice of language ------------------- - -Julia is a new high-level open source language that synthesizes the -best features of Fortran, C, Python, R, MATLAB, and Lisp for -scientific computing. Julia uses a just-in-time compiler with type -inference and multiple dispatch to acheive performance comparable to C -and Fortran. Unlike C and Fortran, Julia can be run interactively -which greatly speeds up code development and allows prototyping in the -same language to be used for performance runs. Julia natively -supports parallelization and can call code from Fortran, C, and -Python. Julia proves that high performance can be achieved with a -dynamically typed interactive language. +Implementation Notes +==================== Abstract Interfaces ------------------- @@ -230,13 +381,6 @@ in Julia (see `Julia issue #6875 on GitHub to ABCs would be to create a macro that checks that a particular class or class instance has all of the required methods. -Python interface ----------------- - -The implementation shall be written entirely in Julia, but shall have -a Python interface. The interface may either be included in the -PlasmaPy core package or as an affiliated package. - Issues, Pull Requests, and Branches =================================== From 798c2cac27bdbe08d96d93dd95cf8a3e067939f4 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 4 Jan 2019 20:12:33 -0500 Subject: [PATCH 06/27] Describe user experience for plasma simulator --- PLEP-0007.rst | 43 ++++++++++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 11 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 650b270..7f68417 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -170,6 +170,9 @@ possible. This separation of responsibilities is necessary to allow users to straightforwardly switch between different systems of equations and computational algorithms. +Defining the problem +-------------------- + Users will first create a problem setup object or module that should contain all of the information necessary to set up a simulation independent of physical model and numerical method. @@ -182,25 +185,33 @@ independent of physical model and numerical method. The choice of coordinate system will provide the dimensionality (i.e., if the simulation is 1D, 2D, or 3D). The physical domain should be -capable of being multiply connected or with a more complicated geometry -(such as a stellarator). The initial and boundary conditions will -provide a list of the dependent variables. The initial conditions -should be able to be specified by +capable of being multiply connected or a more complicated geometry (such +as a stellarator). The initial and boundary conditions will provide a +list of the dependent variables. The initial conditions should be able +to be specified by: * Choosing a pre-defined standard setup * Specifying functions for different fields * Passing in arrays of values * Using helper tools such as a Grad-Shafranov solver -The boundary conditions should be able to be specified by +The boundary conditions should be able to be specified by: * Choosing pre-defined boundary conditions (e.g., periodic or no-slip conducting wall boundaries) -* Specifying functions for different fields along different boundaries +* Specifying functions or conditions that need to be met for different + fields along different boundaries -For fluid simulations, users will create a physical model object that -contains the system of equations. +Specifying the numerical method +------------------------------- +.. Right now I am not certain how to deal with boundary conditions, and + how we could treat them similarly for fluid and PIC simulations. + If we have a problem setup for a fluid case that has Dirichlet BCs + for density, then how do we transfer that to a PIC case? Should we + have BCs defined separately? Or include them in the numerical method + definition maybe? Maybe we should have a way to convert a fluid + simulation setup into a PIC .. Users will next define the system of equations or physical model to be solved. It is at this point that users will choose the style of @@ -306,9 +317,6 @@ Development Principles * **Optimize for both productivity and performance.** With Julia, it is possible to have the performance of a compiled language with the -.. Plasma simulation - should be straightforward - * **Develop the overall architecture under the point of view that we have not decided on the numerical method or physical model yet.** @@ -368,6 +376,19 @@ Development Principles Implementation Notes ==================== +Boundary Conditions +------------------- + +A goal of this effort is to make the setup of fluid, particle, and +hybrid simulations as similar as possible. Ideally, the same problem +setup object should be able to be used to initialize all of these +different types of simulations as similarly as possible. However, the +formulation of boundary conditions between fluid and PIC simulations can +be substantially different and potentially incompatible. + +.. I'm not sure how to handle this yet, particularly because I do not + know enough about boundary conditions for PIC simulations. -Nick + Abstract Interfaces ------------------- From 7b7c80d14623e1b4b03435335c36ed07babbb7d5 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Tue, 12 Feb 2019 00:52:03 -0500 Subject: [PATCH 07/27] Continue to write/revise PLEP on plasma simulator --- PLEP-0007.rst | 361 ++++++++++++++++++++++++++++---------------------- 1 file changed, 201 insertions(+), 160 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 7f68417..2fe8f41 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -11,7 +11,7 @@ PLEP-0007 – A Next Generation Plasma Simulator +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-01-02 | +| date last revised | 2019-02-11 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -24,140 +24,158 @@ PLEP-0007 – A Next Generation Plasma Simulator Abstract ======== -This proposal describes the current status of simulation software in -the plasma physics community and describes a next generation general -purpose plasma simulation framework in the PlasmaPy ecosystem. This -plasma simulator should be developed with sufficient modularity and -flexibility to enable straightforward switching of both physical -models and numerical methods. - -*Due to the magnitude of this project, this PLEP will be implemented -only if sufficient funding for code development becomes available.* +Most plasma simulation codes are written in low-level compiled +languages, solve a particular class of equations, and use a specific +numerical method. These codes often lack sufficient documentation and +tests, and are difficult to install and modify. Codes tend to not be +interoperable with each other: if you wish to change numerical +methods, compare different classes of equations, or perform a +benchmark, then you generally need to set up the same problem in a +different code. + +This proposal describes a plan for a modular plasma simulation +framework within the PlasmaPy ecosystem. The framework will be +written in Julia and also have a Python interface. The initial +conditions and domain specification will be defined separately from +the system of equations and numerical method. Consequently, the same +problem setup can be used for different systems of equations and +numerical methods. The users will be able to input the system of +equations and boundary conditions in strings that will be interpreted +by the framework. Tools to discretize the system of equations +directly will be used when possible to maximize flexibility. The +framework will be capable of performing fluid, kinetic, and hybrid +simulations. The output will be provided in standardized forms. Background ========== Numerical simulation is a powerful research tool that allows us to -investigate plasma phenomena such as disruptions in tokamaks, solar -eruptions, and accretion into black holes. Numerical methods for -plasma simulation are continually advancing. - - -Major problems include: - -* Verification and validation (how do we compare against reality?) - -Open source plasma simulation codes include: - -Current Status of Plasma Simulation Research --------------------------------------------- - -Major successes include: - -* Highly sophisticated simulations of tokamaks - -* Reaching petascale computing - -* Turbulence studies - -* GRMHD, simulation of plasmas in extreme environments - -Sample major topics include: - -* Whole device simulation of tokamaks -* Multi-scale simulations of magnetic reconnection in the solar corona - (where PIC and fluid codes are important) - -Current Status of Plasma Simulation Software +investigate plasma phenomena such as tokamak disruptions, solar +eruptions, geomagnetic storms, and accretion into black holes. +Numerical simulations can be used to understand experiments and act as +a bridge between laboratory, heliospheric, and astrophysical plasma +phenomena. The numerical techniques used to simulate plasmas are +sophisticated and continually advancing. It is essential that the +PlasmaPy ecosystem include flexible plasma simulation capabilities +that allow for high performance but without sacrificing usability. + +Current status of plasma simulation software -------------------------------------------- -* **Numerical methods for plasma simulations are very advanced.** - Plasma scientists have developed highly sophisticated numerical - methods for simulating the behavior of laboratory, heliospheric, - space, and astrophysical plasmas. - -* **Plasma simulation codes are usually written in low level compiled - languages such as Fortran and C.** While these compiled languages - offer excellent performance, code development is significantly slowed - down because the languages are not interactive and variable types must - be declared. - -* **Access to codes is commonly restricted in some way.** Closed - source codes hinder scientific reproducibility. Open source codes - often have a lower density of bugs because more people are able to - review the code and make contributions. - -* **Codes often lack sufficient documentation.** Often it is - necessary to work closely with a core developer of a code in order - to set up a problem. - -* **Code compilation and installation is often difficult and - time-consuming.** This is particularly true for codes that depend on - external libraries or include multiple languages. Compilation often - requires detailed knowledge of makefiles which is a significant +Before + +* **Numerical methods for plasma simulations are highly advanced.** + Plasma scientists have developed and implemented highly + sophisticated numerical methods for simulating the behavior of + laboratory, heliospheric, space, and astrophysical plasmas in a wide + variety of different regimes. + +* **Plasma simulation codes are usually written in compiled languages + such as Fortran and C.** These compiled languages offer excellent + performance, but code development is slower because these languages + are not interactive and types must be declared. + +* **Access to codes is commonly restricted in some way.** Plasma + simulation codes are often not released under an open source + license, especially for codes used to simulate burning plasmas. + Often prospective users must sign a user agreement form that + restricts modification and/or redistribution. + +.. Using open source codes greatly improves scientific + reproducibility. Open source codes often have a lower density of + bugs because more people are able to review the code and make + contributions. + +* **Codes often lack sufficient documentation.** Scientists are + generally under high pressure to publish in order to get tenure, win + grants, or get hired for their next job. Documentation then becomes + a lower priority. When documentation is inadequate, it is often + necessary to work closely with a core developer with limited time in + order to set up or modify a problem. + +* **Installing and compiling codes is often difficult and + time-consuming.** When plasma codes depend on external libraries or + include multiple languages, users must often change compiler flags + or edit make files. This requirement can provide a significant barrier to entry for newcomers. Lack of documentation on how to - compile code exacerbates this problem. + install and compile code can exacerbate this problem. * **Codes frequently lack unit testing and continuous integration - testing frameworks.** These testing strategies help us improve the + testing frameworks.** These testing strategies help to improve the reliability of the code, track down bugs as soon as they appear, and - greatly reduce the likelihood that we will introduce bugs as we - modify or maintain the code. - -* **Code is often difficult to read.** Codes commonly use highly - abbreviated variable names that obfuscate what the variable - represents, especially when there are no comments that describe what + greatly reduce the likelihood that new bugs will be introduced when + the code is modified or maintained. -* **Benchmarking two algorithms or comparing physical models usually - requires setting up the same problem in multiple codes.** Usually - the same case must be set up by an experienced or expert user of - each code. Often this requires careful discussions to make sure - that things like normalizations and boundary conditions are able to - be matched. The data are often stored in different formats, so - -* **The code for the problem setup, physical model, and numerical - method are often intertwined with each other.** When these aspects +* **Code is often difficult to read.** Codes commonly use highly + abbreviated names that obfuscate the meanings of variables, + functions, and classes. Often there are no or inadequate comments + to describe the meanings of these objects. * **Simulation output is rarely stored in a standardized form.** Few - standards exist (with the exception of OpenPMD). The lack of a - standardized format makes developing general purpose plotting - software much more difficult. - -* **Code is often difficult to maintain.** - -* **Research projects involving plasma simulations are usually very - difficult to reproduce.** + standards or specifications exist for the storage of plasma + simulation output (with the exception of OpenPMD). When output is + not kept in a standardized form, then different software is needed + to access and analyze the results. The shortage of standardized + formats makes it more difficult to develop general purpose analysis + and plotting software. + +.. The way yt does it, I think, is to have different software to read + in the outputs of different simulations. We could incorporate our + new standards into yt. + +* **Benchmarking algorithms or comparing physical models usually + requires setting up the same problem for multiple codes.** Each code + generally has unique methods for defining initial conditions, + boundary conditions, and the computational domain. The problem + setup for one code cannot be easily transferred to another code. + Typically the same case must be set up by an experienced user of + each code. Often benchmarks requires careful discussions to make + sure that normalizations and boundary conditions are able to be + matched, and different analysis software must often be used. + +.. * **The code for the problem setup, physical model, and numerical + method are often intertwined with each other.** When these aspects + +.. * **Code is often difficult to maintain.** + +.. * **Research projects involving plasma simulations are usually very + difficult to reproduce.** Project Goals ============= -We propose to develop a flexible open source framework for plasma -simulation. The choice of Julia will allow us to optimize both -performance and productivity. Julia's use of a JIT compiler allows -programs to achieve performance comparable to compiled languages like -Fortran and C. Productivity is enhanced because Julia is a dynamically -typed language that can be used interactively and because -parallelization is a built-in feature of the language. Flexibility will -be achieved by taking a modular approach and using well-defined abstract -interfaces. +We propose to develop a flexible modular open source framework for +plasma simulation as part of the PlasmaPy ecosystem. We will develop +this framework using Julia (with a Python interface) so that we may +optimize both performance and productivity. Julia's use of a JIT +compiler allows programs to achieve performance comparable to compiled +languages like Fortran and C. Productivity is enhanced because Julia +is a dynamically typed language that can be used interactively and +because parallelization is a built-in feature of the language. +Flexibility will be achieved by taking a modular approach and using +well-defined abstract interfaces. + +The highly ambitious development goals are to: * **Allow users to perform high quality plasma simulations with the minimal amount of effort and required knowledge.** -* **Allow users to switch between physical models and numerical methods - with minimal effort.** - * **Optimize for both productivity and performance.** -* **Code should be readable, and self-documenting when possible.** - -* **Provide readable and maintainable code.** Code should be - self-documenting when possible. +* **Provide a package that can be installed in a single line.** + +* **Develop readable and maintainable code that is self-documenting + when possible.** + +* **Allow users to switch to a different physical model or to a + different numerical method with minimal effort.** -* **Make it hard for users to make mistakes.** - -* **Provide a package that can be installed in one or two lines.** +* **Make it hard for users to make mistakes while providing useful + error messages.** +* **Implement fluid, particle-in-cell (PIC), and hybrid simulation + capabilities.** + * **Begin development using well-understood, published numerical methods.** @@ -170,28 +188,28 @@ possible. This separation of responsibilities is necessary to allow users to straightforwardly switch between different systems of equations and computational algorithms. -Defining the problem --------------------- +Defining the problem setup +-------------------------- -Users will first create a problem setup object or module that should -contain all of the information necessary to set up a simulation -independent of physical model and numerical method. +Users will first instantiate a class or create a module that contains +all of the information needed to set up the physical problem, including: -* Coordinate system +* Coordinate system and dimensionality * Physical domain * Initial conditions * Boundary conditions * Time interval -The choice of coordinate system will provide the dimensionality (i.e., -if the simulation is 1D, 2D, or 3D). The physical domain should be -capable of being multiply connected or a more complicated geometry (such -as a stellarator). The initial and boundary conditions will provide a -list of the dependent variables. The initial conditions should be able -to be specified by: +The physical domain should be capable of being multiply connected or a +more complicated geometry (such as a stellarator). + +The initial and boundary conditions will provide a list of the +dependent variables. The initial conditions should be able to be +specified by: * Choosing a pre-defined standard setup -* Specifying functions for different fields +* Specifying functions for different fields (either as callable + objects or string representations of the equations) * Passing in arrays of values * Using helper tools such as a Grad-Shafranov solver @@ -200,18 +218,35 @@ The boundary conditions should be able to be specified by: * Choosing pre-defined boundary conditions (e.g., periodic or no-slip conducting wall boundaries) * Specifying functions or conditions that need to be met for different - fields along different boundaries + fields along different boundaries (either as callable objects or + string representations of the equations) + +Choosing the physical model +--------------------------- + +Users will choose what general class of simulation to perform, +including but not limited to fluid, PIC, and hybrid. + +For simulations using the fluid approximation, users will have the +option to choose pre-defined systems of equations such as resistive +MHD, Hall MHD, and so on. + +If the equations are in conservative form (including with sources and +sinks), then more general numerical methods may be used. Specifying the numerical method ------------------------------- +If the users choose a PIC simulation, then they will get to choose the +particle pusher for the time advance. + .. Right now I am not certain how to deal with boundary conditions, and how we could treat them similarly for fluid and PIC simulations. If we have a problem setup for a fluid case that has Dirichlet BCs for density, then how do we transfer that to a PIC case? Should we have BCs defined separately? Or include them in the numerical method definition maybe? Maybe we should have a way to convert a fluid - simulation setup into a PIC + simulation setup into a PIC simulation. .. Users will next define the system of equations or physical model to be solved. It is at this point that users will choose the style of @@ -225,15 +260,8 @@ Specifying the numerical method physical model (with the caveat that all fields have to be defined or assumed to be zero, and also be physically realizable). The package will check that all three are consistent with each other. - - Physical domain - - Coordinate system - - Dimensionality - - Size - - Allow for multiply connected geometries (like in MRX) - - Initial conditions - - Boundary conditions - - Have pre-set BCs for things like a conducting wall. BCs can be - messy to implement, like putting them into a + + * Users define the physical model. - Options for user input - Pre-defined sets of equations with options to specify different @@ -305,21 +333,28 @@ in Julia and depend only on packages that can be installed using Julia's built-in package manager. Users shall *not* be required to compile or install any external libraries or use any shell scripts. -The implementation shall be written entirely in Julia, but shall have a -Python interface. The interface may either be included in the PlasmaPy -core package or in an affiliated package. The ``Plasma`` class should -be able to handle the output of simulations performed using this plasma -simulation framework. +The implementation shall be written entirely in Julia, and shall have +a Python interface in addition to a Julia interface. The interface +may either be included in the PlasmaPy core package or in an +affiliated package. The ``Plasma`` class should be able to handle the +output of simulations performed using this plasma simulation +framework. Development Principles ====================== -* **Optimize for both productivity and performance.** With Julia, it is - possible to have the performance of a compiled language with the +* **Optimize for both productivity and performance.** Most plasma + simulation codes prioritize performance over user-friendliness and + usability. With Julia, we can achieve high performance without + sacrificing usability. +* **Make the code as modular as possible.** Separate the initial + conditions from the system of equations and the numerical method. + * **Develop the overall architecture under the point of view that we have not decided on the numerical method or physical model yet.** + * **Make the code as modular as possible.** * **Optimize for both productivity and performance.** Plasma simulation @@ -333,13 +368,15 @@ Development Principles this project should all be installable by using Julia's built-in package manager. -* **The software should be easy to install.** The plasma simulation - package should be installable using Julia's built-in package manager. - Users shall not be required to run bash scripts or install libraries. +* **Installation should be simple.** After installing Julia, users + should only be required to type in one line in order to install the + plasma simulation package using Julia's built-in package manager. + Users shall not be required to run shell scripts or install + libraries manually. -* **Prioritize documentation.** The documentation should be - sufficient for a student taking their first plasma physics class to - be able to use it without too much difficulty. +* **Prioritize documentation.** The documentation should be sufficient + for a student taking their first plasma physics class to be able to + use it without too much difficulty. * **Prioritize flexible numerical methods.** There are a number of numerical methods that have been developed for systems of equations @@ -421,23 +458,25 @@ disadvantages are shown in bold. * **Fortran**, **C**, or **C++** - Advantages - - Exceptional performance as a compiled language + - Exceptional performance as compiled languages - Many plasma physicists have considerable knowledge and experience with these languages - - Can call code from Python + - Codes can be called from other languages like Python and Julia - Disadvantages - - **Development is greatly slowed because the language is not + - **Productivity is reduced because these languages are not interactive** - Code in these languages is often several times the length of equivalent code in Julia or Python - Continuous integration testing platforms such as Travis CI do not support direct testing of Fortran code - - Experience with Fortran is less likely to be helpful for plasma - physics students who find jobs outside of research and academia + - Experience with Fortran is less helpful for plasma physics + students searching for jobs outside of research and academia + - Limited metaprogramming capabilities * **Python with NumPy** - Advantages - - Fast development + - Very fast development + - Useful for prototyping - No need to compile code - Disadvantage - **Slow performance as an interpreted language** @@ -445,22 +484,24 @@ disadvantages are shown in bold. * **Python with Cython** - Advantages - - Because Cython allows us to generate C code from Python-like - code, we are able to get compiled speeds - - Allows for a Python interface + - C code generated from Cython provides compiled speeds + - Better usability for end users because they can interact with a + Python interface - Disadvantages + - Does not provide a whole-language solution - **Cython is difficult to work with and maintain** - + * **Python with Numba** - Advantages - Uses a just-in-time compiler to get compiled speeds - Often decorating a function with ``numba.jit`` is enough to get compiled speeds - Disadvantages - - Because Numba compiles one function at a time, it may be unable - to do global optimizations while compiling - - **Does not yet provide a full language solution to the - performance vs. productivity conundrum** + - Because Numba compiles one function at a time, it is unable to + do global optimizations while compiling + - Not currently well-suited for massively parallel computing + - **Does not provide a full language solution to the performance + vs. productivity conundrum** * **LuaJIT** - Advantages From 86bc7c28135c04a694de95fd364ff383c2126b57 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Tue, 12 Feb 2019 20:20:43 -0500 Subject: [PATCH 08/27] Continue writing PLEP on plasma simulator --- PLEP-0007.rst | 171 ++++++++++++++++++++++++++------------------------ 1 file changed, 90 insertions(+), 81 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 2fe8f41..78b262a 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -11,7 +11,7 @@ PLEP-0007 – A Next Generation Plasma Simulator +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-02-11 | +| date last revised | 2019-02-12 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -26,25 +26,25 @@ Abstract Most plasma simulation codes are written in low-level compiled languages, solve a particular class of equations, and use a specific -numerical method. These codes often lack sufficient documentation and -tests, and are difficult to install and modify. Codes tend to not be +numerical method. These codes often lack sufficient documentation and +tests, and are difficult to install and modify. Codes tend to not be interoperable with each other: if you wish to change numerical methods, compare different classes of equations, or perform a benchmark, then you generally need to set up the same problem in a different code. This proposal describes a plan for a modular plasma simulation -framework within the PlasmaPy ecosystem. The framework will be -written in Julia and also have a Python interface. The initial +framework within the PlasmaPy ecosystem. The framework will be +written in Julia and also have a Python interface. The initial conditions and domain specification will be defined separately from -the system of equations and numerical method. Consequently, the same +the system of equations and numerical method. Consequently, the same problem setup can be used for different systems of equations and -numerical methods. The users will be able to input the system of +numerical methods. The users will be able to input the system of equations and boundary conditions in strings that will be interpreted -by the framework. Tools to discretize the system of equations -directly will be used when possible to maximize flexibility. The +by the framework. Tools to discretize the system of equations +directly will be used when possible to maximize flexibility. The framework will be capable of performing fluid, kinetic, and hybrid -simulations. The output will be provided in standardized forms. +simulations. The output will be provided in standardized forms. Background ========== @@ -54,15 +54,17 @@ investigate plasma phenomena such as tokamak disruptions, solar eruptions, geomagnetic storms, and accretion into black holes. Numerical simulations can be used to understand experiments and act as a bridge between laboratory, heliospheric, and astrophysical plasma -phenomena. The numerical techniques used to simulate plasmas are -sophisticated and continually advancing. It is essential that the +phenomena. The numerical techniques used to simulate plasmas are +sophisticated and continually advancing. It is essential that the PlasmaPy ecosystem include flexible plasma simulation capabilities that allow for high performance but without sacrificing usability. Current status of plasma simulation software -------------------------------------------- -Before +Before describing our proposal in detail, we should first describe the +current status of plasma simulation based largely on personal +experience. * **Numerical methods for plasma simulations are highly advanced.** Plasma scientists have developed and implemented highly @@ -70,7 +72,7 @@ Before laboratory, heliospheric, space, and astrophysical plasmas in a wide variety of different regimes. -* **Plasma simulation codes are usually written in compiled languages +* **Plasma simulation codes are usually written in low-level languages such as Fortran and C.** These compiled languages offer excellent performance, but code development is slower because these languages are not interactive and types must be declared. @@ -79,25 +81,20 @@ Before simulation codes are often not released under an open source license, especially for codes used to simulate burning plasmas. Often prospective users must sign a user agreement form that - restricts modification and/or redistribution. - -.. Using open source codes greatly improves scientific - reproducibility. Open source codes often have a lower density of - bugs because more people are able to review the code and make - contributions. + restricts modification and/or redistribution. * **Codes often lack sufficient documentation.** Scientists are generally under high pressure to publish in order to get tenure, win - grants, or get hired for their next job. Documentation then becomes - a lower priority. When documentation is inadequate, it is often + grants, or get hired for their next job. Documentation then becomes + a lower priority. When documentation is inadequate, it is often necessary to work closely with a core developer with limited time in order to set up or modify a problem. * **Installing and compiling codes is often difficult and time-consuming.** When plasma codes depend on external libraries or include multiple languages, users must often change compiler flags - or edit make files. This requirement can provide a significant - barrier to entry for newcomers. Lack of documentation on how to + or edit make files. This requirement can provide a significant + barrier to entry for newcomers. Lack of documentation on how to install and compile code can exacerbate this problem. * **Codes frequently lack unit testing and continuous integration @@ -106,50 +103,62 @@ Before greatly reduce the likelihood that new bugs will be introduced when the code is modified or maintained. -* **Code is often difficult to read.** Codes commonly use highly - abbreviated names that obfuscate the meanings of variables, - functions, and classes. Often there are no or inadequate comments - to describe the meanings of these objects. +* **Code is often difficult to read and therefore maintain.** Codes + commonly use highly abbreviated names that obfuscate the meanings of + variables, functions, and classes. Often there are no or inadequate + comments to describe the meanings of these objects. * **Simulation output is rarely stored in a standardized form.** Few standards or specifications exist for the storage of plasma - simulation output (with the exception of OpenPMD). When output is + simulation output (with the exception of OpenPMD). When output is not kept in a standardized form, then different software is needed - to access and analyze the results. The shortage of standardized + to access and analyze the results. The shortage of standardized formats makes it more difficult to develop general purpose analysis and plotting software. -.. The way yt does it, I think, is to have different software to read - in the outputs of different simulations. We could incorporate our - new standards into yt. - * **Benchmarking algorithms or comparing physical models usually requires setting up the same problem for multiple codes.** Each code - generally has unique methods for defining initial conditions, - boundary conditions, and the computational domain. The problem + generally has unique conventions for defining initial conditions, + boundary conditions, and the computational domain. The problem setup for one code cannot be easily transferred to another code. Typically the same case must be set up by an experienced user of - each code. Often benchmarks requires careful discussions to make + each code. Often benchmarks require careful discussions to make sure that normalizations and boundary conditions are able to be matched, and different analysis software must often be used. +* **Reproducing plasma simulation results usually requires significant + effort.** All source code and raw data must be available and usable + in order for a computational result to be replicable. While the + software packages used to perform simulations are often open source, + the specific version of the software is rarely cited. The code used + to set up a particular simulation and analyze its results is usually + not cited or available for inspection or reuse. + +.. Still working on next bullet point. + +* **Code bases often contain interdependencies.** The code for the + problem setup, physical model, and numerical method are often + intertwined with each other. + +.. Dependency diagrams often look like "hairball" diagrams. + +.. Often code is not broken up into separate functions. Spaghetti code. + .. * **The code for the problem setup, physical model, and numerical - method are often intertwined with each other.** When these aspects + method are often intertwined with each other.** -.. * **Code is often difficult to maintain.** +.. High-level code is often intermixed with low-level code. -.. * **Research projects involving plasma simulations are usually very - difficult to reproduce.** Project Goals ============= We propose to develop a flexible modular open source framework for -plasma simulation as part of the PlasmaPy ecosystem. We will develop +plasma simulation as part of the PlasmaPy ecosystem. We will develop this framework using Julia (with a Python interface) so that we may -optimize both performance and productivity. Julia's use of a JIT +optimize both performance and productivity. Julia's use of a JIT compiler allows programs to achieve performance comparable to compiled -languages like Fortran and C. Productivity is enhanced because Julia +languages like Fortran and C. Productivity is enhanced because Julia is a dynamically typed language that can be used interactively and because parallelization is a built-in feature of the language. Flexibility will be achieved by taking a modular approach and using @@ -163,10 +172,10 @@ The highly ambitious development goals are to: * **Optimize for both productivity and performance.** * **Provide a package that can be installed in a single line.** - + * **Develop readable and maintainable code that is self-documenting when possible.** - + * **Allow users to switch to a different physical model or to a different numerical method with minimal effort.** @@ -175,7 +184,7 @@ The highly ambitious development goals are to: * **Implement fluid, particle-in-cell (PIC), and hybrid simulation capabilities.** - + * **Begin development using well-understood, published numerical methods.** @@ -184,7 +193,7 @@ Anticipated User Experience A requirement of this package is to allow users to specify the problem setup, physical model, and numerical method as independently as -possible. This separation of responsibilities is necessary to allow +possible. This separation of responsibilities is necessary to allow users to straightforwardly switch between different systems of equations and computational algorithms. @@ -204,7 +213,7 @@ The physical domain should be capable of being multiply connected or a more complicated geometry (such as a stellarator). The initial and boundary conditions will provide a list of the -dependent variables. The initial conditions should be able to be +dependent variables. The initial conditions should be able to be specified by: * Choosing a pre-defined standard setup @@ -249,18 +258,18 @@ particle pusher for the time advance. simulation setup into a PIC simulation. .. Users will next define the system of equations or physical model to be - solved. It is at this point that users will choose the style of + solved. It is at this point that users will choose the style of simulation (including but not limited to fluid, particle-in-cell, and - hybrid approaches). The physical model will be checked to be consistent + hybrid approaches). The physical model will be checked to be consistent with the initial and boundary conditions. .. Users will then define the numerical method and specify inputs for the -.. Users define the problem setup. This is done independently of the +.. Users define the problem setup. This is done independently of the physical model (with the caveat that all fields have to be defined or - assumed to be zero, and also be physically realizable). The package + assumed to be zero, and also be physically realizable). The package will check that all three are consistent with each other. - + * Users define the physical model. - Options for user input @@ -310,33 +319,33 @@ Choice of language Julia is a new high-level open source language that synthesizes the best features of Fortran, C, Python, R, MATLAB, and Lisp for scientific -computing. Julia uses a just-in-time (JIT) compiler with type inference +computing. Julia uses a just-in-time (JIT) compiler with type inference and multiple dispatch to achieve performance comparable to C and -Fortran. Unlike C and Fortran, Julia can be run interactively and does -not require type declarations. These features greatly speed up code +Fortran. Unlike C and Fortran, Julia can be run interactively and does +not require type declarations. These features greatly speed up code development by allowing prototyping in the same language to be used for -performance runs. Julia natively supports parallelization, and has been -used to achieve petascale computing. **Julia proves that high +performance runs. Julia natively supports parallelization, and has been +used to achieve petascale computing. **Julia proves that high performance can be achieved with a dynamically typed interactive language without sacrificing usability.** Julia can call code from Fortran and C, and can act as a wrapper for -codes written in these compiled languages. The main drawback of this +codes written in these compiled languages. The main drawback of this approach is that the resulting code would be harder to maintain because -developers would need to know two or three languages. A potential +developers would need to know two or three languages. A potential drawback is that problems can arise in practice when code in one -language is called from a different language. Global optimizations +language is called from a different language. Global optimizations might also not be possible when mixing more than one language (though Julia may become capable of optimizations across language boundaries in -the future. If possible, the package itself should be written entirely +the future. If possible, the package itself should be written entirely in Julia and depend only on packages that can be installed using Julia's -built-in package manager. Users shall *not* be required to compile or +built-in package manager. Users shall *not* be required to compile or install any external libraries or use any shell scripts. The implementation shall be written entirely in Julia, and shall have -a Python interface in addition to a Julia interface. The interface +a Python interface in addition to a Julia interface. The interface may either be included in the PlasmaPy core package or in an -affiliated package. The ``Plasma`` class should be able to handle the +affiliated package. The ``Plasma`` class should be able to handle the output of simulations performed using this plasma simulation framework. @@ -345,26 +354,26 @@ Development Principles * **Optimize for both productivity and performance.** Most plasma simulation codes prioritize performance over user-friendliness and - usability. With Julia, we can achieve high performance without + usability. With Julia, we can achieve high performance without sacrificing usability. * **Make the code as modular as possible.** Separate the initial conditions from the system of equations and the numerical method. - + * **Develop the overall architecture under the point of view that we have not decided on the numerical method or physical model yet.** - + * **Make the code as modular as possible.** * **Optimize for both productivity and performance.** Plasma simulation - should be straightforward. Running plasma simulations should not - require expertise into the details of the code. Code should be + should be straightforward. Running plasma simulations should not + require expertise into the details of the code. Code should be optimized for performance only after it works, and then only after it becomes apparent what the bottlenecks are. * **Prioritize usability, readability, and maintainability.** Code - should be straightforward to install. The packages resulting from + should be straightforward to install. The packages resulting from this project should all be installable by using Julia's built-in package manager. @@ -407,8 +416,8 @@ Development Principles forced to depend upon interfaces that they do not use. - The *dependency inversion principle*: High level modules should - not depend upon low level modules. Abstractions should not depend - upon details. Details should depend upon abstractions. + not depend upon low level modules. Abstractions should not depend + upon details. Details should depend upon abstractions. Implementation Notes ==================== @@ -417,9 +426,9 @@ Boundary Conditions ------------------- A goal of this effort is to make the setup of fluid, particle, and -hybrid simulations as similar as possible. Ideally, the same problem +hybrid simulations as similar as possible. Ideally, the same problem setup object should be able to be used to initialize all of these -different types of simulations as similarly as possible. However, the +different types of simulations as similarly as possible. However, the formulation of boundary conditions between fluid and PIC simulations can be substantially different and potentially incompatible. @@ -432,10 +441,10 @@ Abstract Interfaces `Abstract base classes `_ (ABCs) in Python allow users to define what methods and attributes must be defined in a -subclass of that ABC. This functionality is used in PlasmaPy's -``Plasma`` class. An equivalent to ABCs has not yet been implemented +subclass of that ABC. This functionality is used in PlasmaPy's +``Plasma`` class. An equivalent to ABCs has not yet been implemented in Julia (see `Julia issue #6875 on GitHub -`_). An alternative +`_). An alternative to ABCs would be to create a macro that checks that a particular class or class instance has all of the required methods. @@ -453,7 +462,7 @@ Alternatives ============ Julia is not the only language that could be used for this project. -The main alternatives are listed below. The most significant +The main alternatives are listed below. The most significant disadvantages are shown in bold. * **Fortran**, **C**, or **C++** @@ -490,7 +499,7 @@ disadvantages are shown in bold. - Disadvantages - Does not provide a whole-language solution - **Cython is difficult to work with and maintain** - + * **Python with Numba** - Advantages - Uses a just-in-time compiler to get compiled speeds From c108e2808bcfdd35923ec8b3093876900a0f253e Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 22 Feb 2019 17:56:51 -0500 Subject: [PATCH 09/27] Expand PLEP on plasma simulator --- PLEP-0007.rst | 332 +++++++++++++++++++++++++++----------------------- 1 file changed, 178 insertions(+), 154 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 78b262a..7995d85 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -11,7 +11,7 @@ PLEP-0007 – A Next Generation Plasma Simulator +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-02-12 | +| date last revised | 2019-02-22 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -134,8 +134,6 @@ experience. to set up a particular simulation and analyze its results is usually not cited or available for inspection or reuse. -.. Still working on next bullet point. - * **Code bases often contain interdependencies.** The code for the problem setup, physical model, and numerical method are often intertwined with each other. @@ -144,49 +142,42 @@ experience. .. Often code is not broken up into separate functions. Spaghetti code. -.. * **The code for the problem setup, physical model, and numerical - method are often intertwined with each other.** - .. High-level code is often intermixed with low-level code. Project Goals ============= -We propose to develop a flexible modular open source framework for -plasma simulation as part of the PlasmaPy ecosystem. We will develop -this framework using Julia (with a Python interface) so that we may -optimize both performance and productivity. Julia's use of a JIT -compiler allows programs to achieve performance comparable to compiled -languages like Fortran and C. Productivity is enhanced because Julia -is a dynamically typed language that can be used interactively and -because parallelization is a built-in feature of the language. -Flexibility will be achieved by taking a modular approach and using -well-defined abstract interfaces. - -The highly ambitious development goals are to: - -* **Allow users to perform high quality plasma simulations with the - minimal amount of effort and required knowledge.** - -* **Optimize for both productivity and performance.** - -* **Provide a package that can be installed in a single line.** - -* **Develop readable and maintainable code that is self-documenting - when possible.** +We propose to create an open source framework for plasma simulation +within the PlasmaPy ecosystem. The goal of this framework is to +enable users to perform high quality simulations with the minimum +amount of effort and without requiring knowledge of the details of the +implementation. Code design will prioritize usability, readability, +and maintainability. Optimization of performance should not come at +the cost of decreased user-friendliness. The functionality for +initial conditions, boundary conditions, physical model, and numerical +method should be separated from each other with abstract interfaces. +Users should be provided with the flexibility to switch to a different +physical model or different numerical method with minimal effort. +This framework should include fluid, particle-in-cell (PIC), and +hybrid simulation capabilities using a variety of well-described +numerical methods. + +.. Minimize the amount of time, effort, and frustration to needed to + reach scientific understanding. + +Requirements +============ -* **Allow users to switch to a different physical model or to a - different numerical method with minimal effort.** +* Allow the same problem setup to be used for different systems of + equations and numerical methods. -* **Make it hard for users to make mistakes while providing useful - error messages.** +* Allow users to change the system of equations or numerical method in + the middle of a simulation with minimal effort. -* **Implement fluid, particle-in-cell (PIC), and hybrid simulation - capabilities.** +* Be able to use the same analysis techniques on -* **Begin development using well-understood, published numerical - methods.** +* Stretch goals: Anticipated User Experience =========================== @@ -233,22 +224,34 @@ The boundary conditions should be able to be specified by: Choosing the physical model --------------------------- -Users will choose what general class of simulation to perform, -including but not limited to fluid, PIC, and hybrid. +Users will choose between fluid, PIC, and hybrid simulations. + +For simulations using the fluid approximation, users will either +specify the equations in strings that will be parsed or select +pre-defined systems of equations such as resistive MHD or Hall MHD. +Users will add source and sink terms as necessary and choose models +for dissipation coefficients. + +.. If the equations are in conservative form (including with sources and + sinks), then more general numerical methods may be used. -For simulations using the fluid approximation, users will have the -option to choose pre-defined systems of equations such as resistive -MHD, Hall MHD, and so on. -If the equations are in conservative form (including with sources and -sinks), then more general numerical methods may be used. Specifying the numerical method ------------------------------- -If the users choose a PIC simulation, then they will get to choose the +For simulations using the fluid approximation, users will choose +between different finite difference, finite volume, finite/spectral +element, and spectral methods. If possible, the code for the +numerical method will be generated from the specified equations +(though this may require that the equations be specified in +conservative form). + +If the users choose a PIC simulation, then they will choose the particle pusher for the time advance. +At this point, users will specify the numerical input parameters. + .. Right now I am not certain how to deal with boundary conditions, and how we could treat them similarly for fluid and PIC simulations. If we have a problem setup for a fluid case that has Dirichlet BCs @@ -263,56 +266,147 @@ particle pusher for the time advance. hybrid approaches). The physical model will be checked to be consistent with the initial and boundary conditions. -.. Users will then define the numerical method and specify inputs for the - -.. Users define the problem setup. This is done independently of the - physical model (with the caveat that all fields have to be defined or - assumed to be zero, and also be physically realizable). The package - will check that all three are consistent with each other. - - -* Users define the physical model. - - Options for user input - - Pre-defined sets of equations with options to specify different +.. Users define the physical model. + Options for user input + Pre-defined sets of equations with options to specify different coefficients (like resistive MHD with uniform, Spitzer, anomalous, or a user-defined function) - - List of strings containing the different equations - - If the equations are in conservative form (including with sources + List of strings containing the different equations + If the equations are in conservative form (including with sources and sinks) then - - We can have pre-defined sets of equations - - We can have pre-defined sets of equations as a string (including - unicode characters) following Dedalus approach - - This can be done best if numerical method can be automagically - generated - - Julia allows us to pass functions around as arguments (A function + We can have pre-defined sets of equations + We can have pre-defined sets of equations as a string (including + unicode characters) following Dedalus approach + This can be done best if numerical method can be automagically generated + Julia allows us to pass functions around as arguments (A function can have a function as an argument, and a function can return a function) - - We can sometimes use pre-set equations (like resistive MHD, with - uniform or temperature dependent or anomalous resistivity) - - Numerical method - * Post-processing - - Maybe we could create a function that automatically writes text that - describes the numerical method and such. + We can sometimes use pre-set equations (like resistive MHD, with + uniform or temperature dependent or anomalous resistivity) + Numerical method + Post-processing + Maybe we could create a function that automatically writes text that + describes the numerical method and such. + +Grid generation +--------------- + +.. More detail needed on grid generation. Need to discuss mesh + packing capabilities and how to generate complicated grids. For + finite element simulations, more information on the mapping will be + necessary, but might not be worth discussing here. + +Users will be able to generate the grid after the domain is specified +and the numerical method is chosen. + +Performing the simulation +------------------------- + +Users will have varying amounts of control over how the simulation is +performed. If no special processing is required, then users would be +able to perform the simulation in a single command. + +Development Principles +====================== + +Code development should take place using best practices for scientific +and object-oriented programming. + + * **Optimize for both productivity and performance.** High + performance should not come at the cost of decreased usability. + + * **Provide and maintain thorough documentation.** The documentation + should be sufficient for a student beginning their first research + project in plasma physics. + + * **Make installation simple.** Allow the framework to be quickly and + reliably installed using one command with a package manager. Users + should not be required to manually install external libraries, edit + make files, or run installation scripts. + +.. * **Plan the program architecture in advance.** Separate the + different aspects of the + + * **Define abstract interfaces between the objects that define the + problem setup and numerical method.** + Program each side to the interface. + + +.. **Make the code as modular as possible.** Separate the initial + conditions from the system of equations and the numerical method. + +.. **Develop the overall architecture under the point of view that we + have not decided on the numerical method or physical model yet.** + +.. **Make the code as modular as possible.** + +.. **Optimize for both productivity and performance.** Plasma + simulation should be straightforward. Running plasma simulations + should not require expertise into the details of the code. Code + should be optimized for performance only after it works, and then + only after it becomes apparent what the bottlenecks are. + +.. **Prioritize usability, readability, and maintainability.** Code + should be straightforward to install. The packages resulting from + this project should all be installable by using Julia's built-in + package manager. + + +.. **Prioritize documentation.** The documentation should be + sufficient for a student taking their first plasma physics class to + be able to use it without too much difficulty. + +.. **Prioritize flexible numerical methods.** There are a number of + numerical methods that have been developed for systems of equations + written in conservative form. + +.. **Minimize the amount of work and knowledge required by the end + user.** + +.. **Make it difficult for users to make silent mistakes.** + +.. **Make it straightforward to perform convergence studies and to + test solutions using different numerical methods.** + +.. **Use the SOLID principles for software development.** + + The *single responsibility principle*: There should never be more + than one reason for a class to change. + + The *open-closed principle*: Software entities (classes, modules, + functions, etc.) should be open for extension but closed for + modification. + + The *Liskov substitution principle*: Objects in a program should + be replaceable with instances of their subtypes without altering + the correctness of that program. + + The *interface segregation principle*: Clients should not be + forced to depend upon interfaces that they do not use. + + The *dependency inversion principle*: High level modules should + not depend upon low level modules. Abstractions should not depend + upon details. Details should depend upon abstractions. Proposed Package Structure ========================== .. Mathematical functions - - Basis functions that are not defined in other packages - * Physics coefficients - - Resistivities - - Transport coefficients - - Plasma parameters - * Built-in grid tools - - Methods for creating a grid - - Should be able to define: - - Finite difference grids (including staggered grids) - - Finite volume grids (including staggered grids) - - Finite element and spectral element grids - - Including for multiply defined geometries - - Ways to specify grids for finite difference and finite volume - methods (including on staggered grids) - - Ways to specify grids + Basis functions that are not defined in other packages + Physics coefficients + Resistivities + Transport coefficients + Plasma parameters + Built-in grid tools + Methods for creating a grid + Should be able to define: + Finite difference grids (including staggered grids) + Finite volume grids (including staggered grids) + Finite element and spectral element grids + Including for multiply defined geometries + Ways to specify grids for FD and FV methods (incl. on staggered grids) + Ways to specify grids + Choice of language ================== @@ -349,76 +443,6 @@ affiliated package. The ``Plasma`` class should be able to handle the output of simulations performed using this plasma simulation framework. -Development Principles -====================== - -* **Optimize for both productivity and performance.** Most plasma - simulation codes prioritize performance over user-friendliness and - usability. With Julia, we can achieve high performance without - sacrificing usability. - -* **Make the code as modular as possible.** Separate the initial - conditions from the system of equations and the numerical method. - -* **Develop the overall architecture under the point of view that we - have not decided on the numerical method or physical model yet.** - - -* **Make the code as modular as possible.** - -* **Optimize for both productivity and performance.** Plasma simulation - should be straightforward. Running plasma simulations should not - require expertise into the details of the code. Code should be - optimized for performance only after it works, and then only after it - becomes apparent what the bottlenecks are. - -* **Prioritize usability, readability, and maintainability.** Code - should be straightforward to install. The packages resulting from - this project should all be installable by using Julia's built-in - package manager. - -* **Installation should be simple.** After installing Julia, users - should only be required to type in one line in order to install the - plasma simulation package using Julia's built-in package manager. - Users shall not be required to run shell scripts or install - libraries manually. - -* **Prioritize documentation.** The documentation should be sufficient - for a student taking their first plasma physics class to be able to - use it without too much difficulty. - -* **Prioritize flexible numerical methods.** There are a number of - numerical methods that have been developed for systems of equations - written in conservative form. - -* **Minimize the amount of work and knowledge required by the end - user.** - -* **Make it difficult for users to make silent mistakes.** - -* **Make it straightforward to perform convergence studies and to test - solutions using different numerical methods.** - -* **Use the SOLID principles for software development.** - - - The *single responsibility principle*: There should never be more - than one reason for a class to change. - - - The *open-closed principle*: Software entities (classes, modules, - functions, etc.) should be open for extension but closed for - modification. - - - The *Liskov substitution principle*: Objects in a program should - be replaceable with instances of their subtypes without altering - the correctness of that program. - - - The *interface segregation principle*: Clients should not be - forced to depend upon interfaces that they do not use. - - - The *dependency inversion principle*: High level modules should - not depend upon low level modules. Abstractions should not depend - upon details. Details should depend upon abstractions. - Implementation Notes ==================== From c20b89b3895e37f41a767f6c094d86eb40b8389a Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 22 Feb 2019 21:39:24 -0500 Subject: [PATCH 10/27] Revise abstract of plasma simulator PLEP --- PLEP-0007.rst | 53 +++++++++++++++++++++++++++++++-------------------- 1 file changed, 32 insertions(+), 21 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 7995d85..af09b65 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -24,27 +24,38 @@ PLEP-0007 – A Next Generation Plasma Simulator Abstract ======== -Most plasma simulation codes are written in low-level compiled -languages, solve a particular class of equations, and use a specific -numerical method. These codes often lack sufficient documentation and -tests, and are difficult to install and modify. Codes tend to not be -interoperable with each other: if you wish to change numerical -methods, compare different classes of equations, or perform a -benchmark, then you generally need to set up the same problem in a -different code. - -This proposal describes a plan for a modular plasma simulation -framework within the PlasmaPy ecosystem. The framework will be -written in Julia and also have a Python interface. The initial -conditions and domain specification will be defined separately from -the system of equations and numerical method. Consequently, the same -problem setup can be used for different systems of equations and -numerical methods. The users will be able to input the system of -equations and boundary conditions in strings that will be interpreted -by the framework. Tools to discretize the system of equations -directly will be used when possible to maximize flexibility. The -framework will be capable of performing fluid, kinetic, and hybrid -simulations. The output will be provided in standardized forms. +This proposal describes a plan for a flexible plasma simulation +framework for the PlasmaPy ecosystem. Most plasma simulation codes +solve a particular class of equations and use a specific numerical +method. These codes generally lack interoperability with each other. +In order to change numerical methods, compare different classes of +equations, or perform a benchmark, then it is necessary to set up the +same problem in different codes with different conventions for +defining initial and boundary conditions. By creating a boundary with +a clear interface that separates the problem setup from the physical +model and numerical method, the same problem setup can be used with +different physical models and numerical methods. Such boundaries will +enable the framework to include different numerical method and +standard analysis modules with the same high level user interface that +can therefore be used largely interchangeably. Code development will +equally prioritizer usability, maintainability, and performance. User +documentation will be written to be useful to students and scientists +doing their first research project in plasma physics. + + +.. This proposal describes a plan for a modular plasma simulation + framework within the PlasmaPy ecosystem. The framework will be + written in Julia and also have a Python interface. The initial + conditions and domain specification will be defined separately from + the system of equations and numerical method. Consequently, the + same problem setup can be used for different systems of equations + and numerical methods. The users will be able to input the system + of equations and boundary conditions in strings that will be + interpreted by the framework. Tools to discretize the system of + equations directly will be used when possible to maximize + flexibility. The framework will be capable of performing fluid, + kinetic, and hybrid simulations. The output will be provided in + standardized forms. Background ========== From 45fcf5c26e5b4515945eb7265b7c4379a0ba4166 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Sat, 23 Feb 2019 16:23:59 -0500 Subject: [PATCH 11/27] Re-revise abstract of plasma simulator PLEP The current abstract might be a bit long, but that might be okay since there are a lot of important points I want to make. --- PLEP-0007.rst | 42 ++++++++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index af09b65..626844a 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -24,23 +24,33 @@ PLEP-0007 – A Next Generation Plasma Simulator Abstract ======== -This proposal describes a plan for a flexible plasma simulation -framework for the PlasmaPy ecosystem. Most plasma simulation codes +This document describes a proposed flexible plasma simulation +framework within the PlasmaPy ecosystem. Most plasma simulation codes solve a particular class of equations and use a specific numerical -method. These codes generally lack interoperability with each other. -In order to change numerical methods, compare different classes of -equations, or perform a benchmark, then it is necessary to set up the -same problem in different codes with different conventions for -defining initial and boundary conditions. By creating a boundary with -a clear interface that separates the problem setup from the physical -model and numerical method, the same problem setup can be used with -different physical models and numerical methods. Such boundaries will -enable the framework to include different numerical method and -standard analysis modules with the same high level user interface that -can therefore be used largely interchangeably. Code development will -equally prioritizer usability, maintainability, and performance. User -documentation will be written to be useful to students and scientists -doing their first research project in plasma physics. +method. These codes usually lack interoperability with each other. +Changing to a different physical model or numerical method often +requires setting up the same problem in a different code with +different conventions for defining initial and boundary conditions. +By designing a boundary with a clear interface that separates the +problem setup from the physical model and numerical method, the same +problem setup can be used with different physical models and numerical +methods. Such boundaries will enable the framework to include +different numerical methods and analysis modules with the same high +level user interface that can therefore be used largely +interchangeably. The framework should include its own interchangeable +numerical methods for fluid, particle-in-cell, and hybrid simulations, +and also allow for existing open source codes to be called via +wrappers designed to this interface. Output will be created in +standardized forms. The framework should include general methods for +solution of systems of partial differential equations written in +conservative form that can be inputted by the user in strings that are +processed by the framework. Code development will equally prioritize +readability, usability, reliability, maintainability, and performance. +User documentation will be aimed toward both students taking their +first course in plasma physics as well as experienced researchers. +The documentation will include advice to users on how best choose a +physical model and numerical method and describe potential pitfalls +for different types of simulations. .. This proposal describes a plan for a modular plasma simulation From 305a48c3260869d60b6a231bb37f3171c4a17317 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Mon, 25 Feb 2019 00:18:08 -0500 Subject: [PATCH 12/27] Update current status of plasma simulation --- PLEP-0007.rst | 77 ++++++++++++++++++++++++++------------------------- 1 file changed, 39 insertions(+), 38 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 626844a..9980811 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -1,6 +1,6 @@ -============================================== -PLEP-0007 – A Next Generation Plasma Simulator -============================================== +============================================================================== +PLEP-0007 – Plasma Simulation Capabilities for the PlasmaPy Software Ecosystem +============================================================================== +-------------------+---------------------------------------------+ | PLEP | number | @@ -11,7 +11,7 @@ PLEP-0007 – A Next Generation Plasma Simulator +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-02-22 | +| date last revised | 2019-02-24 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -24,7 +24,7 @@ PLEP-0007 – A Next Generation Plasma Simulator Abstract ======== -This document describes a proposed flexible plasma simulation +This document proposed the development of a flexible plasma simulation framework within the PlasmaPy ecosystem. Most plasma simulation codes solve a particular class of equations and use a specific numerical method. These codes usually lack interoperability with each other. @@ -52,21 +52,6 @@ The documentation will include advice to users on how best choose a physical model and numerical method and describe potential pitfalls for different types of simulations. - -.. This proposal describes a plan for a modular plasma simulation - framework within the PlasmaPy ecosystem. The framework will be - written in Julia and also have a Python interface. The initial - conditions and domain specification will be defined separately from - the system of equations and numerical method. Consequently, the - same problem setup can be used for different systems of equations - and numerical methods. The users will be able to input the system - of equations and boundary conditions in strings that will be - interpreted by the framework. Tools to discretize the system of - equations directly will be used when possible to maximize - flexibility. The framework will be capable of performing fluid, - kinetic, and hybrid simulations. The output will be provided in - standardized forms. - Background ========== @@ -83,20 +68,36 @@ that allow for high performance but without sacrificing usability. Current status of plasma simulation software -------------------------------------------- -Before describing our proposal in detail, we should first describe the -current status of plasma simulation based largely on personal -experience. - -* **Numerical methods for plasma simulations are highly advanced.** - Plasma scientists have developed and implemented highly - sophisticated numerical methods for simulating the behavior of - laboratory, heliospheric, space, and astrophysical plasmas in a wide - variety of different regimes. - +.. For this section, I am wondering if it would be best to structure + it like: + + Problem: We often have spaghetti code. This makes unit testing + difficult because there are no distinct units. + + Solution: Break up code into multiple functions. Make sure there + are tests for each function. + +Before making a proposal for the future, it is worth considering +common programming practices and the current culture within plasma +physics. A critical evaluation of the current status of plasma +simulation software based on experiences with multiple codes will +provide insight into how the plasma simulation community can improve +in the future. + +* **Numerical methods for plasma simulations are highly advanced and + continually advancing.** Plasma scientists have developed and + implemented highly sophisticated numerical methods for simulating + the behavior of laboratory, heliospheric, space, and astrophysical + plasmas in a wide variety of different regimes. + +.. The above point should make sure to strongly value the + contributions that have been made by numerous scientists on how to + improve plasma simulation. + * **Plasma simulation codes are usually written in low-level languages such as Fortran and C.** These compiled languages offer excellent performance, but code development is slower because these languages - are not interactive and types must be declared. + are not interactive and types must be declared. * **Access to codes is commonly restricted in some way.** Plasma simulation codes are often not released under an open source @@ -155,17 +156,17 @@ experience. to set up a particular simulation and analyze its results is usually not cited or available for inspection or reuse. -* **Code bases often contain interdependencies.** The code for the - problem setup, physical model, and numerical method are often - intertwined with each other. +* **Codes often lack clear boundaries between different components.** + The code for the problem setup, physical model, and numerical method + are often intertwined with each other. Different components + sometimes contain interdependencies with each other. -.. Dependency diagrams often look like "hairball" diagrams. +* **High-level code is often intermixed with low-level code.** Code + related to the time advance can be intermixed with MPI calls related + to transferring data between processors. .. Often code is not broken up into separate functions. Spaghetti code. -.. High-level code is often intermixed with low-level code. - - Project Goals ============= From cbcee710a78652c5aa5cf7df07c9afb29a551a70 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Mon, 25 Feb 2019 21:59:08 -0500 Subject: [PATCH 13/27] Restructure section on current status of plasma simulation I changed this section so that it includes a list of problems or current practices, each of which is followed by solutions or alternatives. I think this approach is more constructive. The downside of this section is that it is becoming very long. However, many of these problems may not be recognized as problems by people doing plasma simulations, so it is important to name them in the language the dragons speak and offer potential solutions that can be implemented in this framework. --- PLEP-0007.rst | 530 ++++++++++++++++++++++++++------------------------ 1 file changed, 278 insertions(+), 252 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 9980811..cf30f17 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -1,6 +1,6 @@ -============================================================================== -PLEP-0007 – Plasma Simulation Capabilities for the PlasmaPy Software Ecosystem -============================================================================== +======================================================= +PLEP-0007 – Plasma Simulation Capabilities for PlasmaPy +======================================================= +-------------------+---------------------------------------------+ | PLEP | number | @@ -11,7 +11,7 @@ PLEP-0007 – Plasma Simulation Capabilities for the PlasmaPy Software Ecosystem +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-02-24 | +| date last revised | 2019-02-25 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -24,33 +24,30 @@ PLEP-0007 – Plasma Simulation Capabilities for the PlasmaPy Software Ecosystem Abstract ======== -This document proposed the development of a flexible plasma simulation -framework within the PlasmaPy ecosystem. Most plasma simulation codes -solve a particular class of equations and use a specific numerical -method. These codes usually lack interoperability with each other. +We propose to develop a flexible plasma simulation framework for the +PlasmaPy software ecosystem. Most plasma simulation codes solve a +particular class of equations and use a specific numerical method. +Different codes generally lack interoperability with each other. Changing to a different physical model or numerical method often requires setting up the same problem in a different code with different conventions for defining initial and boundary conditions. By designing a boundary with a clear interface that separates the problem setup from the physical model and numerical method, the same problem setup can be used with different physical models and numerical -methods. Such boundaries will enable the framework to include -different numerical methods and analysis modules with the same high -level user interface that can therefore be used largely -interchangeably. The framework should include its own interchangeable -numerical methods for fluid, particle-in-cell, and hybrid simulations, -and also allow for existing open source codes to be called via -wrappers designed to this interface. Output will be created in -standardized forms. The framework should include general methods for -solution of systems of partial differential equations written in -conservative form that can be inputted by the user in strings that are -processed by the framework. Code development will equally prioritize -readability, usability, reliability, maintainability, and performance. -User documentation will be aimed toward both students taking their -first course in plasma physics as well as experienced researchers. -The documentation will include advice to users on how best choose a -physical model and numerical method and describe potential pitfalls -for different types of simulations. +methods. The framework should include interchangeable numerical +methods for fluid, particle-in-cell, gyrokinetic, and hybrid +simulations. Boundaries and interfaces should be designed to allow +existing codes to implement them. The framework should include general +methods for solution of systems of partial differential equations +written in conservative form that can be inputted by the user in +strings that are processed by the framework. Code development will +equally prioritize readability, usability, reliability, +maintainability, and performance. User documentation will be aimed +toward both students taking their first course in plasma physics as +well as experienced researchers. The documentation will include +advice to users on how best choose a physical model and numerical +method and describe potential pitfalls for different types of +simulations. Background ========== @@ -60,123 +57,231 @@ investigate plasma phenomena such as tokamak disruptions, solar eruptions, geomagnetic storms, and accretion into black holes. Numerical simulations can be used to understand experiments and act as a bridge between laboratory, heliospheric, and astrophysical plasma -phenomena. The numerical techniques used to simulate plasmas are -sophisticated and continually advancing. It is essential that the -PlasmaPy ecosystem include flexible plasma simulation capabilities -that allow for high performance but without sacrificing usability. - -Current status of plasma simulation software --------------------------------------------- - -.. For this section, I am wondering if it would be best to structure - it like: - - Problem: We often have spaghetti code. This makes unit testing - difficult because there are no distinct units. - - Solution: Break up code into multiple functions. Make sure there - are tests for each function. - -Before making a proposal for the future, it is worth considering -common programming practices and the current culture within plasma -physics. A critical evaluation of the current status of plasma -simulation software based on experiences with multiple codes will -provide insight into how the plasma simulation community can improve -in the future. - -* **Numerical methods for plasma simulations are highly advanced and - continually advancing.** Plasma scientists have developed and - implemented highly sophisticated numerical methods for simulating - the behavior of laboratory, heliospheric, space, and astrophysical - plasmas in a wide variety of different regimes. - -.. The above point should make sure to strongly value the - contributions that have been made by numerous scientists on how to - improve plasma simulation. - -* **Plasma simulation codes are usually written in low-level languages - such as Fortran and C.** These compiled languages offer excellent - performance, but code development is slower because these languages - are not interactive and types must be declared. - -* **Access to codes is commonly restricted in some way.** Plasma - simulation codes are often not released under an open source - license, especially for codes used to simulate burning plasmas. - Often prospective users must sign a user agreement form that - restricts modification and/or redistribution. - -* **Codes often lack sufficient documentation.** Scientists are - generally under high pressure to publish in order to get tenure, win - grants, or get hired for their next job. Documentation then becomes - a lower priority. When documentation is inadequate, it is often - necessary to work closely with a core developer with limited time in - order to set up or modify a problem. - -* **Installing and compiling codes is often difficult and - time-consuming.** When plasma codes depend on external libraries or - include multiple languages, users must often change compiler flags - or edit make files. This requirement can provide a significant - barrier to entry for newcomers. Lack of documentation on how to - install and compile code can exacerbate this problem. - -* **Codes frequently lack unit testing and continuous integration - testing frameworks.** These testing strategies help to improve the - reliability of the code, track down bugs as soon as they appear, and - greatly reduce the likelihood that new bugs will be introduced when - the code is modified or maintained. - -* **Code is often difficult to read and therefore maintain.** Codes - commonly use highly abbreviated names that obfuscate the meanings of - variables, functions, and classes. Often there are no or inadequate - comments to describe the meanings of these objects. - -* **Simulation output is rarely stored in a standardized form.** Few - standards or specifications exist for the storage of plasma - simulation output (with the exception of OpenPMD). When output is - not kept in a standardized form, then different software is needed - to access and analyze the results. The shortage of standardized - formats makes it more difficult to develop general purpose analysis - and plotting software. - -* **Benchmarking algorithms or comparing physical models usually - requires setting up the same problem for multiple codes.** Each code - generally has unique conventions for defining initial conditions, - boundary conditions, and the computational domain. The problem - setup for one code cannot be easily transferred to another code. - Typically the same case must be set up by an experienced user of - each code. Often benchmarks require careful discussions to make - sure that normalizations and boundary conditions are able to be - matched, and different analysis software must often be used. - -* **Reproducing plasma simulation results usually requires significant - effort.** All source code and raw data must be available and usable - in order for a computational result to be replicable. While the - software packages used to perform simulations are often open source, - the specific version of the software is rarely cited. The code used - to set up a particular simulation and analyze its results is usually - not cited or available for inspection or reuse. - -* **Codes often lack clear boundaries between different components.** - The code for the problem setup, physical model, and numerical method - are often intertwined with each other. Different components - sometimes contain interdependencies with each other. - -* **High-level code is often intermixed with low-level code.** Code - related to the time advance can be intermixed with MPI calls related - to transferring data between processors. - -.. Often code is not broken up into separate functions. Spaghetti code. +phenomena. Numerical methods have developed and implemented highly +sophisticated numerical methods for simulating the behavior of plasmas +in a wide variety of regimes. It is essential that the PlasmaPy +ecosystem include flexible plasma simulation capabilities that allow +for high performance but without sacrificing usability. + +Addressing technical debt in plasma simulation software +------------------------------------------------------- + +Most scientific programmers are self-taught. Graduate programs in +plasma physics generally lack courses on scientific programming, so +students often learn programming on their own unless they had taken a +computer science course while an undergraduate. Students and +scientists are under high pressure to publish, which incentivizes +building up long-term technical debt to enable faster short-term +progress. Critically evaluating the current status of plasma +simulation software provides insight into how the plasma simulation +community can improve its practices in the future. + +**Common practice: Plasma simulation codes are usually written in +low-level compiled languages such as Fortran and C.** While these +languages allow for excellent performance, code development is slower +because these languages are not interactive and types must be +declared. Making a change often requires additional steps such as +declaring a type, communicating that variable to different processors, +and compiling the code. Using Fortran and C prioritizes performance at +the cost of usability and maintainability. + +**Alternative: Develop using modern languages that do not require +sacrificing usability to achieve high performance.** Dynamically typed +languages that can be used interactively allow for faster development, +but often at the cost of reduced performance. Modern languages such +as Julia make use of just-in-time (JIT) compilation with type +inference and multiple dispatch so that users do not need to choose +between usability and performance. + +**Problem: Access to existing plasma physics codes is frequently +restricted in some way.** Plasma simulation codes are often not +released under an open source license, especially for codes used to +simulate burning plasmas. Often prospective users must sign a user +agreement form that restricts modification and/or redistribution. +Such restrictions reduce scientific reproducibility and transparency +of research. + +**Solution: Develop openly under a permissive open source license.** +Open development maximizes scientific reproducibility and +transparency. Open source software often contains fewer bugs because +more people end up using and reading the code. Choosing a permissive +rather than copyleft license will allow anyone to freely use, modify, +and redistribute the package. + +**Problem: Plasma simulation software is often difficult to acquire, +compile, and install.** When plasma codes depend on external libraries +or include multiple languages, users must often edit make files and +change compiler flags. Scripts to install packages sometimes do not +work on all platforms, and thus may need to be edited. Difficulty in +getting a code to work in the first place can provide a significant +barrier to entry for newcomers who may have little experience with +make files and shell scripts. Lack of documentation exacerbates this +problem. + +**Solution: Make installation simple.** Modern package managers allow +open source packages to be installed quickly and reliably. Users often +only need to run a single command once the package manager and +language are installed. Users should never be required to manually +install external libraries, edit make files, or run installation +scripts. + +**Problem: Documentation is usually insufficient.** High pressure to +publish often makes writing documentation a lower priority. When +documentation is inadequate, it is often necessary to work closely +with a core developer with limited time in order to begin setting up +or modifying a problem. + +**Solution: Prioritize user documentation.** Each function and class +should have its own documentation. Each subpackage should have its own +narrative documentation. Documentation should be written so as to be +sufficient for a student taking their first class or starting their +first research project in plasma physics. Well-written documentation +provides users with a chance to get started with their project +quickly. + +**Problem: Codes often lack sufficient tests.** Legacy code is +sometimes defined as code without tests. With tests, we can modify +code and be confident that we are not introducing breaking changes. +Without tests, we will not know if our changes lead to new errors. + +**Solution: Implement unit and continuous integration testing while +using test-driven development.** These testing strategies help to +improve the reliability of the code, track down bugs as soon as they +appear, and greatly reduce the likelihood that new bugs will be +introduced when the code is modified or maintained. Test-driven +development involves writing tests first and then developing the code +until the tests pass. Tests should be developed according to the same +standards as the main part of the code. + +**Problem: Code is often difficult to read, maintain, and modify.** +Codes commonly use highly abbreviated names that obfuscate the +meanings of variables, functions, and classes. Functions and classes +often lack documentation. Comments are not always helpful and may be +obsolete. + +**Solution: Develop using modern best practices for scientific +computing.** Take the time to learn about how to improve programming +skills and develop sustainable software from different resources. + +.. + +.. TODO: add citations above to Wilson article, Clean Code/Clean + Architecture. + +.. Have coding standards and stick to them (like PEP8) + +**Problem: Benchmarking algorithms or comparing physical models +requires setting up the same problem for multiple codes.** Each code +generally has unique conventions for defining initial conditions, +boundary conditions, and the computational domain. The problem setup +for one code cannot be easily transferred to another code. Typically +the same case must be set up by an experienced user of each +code. Often benchmarks require careful discussions to make sure that +normalizations and boundary conditions are able to be matched, and +different analysis software must often be used. + +**Solution: Develop a standard mechanism for representing the problem +setup that can be implemented by multiple codes.** Create a boundary +with a well-defined interface that separates the problem setup from +the physical model and numerical method. Develop this interface so +that it can be implemented with existing codes. Implement multiple +numerical methods that can be adapted to different physical models. + +**Problem: Simulation output is rarely stored in a standardized +form.** Few standards or specifications exist for the storage of +plasma simulation output (with the exception of OpenPMD). When output +is not kept in a standardized form, then different software is needed +to access and analyze the results. The shortage of standardized +formats makes it more difficult to develop general purpose analysis +and plotting software. + +**Problem: Reproducing plasma simulation results usually requires +significant effort.** All source code and raw data must be available +and usable in order for a computational result to be replicable. While +the software packages used to perform simulations are often open +source, the specific version of the software is rarely cited. The code +used to set up a particular simulation and analyze its results is +usually not cited or available for inspection or reuse. + +**Solution: Make reproducibility part of the standard workflow.** + +.. TODO: Clarify and add more detail. + +**Problem: Codes often lack clear boundaries between different +components.** The code for the problem setup, physical model, and +numerical method are often intertwined with each other. Different +components sometimes contain interdependencies with each other. + +**Solution:** + +**Problem: High-level code is often intermixed with low-level code.** +Code related to the time advance can be intermixed with MPI calls +related to transferring data between processors. + +**Solution:** + +.. Code development should take place using best practices for scientific + and object-oriented programming. + +.. * **Optimize for both productivity and performance.** High + performance should not come at the cost of decreased usability. + +.. * **Provide and maintain thorough documentation.** The documentation + should be sufficient for a student beginning their first research + project in plasma physics. + +.. * **Plan the program architecture in advance.** Separate the + different aspects of the + +.. * **Define abstract interfaces between the objects that define the + problem setup and numerical method.** + Program each side to the interface. + +.. **Make the code as modular as possible.** Separate the initial + conditions from the system of equations and the numerical method. + +.. **Develop the overall architecture under the point of view that we + have not decided on the numerical method or physical model yet.** + +.. **Make the code as modular as possible.** + +.. **Optimize for both productivity and performance.** Plasma + simulation should be straightforward. Running plasma simulations + should not require expertise into the details of the code. Code + should be optimized for performance only after it works, and then + only after it becomes apparent what the bottlenecks are. + +.. **Prioritize usability, readability, and maintainability.** Code + should be straightforward to install. The packages resulting from + this project should all be installable by using Julia's built-in + package manager. + +.. **Prioritize documentation.** The documentation should be + sufficient for a student taking their first plasma physics class to + be able to use it without too much difficulty. + +.. **Prioritize flexible numerical methods.** There are a number of + numerical methods that have been developed for systems of equations + written in conservative form. + +.. **Minimize the amount of work and knowledge required by the end + user.** + +.. **Make it difficult for users to make silent mistakes.** + +.. **Make it straightforward to perform convergence studies and to + test solutions using different numerical methods.** Project Goals ============= We propose to create an open source framework for plasma simulation -within the PlasmaPy ecosystem. The goal of this framework is to +within the PlasmaPy ecosystem. The goal of this framework is to enable users to perform high quality simulations with the minimum amount of effort and without requiring knowledge of the details of the -implementation. Code design will prioritize usability, readability, -and maintainability. Optimization of performance should not come at -the cost of decreased user-friendliness. The functionality for +implementation. Code design will prioritize usability, readability, +and maintainability. Optimization of performance should not come at +the cost of decreased user-friendliness. The functionality for initial conditions, boundary conditions, physical model, and numerical method should be separated from each other with abstract interfaces. Users should be provided with the flexibility to switch to a different @@ -197,9 +302,9 @@ Requirements * Allow users to change the system of equations or numerical method in the middle of a simulation with minimal effort. -* Be able to use the same analysis techniques on +* Be able to use the same analysis techniques on -* Stretch goals: +* Stretch goals: Anticipated User Experience =========================== @@ -252,7 +357,7 @@ For simulations using the fluid approximation, users will either specify the equations in strings that will be parsed or select pre-defined systems of equations such as resistive MHD or Hall MHD. Users will add source and sink terms as necessary and choose models -for dissipation coefficients. +for dissipation coefficients. .. If the equations are in conservative form (including with sources and sinks), then more general numerical methods may be used. @@ -264,7 +369,7 @@ Specifying the numerical method For simulations using the fluid approximation, users will choose between different finite difference, finite volume, finite/spectral -element, and spectral methods. If possible, the code for the +element, and spectral methods. If possible, the code for the numerical method will be generated from the specified equations (though this may require that the equations be specified in conservative form). @@ -313,8 +418,8 @@ At this point, users will specify the numerical input parameters. Grid generation --------------- -.. More detail needed on grid generation. Need to discuss mesh - packing capabilities and how to generate complicated grids. For +.. More detail needed on grid generation. Need to discuss mesh + packing capabilities and how to generate complicated grids. For finite element simulations, more information on the mapping will be necessary, but might not be worth discussing here. @@ -325,90 +430,9 @@ Performing the simulation ------------------------- Users will have varying amounts of control over how the simulation is -performed. If no special processing is required, then users would be +performed. If no special processing is required, then users would be able to perform the simulation in a single command. -Development Principles -====================== - -Code development should take place using best practices for scientific -and object-oriented programming. - - * **Optimize for both productivity and performance.** High - performance should not come at the cost of decreased usability. - - * **Provide and maintain thorough documentation.** The documentation - should be sufficient for a student beginning their first research - project in plasma physics. - - * **Make installation simple.** Allow the framework to be quickly and - reliably installed using one command with a package manager. Users - should not be required to manually install external libraries, edit - make files, or run installation scripts. - -.. * **Plan the program architecture in advance.** Separate the - different aspects of the - - * **Define abstract interfaces between the objects that define the - problem setup and numerical method.** - Program each side to the interface. - - -.. **Make the code as modular as possible.** Separate the initial - conditions from the system of equations and the numerical method. - -.. **Develop the overall architecture under the point of view that we - have not decided on the numerical method or physical model yet.** - -.. **Make the code as modular as possible.** - -.. **Optimize for both productivity and performance.** Plasma - simulation should be straightforward. Running plasma simulations - should not require expertise into the details of the code. Code - should be optimized for performance only after it works, and then - only after it becomes apparent what the bottlenecks are. - -.. **Prioritize usability, readability, and maintainability.** Code - should be straightforward to install. The packages resulting from - this project should all be installable by using Julia's built-in - package manager. - - -.. **Prioritize documentation.** The documentation should be - sufficient for a student taking their first plasma physics class to - be able to use it without too much difficulty. - -.. **Prioritize flexible numerical methods.** There are a number of - numerical methods that have been developed for systems of equations - written in conservative form. - -.. **Minimize the amount of work and knowledge required by the end - user.** - -.. **Make it difficult for users to make silent mistakes.** - -.. **Make it straightforward to perform convergence studies and to - test solutions using different numerical methods.** - -.. **Use the SOLID principles for software development.** - - The *single responsibility principle*: There should never be more - than one reason for a class to change. - - The *open-closed principle*: Software entities (classes, modules, - functions, etc.) should be open for extension but closed for - modification. - - The *Liskov substitution principle*: Objects in a program should - be replaceable with instances of their subtypes without altering - the correctness of that program. - - The *interface segregation principle*: Clients should not be - forced to depend upon interfaces that they do not use. - - The *dependency inversion principle*: High level modules should - not depend upon low level modules. Abstractions should not depend - upon details. Details should depend upon abstractions. Proposed Package Structure ========================== @@ -433,37 +457,36 @@ Proposed Package Structure Choice of language ================== -Julia is a new high-level open source language that synthesizes the best -features of Fortran, C, Python, R, MATLAB, and Lisp for scientific -computing. Julia uses a just-in-time (JIT) compiler with type inference -and multiple dispatch to achieve performance comparable to C and -Fortran. Unlike C and Fortran, Julia can be run interactively and does -not require type declarations. These features greatly speed up code -development by allowing prototyping in the same language to be used for -performance runs. Julia natively supports parallelization, and has been -used to achieve petascale computing. **Julia proves that high -performance can be achieved with a dynamically typed interactive +Julia is a new high-level open source language that synthesizes the +best features of Fortran, C, Python, R, MATLAB, and Lisp for +scientific computing. Julia uses a just-in-time (JIT) compiler with +type inference and multiple dispatch to achieve performance comparable +to C and Fortran. Unlike C and Fortran, Julia can be run interactively +and does not require type declarations. These features greatly speed +up code development by allowing prototyping in the same language to be +used for performance runs. Julia natively supports parallelization, +and has been used to achieve petascale computing. **Julia proves that +high performance can be achieved with a dynamically typed interactive language without sacrificing usability.** Julia can call code from Fortran and C, and can act as a wrapper for codes written in these compiled languages. The main drawback of this -approach is that the resulting code would be harder to maintain because -developers would need to know two or three languages. A potential -drawback is that problems can arise in practice when code in one -language is called from a different language. Global optimizations +approach is that the resulting code would be harder to maintain +because developers would need to know two or three languages. A +potential drawback is that problems can arise in practice when code in +one language is called from a different language. Global optimizations might also not be possible when mixing more than one language (though -Julia may become capable of optimizations across language boundaries in -the future. If possible, the package itself should be written entirely -in Julia and depend only on packages that can be installed using Julia's -built-in package manager. Users shall *not* be required to compile or -install any external libraries or use any shell scripts. +Julia may become capable of optimizations across language boundaries +in the future. If possible, the package itself should be written +entirely in Julia and depend only on packages that can be installed +using Julia's built-in package manager. Users shall *not* be required +to compile or install any external libraries or use any shell scripts. The implementation shall be written entirely in Julia, and shall have -a Python interface in addition to a Julia interface. The interface -may either be included in the PlasmaPy core package or in an -affiliated package. The ``Plasma`` class should be able to handle the -output of simulations performed using this plasma simulation -framework. +a Python interface in addition to a Julia interface. The interface may +either be included in the PlasmaPy core package or in an affiliated +package. The ``Plasma`` class should be able to handle the output of +simulations performed using this plasma simulation framework. Implementation Notes ==================== @@ -475,8 +498,8 @@ A goal of this effort is to make the setup of fluid, particle, and hybrid simulations as similar as possible. Ideally, the same problem setup object should be able to be used to initialize all of these different types of simulations as similarly as possible. However, the -formulation of boundary conditions between fluid and PIC simulations can -be substantially different and potentially incompatible. +formulation of boundary conditions between fluid and PIC simulations +can be substantially different and potentially incompatible. .. I'm not sure how to handle this yet, particularly because I do not know enough about boundary conditions for PIC simulations. -Nick @@ -490,9 +513,9 @@ allow users to define what methods and attributes must be defined in a subclass of that ABC. This functionality is used in PlasmaPy's ``Plasma`` class. An equivalent to ABCs has not yet been implemented in Julia (see `Julia issue #6875 on GitHub -`_). An alternative -to ABCs would be to create a macro that checks that a particular class -or class instance has all of the required methods. +`_). An alternative to +ABCs would be to create a macro that checks that a particular class or +class instance has all of the required methods. Issues, Pull Requests, and Branches =================================== @@ -507,6 +530,9 @@ development. Alternatives ============ +Choice of language +------------------ + Julia is not the only language that could be used for this project. The main alternatives are listed below. The most significant disadvantages are shown in bold. From 3876e24a0bd326bd158ac9f053b4c46413384ef0 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Tue, 26 Feb 2019 10:39:39 -0500 Subject: [PATCH 14/27] Update PLEP on plasma simulator --- PLEP-0007.rst | 76 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 45 insertions(+), 31 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index cf30f17..baaf4c9 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -63,23 +63,27 @@ in a wide variety of regimes. It is essential that the PlasmaPy ecosystem include flexible plasma simulation capabilities that allow for high performance but without sacrificing usability. -Addressing technical debt in plasma simulation software -------------------------------------------------------- - +Motivation +========== + Most scientific programmers are self-taught. Graduate programs in plasma physics generally lack courses on scientific programming, so -students often learn programming on their own unless they had taken a -computer science course while an undergraduate. Students and -scientists are under high pressure to publish, which incentivizes -building up long-term technical debt to enable faster short-term -progress. Critically evaluating the current status of plasma -simulation software provides insight into how the plasma simulation -community can improve its practices in the future. +students are often left to learn these skills on their own. Students +and scientists often have not had the time or opportunity to learn +techniques and best practices from computer science and software +engineering that can greatly improve the reliability, maintainability, +and usability of software. The high pressure to publish and get +results out quickly incentivizes building up long-term technical debt +to enable faster short-term progress. A critical evaluation of the +current status of plasma simulation software based on years of +experience with multiple codes provides motivation to adopt strategies +that reduce technical debt and insight into how the plasma community +can improve its practices into the future. **Common practice: Plasma simulation codes are usually written in -low-level compiled languages such as Fortran and C.** While these -languages allow for excellent performance, code development is slower -because these languages are not interactive and types must be +low-level compiled languages such as Fortran, C, and C++.** While +these languages allow for excellent performance, code development is +slower because these languages are not interactive and types must be declared. Making a change often requires additional steps such as declaring a type, communicating that variable to different processors, and compiling the code. Using Fortran and C prioritizes performance at @@ -163,29 +167,35 @@ obsolete. computing.** Take the time to learn about how to improve programming skills and develop sustainable software from different resources. -.. - .. TODO: add citations above to Wilson article, Clean Code/Clean Architecture. .. Have coding standards and stick to them (like PEP8) -**Problem: Benchmarking algorithms or comparing physical models -requires setting up the same problem for multiple codes.** Each code -generally has unique conventions for defining initial conditions, -boundary conditions, and the computational domain. The problem setup -for one code cannot be easily transferred to another code. Typically -the same case must be set up by an experienced user of each -code. Often benchmarks require careful discussions to make sure that -normalizations and boundary conditions are able to be matched, and -different analysis software must often be used. - -**Solution: Develop a standard mechanism for representing the problem -setup that can be implemented by multiple codes.** Create a boundary -with a well-defined interface that separates the problem setup from -the physical model and numerical method. Develop this interface so -that it can be implemented with existing codes. Implement multiple -numerical methods that can be adapted to different physical models. +**Problem: Changing numerical methods, comparing physical models, and +performing benchmarks requires setting up the same problem for +multiple codes.** Each plasma simulation code generally has unique +conventions for defining initial conditions, boundary conditions, and +the computational domain. The problem setup for one code cannot be +easily transferred to another code. Typically the same case must be +set up by an experienced user of each code. Often benchmarks require +careful discussions to make sure that normalizations and boundary +conditions are able to be matched, and different analysis software +must often be used. + +**Solution: Develop a standard for representing the problem setup +independently of the physical model and numerical method.** Creating a +boundary with a well-defined interface will allow a clear separation +of responsibilities between the specification of the problem and the +numerics. This strategy will enable the same problem setup to be used +interchangeably with different physical models and numerical methods. +A boundary between the physical model and numerical method would +provide a means of switching just one of the physical model and +numerical method while keeping the other constant. Such a boundary is +possible with code generation and/or some numerical methods for +conservation laws, but often the numerical method must be fine-tuned +to the physical model. These interfaces should be developed so that +they can be implemented by existing codes. **Problem: Simulation output is rarely stored in a standardized form.** Few standards or specifications exist for the storage of @@ -195,6 +205,10 @@ to access and analyze the results. The shortage of standardized formats makes it more difficult to develop general purpose analysis and plotting software. +**Solution:** + +.. + **Problem: Reproducing plasma simulation results usually requires significant effort.** All source code and raw data must be available and usable in order for a computational result to be replicable. While From 1285202521bafe76d977e01ab36b9b6f2ff95939 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Thu, 28 Feb 2019 01:51:22 -0500 Subject: [PATCH 15/27] Update problems & solutions section of plasma simulator PLEP --- PLEP-0007.rst | 115 ++++++++++++++++++++------------------------------ 1 file changed, 45 insertions(+), 70 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index baaf4c9..97a7a8d 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -152,7 +152,7 @@ Without tests, we will not know if our changes lead to new errors. using test-driven development.** These testing strategies help to improve the reliability of the code, track down bugs as soon as they appear, and greatly reduce the likelihood that new bugs will be -introduced when the code is modified or maintained. Test-driven +introduced when the code is modified or maintained. Test-driven development involves writing tests first and then developing the code until the tests pass. Tests should be developed according to the same standards as the main part of the code. @@ -187,93 +187,68 @@ must often be used. independently of the physical model and numerical method.** Creating a boundary with a well-defined interface will allow a clear separation of responsibilities between the specification of the problem and the -numerics. This strategy will enable the same problem setup to be used +numerics. This strategy will enable the same problem setup to be used interchangeably with different physical models and numerical methods. -A boundary between the physical model and numerical method would -provide a means of switching just one of the physical model and -numerical method while keeping the other constant. Such a boundary is -possible with code generation and/or some numerical methods for -conservation laws, but often the numerical method must be fine-tuned -to the physical model. These interfaces should be developed so that -they can be implemented by existing codes. +Such interfaces should be developed so that they can be implemented +for existing codes. Prioritizing the implementation of numerical +methods that can be separated from the physical model will allow +additional boundaries between these two aspects of the problem. This +separation will be possible with some numerical methods designed for +conservation laws or with code generation, though in some cases the +numerical method will need to be fine-tuned to the physical model. **Problem: Simulation output is rarely stored in a standardized form.** Few standards or specifications exist for the storage of -plasma simulation output (with the exception of OpenPMD). When output -is not kept in a standardized form, then different software is needed -to access and analyze the results. The shortage of standardized -formats makes it more difficult to develop general purpose analysis -and plotting software. - -**Solution:** - -.. - -**Problem: Reproducing plasma simulation results usually requires -significant effort.** All source code and raw data must be available -and usable in order for a computational result to be replicable. While -the software packages used to perform simulations are often open -source, the specific version of the software is rarely cited. The code -used to set up a particular simulation and analyze its results is -usually not cited or available for inspection or reuse. - -**Solution: Make reproducibility part of the standard workflow.** +plasma simulation data. When output is not kept in a standardized +form, then different software is needed to access and analyze the +results. The shortage of standardized formats makes simulations less +reproducible and increases the difficulty of developing general +purpose analysis and plotting software. + +**Solution: Implement standard formats for simulation output.** A few +standardized formats or metadata and naming standards do exist such as +`OpenPMD `_ for particle-mesh +data. Implementing existing standards will generally be advantageous +when possible. New standards should be developed and made openly +available as necessary. A high-level interface can be developed and +matched to different low-level routines that are designed to access +different formats. + +**Problem: Plasma research is usually very difficult to replicate or +reproduce.** Many of the best practices for scientific reproducibility +being adopted in other fields have yet to be implemented for plasma +research. It is uncommon to make experimental and numerical results +openly available, and the specific source code to perform and analyze +a simulation is rarely make openly available and cited. + +**Solution: Build reproducibility into the standard workflow for +performing and analyzing numerical simulations.** Make it +straightforward to create a container that includes all of the source +code and associated materials to perform and analyze the simulation. .. TODO: Clarify and add more detail. -**Problem: Codes often lack clear boundaries between different -components.** The code for the problem setup, physical model, and -numerical method are often intertwined with each other. Different -components sometimes contain interdependencies with each other. - -**Solution:** - -**Problem: High-level code is often intermixed with low-level code.** -Code related to the time advance can be intermixed with MPI calls -related to transferring data between processors. - -**Solution:** - -.. Code development should take place using best practices for scientific - and object-oriented programming. +**Problem: Codes often lack boundaries between different components.** +The code for the problem setup, physical model, and numerical method +are often intertwined with each other. Different components sometimes +contain interdependencies with each other. High-level code is often +intermixed with low-level code, such as when MPI routines to transfer +data to different subprocesses are next to high-level calls for the +numerical method. -.. * **Optimize for both productivity and performance.** High - performance should not come at the cost of decreased usability. - -.. * **Provide and maintain thorough documentation.** The documentation - should be sufficient for a student beginning their first research - project in plasma physics. - -.. * **Plan the program architecture in advance.** Separate the - different aspects of the - -.. * **Define abstract interfaces between the objects that define the - problem setup and numerical method.** - Program each side to the interface. - -.. **Make the code as modular as possible.** Separate the initial - conditions from the system of equations and the numerical method. +**Solution: Implement well-defined software architecture with clear +boundaries.** Use the SOLID principles for software development. Do +not allow circular software dependencies. .. **Develop the overall architecture under the point of view that we have not decided on the numerical method or physical model yet.** -.. **Make the code as modular as possible.** - .. **Optimize for both productivity and performance.** Plasma simulation should be straightforward. Running plasma simulations should not require expertise into the details of the code. Code should be optimized for performance only after it works, and then only after it becomes apparent what the bottlenecks are. -.. **Prioritize usability, readability, and maintainability.** Code - should be straightforward to install. The packages resulting from - this project should all be installable by using Julia's built-in - package manager. - -.. **Prioritize documentation.** The documentation should be - sufficient for a student taking their first plasma physics class to - be able to use it without too much difficulty. - .. **Prioritize flexible numerical methods.** There are a number of numerical methods that have been developed for systems of equations written in conservative form. From bcc5a74bbf0ec69d1365a9502572f3c6570c0a04 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Thu, 28 Feb 2019 13:14:46 -0500 Subject: [PATCH 16/27] Update plasma simulator PLEP --- PLEP-0007.rst | 108 ++++++++++++++++---------------------------------- 1 file changed, 35 insertions(+), 73 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index cf30f17..80f9ff1 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -160,15 +160,31 @@ often lack documentation. Comments are not always helpful and may be obsolete. **Solution: Develop using modern best practices for scientific -computing.** Take the time to learn about how to improve programming -skills and develop sustainable software from different resources. +computing.** Take the time to learn and improve programming skills +using lessons learned by researchers and computer scientists +(Wilson2008_, Martin2009_, Martin2018_). Keep in mind that many +students and scientists who will be reading the code will not be +expert programmers. Have a coding standard and stick to it. Adopt +practices that are favorable to software sustainability +(Hettrick2016_, Wilkinson2016_). -.. +.. [Wilson2014] G. Wilson (2014), `Best Practices for Scientific + Computing `_ -.. TODO: add citations above to Wilson article, Clean Code/Clean - Architecture. +.. [Hettrick2016] S. Hettrick (2016), `Research Software + Sustainability: Report on a Knowledge Exchange Workshop + `_ -.. Have coding standards and stick to them (like PEP8) +.. [Wilkinson2016] M. D. Wilkinson et al. (2016), `The FAIR Guiding + Principles for scientific data management and stewardship + `_, Scientific Data, 3, + 160018 + +.. [Martin2009] R. C. Martin, *Clean Code: A Handbook of Agile + Software Craftsmanship* (2009, Prentice Hall) + +.. [Martin2018] R. C. Martin, *Clean Architecture: A Craftman's Guide + to Software Structure and Design* (2018, Prentice Hall) **Problem: Benchmarking algorithms or comparing physical models requires setting up the same problem for multiple codes.** Each code @@ -189,23 +205,11 @@ numerical methods that can be adapted to different physical models. **Problem: Simulation output is rarely stored in a standardized form.** Few standards or specifications exist for the storage of -plasma simulation output (with the exception of OpenPMD). When output -is not kept in a standardized form, then different software is needed -to access and analyze the results. The shortage of standardized -formats makes it more difficult to develop general purpose analysis -and plotting software. - -**Problem: Reproducing plasma simulation results usually requires -significant effort.** All source code and raw data must be available -and usable in order for a computational result to be replicable. While -the software packages used to perform simulations are often open -source, the specific version of the software is rarely cited. The code -used to set up a particular simulation and analyze its results is -usually not cited or available for inspection or reuse. - -**Solution: Make reproducibility part of the standard workflow.** - -.. TODO: Clarify and add more detail. +plasma simulation output (with the exception of `OpenPMD <>`_). When +output is not kept in a standardized form, then different software is +needed to access and analyze the results. The limited use of +standardized formats makes it more difficult to develop general +purpose analysis and plotting software. **Problem: Codes often lack clear boundaries between different components.** The code for the problem setup, physical model, and @@ -220,57 +224,15 @@ related to transferring data between processors. **Solution:** -.. Code development should take place using best practices for scientific - and object-oriented programming. - -.. * **Optimize for both productivity and performance.** High - performance should not come at the cost of decreased usability. - -.. * **Provide and maintain thorough documentation.** The documentation - should be sufficient for a student beginning their first research - project in plasma physics. - -.. * **Plan the program architecture in advance.** Separate the - different aspects of the - -.. * **Define abstract interfaces between the objects that define the - problem setup and numerical method.** - Program each side to the interface. - -.. **Make the code as modular as possible.** Separate the initial - conditions from the system of equations and the numerical method. - -.. **Develop the overall architecture under the point of view that we - have not decided on the numerical method or physical model yet.** - -.. **Make the code as modular as possible.** - -.. **Optimize for both productivity and performance.** Plasma - simulation should be straightforward. Running plasma simulations - should not require expertise into the details of the code. Code - should be optimized for performance only after it works, and then - only after it becomes apparent what the bottlenecks are. - -.. **Prioritize usability, readability, and maintainability.** Code - should be straightforward to install. The packages resulting from - this project should all be installable by using Julia's built-in - package manager. - -.. **Prioritize documentation.** The documentation should be - sufficient for a student taking their first plasma physics class to - be able to use it without too much difficulty. - -.. **Prioritize flexible numerical methods.** There are a number of - numerical methods that have been developed for systems of equations - written in conservative form. - -.. **Minimize the amount of work and knowledge required by the end - user.** - -.. **Make it difficult for users to make silent mistakes.** +**Problem: Reproducing plasma simulation results usually requires +significant effort.** All source code and raw data must be available +and usable in order for a computational result to be replicable. While +the software packages used to perform simulations are often open +source, the specific version of the software is rarely cited. The code +used to set up a particular simulation and analyze its results is +usually not cited or available for inspection or reuse. -.. **Make it straightforward to perform convergence studies and to - test solutions using different numerical methods.** +**Solution: Make reproducibility part of the standard workflow.** Project Goals ============= From ae9540fdb8a8e4d08ce1fa991029b57e098f46dd Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 1 Mar 2019 15:42:39 -0500 Subject: [PATCH 17/27] Update use cases in plasma simulator PLEP --- PLEP-0007.rst | 37 ++++++++++++------------------------- 1 file changed, 12 insertions(+), 25 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index e3aea92..9be0ce8 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -11,7 +11,7 @@ PLEP-0007 – Plasma Simulation Capabilities for PlasmaPy +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-02-25 | +| date last revised | 2019-03-01 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -367,27 +367,17 @@ physical situations. Integrated tokamak modeling --------------------------- -A major goal of magnetic fusion energy science is to create robust -predictive whole-device models for tokamaks and other magnetic -confinement devices (Poli2018_). Whole-device models will be used by -scientists who wish to understand fundamental plasma behavior, -engineers who wish to design new devices, and control room operators -who are overseeing discharges. The model must account for a wide -variety of effects ranging from - - -All of these users have different -requirements. The model must account for a wide variety of effects -ranging from current drive - - - -The model must include ohmic current drive, radio frequency (RF) -heating, neutral beam injection, and several other effects. - -Simulation results will need to be tested against multiple plasma -diagnostics. - +A major goal of magnetic fusion energy science is to create predictive +whole-device models for magnetic confinement devices such as tokamaks +(Poli2018_). Whole-device models will be used by scientists who wish +to understand fundamental plasma behavior, engineers who wish to +design new devices, and control room operators who are overseeing +discharges. Different physical processes will be important in +different parts of the device and at different time and length scales. +The model would need to include a variety of effects including but not +limited to plasma-wall interactions, radio frequency heating, and +neutral beam injection. Simulation results will need to be tested +against multiple plasma diagnostics. .. [Poli2018] F. M. Poli (2018), `*Integrated Tokamak modeling: When physics informs engineering and research planning* @@ -403,9 +393,6 @@ Modeling the solar chromosphere Software Requirements Specification =================================== - - - * Allow the same problem setup to be used for different systems of equations and numerical methods. From 14b276493e2499bb00d04e9494d7bdbba9135778 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 1 Mar 2019 15:43:07 -0500 Subject: [PATCH 18/27] Begin use case on simulating solar chromosphere --- PLEP-0007.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 9be0ce8..9fa030c 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -387,7 +387,7 @@ against multiple plasma diagnostics. Modeling the solar chromosphere ------------------------------- - +The solar chromosphere is the worst layer of the solar atmosphere. Software Requirements Specification From e6c824cbaed4ea918f71f5a3d54020ddb83826e5 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 1 Mar 2019 15:49:54 -0500 Subject: [PATCH 19/27] Describe chromosphere more tactfully --- PLEP-0007.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 9fa030c..a1f277d 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -387,7 +387,7 @@ against multiple plasma diagnostics. Modeling the solar chromosphere ------------------------------- -The solar chromosphere is the worst layer of the solar atmosphere. +The solar chromosphere is host to a rich variety of physical processes. Software Requirements Specification From 12ccd30fb758f2d59c4256a8a9ce4ef698ea7f27 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 1 Mar 2019 20:14:29 -0500 Subject: [PATCH 20/27] Begin software requirements specification --- PLEP-0007.rst | 214 +++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 176 insertions(+), 38 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index a1f277d..36d0a9a 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -30,24 +30,24 @@ particular class of equations and use a specific numerical method. Different codes generally lack interoperability with each other. Changing to a different physical model or numerical method often requires setting up the same problem in a different code with -different conventions for defining initial and boundary conditions. -By designing a boundary with a clear interface that separates the -problem setup from the physical model and numerical method, the same -problem setup can be used with different physical models and numerical -methods. The framework should include interchangeable numerical -methods for fluid, particle-in-cell, gyrokinetic, and hybrid -simulations. Boundaries and interfaces should be designed to allow -existing codes to implement them. The framework should include general -methods for solution of systems of partial differential equations -written in conservative form that can be inputted by the user in -strings that are processed by the framework. Code development will -equally prioritize readability, usability, reliability, -maintainability, and performance. User documentation will be aimed -toward both students taking their first course in plasma physics as -well as experienced researchers. The documentation will include -advice to users on how best choose a physical model and numerical -method and describe potential pitfalls for different types of -simulations. +different conventions for defining initial conditions (ICs) and +boundary conditions (BCs). By designing a boundary with a clear +interface that separates the problem setup from the physical model and +numerical method, the same problem setup can be used with different +physical models and numerical methods. The framework should include +interchangeable numerical methods for fluid, particle-in-cell, +gyrokinetic, and hybrid simulations. Boundaries and interfaces should +be designed to allow existing codes to implement them. The framework +should include general methods for solution of systems of partial +differential equations written in conservative form that can be +inputted by the user in strings that are processed by the +framework. Code development will equally prioritize readability, +usability, reliability, maintainability, and performance. User +documentation will be aimed toward both students taking their first +course in plasma physics as well as experienced researchers. The +documentation will include advice to users on how best choose a +physical model and numerical method and describe potential pitfalls +for different types of simulations. Background ========== @@ -206,13 +206,12 @@ not allow circular software dependencies. **Problem: Changing numerical methods, comparing physical models, and performing benchmarks requires setting up the same problem for multiple codes.** Each plasma simulation code generally has unique -conventions for defining initial conditions, boundary conditions, and -the computational domain. The problem setup for one code cannot be -easily transferred to another code. Typically the same case must be -set up by an experienced user of each code. Often benchmarks require -careful discussions to make sure that normalizations and boundary -conditions are able to be matched, and different analysis software -must often be used. +conventions for defining ICs, BCs, and the computational domain. The +problem setup for one code cannot be easily transferred to another +code. Typically the same case must be set up by an experienced user of +each code. Often benchmarks require careful discussions to make sure +that normalizations and boundary conditions are able to be matched, +and different analysis software must often be used. **Solution: Develop a standard for representing the problem setup independently of the physical model and numerical method.** Creating a @@ -319,6 +318,7 @@ Sample Use Cases We describe some potential use cases for students, scientists, and engineers doing plasma simulations roughly in order of difficulty (from *low* at the top to *walking into Mordor* near the bottom). +These use cases show the variety of .. _waves: @@ -341,11 +341,10 @@ Reproducing the GEM Challenge The Geospace Environmental Modeling (GEM) Challenge was performed to isolate the essential physics required to model collisionless magnetic reconnection (Birn2001_). The strategy was to perform simulations with -the same initial and boundary conditions using multiple codes with -different physical models, ranging from resistive MHD to fully -electromagnetic PIC. A scientist decides to attempt to reproduce the -results of the GEM challenge using a larger computational domain and -higher resolution. +the same ICs and BCs using multiple codes with different physical +models, ranging from resistive MHD to fully electromagnetic PIC. A +scientist decides to attempt to reproduce the results of the GEM +challenge using a larger computational domain and higher resolution. .. [Birn2001] J. Birn et al. (2001), `*Geospace Environmental Modeling (GEM) Magnetic Reconnection Challenge* @@ -387,21 +386,155 @@ against multiple plasma diagnostics. Modeling the solar chromosphere ------------------------------- -The solar chromosphere is host to a rich variety of physical processes. - +The solar chromosphere hosts a rich variety of physical processes. +Many of the simplifying assumptions that are valid in either the +photosphere below or the corona above cannot be applied to the +chromosphere [e.g., the assumption of local thermodynamic equilibrium +(LTE)]. The plasma is partially ionized, so interactions between +neutrals and charged particles are important. Non-LTE radiative +transfer is required. Shocks contribute to the heating. Some solar +physicists will want to model a beam of energetic electrons +precipitating from the corona into the chromosphere during solar +flares. Synthetic observables will be required to validate simulation +results against reality. Software Requirements Specification =================================== -* Allow the same problem setup to be used for different systems of - equations and numerical methods. +Availability requirements +------------------------- + +* Develop openly under the Open Source Initiative approved BSD+Patent + license. + +* Allow installation in a single line with a package manager. + + - Do not require users to compile external libraries, edit make + files, or run bash scripts. + +* Design the package to run in Linux, macOS, and Windows environments + and on scales ranging from a single core to the most powerful + supercomputers. + +Language requirements +--------------------- + +* Develop the core functionality using Julia because it is an + interactive language with dynamic typing that uses a just-in-time + compiler with type inference and multiple dispatch and has + parallelism built into the language. + + - Do not include any statically typed or non-interactive languages + because that will decrease long-term maintainability and because + mixing languages can cause problems with compilers. + +* Provide a Python interface to the core functionality. + +Base requirements +----------------- + +* Create a standardized representation for the problem setup so that + the same problem setup can be used interchangeably for different + physical models and numerical methods. + + - Include distinct representations for the domain, ICs, and BCs. + + - Check that different domain, IC, and BC representations are + consistent with each other. + + - Provide clear, well-defined interfaces for each of these + representations. + + - Allow distinct representation of volumetric source terms that are + needed to drive turbulence. + + - Implement checks that the domain, ICs, and BCs are consistent with + each other. + + - Use exception handling to provide useful error messages that + help users quickly pinpoint problems. + + - Issue a warning when the initial conditions are far out of + equilibrium. + + - Allow users to select pre-defined initial conditions that are + commonly used. + +* Incorporate grid generation capabilities. + + - Implement general tools to create non-uniform logically + rectangular grids. + + - Implement or use tools to create a finite element grid for + experimental geometries. + +* Prioritize flexible numerical methods that can solve almost + arbitrary systems of equations written in conservative form. + + - Allow users to select pre-defined systems of equations. + + - Allow users to provide equations as strings. + + - Parse each string to extract the flux and source terms for each + equation. + + - Create functions for each of the fluxes and sources that can be + compiled at runtime (with automatic differentiation to calculate + Jacobians, when needed). -* Allow users to change the system of equations or numerical method in - the middle of a simulation with minimal effort. + - Implement finite difference capabilities. -* Be able to use the same analysis techniques on + - Implement finite volume capabilities. + + - Implement finite element/spectral element capabilities. + + - Provide shock capturing algorithms. + +* Implement auxiliary functionality. + + - Grad-Shafranov solver + + - Synthetic diagnostic generators + +* Implement particle-in-cell simulation capabilities. + + - Define abstract interfaces in the implementation to allow + different particle-pushers and other simulation components to be + used interchangeably. + +* Implement standardized formats and metadata naming conventions for + output data. + + - Use existing standards when reasonable to do so. + + - Develop new open standards when necessary. + + +Testing requirements +-------------------- + +* Adopt test-driven development. + +* Develop a suite of tests + +* + + +Code quality requirements +------------------------- + + +Documentation requirements +-------------------------- + +* Provide narrative documentation + +* All user-facing functions and classes should have + +* Provide a cookbook of recipes + +* Documentation should be developed to be -* Stretch goals: Anticipated User Experience =========================== @@ -484,6 +617,11 @@ At this point, users will specify the numerical input parameters. definition maybe? Maybe we should have a way to convert a fluid simulation setup into a PIC simulation. +.. Thinking more: if the boundary conditions are different between + fluid and PIC, then they should be denoted in different ways. We + could provide some suggestions on adapting a problem for PIC vs + fluid approximation simulations. + .. Users will next define the system of equations or physical model to be solved. It is at this point that users will choose the style of simulation (including but not limited to fluid, particle-in-cell, and From e0791c704ac9bafd23030d4430d6193b1adb7e4b Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Mon, 4 Mar 2019 01:19:38 -0500 Subject: [PATCH 21/27] Update API requirements, etc. in plasma simulator PLEP --- PLEP-0007.rst | 60 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 13 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 36d0a9a..92703f5 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -57,7 +57,7 @@ investigate plasma phenomena such as tokamak disruptions, solar eruptions, geomagnetic storms, and accretion into black holes. Numerical simulations can be used to understand experiments and act as a bridge between laboratory, heliospheric, and astrophysical plasma -phenomena. Numerical methods have developed and implemented highly +phenomena. Plasma physicists have developed and implemented highly sophisticated numerical methods for simulating the behavior of plasmas in a wide variety of regimes. It is essential that the PlasmaPy ecosystem include flexible plasma simulation capabilities that allow @@ -65,7 +65,17 @@ for high performance but without sacrificing usability. Motivation ========== - + +.. The points being made in this section are really worth making, but + the section has become really long, and the suggestions are much + more general than this. + +.. Plasma simulation in the 24½th century + +.. There are important points being made in this section, but it's + long and not directly related to the + + Most scientific programmers are self-taught. Graduate programs in plasma physics generally lack courses on scientific programming, so students are often left to learn these skills on their own. Students @@ -318,7 +328,8 @@ Sample Use Cases We describe some potential use cases for students, scientists, and engineers doing plasma simulations roughly in order of difficulty (from *low* at the top to *walking into Mordor* near the bottom). -These use cases show the variety of +These use cases show the variety of needs that general plasma +simulation software must meet. .. _waves: @@ -404,13 +415,13 @@ Software Requirements Specification Availability requirements ------------------------- -* Develop openly under the Open Source Initiative approved BSD+Patent - license. +* Develop openly under the Open Source Initiative approved `BSD+Patent + `_ license. -* Allow installation in a single line with a package manager. +* Allow installation in a single line using a package manager. - Do not require users to compile external libraries, edit make - files, or run bash scripts. + files, change compiler flags, or run bash scripts. * Design the package to run in Linux, macOS, and Windows environments and on scales ranging from a single core to the most powerful @@ -430,13 +441,36 @@ Language requirements * Provide a Python interface to the core functionality. -Base requirements ------------------ +Application programming interface requirements +---------------------------------------------- + +* Create APIs for multiple levels of abstraction. Greater abstraction + will make the code easier to use, whereas less abstraction will + provide more options for customizability. + + - Enable a simple simulation (e.g., analytical initial conditions on + a uniform rectangular grid with periodic boundary conditions) to + be set up in the minimal number of lines of code while maintaining + readability and not hiding important details. + + - Provide narrative documentation that progresses from a simulation + set up using the highest level of abstraction to lower levels of + abstraction. + +* Each numerical method should be programmed to a well-defined + interface. PIC and fluid simulations should have identical + interfaces at the highest level of abstraction. + +Functionality requirements +-------------------------- * Create a standardized representation for the problem setup so that the same problem setup can be used interchangeably for different physical models and numerical methods. +.. Create a class or object that represents the dependent variables so + that it can be used when creating the other classes when needed? + - Include distinct representations for the domain, ICs, and BCs. - Check that different domain, IC, and BC representations are @@ -509,15 +543,15 @@ Base requirements - Develop new open standards when necessary. - Testing requirements -------------------- -* Adopt test-driven development. +* Use test-driven development as appropriate. Write tests first and + then develop code until tests pass. -* Develop a suite of tests +* Create -* +* Test Code quality requirements From e56ae4126a8e5d73db17908b5d5b3f455bfb15cb Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Tue, 5 Mar 2019 01:36:35 -0500 Subject: [PATCH 22/27] Reorganize plasma simulator PLEP --- PLEP-0007.rst | 223 +++++++++++++++++++++++--------------------------- 1 file changed, 101 insertions(+), 122 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 92703f5..8878bb4 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -49,6 +49,20 @@ documentation will include advice to users on how best choose a physical model and numerical method and describe potential pitfalls for different types of simulations. +Project Goals +============= + +We propose to create an open source framework for plasma simulation +within the PlasmaPy ecosystem. The goals of this framework are to: + +* Enable users to perform high quality numerical simulations of + diverse plasma phenomena; + +* Minimize the time, effort, and frustration needed to achieve + scientific understanding through numerical simulation; and + +* Maximize scientific and computational reproducibility. + Background ========== @@ -66,15 +80,9 @@ for high performance but without sacrificing usability. Motivation ========== -.. The points being made in this section are really worth making, but - the section has become really long, and the suggestions are much - more general than this. - -.. Plasma simulation in the 24½th century - -.. There are important points being made in this section, but it's - long and not directly related to the - +.. A possibility for this section would be to transform it into its + own article and call it something like "Plasma simulation in the + 24½th century" Most scientific programmers are self-taught. Graduate programs in plasma physics generally lack courses on scientific programming, so @@ -182,25 +190,6 @@ expert programmers. Have a coding standard and stick to it. Adopt practices that are favorable to software sustainability (Hettrick2016_, Wilkinson2016_). -.. [Wilson2014] G. Wilson (2014), `Best Practices for Scientific - Computing `_, PLoS - Biology, 12, e1001745, doi: 10.1371/journal.pbio.1001745 - -.. [Hettrick2016] S. Hettrick (2016), `Research Software - Sustainability: Report on a Knowledge Exchange Workshop - `_ - -.. [Wilkinson2016] M. D. Wilkinson et al. (2016), `The FAIR Guiding - Principles for scientific data management and stewardship - `_, Scientific Data, 3, - 160018, doi: 10.1038/sdata.2016.18 - -.. [Martin2009] R. C. Martin, *Clean Code: A Handbook of Agile - Software Craftsmanship* (2009, Prentice Hall) - -.. [Martin2018] R. C. Martin, *Clean Architecture: A Craftman's Guide - to Software Structure and Design* (2018, Prentice Hall) - **Problem: Codes often lack boundaries between different components.** The code for the problem setup, physical model, and numerical method are often intertwined with each other. Different components sometimes @@ -209,6 +198,8 @@ intermixed with low-level code, such as when MPI routines to transfer data to different subprocesses are next to high-level calls for the numerical method. +.. TODO: Expand the following point. + **Solution: Implement well-defined software architecture with clear boundaries.** Use the SOLID principles for software development. Do not allow circular software dependencies. @@ -275,59 +266,12 @@ create containers that include all of the source code, dependency version numbers, and other materials needed to perform and analyze a numerical simulation. -Project Goals -============= - -.. The goal of this framework is to minimize the time, effort, and - energy necessary for users to perform high quality numerical - simulations of diverse plasma phenomena while maximizing scientific - and computational reproducibility. - -We propose to create an open source framework for plasma simulation -within the PlasmaPy ecosystem. The goals of this framework are to: - -* Enable users to perform high quality numerical simulations of - diverse plasma phenomena; - -* Minimize the time, effort, and frustration needed to achieve - scientific understanding through numerical simulation; and - -* Maximize scientific and computational reproducibility. - -.. The framework specifications - -.. We propose to create an open source framework for plasma simulation - within the PlasmaPy ecosystem. The goal of this framework is to - minimize the time, effort, and frustration needed to achieve - scientific understanding via numerical simulation. Code design will - prioritize usability, readability, and - maintainability. Optimization for performance should not come at - the cost of decreased user-friendliness. The - -.. We propose to create an open source framework for plasma simulation - within the PlasmaPy ecosystem. The goal of this framework is to - enable users to perform high quality simulations with the minimum - amount of effort and without requiring knowledge of the details of - the implementation. Code design will prioritize usability, - readability, and maintainability. Optimization of performance - should not come at the cost of decreased user-friendliness. The - functionality for initial conditions, boundary conditions, physical - model, and numerical method should be separated from each other - with abstract interfaces. Users should be provided with the - flexibility to switch to a different physical model or different - numerical method with minimal effort. This framework should - include fluid, particle-in-cell (PIC), and hybrid simulation - capabilities using a variety of well-described numerical methods. - -.. Minimize the amount of time, effort, and frustration to needed to - reach scientific understanding. - Sample Use Cases ================ We describe some potential use cases for students, scientists, and engineers doing plasma simulations roughly in order of difficulty -(from *low* at the top to *walking into Mordor* near the bottom). +(from *low* near the top to *walking into Mordor* near the bottom). These use cases show the variety of needs that general plasma simulation software must meet. @@ -357,11 +301,6 @@ models, ranging from resistive MHD to fully electromagnetic PIC. A scientist decides to attempt to reproduce the results of the GEM challenge using a larger computational domain and higher resolution. -.. [Birn2001] J. Birn et al. (2001), `*Geospace Environmental Modeling - (GEM) Magnetic Reconnection Challenge* - `_, Journal of Geophysical - Research, 106, 3715, doi: 10.1029/1999JA900449 - .. _testPIC: Testing PIC algorithms @@ -418,11 +357,15 @@ Availability requirements * Develop openly under the Open Source Initiative approved `BSD+Patent `_ license. -* Allow installation in a single line using a package manager. +* Allow installation in a single line with widely-used package + managers. - Do not require users to compile external libraries, edit make files, change compiler flags, or run bash scripts. + - Depend only on packages that can be installed using that package + manager. + * Design the package to run in Linux, macOS, and Windows environments and on scales ranging from a single core to the most powerful supercomputers. @@ -430,10 +373,7 @@ Availability requirements Language requirements --------------------- -* Develop the core functionality using Julia because it is an - interactive language with dynamic typing that uses a just-in-time - compiler with type inference and multiple dispatch and has - parallelism built into the language. +* Develop the base functionality using Julia. - Do not include any statically typed or non-interactive languages because that will decrease long-term maintainability and because @@ -441,17 +381,19 @@ Language requirements * Provide a Python interface to the core functionality. -Application programming interface requirements + - Include this interface in PlasmaPy or an affiliated package in the + PlasmaPy ecosystem. + +Application programming interface (API) requirements ---------------------------------------------- -* Create APIs for multiple levels of abstraction. Greater abstraction - will make the code easier to use, whereas less abstraction will - provide more options for customizability. +* Create APIs that enable multiple levels of abstraction. Greater + abstraction will make the code easier to use, whereas less + abstraction will provide more options for customizability and + fine-tuning. - - Enable a simple simulation (e.g., analytical initial conditions on - a uniform rectangular grid with periodic boundary conditions) to - be set up in the minimal number of lines of code while maintaining - readability and not hiding important details. + - Enable a simple simulation to be set up in ten to twenty lines of + code. - Provide narrative documentation that progresses from a simulation set up using the highest level of abstraction to lower levels of @@ -524,11 +466,15 @@ Functionality requirements - Provide shock capturing algorithms. -* Implement auxiliary functionality. +* Implement auxiliary functionality, including: - Grad-Shafranov solver - - Synthetic diagnostic generators + - Synthetic diagnostics + + - Magnetic topology analysis tools + + - Turbulence analysis tools * Implement particle-in-cell simulation capabilities. @@ -536,23 +482,28 @@ Functionality requirements different particle-pushers and other simulation components to be used interchangeably. + - Keep a similar API to fluid-approximation simulations at high + levels of abstraction. + * Implement standardized formats and metadata naming conventions for output data. - - Use existing standards when reasonable to do so. + - Use existing standards as appropriate. - - Develop new open standards when necessary. + - Develop new open standards as necessary. Testing requirements -------------------- -* Use test-driven development as appropriate. Write tests first and - then develop code until tests pass. +* Use test-driven development. -* Create +* Create unit tests for all core functionality. -* Test +* Create continuous integration tests to make sure that different + parts of the code work with each other as required. +* Test that numerical methods have the same order of convergence as + expected theoretically. Code quality requirements ------------------------- @@ -561,6 +512,8 @@ Code quality requirements Documentation requirements -------------------------- +* Provide narrative documentation that allows users to + * Provide narrative documentation * All user-facing functions and classes should have @@ -684,27 +637,26 @@ At this point, users will specify the numerical input parameters. Maybe we could create a function that automatically writes text that describes the numerical method and such. -Grid generation ---------------- +.. Grid generation + --------------- .. More detail needed on grid generation. Need to discuss mesh packing capabilities and how to generate complicated grids. For finite element simulations, more information on the mapping will be necessary, but might not be worth discussing here. -Users will be able to generate the grid after the domain is specified -and the numerical method is chosen. - -Performing the simulation -------------------------- +.. Users will be able to generate the grid after the domain is + specified and the numerical method is chosen. -Users will have varying amounts of control over how the simulation is -performed. If no special processing is required, then users would be -able to perform the simulation in a single command. +.. Performing the simulation + ------------------------- +.. Users will have varying amounts of control over how the simulation + is performed. If no special processing is required, then users + would be able to perform the simulation in a single command. -Proposed Package Structure -========================== +.. Proposed Package Structure + ========================== .. Mathematical functions Basis functions that are not defined in other packages @@ -726,13 +678,13 @@ Proposed Package Structure Choice of language ================== -Julia is a new high-level open source language that synthesizes the -best features of Fortran, C, Python, R, MATLAB, and Lisp for -scientific computing. Julia uses a just-in-time (JIT) compiler with -type inference and multiple dispatch to achieve performance comparable -to C and Fortran. Unlike C and Fortran, Julia can be run interactively -and does not require type declarations. These features greatly speed -up code development by allowing prototyping in the same language to be +Julia is a high-level open source language that synthesizes the best +features of Fortran, C, Python, R, MATLAB, and Lisp for scientific +computing (JuliaIntro_). Julia uses a just-in-time (JIT) compiler with type +inference and multiple dispatch to achieve performance comparable to C +and Fortran. Unlike C and Fortran, Julia can be run interactively and +does not require type declarations. These features greatly speed up +code development by allowing prototyping in the same language to be used for performance runs. Julia natively supports parallelization, and has been used to achieve petascale computing. **Julia proves that high performance can be achieved with a dynamically typed interactive @@ -814,7 +766,7 @@ disadvantages are shown in bold. - Codes can be called from other languages like Python and Julia - Disadvantages - **Productivity is reduced because these languages are not - interactive** + interactive or dynamically typed** - Code in these languages is often several times the length of equivalent code in Julia or Python - Continuous integration testing platforms such as Travis CI do @@ -866,3 +818,30 @@ Decision Rationale ================== This PLEP has not been decided upon yet. + +References +========== + +.. [Birn2001] J. Birn et al. (2001), `*Geospace Environmental Modeling + (GEM) Magnetic Reconnection Challenge* + `_, Journal of Geophysical + Research, 106, 3715, doi: 10.1029/1999JA900449 + +.. [Hettrick2016] S. Hettrick (2016), `Research Software + Sustainability: Report on a Knowledge Exchange Workshop + `_ + +.. [Martin2009] R. C. Martin, *Clean Code: A Handbook of Agile + Software Craftsmanship* (2009, Prentice Hall) + +.. [Martin2018] R. C. Martin, *Clean Architecture: A Craftman's Guide + to Software Structure and Design* (2018, Prentice Hall) + +.. [Wilkinson2016] M. D. Wilkinson et al. (2016), `The FAIR Guiding + Principles for scientific data management and stewardship + `_, Scientific Data, 3, + 160018, doi: 10.1038/sdata.2016.18 + +.. [Wilson2014] G. Wilson (2014), `Best Practices for Scientific + Computing `_, PLoS + Biology, 12, e1001745, doi: 10.1371/journal.pbio.1001745 From 90d00c0636d6eea68e815a7f0aaf5910eb271c86 Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Wed, 6 Mar 2019 01:21:31 -0500 Subject: [PATCH 23/27] Update PLEP on plasma simulator --- PLEP-0007.rst | 65 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 8878bb4..3db84cf 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -255,17 +255,14 @@ are often customized for a particular project. The specific software and input files used to perform and analyze a simulation are rarely made openly available for inspection or reuse. -.. [Murphy2019] N. A. Murphy, B. Alterman, and D. Stansby (2019), `Making - plasma research reproducible - `_, Zenodo, - doi: 10.5281/zenodo.2578291 - **Solution: Build reproducibility into the standard workflow for performing and analyzing numerical simulations.** Allow users to create containers that include all of the source code, dependency version numbers, and other materials needed to perform and analyze a numerical simulation. +.. _usecases + Sample Use Cases ================ @@ -392,17 +389,21 @@ Application programming interface (API) requirements abstraction will provide more options for customizability and fine-tuning. - - Enable a simple simulation to be set up in ten to twenty lines of + - Enable a simple simulation to be set up in as few as ten lines of code. - Provide narrative documentation that progresses from a simulation set up using the highest level of abstraction to lower levels of abstraction. -* Each numerical method should be programmed to a well-defined - interface. PIC and fluid simulations should have identical - interfaces at the highest level of abstraction. - +* Program each numerical method to the same well-defined high level + interface. + + - PIC and fluid simulations should use identical interfaces at the + highest level of abstraction. + + - The interface should be expandable. + Functionality requirements -------------------------- @@ -512,16 +513,23 @@ Code quality requirements Documentation requirements -------------------------- -* Provide narrative documentation that allows users to +* Provide narrative documentation -* Provide narrative documentation + - Write a quickstart guide for new contributors, including people + who are new to plasma physics. -* All user-facing functions and classes should have + - Develop more detailed documentation for experienced users who may + wish to engage in more complex tasks like implement a new + numerical method. -* Provide a cookbook of recipes +* All user-facing functions and classes should have a numpydoc style + docstring. -* Documentation should be developed to be +* All private functions and classes should have a docstring unless it + is simple +* Provide a cookbook of sample programs that do different things that + users may end up trying to implement (akin to the matplotlib gallery). Anticipated User Experience =========================== @@ -680,7 +688,7 @@ Choice of language Julia is a high-level open source language that synthesizes the best features of Fortran, C, Python, R, MATLAB, and Lisp for scientific -computing (JuliaIntro_). Julia uses a just-in-time (JIT) compiler with type +computing (JuliaIntro_). Julia uses a JIT compiler with type inference and multiple dispatch to achieve performance comparable to C and Fortran. Unlike C and Fortran, Julia can be run interactively and does not require type declarations. These features greatly speed up @@ -751,6 +759,14 @@ development. Alternatives ============ +Numerical method code development +--------------------------------- + +.. When going between + +.. Additionally, some plasma simulation codes like BOUT++ and PLUTO + are licensed under the GPLv3. + Choice of language ------------------ @@ -773,7 +789,8 @@ disadvantages are shown in bold. not support direct testing of Fortran code - Experience with Fortran is less helpful for plasma physics students searching for jobs outside of research and academia - - Limited metaprogramming capabilities + - Limited metaprogramming capabilities and cannot compile code at + runtime * **Python with NumPy** - Advantages @@ -795,7 +812,7 @@ disadvantages are shown in bold. * **Python with Numba** - Advantages - - Uses a just-in-time compiler to get compiled speeds + - Uses a JIT compiler to get compiled speeds - Often decorating a function with ``numba.jit`` is enough to get compiled speeds - Disadvantages @@ -807,11 +824,10 @@ disadvantages are shown in bold. * **LuaJIT** - Advantages - - Uses a just-in-time compiler to get compiled speeds - - Offers exceptional performance, including for graphical - processing units (GPUs) + - Uses a JIT compiler to get compiled speeds + - Offers great performance - Disadvantages - - **Less active community surrounding scientific LuaJIT** + - **Smaller scientific community surrounding LuaJIT** - Fewer scientific libraries written in LuaJIT Decision Rationale @@ -836,6 +852,11 @@ References .. [Martin2018] R. C. Martin, *Clean Architecture: A Craftman's Guide to Software Structure and Design* (2018, Prentice Hall) + +.. [Murphy2019] N. A. Murphy, B. Alterman, and D. Stansby (2019), + `Making plasma research reproducible + `_, Zenodo, doi: + 10.5281/zenodo.2 .. [Wilkinson2016] M. D. Wilkinson et al. (2016), `The FAIR Guiding Principles for scientific data management and stewardship From 0511cc56e8c9a79212ef6e7559531fb0ee19d5b6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dominik=20Sta=C5=84czak?= Date: Wed, 6 Mar 2019 21:23:20 -0500 Subject: [PATCH 24/27] Update PLEP-0007.rst Co-Authored-By: namurphy --- PLEP-0007.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 3db84cf..f09c906 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -527,7 +527,7 @@ Documentation requirements * All private functions and classes should have a docstring unless it is simple - +* provide Jupyter notebook examples for ease of use * Provide a cookbook of sample programs that do different things that users may end up trying to implement (akin to the matplotlib gallery). From f3ed39c35db65749cb5cf800d3d8ec77cb34677c Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Sun, 17 Mar 2019 15:30:42 -0400 Subject: [PATCH 25/27] A few minor changes to the plasma simulator PLEP I still need to address most comments from GitHub. --- PLEP-0007.rst | 138 ++++++++++++++++++++++++++------------------------ 1 file changed, 73 insertions(+), 65 deletions(-) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index f09c906..5d172ac 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -1,6 +1,6 @@ -======================================================= -PLEP-0007 – Plasma Simulation Capabilities for PlasmaPy -======================================================= +================================================== +PLEP-0007 – Plasma simulation for the 24½th century +================================================== +-------------------+---------------------------------------------+ | PLEP | number | @@ -11,7 +11,7 @@ PLEP-0007 – Plasma Simulation Capabilities for PlasmaPy +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-03-01 | +| date last revised | 2019-03-17 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -58,9 +58,12 @@ within the PlasmaPy ecosystem. The goals of this framework are to: * Enable users to perform high quality numerical simulations of diverse plasma phenomena; +* Enable users to straightforwardly compare numerical results to + experimental and observational data; + * Minimize the time, effort, and frustration needed to achieve scientific understanding through numerical simulation; and - + * Maximize scientific and computational reproducibility. Background @@ -80,32 +83,30 @@ for high performance but without sacrificing usability. Motivation ========== -.. A possibility for this section would be to transform it into its - own article and call it something like "Plasma simulation in the - 24½th century" - -Most scientific programmers are self-taught. Graduate programs in -plasma physics generally lack courses on scientific programming, so -students are often left to learn these skills on their own. Students -and scientists often have not had the time or opportunity to learn -techniques and best practices from computer science and software -engineering that can greatly improve the reliability, maintainability, -and usability of software. The high pressure to publish and get -results out quickly incentivizes building up long-term technical debt -to enable faster short-term progress. A critical evaluation of the -current status of plasma simulation software based on years of -experience with multiple codes provides motivation to adopt strategies -that reduce technical debt and insight into how the plasma community -can improve its practices into the future. +Most scientific programmers are self-taught. Undergraduate and +graduate programs in physics and astronomy generally lack courses on +scientific programming, so students are often left to learn these +skills on their own. Students and scientists often have not had the +time or opportunity to learn techniques and best practices from +computer science and software engineering that can greatly improve the +reliability, maintainability, and usability of software and prevent +future headaches. The high pressure to publish results quickly +incentivizes building up long-term technical debt to enable faster +short-term progress. A candid evaluation of the current status of +plasma simulation software based on experiences using a variety of +codes over multiple years provides motivation to adopt strategies that +reduce technical debt and insight into how the plasma community can +improve its practices into the future. **Common practice: Plasma simulation codes are usually written in -low-level compiled languages such as Fortran, C, and C++.** While +low-level pre-compiled languages such as Fortran, C, and C++.** While these languages allow for excellent performance, code development is -slower because these languages are not interactive and types must be -declared. Making a change often requires additional steps such as -declaring a type, communicating that variable to different processors, -and compiling the code. Using Fortran and C prioritizes performance at -the cost of usability and maintainability. +slower because these languages are not interactive and the types of +variables must be declared. Making a change often requires additional +steps such as declaring a type, communicating that variable to +different processors, and re-compiling the code. Using pre-compiled +languages prioritizes performance at the cost of usability and +maintainability. **Alternative: Develop using modern languages that do not require sacrificing usability to achieve high performance.** Dynamically typed @@ -113,7 +114,7 @@ languages that can be used interactively allow for faster development, but often at the cost of reduced performance. Modern languages such as Julia make use of just-in-time (JIT) compilation with type inference and multiple dispatch so that users do not need to choose -between usability and performance. +between usability and performance. **Problem: Access to existing plasma physics codes is frequently restricted in some way.** Plasma simulation codes are often not @@ -195,8 +196,8 @@ The code for the problem setup, physical model, and numerical method are often intertwined with each other. Different components sometimes contain interdependencies with each other. High-level code is often intermixed with low-level code, such as when MPI routines to transfer -data to different subprocesses are next to high-level calls for the -numerical method. +data to different subprocesses are next to high-level calls related to +the numerical method. .. TODO: Expand the following point. @@ -354,19 +355,21 @@ Availability requirements * Develop openly under the Open Source Initiative approved `BSD+Patent `_ license. -* Allow installation in a single line with widely-used package - managers. - +* Allow installation in a single command with the appropriate standard + package managers. + - Do not require users to compile external libraries, edit make files, change compiler flags, or run bash scripts. - - Depend only on packages that can be installed using that package - manager. + - Depend only on packages that can be installed automatically with + the package manager when running the installation command. * Design the package to run in Linux, macOS, and Windows environments and on scales ranging from a single core to the most powerful supercomputers. +* Provide thorough online documentation. + Language requirements --------------------- @@ -393,16 +396,19 @@ Application programming interface (API) requirements code. - Provide narrative documentation that progresses from a simulation - set up using the highest level of abstraction to lower levels of - abstraction. - + set up using the highest level of abstraction (e.g., a quick start + guide) to lower levels of abstraction. + * Program each numerical method to the same well-defined high level interface. - PIC and fluid simulations should use identical interfaces at the highest level of abstraction. - - The interface should be expandable. + - The interface must be expandable. + +* Use exception handling and provide useful error messages to help + users pinpoint problems quickly. Functionality requirements -------------------------- @@ -410,29 +416,22 @@ Functionality requirements * Create a standardized representation for the problem setup so that the same problem setup can be used interchangeably for different physical models and numerical methods. - -.. Create a class or object that represents the dependent variables so - that it can be used when creating the other classes when needed? - - Include distinct representations for the domain, ICs, and BCs. - - - Check that different domain, IC, and BC representations are - consistent with each other. - - - Provide clear, well-defined interfaces for each of these - representations. + - Provide standardized representations with well-defined interfaces + for the domain, ICs, and BCs. - - Allow distinct representation of volumetric source terms that are - needed to drive turbulence. + - Implement checks that different domain, IC, and BC + representations are consistent with each other. - - Implement checks that the domain, ICs, and BCs are consistent with - each other. + - Issue a warning when the initial conditions are far out of + equilibrium. - - Use exception handling to provide useful error messages that - help users quickly pinpoint problems. - - - Issue a warning when the initial conditions are far out of - equilibrium. + - Raise an exception when the magnetic field is not approximately + divergence-free. + + - Provide a standardized representation for how to define volumetric + source terms that, for example, may be needed to drive turbulence + or initiate magnetic reconnection. - Allow users to select pre-defined initial conditions that are commonly used. @@ -445,12 +444,12 @@ Functionality requirements - Implement or use tools to create a finite element grid for experimental geometries. -* Prioritize flexible numerical methods that can solve almost - arbitrary systems of equations written in conservative form. +* Prioritize flexible numerical methods intended to solve systems of + equations written in conservative form (LeVeque1992_, Lukin2015_). - - Allow users to select pre-defined systems of equations. + - Enable users to select pre-defined systems of equations. - - Allow users to provide equations as strings. + - Enable users to provide equations as strings. - Parse each string to extract the flux and source terms for each equation. @@ -528,6 +527,7 @@ Documentation requirements * All private functions and classes should have a docstring unless it is simple * provide Jupyter notebook examples for ease of use + * Provide a cookbook of sample programs that do different things that users may end up trying to implement (akin to the matplotlib gallery). @@ -838,8 +838,8 @@ This PLEP has not been decided upon yet. References ========== -.. [Birn2001] J. Birn et al. (2001), `*Geospace Environmental Modeling - (GEM) Magnetic Reconnection Challenge* +.. [Birn2001] J. Birn et al. (2001), `Geospace Environmental Modeling + (GEM) Magnetic Reconnection Challenge `_, Journal of Geophysical Research, 106, 3715, doi: 10.1029/1999JA900449 @@ -847,6 +847,14 @@ References Sustainability: Report on a Knowledge Exchange Workshop `_ +.. [Lukin2016] V. S. Lukin et al. (2016), `Overview of HiFi - implicit + spectral element code framework for multi-fluid plasma applications + `_, arXiv:1608.06030 + +.. [LeVeque1992] R. J. LeVeque, `Numerical Methods for Conservation + Laws `_ (1992, + Birkhäuser Basel) + .. [Martin2009] R. C. Martin, *Clean Code: A Handbook of Agile Software Craftsmanship* (2009, Prentice Hall) From 16eed59528ba9117240122d5ad2c4bb5851e1d2e Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Fri, 22 Mar 2019 09:04:16 -0400 Subject: [PATCH 26/27] Minor updates to plasma simulator PLEP --- PLEP-0000.rst | 12 ++++++------ PLEP-0007.rst | 49 +++++++++++++++++++++++++++---------------------- 2 files changed, 33 insertions(+), 28 deletions(-) diff --git a/PLEP-0000.rst b/PLEP-0000.rst index 3726880..2a8eade 100644 --- a/PLEP-0000.rst +++ b/PLEP-0000.rst @@ -66,12 +66,12 @@ Declined PLEPs PLEPs in preparation or under consideration =========================================== -+--------+---------------------------------------------------------------------------+---------------------------+---------------+ -| number | title | author(s) | type | -+========+===========================================================================+===========================+===============+ -| 7 | `A Next Generation Plasma Simulator | Nicholas A. Murphy | | -| | `__ | | | -+--------+---------------------------------------------------------------------------+---------------------------+---------------+ ++--------+---------------------------------------------------------------------------+---------------------------+---------------+---------------------------------------------+ +| number | title | author(s) | type | DOI | ++========+===========================================================================+===========================+===============+=============================================+ +| 7 | `Plasma Simulation for the 24½th Century | Nicholas A. Murphy | standard | | +| | `__ | | | | ++--------+---------------------------------------------------------------------------+---------------------------+---------------+---------------------------------------------+ Tentatively Reserved PLEPs ========================== diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 5d172ac..949c379 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -1,6 +1,6 @@ -================================================== -PLEP-0007 – Plasma simulation for the 24½th century -================================================== +=================================================== +PLEP-0007 – Plasma Simulation for the 24½th Century +=================================================== +-------------------+---------------------------------------------+ | PLEP | number | @@ -11,7 +11,7 @@ PLEP-0007 – Plasma simulation for the 24½th century +-------------------+---------------------------------------------+ | date created | 2018-12-17 | +-------------------+---------------------------------------------+ -| date last revised | 2019-03-17 | +| date last revised | 2019-03-22 | +-------------------+---------------------------------------------+ | type | standard | +-------------------+---------------------------------------------+ @@ -63,7 +63,7 @@ within the PlasmaPy ecosystem. The goals of this framework are to: * Minimize the time, effort, and frustration needed to achieve scientific understanding through numerical simulation; and - + * Maximize scientific and computational reproducibility. Background @@ -114,7 +114,7 @@ languages that can be used interactively allow for faster development, but often at the cost of reduced performance. Modern languages such as Julia make use of just-in-time (JIT) compilation with type inference and multiple dispatch so that users do not need to choose -between usability and performance. +between usability and performance. **Problem: Access to existing plasma physics codes is frequently restricted in some way.** Plasma simulation codes are often not @@ -152,7 +152,7 @@ scripts. publish often makes writing documentation a lower priority. When documentation is inadequate, it is often necessary to work closely with a core developer with limited time in order to begin setting up -or modifying a problem. +or modifying a problem. **Solution: Prioritize user documentation.** Each function and class should have its own documentation. Each subpackage should have its own @@ -180,7 +180,7 @@ standards as the main part of the code. Codes commonly use highly abbreviated names that obfuscate the meanings of variables, functions, and classes. Functions and classes often lack documentation. Comments are not always helpful and may be -obsolete. +obsolete. **Solution: Develop using modern best practices for scientific computing.** Take the time to learn and improve programming skills @@ -260,9 +260,9 @@ made openly available for inspection or reuse. performing and analyzing numerical simulations.** Allow users to create containers that include all of the source code, dependency version numbers, and other materials needed to perform and analyze a -numerical simulation. +numerical simulation. -.. _usecases +.. _usecases: Sample Use Cases ================ @@ -284,7 +284,7 @@ magnetosonic waves, and fast magnetosonic waves. She assigns a homework assignment to perform MHD simulations of these three different types of waves and produce movies of different quantities. Many of the students in her class have never performed a -numerical simulation. +numerical simulation. .. _GEM: @@ -357,7 +357,7 @@ Availability requirements * Allow installation in a single command with the appropriate standard package managers. - + - Do not require users to compile external libraries, edit make files, change compiler flags, or run bash scripts. @@ -398,7 +398,7 @@ Application programming interface (API) requirements - Provide narrative documentation that progresses from a simulation set up using the highest level of abstraction (e.g., a quick start guide) to lower levels of abstraction. - + * Program each numerical method to the same well-defined high level interface. @@ -416,7 +416,7 @@ Functionality requirements * Create a standardized representation for the problem setup so that the same problem setup can be used interchangeably for different physical models and numerical methods. - + - Provide standardized representations with well-defined interfaces for the domain, ICs, and BCs. @@ -428,7 +428,7 @@ Functionality requirements - Raise an exception when the magnetic field is not approximately divergence-free. - + - Provide a standardized representation for how to define volumetric source terms that, for example, may be needed to drive turbulence or initiate magnetic reconnection. @@ -450,7 +450,7 @@ Functionality requirements - Enable users to select pre-defined systems of equations. - Enable users to provide equations as strings. - + - Parse each string to extract the flux and source terms for each equation. @@ -484,7 +484,7 @@ Functionality requirements - Keep a similar API to fluid-approximation simulations at high levels of abstraction. - + * Implement standardized formats and metadata naming conventions for output data. @@ -525,7 +525,7 @@ Documentation requirements docstring. * All private functions and classes should have a docstring unless it - is simple + is simple * provide Jupyter notebook examples for ease of use * Provide a cookbook of sample programs that do different things that @@ -854,7 +854,7 @@ References .. [LeVeque1992] R. J. LeVeque, `Numerical Methods for Conservation Laws `_ (1992, Birkhäuser Basel) - + .. [Martin2009] R. C. Martin, *Clean Code: A Handbook of Agile Software Craftsmanship* (2009, Prentice Hall) @@ -864,13 +864,18 @@ References .. [Murphy2019] N. A. Murphy, B. Alterman, and D. Stansby (2019), `Making plasma research reproducible `_, Zenodo, doi: - 10.5281/zenodo.2 - + 10.5281/zenodo.2578291 + +.. [Poli2018] F. M. Poli (2018), `*Integrated Tokamak modeling: When + physics informs engineering and research planning* + `_, Physics of Plasmas, 25, + 055602, doi: 10.1063/1.5021489 + .. [Wilkinson2016] M. D. Wilkinson et al. (2016), `The FAIR Guiding Principles for scientific data management and stewardship `_, Scientific Data, 3, 160018, doi: 10.1038/sdata.2016.18 - + .. [Wilson2014] G. Wilson (2014), `Best Practices for Scientific Computing `_, PLoS Biology, 12, e1001745, doi: 10.1371/journal.pbio.1001745 From 804ac51153b58c956dafb7c085cc01d98de7a0df Mon Sep 17 00:00:00 2001 From: Nick Murphy Date: Sun, 24 Mar 2019 07:51:34 -0400 Subject: [PATCH 27/27] Minor updates to plasma simulation PLEP --- PLEP-0007.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/PLEP-0007.rst b/PLEP-0007.rst index 949c379..4eb3e47 100644 --- a/PLEP-0007.rst +++ b/PLEP-0007.rst @@ -49,6 +49,11 @@ documentation will include advice to users on how best choose a physical model and numerical method and describe potential pitfalls for different types of simulations. +*Due to the scope of the proposed work, the decision on and +implementation of this PLEP have been postponed until sufficient +funding to support the development of the PlasmaPy software ecosystem +has been made available.* + Project Goals =============