-
Notifications
You must be signed in to change notification settings - Fork 314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(createpackages): use jinja for mf6 module code generation #2333
base: develop
Are you sure you want to change the base?
Conversation
a3b7c2e
to
af1e7a5
Compare
d1e8a21
to
a7f2644
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't looked through everything, but I left you a PR on your own branch with indications on how I would probably approach this. Let's take a look at it together and see if we can get rid of the shim :)
bab83ac
to
cf68432
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #2333 +/- ##
=========================================
+ Coverage 74.5% 74.7% +0.1%
=========================================
Files 292 297 +5
Lines 59624 60654 +1030
=========================================
+ Hits 44463 45350 +887
- Misses 15161 15304 +143
|
f95ad8c
to
6d57fae
Compare
This reverts commit 44f5db4.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a suggestion for removing indentation out of the macro
@jdhughes-usgs @langevin-usgs I think this is about ready, but maybe it should wait til after the class if the generate classes script will be used there |
Carved out of #2317 — just move to Jinja as an initial step, type hints can come afterwards as they will take some effort to get right. The aim here is to match the old
createpackages.py
capabilities without relying onmfstructure.py
. This is an initial step to movemfstructure.py
to a leaner representation of the input spec.mfstructure.py
is still used at runtime which will need to be unraveled in followup work.Introduce a
flopy.mf6.utils.codegen
module with some new machinery for loading an input specification from DFN files into an intermediate representation, and generating source code with Jinja. This is fairly general, handling quirks of the existing framework in a "shim" of Jinja filters that transform the template context before rendering. These can be removed as we rework things. The templates should also get simpler over time.For now the load routine uses a data structure from the
boltons
library to represent a DFN as an unstructured (flat) map of variables before structural parsing, where variables can have duplicate names. If we could guarantee all variable names were unique by changing a few DFNs this wouldn't be needed. In any case it will not be necessary once we have structured TOML definition files.Jinja2
,boltons
,tomlkit
andmodflow-devtools
are added to a new optional dependency groupcodegen
. These are required to use the code generation utilities — this may require CI tweaks in related repositories and should be noted by developers too.Tentative next steps after this PR:
Refactor input spec introspection
mfstructure.py
to inspect this instead of parsing lists of stringsStart using structured DFN files. We have discussed prototyping this for the time being in a way that is not visible to the user, and refining the format until ready to adopt upstream in MF6. A first draft TOML conversion script has been added — we could later convert DFNs -> TOML at code generation time, then exercise the TOML load.
Explicit versioning for the DFN specification language. TOML could be considered v2 where V1 is the original DFN format.
Add type hints as per feature: Adding Python type hints for input specifications #2298
Drop
*TemplateGenerator
mechanism if possibleMiscellaneous:
flopy/mf6/data/mfdatastorage.py
to avoid referencing a variable before it exists