Merge pull request #22 from piyueh/dev

address JOSS reviews

README.md

@@ -5,6 +5,7 @@ geoclaw-landspill
 [![Travis CI](https://img.shields.io/travis/com/barbagroup/geoclaw-landspill/master?label=Travis%20CI)](https://travis-ci.com/barbagroup/geoclaw-landspill)
 [![GitHub Action CI](https://img.shields.io/github/workflow/status/barbagroup/geoclaw-landspill/CI/master?label=GitHub%20Action%20CI)](https://github.com/barbagroup/geoclaw-landspill/actions?query=workflow%3ACI)
 [![status](https://joss.theoj.org/papers/fb7b012799a70c9b4c55eb4bb0f36f97/status.svg)](https://joss.theoj.org/papers/fb7b012799a70c9b4c55eb4bb0f36f97)
+[![Conda](https://anaconda.org/barbagroup/geoclaw-landspill/badges/installer/conda.svg)](https://anaconda.org/barbagroup/geoclaw-landspill)
 
 ***Note: if looking for content of `geoclaw-landspill-cases`, please checkout tag
 `v0.1`. This repository has been converted to a fully working solver package.***
@@ -34,42 +35,36 @@ ruptures. These features include:
 1. [Dependencies, installation, and tests](doc/deps_install_tests.md)
 2. [Usage](doc/usage.md)
 3. [Configuration file: `setrun.py`](doc/configuration.md)
-3. [Example cases](cases/README.md)
-4. [Containers: Docker and Singularity](doc/container.md)
+4. [Example cases](cases/README.md)
+5. [Containers: Docker and Singularity](doc/container.md)
 
 ------------------------------------------------------------------------
 ## Quick start
 
-Beyond this quick start, to see more details, please refer to
-the [documentation](#documentation) section.
+We only maintain compatibility with Linux. Though using `pip` or building from
+source may still work in Mac OS or Windows (e.g., through WSL), we are not able
+to help with the installation issues on these two systems.
 
-### 1. Installation
-
-Install `gfortran`, `python>=3.7`, and `pip`. For example, in Arch Linux:
-```
-# pacman -S gcc-gfortran python python-pip
-```
+Beyond this quick start, to see more details, please refer to the
+[documentation](#documentation) section.
 
-Or, in Ubuntu Focal:
-```
-# apt install gfortran python3 python3-pip
-```
+### 1. Installation
 
-Then install *geoclaw-landspill* from PyPI
-```
-# pip install geoclaw-landspill
-```
+The fast way to install *geoclaw-landspill* is through
+[Anaconda](https://www.anaconda.com/)'s `conda` command. The following command
+creates a conda environment (called `landspill`) and installs the package and
+dependencies:
 
-Using system Python and system `pip` requires root privilege and installs the
-package to the system path. To install to the user's local path without root
-privilege, instead, do
 ```
-$ pip install --user geoclaw-landspill
+$ conda create \
+    -n landspill -c barbagroup -c conda-forge \
+    python=3.8 geoclaw-landspill
 ```
-However, if using the `--user` flag, users should make sure `pip`'s local `bin`
-path is in `PATH`.
 
-Alternatively, users can consider using virtual environments.
+Then use `conda activate landspill` or
+`source <conda installation prefix>/bin/activate landspill` to activate the
+environment. Type `geoclaw-landspill --help` in the terminal to see if
+*geoclaw-landspill* is correctly installed.
 
 ### 2. Running an example case
 
@@ -111,6 +106,11 @@ QGIS and ArcGIS should be able to read the resulting NetCDF raster file.
 * riemann: https://github.com/clawpack/riemann
   ([BSD 3-Clause License](https://github.com/clawpack/riemann/blob/597824c051d56fa0c8818e00d740867283329b24/LICENSE))
 
+------------------------------------------------------------------------
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md).
+
 ------------------------------------------------------------------------
 ## Contact
 

conda/conda_build_config.yaml

@@ -0,0 +1,4 @@
+python:
+  - 3.7
+  - 3.8
+  - 3.9

conda/meta.yaml

@@ -0,0 +1,51 @@
+{% set name = "geoclaw-landspill" %}
+{% set version = "1.0.dev4" %}
+
+package:
+  name: "{{ name|lower }}"
+  version: "{{ version }}"
+
+source:
+  url: "https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz"
+  sha256: fc37a636e97327fcb13d4a94d55575beaa0cf4baf50ba7fadef074a8e6f93d6f
+
+build:
+  number: 0
+  entry_points:
+    - geoclaw-landspill = gclandspill.__main__:main
+  script: "{{ PYTHON }} -m pip install . -vv"
+
+requirements:
+  build:
+    - {{ compiler('fortran') }}
+  host:
+    - python {{ python }}
+    - pip >=21.0
+    - scikit-build
+  run:
+    - python >=3.7
+    - six >=1.15.0
+    - psutil >=5.8.0
+    - requests >=2.25.1
+    - numpy >=1.19.4
+    - matplotlib >=3.3.3
+    - netcdf4 >=1.5.5.1
+    - rasterio >=1.1.8
+    - typing_extensions >=3.7  # [py==37]
+
+test:
+  imports:
+    - gclandspill
+  commands:
+    - geoclaw-landspill --help
+
+about:
+  home: "https://github.com/barbagroup/geoclaw-landspill"
+  license: BSD 3-Clause
+  license_family: BSD
+  license_file: LICENSE
+  summary: "A hydrocarbon overland flow simulator for pipeline rupture events."
+
+extra:
+  recipe-maintainers:
+    - Pi-Yueh Chuang <pychuang@gwu.edu>

doc/deps_install_tests.md

@@ -1,59 +1,77 @@
 # Dependencies, installation, and tests
 
-The only operating system supported is Linux. We are not maintaining
+The only operating system officially supported is Linux. We are not maintaining
 compatibility with other systems, though they may still work.
 
 ---------------
 ## Dependencies
 
-`pip` can install all except one dependency. File `requirements.txt` describes
-runtime dependencies, and file `requirements-build.txt` describes additional
-dependencies if building the package locally. When installing
-*geoclaw-landspill* through `pip`, `pip` takes care of dependencies
-automatically.
+Build time and runtime dependencies are described in `requirements-build.txt`
+and `requirements.txt`, respectively. `gfortran >= 8.0` is the only build time
+dependency not included in the file, and, correspondingly, `libgfortran5 >= 8.0`
+is the only runtime dependency not included in the file.
 
-The only dependency that may need manual installation is the Fortran compiler.
-We have only tested `gfortran`. After installation, users can remove the
-compiler itself (i.e., `gfortran`). However, `libgfortran` may be needed during
-runtime if using dynamic linking. For `gfortran` (and also `libgfortran`), the
-minimum version is 8.
+Anaconda users do not need to worry about dependencies at all.
 
-Installation of `gfortran` depends on the operating systems and virtual
-environments. For example, in Arch Linux:
+On the other hand, `pip` users have to install `gfortran` or/and `libgfortran5`
+in advance using the package managers of their Linux distributions. For example,
+in Arch Linux, use:
 ```
 # pacman -S gcc-fortran
 ```
-Ubuntu 20.04:
+And in Ubuntu 20.04:
 ```
 # apt install gfortran
 ```
 
-Alternatively, one can use `conda` to get `gfortran` if using Anaconda virtual
-environment:
+Alternatively, though not recommended, one can use `conda` to get `gfortran` and
+then continue using `pip` for other dependencies. The command to get `gfortran`
+from Anaconda is
 ```
 $ conda install -c conda-forge "gfortran_linux-64>=8.0"
 ```
 `conda` renames the compiler executable to `x86_64-conda-linux-gnu-gfortran`.
 However, this should not concern users because CMake should be able to find it
 automatically.
 
+After installing `gfortran` manually, `pip` users can continue on the
+installation of *geoclaw-landspill* (the next section).
+
 ---------------
 ## Installation
 
-Note, for the `pip` command and `python setup.py install`, users can always add
-the `--user` flag to install to users' local paths and avoid root privilege.
+### Option 1: use `conda` and install binary files from Anaconda
+
+As described in README, the following command creates an environment called
+`landspill`, and it has *geoclaw-landspill* installed:
+```
+$ conda create \
+    -n landspill -c barbagroup -c conda-forge \
+    python=3.8 geoclaw-landspill
+```
+Once activate the environment, the executable `geoclaw-landspill` should already
+be available.
+
+### Option 2: use `pip` to install from PyPI or from source
 
-### 1. Install from PyPI
+Note, when using the `pip` command, users can always add the `--user` flag to
+install to users' local paths and avoid root privilege. However, if using the
+`--user` flag, users should make sure `pip`'s local `bin` path is in `PATH`.
 
-The easiest way to install *geoclaw-landspill* is through PyPI. 
+#### Option 2.1: install from PyPI
+
+To install the package from PyPI. 
 ```
 $ pip install geoclaw-landspill
 ```
 
 We only distribute source tarballs on PyPI due to the requirement of a Fortran
-compiler. Wheels or binary releases of this package are not available.
+compiler. Wheels or binary releases of this package are not available. `pip`
+will download the source tarball, compile/build the package, and then install
+it. `gfortran` has to be installed in advance as described in the previous
+section.
 
-### 2. Install with a source tarball from GitHub
+#### Option 2.2: install with a source tarball from GitHub
 
 Download a release tarball from the repository's
 [release page](https://github.com/barbagroup/geoclaw-landspill/releases) on GitHub,
@@ -62,7 +80,7 @@ and install the package directly with pip and the tarball:
 $ pip install <tarball name>.tar.gz
 ```
 
-### 3. Install with the repository
+#### Option 2.3: install with the repository in developer mode
 
 Clone/pull the repository from GitHub:
 ```
@@ -75,9 +93,13 @@ $ pip install -r requirements.txt -r requirements-build.txt
 ```
 Then, install *geoclaw-landspill*:
 ```
-$ python setup.py install
+$ pip install --editable .
 ```
 
+Under the developer mode, installation is just a link referencing the source
+directory, so any changes in the source Python files take effect immediately (
+but not the Fortran files because they have to be re-compiled).
+
 --------
 ## Tests
 
@@ -89,9 +111,8 @@ $ tox
 ```
 
 End-users can run tests against the installed package if they install
-*geoclaw-landspill* through method
-[2](#2.-install-with-a-source-tarball-from-github) and method
-[3](#3.-install-with-the-repository). Use `pytest`:
+*geoclaw-landspill* through `pip` and using the source tarball or code
+repository. Use `pytest`:
 ```
 $ pytest -v tests
 ```

gclandspill/__main__.py

@@ -8,9 +8,11 @@
 
 """Main function of geoclaw-landspill.
 """
+import os
 import pathlib
 import argparse
 import subprocess
+import psutil
 import gclandspill
 from gclandspill._preprocessing import create_data
 from gclandspill._postprocessing.netcdf import convert_to_netcdf
@@ -46,6 +48,9 @@ def main():
         "case", action="store", type=pathlib.Path, metavar="CASE",
         help="The path to the target case directory."
     )
+    parser_run.add_argument(
+        '--log-level', dest="log_level", action="store", type=int, default=None,
+        help='Overwrite the log verbosity in the config file `setrun.py` (default: no overwrite)')
     parser_run.set_defaults(func=run)  # set the corresponding callback for the `run` command
 
     # `createnc` command
@@ -297,6 +302,10 @@ def run(args: argparse.Namespace):
     Execution code. 0 means all good. Other values means something wrong.
     """
 
+    # set up default openmp threads
+    if "OMP_NUM_THREADS" not in os.environ:
+        os.environ["OMP_NUM_THREADS"] = "{}".format(psutil.cpu_count(False))
+
     # process path
     args.case = args.case.expanduser().resolve()
     assert args.case.is_dir()
@@ -305,7 +314,7 @@ def run(args: argparse.Namespace):
     args.output = args.case.joinpath("_output")
 
     # create *.data files, topology files, and hydrological file
-    create_data(args.case, args.output)
+    create_data(args.case, args.log_level, args.output)
 
     # get the Fortran solver binary
     solver = pathlib.Path(gclandspill.__file__).parent.joinpath("bin", "geoclaw-landspill-bin")

gclandspill/_preprocessing.py

@@ -26,13 +26,18 @@
 from gclandspill import clawutil
 
 
-def create_data(case_dir: os.PathLike, out_dir: os.PathLike = "_output", overwrite: bool = False):
+def create_data(
+    case_dir: os.PathLike, log_level: int = None,
+    out_dir: os.PathLike = "_output", overwrite: bool = False
+):
     """Create *.data files (and topography & hydrological files) in case folder.
 
     Arguments
     ---------
     case_dir : PathLike
         The (absolute) path to the target case folder, where setrun.py can be found.
+    log_level : int or None
+        To overwrite the log verbosity in the original config file.
     out_dir : str or PathLike
         Folder to put output files for this particular run. If not an absolute path, assume it's
         relative to `case_dir`
@@ -51,6 +56,12 @@ def create_data(case_dir: os.PathLike, out_dir: os.PathLike = "_output", overwri
     setrun = _misc.import_setrun(case_dir)
     rundata = setrun.setrun()  # get ClawRunData object
 
+    # if we need to overwrite the log verbosity
+    if log_level is not None:
+        rundata.clawdata.verbosity = log_level
+        rundata.clawdata.verbosity_regrid = log_level
+        rundata.amrdata.verbosity_regrid = log_level
+
     # geoclaw' `TopographyData` assumes topography filenames are relative to the output folder,
     # while we assuem the filenames are always relative to the case folder. This forces us to write
     # *.data to the case folder first and move to our target output folder

requirements.txt

@@ -1,4 +1,5 @@
 six>=1.15.0
+psutil>=5.8.0
 requests>=2.25.1
 numpy>=1.19.4
 rasterio>=1.1.8