Skip to content

Commit

Permalink
Simplify CI gdal logic. Improve documentation around MPI and gdal issues
Browse files Browse the repository at this point in the history
  • Loading branch information
Chrismarsh committed Dec 13, 2023
1 parent 8826f17 commit 3da5b36
Show file tree
Hide file tree
Showing 4 changed files with 54 additions and 8 deletions.
17 changes: 12 additions & 5 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,11 +152,18 @@ jobs:
if [[ $CI_REF_NAME == "stable/"* ]]; then
export PATH="$pythonLocation:$PATH"
python -m pip install windmapper --no-cache
is_gte35=$(python -c "from osgeo import gdal; import packaging.version; exit(0) if packaging.version.parse('3.5.0') <= packaging.version.parse(gdal.__version__) else exit(1)")
if [ "$is_gte35" -eq 0 ]; then
echo "GDAL version >= 3.5.0"
python -m pip install --no-cache-dir --force-reinstall gdal[numpy] # skip bizarre gdal issues
# with GDAL, esp on macos, the base package seems sensitive to gdal having a broken numpy connection
#
# python -c "from osgeo import gdal; import packaging.version; exit(0) if packaging.version.parse('3.5.0') <= packaging.version.parse(gdal.__version__) else exit(1)"
# if [[ "$?" -eq 0 ]]; then
#
# echo "GDAL version >= 3.5.0"
# python -m pip install --no-cache-dir --force-reinstall gdal[numpy]
# fi

if [ "$RUNNER_OS" = "macOS" ]; then
python -m pip install --no-cache-dir --force-reinstall gdal[numpy]
fi

cd examples/existing_DEM
Expand Down
29 changes: 28 additions & 1 deletion docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,31 @@ To not automatically build WindMapper,

::

BUILD_WINDNINJA=FALSE pip install windmapper
BUILD_WINDNINJA=FALSE pip install windmapper


Troublshooting
--------------------
If you get an error about ``_gdal_array``

::

Traceback (most recent call last):
File "/Users/runner/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/windmapper_utls/MPI_call_WN_1dir.py", line 190, in <module>
main(*sys.argv[1:])
File "/Users/runner/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/windmapper_utls/MPI_call_WN_1dir.py", line 178, in main
call_WN_1dir(WINDNINJA_DATA, gdal_prefix, user_output_dir, fic_config_WN, list_tif_2_vrt, nopt_x, nopt_y, nx,
File "/Users/runner/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/windmapper_utls/MPI_call_WN_1dir.py", line 105, in call_WN_1dir
ang = gtif.GetRasterBand(1).ReadAsArray()
File "/Users/runner/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/osgeo/gdal.py", line 5278, in ReadAsArray
from osgeo import gdal_array
File "/Users/runner/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/osgeo/gdal_array.py", line 10, in <module>
from . import _gdal_array
ImportError: cannot import name '_gdal_array' from 'osgeo' (/Users/runner/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/osgeo/__init__.py)


Then reinstall gdal with numpy

::

python -m pip install --no-cache-dir --force-reinstall gdal[numpy]
14 changes: 13 additions & 1 deletion docs/use.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,19 @@ Troubleshooting
MPI slots
**********
If an error about MPI slots is raised, set the MPI_nworkers configuration parameter to a smaller number. If it hasn't been
set, set it to 1 or 2 less than the slots reported in the error.
set, set it to 1 or 2 less than the slots reported in the error. On single-core machines you will need to enable
oversubscription. This can be done for OpenMPI

::

export OMPI_MCA_rmaps_base_oversubscribe=true; windmapper.py my-config.py

or by pernamentally enabling it

::

echo rmaps_base_oversubscribe=true >> $HOME/.openmpi/mca-params.conf



GDAL
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def gdal_dependency():
from setuptools import setup

args = {'name': 'windmapper',
'version': '2.1.14',
'version': '2.1.15',
'description': 'Windfield library generation',
'long_description': "Generates windfields",
'author': 'Chris Marsh',
Expand Down

0 comments on commit 3da5b36

Please sign in to comment.