Skip to content

Commit

Permalink
feat: use attempt login for nsidc utilities (#55)
Browse files Browse the repository at this point in the history
* feat:  add NASA CMR spatial bounding box queries
* feat: update classify photons parameters to match current GSFC version
* feat: updates for uploading to pypi
* refactor: place some imports behind try/except statements
  • Loading branch information
tsutterley committed Jun 15, 2022
1 parent 21f434b commit b8bdb49
Show file tree
Hide file tree
Showing 17 changed files with 302 additions and 273 deletions.
1 change: 1 addition & 0 deletions .github/workflows/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ RUN pip3 install --no-cache-dir --no-binary=h5py,cartopy \
paramiko \
pyproj \
python-dateutil \
pyYAPC \
scikit-learn \
scipy \
scp \
Expand Down
4 changes: 4 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ read-ICESat-2

|Language|
|License|
|PyPI Version|
|Documentation Status|
|Binder|
|Pangeo|
Expand All @@ -15,6 +16,9 @@ read-ICESat-2
.. |License| image:: https://img.shields.io/badge/license-MIT-green.svg
:target: https://github.com/tsutterley/read-ICESat-2/blob/main/LICENSE

.. |PyPI Version| image:: https://img.shields.io/pypi/v/icesat2-toolkit.svg
:target: https://pypi.python.org/pypi/icesat2-toolkit/

.. |Documentation Status| image:: https://readthedocs.org/projects/read-icesat-2/badge/?version=latest
:target: https://read-icesat-2.readthedocs.io/en/latest/?badge=latest

Expand Down
15 changes: 10 additions & 5 deletions doc/source/getting_started/Install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ The ``read-ICESat-2`` installation uses the ``gdal-config`` routines to set the
Installation
############

Presently ``read-ICESat-2`` is only available for use as a
`GitHub repository <https://github.com/tsutterley/read-ICESat-2>`_.
``read-ICESat-2`` is available for download from the `GitHub repository <https://github.com/tsutterley/read-ICESat-2>`_,
and the `Python Package Index (pypi) <https://pypi.org/project/icesat2-toolkit/>`_,
The contents of the repository can be download as a
`zipped file <https://github.com/tsutterley/read-ICESat-2/archive/main.zip>`_ or cloned.
To use this repository, please fork into your own account and then clone onto your system.
Expand All @@ -55,6 +55,11 @@ Alternatively can install the utilities directly from GitHub with ``pip``:
python3 -m pip install --user git+https://github.com/tsutterley/read-ICESat-2.git
Executable versions of this repository can also be tested using
`Binder <https://mybinder.org/v2/gh/tsutterley/read-ICESat-2/main>`_ or
`Pangeo <https://aws-uswest2-binder.pangeo.io/v2/gh/tsutterley/read-ICESat-2/main?urlpath=lab>`_.
| This repository can be also tested using `BinderHub <https://github.com/jupyterhub/binderhub>`_ platforms:
| |Binder| |Pangeo|
.. |Binder| image:: https://mybinder.org/badge_logo.svg
:target: https://mybinder.org/v2/gh/tsutterley/read-ICESat-2/main

.. |Pangeo| image:: https://img.shields.io/static/v1.svg?logo=Jupyter&label=PangeoBinderAWS&message=us-west-2&color=orange
:target: https://aws-uswest2-binder.pangeo.io/v2/gh/tsutterley/read-ICESat-2/main?urlpath=lab
22 changes: 11 additions & 11 deletions doc/source/getting_started/Parallel-HDF5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,27 +41,27 @@ Dependencies

.. code-block:: bash
curl -O http://zlib.net/zlib-1.2.11.tar.gz
curl -O http://zlib.net/zlib-1.2.12.tar.gz
export CFLAGS=-fPIC
mkdir -p $HOME/packages/zlib/1.2.11
./configure --prefix=$HOME/packages/zlib/1.2.11
mkdir -p $HOME/packages/zlib/1.2.12
./configure --prefix=$HOME/packages/zlib/1.2.12
make
make install
modulefile for local installation of zlib (``~/privatemodules/zlib/1.2.11``):
modulefile for local installation of zlib (``~/privatemodules/zlib/1.2.12``):

.. code-block:: tcl
#%Module 1.0
#
# zlib module for use with 'environment-modules' package:
#
module-whatis "Provides zlib 1.2.11 (local)"
module-whatis "Provides zlib 1.2.12 (local)"
global env
prepend-path PATH $env(HOME)/packages/zlib/1.2.11/bin
prepend-path LD_LIBRARY_PATH $env(HOME)/packages/zlib/1.2.11/lib
prepend-path MANPATH $env(HOME)/packages/zlib/1.2.11/share/man/
append-path ZLIB_DIR $env(HOME)/packages/zlib/1.2.11/
prepend-path PATH $env(HOME)/packages/zlib/1.2.12/bin
prepend-path LD_LIBRARY_PATH $env(HOME)/packages/zlib/1.2.12/lib
prepend-path MANPATH $env(HOME)/packages/zlib/1.2.12/share/man/
append-path ZLIB_DIR $env(HOME)/packages/zlib/1.2.12/
- `szip <https://support.hdfgroup.org/doc_resource/SZIP/>`_
Expand Down Expand Up @@ -127,7 +127,7 @@ modulefile for local installation of OpenMPI (``~/privatemodules/mpi/openmpi/4.0
CC=~/packages/mpi/openmpi/4.0.3/bin/mpicc ./configure \
--enable-parallel --enable-hl --enable-shared \
--prefix=$HOME/packages/hdf5/1.10.5 \
--with-zlib=$HOME/packages/zlib/1.2.11 \
--with-zlib=$HOME/packages/zlib/1.2.12 \
--with-szip=$HOME/packages/szip/2.1.1
make
make check
Expand All @@ -143,7 +143,7 @@ modulefile for local installation of HDF5 (``~/privatemodules/hdf5/1.10.5``):
#
module-whatis "Provides hdf5 1.10.5 (local)"
global env
prereq $env(HOME)/privatemodules/zlib/1.2.11 $env(HOME)/privatemodules/szip/2.1.1 $env(HOME)/privatemodules/mpi/openmpi/4.0.3
prereq $env(HOME)/privatemodules/zlib/1.2.12 $env(HOME)/privatemodules/szip/2.1.1 $env(HOME)/privatemodules/mpi/openmpi/4.0.3
prepend-path PATH $env(HOME)/packages/hdf5/1.10.5/bin
prepend-path LD_LIBRARY_PATH $env(HOME)/packages/hdf5/1.10.5/lib
prepend-path MANPATH $env(HOME)/packages/hdf5/1.10.5/share/man/
Expand Down
26 changes: 13 additions & 13 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
name: read-ICESat-2
name: icesat2_toolkit
channels:
- conda-forge
dependencies:
- python>=3.6
- notebook
- numpy
- scipy
- scikit-learn
- python-dateutil
- pandas
- pyproj
- matplotlib
- boto3
- cartopy
- mpi4py
- future
- gdal
- h5py>=2.9=mpi*
- netCDF4
- zarr
- future
- boto3
- lxml
- matplotlib
- mpi4py
- netCDF4
- numpy
- pandas
- paramiko
- pyproj
- python-dateutil
- scikit-learn
- scipy
- scp
- zarr
- pip:
- git+https://github.com/tsutterley/yapc.git
3 changes: 1 addition & 2 deletions icesat2_toolkit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
import icesat2_toolkit.time
import icesat2_toolkit.utilities
import icesat2_toolkit.version
from icesat2_toolkit.convert import convert
from icesat2_toolkit.convert_delta_time import convert_delta_time
from icesat2_toolkit.read_ICESat2_ATL03 import read_HDF5_ATL03, \
find_HDF5_ATL03_beams, read_HDF5_ATL09, read_HDF5_ATL03_main, read_HDF5_ATL03_beam
Expand All @@ -33,4 +32,4 @@
from icesat2_toolkit.read_ICESat2_ATL12 import read_HDF5_ATL12, \
find_HDF5_ATL12_beams
# get version number
__version__ = icesat2_toolkit.version.version
__version__ = icesat2_toolkit.version.version
20 changes: 17 additions & 3 deletions icesat2_toolkit/convert.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""
convert.py
Written by Tyler Sutterley (04/2022)
Written by Tyler Sutterley (06/2022)
Utilities for converting ICESat-2 HDF5 files into different formats
PYTHON DEPENDENCIES:
Expand All @@ -21,6 +21,7 @@
time.py: Utilities for calculating time operations
UPDATE HISTORY:
Updated 06/2022: place zarr and pandas imports behind try/except statements
Updated 04/2022: updated docstrings to numpy documentation format
Updated 01/2022: added ascii and dataframe outputs for ATL07
Updated 09/2021: added ground track and time to output dataframes
Expand All @@ -33,13 +34,26 @@
import os
import re
import h5py
import zarr
import pandas
import warnings
import itertools
import posixpath
import numpy as np
from icesat2_toolkit.convert_delta_time import convert_delta_time

try:
import pandas
except ModuleNotFoundError:
warnings.filterwarnings("always")
warnings.warn("pandas not available")
warnings.warn("Some functions will throw an exception if called")

try:
import zarr
except ModuleNotFoundError:
warnings.filterwarnings("always")
warnings.warn("zarr not available")
warnings.warn("Some functions will throw an exception if called")

class convert():
np.seterr(invalid='ignore')
def __init__(self, filename=None, reformat=None):
Expand Down
13 changes: 11 additions & 2 deletions icesat2_toolkit/spatial.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python
u"""
spatial.py
Written by Tyler Sutterley (04/2022)
Written by Tyler Sutterley (06/2022)
Utilities for reading and operating on spatial data
Expand All @@ -17,6 +17,7 @@
https://pypi.python.org/pypi/GDAL
UPDATE HISTORY:
Updated 06/2022: place netCDF4 import behind try/except statements
Updated 04/2022: updated docstrings to numpy documentation format
Updated 01/2022: use iteration breaks in convert ellipsoid function
Written 11/2021
Expand All @@ -28,14 +29,22 @@
import uuid
import h5py
import logging
import netCDF4
import warnings
import numpy as np

try:
import netCDF4
except ModuleNotFoundError:
warnings.filterwarnings("always")
warnings.warn("netCDF4 not available")
warnings.warn("Some functions will throw an exception if called")

try:
import osgeo.gdal, osgeo.osr, osgeo.gdalconst
except ModuleNotFoundError:
warnings.filterwarnings("always")
warnings.warn("GDAL not available")
warnings.warn("Some functions will throw an exception if called")

def case_insensitive_filename(filename):
"""
Expand Down
Loading

0 comments on commit b8bdb49

Please sign in to comment.