Skip to content

Commit

Permalink
Merge branch 'next'
Browse files Browse the repository at this point in the history
Conflicts:
	sh/build_documentation.sh
  • Loading branch information
bekozi committed Feb 9, 2015
2 parents 48ba4b8 + feafc33 commit 155b79d
Show file tree
Hide file tree
Showing 160 changed files with 19,344 additions and 10,312 deletions.
2 changes: 1 addition & 1 deletion LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
OpenClimateGIS (OCGIS)

Copyright 2012-2014: University of Michigan-Ann Arbor, Michigan Technological
Copyright 2012-2015: University of Michigan-Ann Arbor, Michigan Technological
University, University of Colorado-Boulder

Licensed under the University of Illinois-NCSA License.
Expand Down
419 changes: 229 additions & 190 deletions doc/api.rst

Large diffs are not rendered by default.

12 changes: 12 additions & 0 deletions doc/appendix.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
Appendix
--------

Output Formats
~~~~~~~~~~~~~~

.. _2d-flexible-mesh-label:

2D Flexible Mesh
++++++++++++++++

The two-dimensional flexible mesh format is a NetCDF4-based format adhering to `UGRID (v0.9.0) <https://github.com/ugrid-conventions/ugrid-conventions/blob/v0.9.0/ugrid-conventions.md>`_ convention. It is used to represent a set of arbitrary boundaries (polygons) with no `gaps <http://manual.linfiniti.com/en/create_vector_data/topo_editing.html#moderate-fa-closing-the-gaps>`_. Documentation for the format is here: https://github.com/ugrid-conventions/ugrid-conventions/blob/v0.9.0/ugrid-conventions.md#2d-flexible-mesh-mixed-triangles-quadrilaterals-etc-topology.

See :ref:`2d-flexible-mesh-example-label` for example code converting a shapefile to a UGRID NetCDF file.

Spatial Operations
~~~~~~~~~~~~~~~~~~

Expand Down
9 changes: 9 additions & 0 deletions doc/computation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -240,4 +240,13 @@ Thresholds
:members: calculate
:undoc-members:

Calculation using ``icclim`` for ECA Indices
============================================

The optional Python library ``icclim`` (http://icclim.readthedocs.org/en/latest) may be used to calculate the full suite of European Climate Assessment (ECA) indices. To select an ``icclim`` calculation, prefix the name of the indice with the prefix ``'icclim_'``. A list of indices computable with ``icclim`` is available here: http://icclim.readthedocs.org/en/latest/python_api.html#icclim-indice-compute-indice.

For example, to calculate the *TG* indice (mean of daily mean temperature), select the calculation like:

>>> calc = [{'func': 'icclim_TG', 'name': 'TG'}]

.. _NumPy masked array functions: http://docs.scipy.org/doc/numpy/reference/maskedarray.html
4 changes: 2 additions & 2 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,9 @@
#
# The short X.Y version.
import ocgis
version = str(ocgis.__VER__)
version = str(ocgis.__version__)
# The full version, including alpha/beta/rc tags.
release = str(ocgis.__RELEASE__)
release = str(ocgis.__release__)

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down
39 changes: 23 additions & 16 deletions doc/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,21 +89,21 @@ Now, the directory structure for `/tmp/foo` will look like:
/tmp/foo/nc_output/nc_output_did.csv
/tmp/foo/nc_output/nc_output.nc
/tmp/foo/nc_output/nc_output_meta.txt
/tmp/foo/csv+_output
/tmp/foo/csv+_output/csv+_output_meta.txt
/tmp/foo/csv+_output/csv+_output_did.csv
/tmp/foo/csv+_output/csv+_output.csv
/tmp/foo/csv+_output/shp
/tmp/foo/csv+_output/shp/csv+_output_gid.csv
/tmp/foo/csv+_output/shp/csv+_output_gid.shp
/tmp/foo/csv+_output/shp/csv+_output_ugid.prj
/tmp/foo/csv+_output/shp/csv+_output_gid.dbf
/tmp/foo/csv+_output/shp/csv+_output_ugid.shp
/tmp/foo/csv+_output/shp/csv+_output_gid.prj
/tmp/foo/csv+_output/shp/csv+_output_ugid.shx
/tmp/foo/csv+_output/shp/csv+_output_gid.shx
/tmp/foo/csv+_output/shp/csv+_output_ugid.csv
/tmp/foo/csv+_output/shp/csv+_output_ugid.dbf
/tmp/foo/csv-shp_output
/tmp/foo/csv-shp_output/csv-shp_output_meta.txt
/tmp/foo/csv-shp_output/csv-shp_output_did.csv
/tmp/foo/csv-shp_output/csv-shp_output.csv
/tmp/foo/csv-shp_output/shp
/tmp/foo/csv-shp_output/shp/csv-shp_output_gid.csv
/tmp/foo/csv-shp_output/shp/csv-shp_output_gid.shp
/tmp/foo/csv-shp_output/shp/csv-shp_output_ugid.prj
/tmp/foo/csv-shp_output/shp/csv-shp_output_gid.dbf
/tmp/foo/csv-shp_output/shp/csv-shp_output_ugid.shp
/tmp/foo/csv-shp_output/shp/csv-shp_output_gid.prj
/tmp/foo/csv-shp_output/shp/csv-shp_output_ugid.shx
/tmp/foo/csv-shp_output/shp/csv-shp_output_gid.shx
/tmp/foo/csv-shp_output/shp/csv-shp_output_ugid.csv
/tmp/foo/csv-shp_output/shp/csv-shp_output_ugid.dbf
/tmp/foo/shp_output
/tmp/foo/shp_output/shp_output_ugid.shx
/tmp/foo/shp_output/shp_output_ugid.prj
Expand Down Expand Up @@ -153,4 +153,11 @@ For example, the code below will return all data from the year 2000 for the firs
Regridding
----------

.. literalinclude:: sphinx_examples/regridding.py
.. literalinclude:: sphinx_examples/regridding.py

.. _2d-flexible-mesh-example-label:

Converting an ESRI Shapefile to UGRID
-------------------------------------

.. literalinclude:: sphinx_examples/to_ugrid_2d_flexible_mesh.py
23 changes: 18 additions & 5 deletions doc/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,26 @@ Required Dependencies
Package Name Version URL
============== ======= =======================================================================
Python 2.7.6 http://www.python.org/download/releases/2.7.6/
``osgeo`` 1.9.1 http://pypi.python.org/pypi/GDAL/
``shapely`` 1.2 http://pypi.python.org/pypi/Shapely
``fiona`` 1.0.2 https://pypi.python.org/pypi/Fiona
``osgeo`` 1.11.1 https://pypi.python.org/pypi/GDAL/
``setuptools`` 12.0.5 https://pypi.python.org/pypi/setuptools
``shapely`` 1.4.3 https://pypi.python.org/pypi/Shapely
``fiona`` 1.4.5 https://pypi.python.org/pypi/Fiona
``numpy`` 1.8.2 http://sourceforge.net/projects/numpy/files/NumPy/1.8.2/
``netCDF4`` 1.1.1 http://unidata.github.io/netcdf4-python/
============== ======= =======================================================================

Optional Dependencies
---------------------

There are two optional dependencies. OpenClimateGIS will still operate without these libraries installed but functionality and performance may change.
Optional dependencies are listed below. OpenClimateGIS will still operate without these libraries installed but functionality and performance may change.

============= ======== ====================================================== =================================================================================================================================
Package Name Version URL Usage
============= ======== ====================================================== =================================================================================================================================
``rtree`` 2.7.2 https://pypi.python.org/pypi/Rtree/ Constructs spatial indexes at runtime. Useful for complicated GIS operations (i.e. large or complex polygons for subsetting)
``rtree`` 0.8.0 https://pypi.python.org/pypi/Rtree/ Constructs spatial indexes at runtime. Useful for complicated GIS operations (i.e. large or complex polygons for subsetting)
``cfunits`` 0.9.6 https://code.google.com/p/cfunits-python/ Allows unit transformations for ``conform_units_to`` argument to :class:`~ocgis.RequestDataset` or :class:`~ocgis.OcgOperations`.
``ESMPy`` 6.3.0rp1 https://www.earthsystemcog.org/projects/esmpy/releases Supports regridding operations.
``icclim`` 3.0 http://icclim.readthedocs.org/en/latest/ Calculation of the full suite of European Climate Assessment (ECA) indices with optimized code implementation.
============= ======== ====================================================== =================================================================================================================================

Ubuntu Linux
Expand Down Expand Up @@ -99,6 +101,17 @@ or
python -c 'import ocgis'
Testing the Installation
~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: There are tests for the optional dependencies. These will fail if the optional dependencies are not installed!

It is recommended that a simple suite of tests are run to verify the new installation:

>>> python setup.py test

Please report any errors to the support email address.

Configuring the :class:`~ocgis.ShpCabinet`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
58 changes: 28 additions & 30 deletions doc/sphinx_examples/advanced_subset.py
Original file line number Diff line number Diff line change
@@ -1,55 +1,53 @@
from ocgis import OcgOperations, RequestDataset, RequestDatasetCollection, env
import os.path

from ocgis import OcgOperations, RequestDataset, RequestDatasetCollection, env


## Directory holding climate data.
# Directory holding climate data.
DATA_DIR = '/usr/local/climate_data/CanCM4'
## Filename to variable name mapping.
NCS = {'tasmin_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc':'tasmin',
'tas_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc':'tas',
'tasmax_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc':'tasmax'}
## Always start with a snippet (if there are no calculations!).
# Filename to variable name mapping.
NCS = {'tasmin_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc': 'tasmin',
'tas_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc': 'tas',
'tasmax_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc': 'tasmax'}
# Always start with a snippet (if there are no calculations!).
SNIPPET = True
## Data returns will overwrite in this case. Use with caution!!
# Data returns will overwrite in this case. Use with caution!!
env.OVERWRITE = True


## RequestDatasetCollection ####################################################
# RequestDatasetCollection #############################################################################################

rdc = RequestDatasetCollection([RequestDataset(
os.path.join(DATA_DIR,uri),var) for uri,var in NCS.iteritems()])
os.path.join(DATA_DIR, uri), var) for uri, var in NCS.iteritems()])

## Return In-Memory ############################################################
# Return In-Memory #####################################################################################################

## Data is returned as a dictionary-like object (SpatialCollection) with 51 keys
## (don't forget Puerto Rico...). A key in the returned dictionary corresponds
## to a geometry "ugid" with the value of type OcgCollection.
# Data is returned as a dictionary-like object (SpatialCollection) with 51 keys (don't forget Puerto Rico...). A key in
# the returned dictionary corresponds to a geometry "ugid" with the value of type OcgCollection.
print('returning numpy...')
ops = OcgOperations(dataset=rdc,spatial_operation='clip',aggregate=True,
snippet=SNIPPET,geom='state_boundaries')
ops = OcgOperations(dataset=rdc, spatial_operation='clip', aggregate=True, snippet=SNIPPET, geom='state_boundaries')
ret = ops.execute()

## Return a SpatialCollection, but only for a target state in a U.S. state
## boundaries shapefile. In this case, the UGID attribute value of 23 is associated
## with Nebraska.
# Return a SpatialCollection, but only for a target state in a U.S. state boundaries shapefile. In this case, the UGID
# attribute value of 23 is associated with Nebraska.

print('returning numpy for a state...')
ops = OcgOperations(dataset=rdc,spatial_operation='clip',aggregate=True,
snippet=SNIPPET,geom='state_boundaries',select_ugid=[23])
ops = OcgOperations(dataset=rdc, spatial_operation='clip', aggregate=True, snippet=SNIPPET, geom='state_boundaries',
select_ugid=[23])
ret = ops.execute()

## Write to Shapefile ##########################################################
# Write to Shapefile ###################################################################################################

print('returning shapefile...')
ops = OcgOperations(dataset=rdc,spatial_operation='clip',aggregate=True,
snippet=SNIPPET,geom='state_boundaries',output_format='shp')
ops = OcgOperations(dataset=rdc, spatial_operation='clip', aggregate=True, snippet=SNIPPET, geom='state_boundaries',
output_format='shp')
path = ops.execute()

## Write All Data to Keyed Format ##############################################
# Write All Data to Keyed Format #######################################################################################

## Without the snippet, we are writing all data to the linked CSV-Shapefile
## output format. The operation will take considerably longer.
print('returning csv+...')
ops = OcgOperations(dataset=rdc,spatial_operation='clip',aggregate=True,
snippet=False,geom='state_boundaries',output_format='csv+')
# Without the snippet, we are writing all data to the linked CSV-Shapefile output format. The operation will take
# considerably longer.
print('returning csv-shp...')
ops = OcgOperations(dataset=rdc, spatial_operation='clip', aggregate=True, snippet=False, geom='state_boundaries',
output_format='csv-shp')
path = ops.execute()
49 changes: 30 additions & 19 deletions doc/sphinx_examples/install_dependencies_ubuntu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ sudo pip install numpy netCDF4 shapely fiona rtree
## osgeo ##
###########

## If this fails, try the apt-get install below...

## http://stackoverflow.com/questions/11336153/python-gdal-package-missing-header-file-when-installing-via-pip
pip install --no-install GDAL
cd /tmp/pip_build_ubuntu/GDAL
Expand All @@ -25,22 +23,22 @@ sudo python setup.py install

CFUNITS_SRCDIR=/tmp/build_cfunits
CFUNITS_VER=0.9.6
CFUNITS_SRC=$CFUNITS_SRCDIR/cfunits-python/v$CFUNITS_VER
CFUNITS_TARBALL=cfunits-$CFUNITS_VER.tar.gz
CFUNITS_URL=https://cfunits-python.googlecode.com/files/$CFUNITS_TARBALL

mkdir -p $CFUNITS_SRC
cd $CFUNITS_SRC
wget $CFUNITS_URL
tar -xzvf $CFUNITS_TARBALL
cd cfunits-$CFUNITS_VER
CFUNITS_SRC=${CFUNITS_SRCDIR}/cfunits-python/v${CFUNITS_VER}
CFUNITS_TARBALL=cfunits-${CFUNITS_VER}.tar.gz
CFUNITS_URL=https://cfunits-python.googlecode.com/files/${CFUNITS_TARBALL}

mkdir -p ${CFUNITS_SRC}
cd ${CFUNITS_SRC}
wget ${CFUNITS_URL}
tar -xzvf ${CFUNITS_TARBALL}
cd cfunits-${CFUNITS_VER}
sudo python setup.py install
## installation does not copy UDUNITS database
CFUNITS_SETUP_DIR=`pwd`
## assumes a standard location. the installation directory may be retrieved by running the command:
## python -c "import cfunits, os;print(os.path.split(cfunits.__file__)[0])"
CFUNITS_INSTALL_DIR=/usr/local/lib/python2.7/dist-packages/cfunits
sudo cp -r $CFUNITS_SETUP_DIR/cfunits/etc $CFUNITS_INSTALL_DIR
sudo cp -r ${CFUNITS_SETUP_DIR}/cfunits/etc ${CFUNITS_INSTALL_DIR}

#########
# ESMPy #
Expand All @@ -63,19 +61,32 @@ ESMF_INSTALL_PREFIX=<path to esmf install directory>
## ESMF framework install ##

sudo apt-get install gfortran g++
mkdir -p $ESMF_SRCDIR
cd $ESMF_SRCDIR
mkdir -p ${ESMF_SRCDIR}
cd ${ESMF_SRCDIR}
<make sure ESMF tarball is available in the source directory>
tar -xzvf $ESMF_TAR
tar -xzvf ${ESMF_TAR}
cd esmf
export ESMF_DIR=`pwd`
make
export ESMF_INSTALL_PREFIX=$ESMF_INSTALL_PREFIX
export ESMF_INSTALL_LIBDIR=$ESMF_INSTALL_PREFIX/lib
export ESMF_INSTALL_PREFIX=${ESMF_INSTALL_PREFIX}
export ESMF_INSTALL_LIBDIR=${ESMF_INSTALL_PREFIX}/lib
sudo -E make install

## ESMPy install ##

cd $ESMF_INSTALL_PREFIX/esmf/src/addon/ESMPy
python setup.py build --ESMFMKFILE=$ESMF_INSTALL_PREFIX/lib/esmf.mk
cd ${ESMF_SRCDIR}/esmf/src/addon/ESMPy
python setup.py build --ESMFMKFILE=${ESMF_INSTALL_PREFIX}/lib/esmf.mk
sudo python setup.py install

##########
# ICCLIM #
##########

ICCLIM_SRCDIR=~/src/icclim/git

git clone https://github.com/tatarinova/icclim.git
pushd ${ICCLIM_SRCDIR}/icclim
gcc -fPIC -g -c -Wall ./icclim/libC.c -o ./icclim/libC.o
gcc -shared -o ./icclim/libC.so ./icclim/libC.o
sudo python setup.py install
popd
41 changes: 20 additions & 21 deletions doc/sphinx_examples/simple_subset.py
Original file line number Diff line number Diff line change
@@ -1,40 +1,39 @@
import ocgis
import os

import ocgis



## Directory holding climate data.
# Directory holding climate data.
DATA_DIR = '/usr/local/climate_data/CanCM4'
## Location and variable name for a daily decadal temperature simulation.
URI_TAS = os.path.join(DATA_DIR,'tas_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc')
# Location and variable name for a daily decadal temperature simulation.
URI_TAS = os.path.join(DATA_DIR, 'tas_day_CanCM4_decadal2000_r2i1p1_20010101-20101231.nc')
VAR_TAS = 'tas'
## Make it easy to switch to non-snippet requests.
# Make it easy to switch to non-snippet requests.
SNIPPET = True
## Set output directory for shapefile and keyed formats. (MAKE SURE IT EXISTS!)
# Set output directory for shapefile and keyed formats. (MAKE SURE IT EXISTS!)
ocgis.env.DIR_OUTPUT = '/tmp/foo'
os.mkdir('/tmp/foo')
## The bounding box coordinates [minx, miny, maxx, maxy] for the state of
## Colorado in WGS84 latitude/longitude coordinates.
# The bounding box coordinates [minx, miny, maxx, maxy] for the state of Colorado in WGS84 latitude/longitude
# coordinates.
BBOX = [-109.1, 36.9, -102.0, 41.0]


## Construct RequestDataset Object #############################################
# Construct RequestDataset Object ######################################################################################

## This object will be reused so just build it once.
rd = ocgis.RequestDataset(URI_TAS,VAR_TAS)
# This object will be reused so just build it once.
rd = ocgis.RequestDataset(URI_TAS, VAR_TAS)

## Returning NumPy Data Objects ################################################
# Returning NumPy Data Objects #########################################################################################

## The NumPy data type return is the default. Only the geometry and
## RequestDataset are required (except "snippet" of course...). See the
## documentation for the OcgCollection object to understand the return
## structure.
ret = ocgis.OcgOperations(dataset=rd,geom=BBOX,snippet=SNIPPET).execute()
# The NumPy data type return is the default. Only the geometry and RequestDataset are required (except "snippet" of
# course...). See the documentation for the OcgCollection object to understand the return structure.
ret = ocgis.OcgOperations(dataset=rd, geom=BBOX, snippet=SNIPPET).execute()

## Returning Converted Files ###################################################
# Returning Converted Files ############################################################################################

output_formats = ['shp','csv','csv+','nc']
output_formats = ['shp', 'csv', 'csv-shp', 'nc']
for output_format in output_formats:
prefix = output_format + '_output'
ops = ocgis.OcgOperations(dataset=rd,geom=BBOX,snippet=SNIPPET,
output_format=output_format,prefix=prefix)
ops = ocgis.OcgOperations(dataset=rd, geom=BBOX, snippet=SNIPPET, output_format=output_format, prefix=prefix)
ret = ops.execute()
Loading

0 comments on commit 155b79d

Please sign in to comment.