Awesome earthobservation code

A curated list of awesome resources on Earth observation coding libraries and open data

Please contribute to make this guide better! Please follow the Contributing Guidelines. Inspired by Awesome Python and Awesome GIS

Table Of Contents:

• Learning resources and platforms
  • Data formats
  • MOOC
  • Youtube channels
• Software & SaaS
  • Jupyter based training resources
  • Web Map Servers
  • Spatial Databases
• Language based EO Libraries
  • Jupyter notebooks
  • Python
  • C++
• Data and Utilities
• Open Forums and communities
• Awesome-Awesomeness

---

Learning resources and platforms

▲ Top

Data formats

• NetCDF and CF - the basics - This workshop will teach some of the basics of Climate and Forecasting metadata for netCDF data files with some hands-on work available in Jupyter Notebooks using Python.

MOOC

• Coursera's GIS Specialization - Including Fundamentals of GIS, GIS Data Formats, Design and Quality, Geospatial and Environmental Analysis, Imagery, Automation, and Applications and Capstone: Geospatial Analysis.
• EUMetLab training portal - The European Space Agency (ESA), the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), and the European Center for Medium-Range Weather Forecast (ECMWF) with the Copernicus Atmosphere Monitoring Service (CAMS) organize joint training webinars sessions.
• Monitoring Atmospheric Composition - A MOOC about atmospheric monitoring using satellites, in-situ data, and models.
• Unidata training workshop - Unidata training workshop on CF conventions and data manipulation.
• Unidata python workshop - Would you like some in-depth training on the scientific Python ecosystem for atmospheric science and meteorology? Work through our workshop materials at your own pace to learn and practice the syntax, functionality, and utility of this powerful programming language, or return to the material after taking the workshop in-person to further your understanding of the material you were taught.
• Use Open source EO data with python - Use Data for Earth and Environmental Science in Open Source Python. Intermediate workshop course.

Youtube channels

• metPy mondays - metPy mondays youtube videos
• Unidata youtube channel - Unidata youtube channel on Integrated Data Viewer (IDV) and metPy

Software and saas

▲ Top

Jupyter based training resources

▲ Top

Many Jupyter based notebooks involve data access using free or paid API services. For more information, jump to the EO data services section of this guide.

• Kaggle - Inside Kaggle you’ll find all the code & data you need to do your data science work. Use over 50,000 public datasets and 400,000 public notebooks to conquer any analysis in no time.
• LTPy - LTPy, a learning Tool for Python on Atmospheric Composition Data.
• ECMWF : introduction to Magics - Magics is the latest generation of the ECMWF's meteorological plotting software. It offers an easy way to visualise data coded in meteorological formats such as GRIB, NetCDF and BUFR. The same resource, in ECMWF Atlassian Confluence collaboration site.
• ECMWF examples - The examples in this space should give you a good starting point how you can work with ECMWF services and data through Python using Jupyter notebooks. A Binder space is also available for immediate playground access.
  • Visualisation meteorological data using ECMWF's Magics plotting package for meteorological data.
  • Retrieving and processing meteorological data using Metview and related Python packages.
• Kaggle MeteoNet North-West France - MeteoNet is a meteorological dataset developed and made available by METEO FRANCE, the French national meteorological service. Let’s start playing with the dataset! You can find playful introductory Notebooks and in the Tasks tab some challenges proposals!
• MeteoNet : Data exploration toolbox - This repository is intended as a toolbox to handle the MeteoNet dataset. It's also a communication interface with MeteoNet's users : if you have a request or a problem concerning MeteoNet, you can post an issue on this project.
• Unidata python training resources - This site is meant to be a one-stop website for learning how to use Python for earth-science education and research for any experience level.
• Unidata python examples gallery - Check out detailed examples for atmospheric science and meteorology, from data analysis to publication-quality figures. Feel free to use code from these examples in your own work.
• Working with Spatio-temporal data in Python - With Software Carpentry lessons and Data Carpentry lessons you learn the fundamental data skills needed to conduct research in your field and learn to write simple programs. This one-day workshop will introduce you to Python for analyzing and visualizing spatial-temporal data. We will be using datasets from the environmental sciences that are freely available.

Web Map Servers

▲ Top

Spatial Databases

▲ Top

Language based EO Libraries

▲ Top

Jupyter notebooks librairies

• appmode - A Jupyter extensions that turns notebooks into web applications.
• ipywidgets - Widgets are eventful python objects that have a representation in the browser, often as a control like a slider, textbox, etc.
• ipyleaflet - Interactive maps in the Jupyter notebook.
• jupyterlab-manager - A JupyterLab extension for Jupyter/IPython widgets.
• jupyter_bokeh - An extension for rendering Bokeh content in JupyterLab notebooks.
• jupyter-matplotlib - An extension for rendering Matplotlib content in JupyterLab notebooks.
• jupyterlab-plotly - The plotly Python library is an interactive, open-source plotting library that supports over 40 unique chart types covering a wide range of statistical, financial, geographic, scientific, and 3-dimensional use-cases.
• jupyterlab-voyager - JupyterLab extension visualize data with Voyager.

Python

▲ Top

Interactive notebooks

• binder - Have a repository full of Jupyter notebooks? With Binder, open those notebooks in an executable environment, making your code immediately reproducible by anyone, anywhere.
• Jupyter - JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data.
• papermill - Papermill is a tool for parameterizing and executing Jupyter Notebooks.

Visualization librairies

• bokeh - The Bokeh Visualization Library
• cmocean - This package contains colormaps for commonly-used oceanographic variables.
• matplotlib-base - Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python.
• panel - A high-level app and dashboarding solution for Python.
• seaborn - Seaborn is a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.

EO data access & computations

• cartopy - Cartopy is a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses.
• ecwmf-api-client - ECMWF WebAPI is a set of services developed by ECMWF to allow users from the outside to access some internal features and data of the centre.
• iris - A powerful, format-agnostic, community-driven Python library for analysing and visualising Earth science data.
• iris-grib - The library iris-grib provides functionality for converting between weather and climate datasets that are stored as GRIB files and Iris cubes.
• magics - Python interface to Magics meteorological plotting package.
• metpy - MetPy is a collection of tools in Python for reading, visualizing, and performing calculations with weather data.
• metview - Python interface to Metview meteorological workstation and batch system.
• netcdf4 - netcdf4-python is a Python interface to the netCDF C library.
• pynio - PyNIO is a multi-format data I/O package with a NetCDF-style interface.
• scipy - SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics, science, and engineering. In particular scikit-image for Image processing in Python and scikit-learn for Machine Learning in Python
• siphon - A collection of Python utilities for retrieving atmospheric and oceanic data from remote sources, focusing on being able to retrieve data from Unidata data technologies, such as the THREDDS data server.

Generic useful librairies

• bottleneck - Bottleneck is a collection of fast, NaN-aware NumPy array functions written in C. Working with pandas and xarray
• dask - Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love.
• descartes - Use Shapely or GeoJSON-like geometric objects as matplotlib paths and patches
• numpy - The fundamental package for scientific computing with Python.
• protobuf - Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data.
• psycopg2 - Psycopg is the most popular PostgreSQL database adapter for the Python programming language.
• pygrib - pyGrib provides a high-level interface to the ECWMF ECCODES C library for reading GRIB files.
• pytables - PyTables is a package for managing hierarchical datasets and designed to efficiently and easily cope with extremely large amounts of data.
• shapely - Manipulation and analysis of geometric objects in the Cartesian plane.
• xarray - xarray (formerly xray) is an open source project and Python package that makes working with labelled multi-dimensional arrays simple, efficient, and fun!

C++

▲ Top

• GEOS - GEOS (Geometry Engine - Open Source) is a C++ port of the ​JTS Topology Suite (JTS). It aims to contain the complete functionality of JTS in C++.
• GeoPandas - GeoPandas is an open source project to make working with geospatial data in python easier. GeoPandas extends the datatypes used by pandas to allow spatial operations on geometric types.
• [MDAL]https://www.mdal.xyz/) - Mesh Data Abstraction Library (MDAL)

Data and Utilities

▲ Top
▲ Jupyter based training resources

Utilities

• BBox finder - Select an area and get BBox information in vairous projections and formats.
• epsg.io - Coordinate systems worldwide
• ecwmf-api-client - ECMWF WebAPI is a set of services developed by ECMWF to allow users from the outside to access some internal features and data of the centre.

Data download services

• Climate Data Online - NOAA Climate Data Online is a collection of climatic data that offers public access and consumption via discovery and ordering services. The data available through CDO is available at no charge and can be viewed online or ordered and delivered to your email inbox.
• ECMWF public datasets - Access to these datasets is provided free of charge. Terms and conditions may apply, please check with each individual dataset.
• ECMWF Web API - ECMWF WebAPI is a set of services developed by ECMWF to allow users from the outside to access some internal features and data of the centre.
• GEBCO Gridded Bathymetry Data - The GEBCO_2020 grid is a global terrain model for ocean and land at 15 arc-second intervals.
• Natural Earth - A public domain dataset available at 1:10m, 1:50m, and 1:110 million scales. Featuring tightly integrated vector and raster data, with Natural Earth you can make a variety of visually pleasing, well-crafted maps with cartography or GIS software.
• NCEI Integrated Surface Dataset (Global) - The ISD is composed of worldwide surface weather observations from over 35,000 stations, though the best spatial coverage is evident in North America, Europe, Australia, and parts of Asia. Parameters included are: air quality, atmospheric pressure, atmospheric temperature/dew point, atmospheric winds, clouds, precipitation, ocean waves, tides and more.
• Open Weather Map - OpenWeatherMap provides many kinds of weather maps including Precipitation, Clouds, Pressure, Temperature, Wind. You can connect them to mobile and web apps.
• Worldclim - WorldClim is a database of high spatial resolution global weather and climate data. These data can be used for mapping and spatial modeling.

EO data API and web services

• Climate Data Online : Web Services - NCDC's Climate Data Online (CDO) offers web services that provide access to current data. This API is for developers looking to create their own scripts or programs that use the CDO database of weather and climate data.
• Copernicus Sentinel-5 Precursor (Sentinel-5P)
• ECWMF Datasets - ECWMF offcial public datasets (Global/Regional Reanalyses, Multi-model, Atmospheric composition, Observation Feedback, etc.)
• ECWF Datasets Web API - Access ECMWF Public Datasets through the official API, using the ecwmf-api-client toolkit.
• EUMETSAT AC SAF - EUMETSAT Satellite Application Facility on Atmospheric Composition Monitoring
• EUMETSAT EO Portal - Free of charge access to EUMETSAT datasets.
• Copernicus Open Access hub - The Copernicus Open Access Hub (previously known as Sentinels Scientific Data Hub) provides complete, free and open access to Sentinel-1, Sentinel-2, -Sentinel-3 and Sentinel-5P user products, starting from the In-Orbit Commissioning Review (IOCR).
• OpenWeatherMap API - Historical, Current & Forecast worldwide weather data collection. Includes Maps collections.
• WEkEO's Harmonized Data Access API - Copernicus and Sentinel data at your fingertips alongside cloud computing resources and tools. A REST API allows users to subset data using a programming interface.

Open Forums and communities

▲ Top

• EUMETSAT - European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT)
• ESA - European Space Agency main site

Awesome Awesomeness

▲ Top