Skip to content

Latest commit

 

History

History
293 lines (183 loc) · 10.3 KB

USAGE.rst

File metadata and controls

293 lines (183 loc) · 10.3 KB
Raw

Usage

Command-Line Interface (CLI)

The BDC-Collectors extension installs a command line tool named bdc-collector:

  • show-providers: List all the supported providers.
  • search: Search for products on remote server.
  • download: Download scenes from remote server.

Search Data sets

The command search has the following parameters:

Search for data set in the given provider.

Options:
    -p --provider TEXT [required]  Provider name to search.
    -d --dataset  TEXT [required]  Data set name  in provider.
    -b --bbox     TEXT [required]  Bounding box definition (west, south, east, north).
    -t --time     TEXT [required]  Time interval. (start/end). Format should be (YYYY-mm-dd)
    -u --username TEXT             Optional username used to search in provider.
       --password TEXT             Optional password used to search in provider.
       --config   PATH             Path to the configuration file to extend options for provider creation
       --platform TEXT             Platform sensor (if required)
       --help                      Show this message and exit.

SciHub

To search for Sentinel-2 L1 in SciHub catalog:

bdc-collector search --provider=SciHub \
                     --dataset=S2MSI1C \
                     --platform=Sentinel-2 \
                     --time=2020-01-01/2020-01-15 \
                     --bbox=-54,-12,-50,-10 \
                     --username=user \
                     --password=password

To search for Sentinel-1 GRD in SciHub catalog:

bdc-collector search --provider=SciHub \
                     --dataset=GRD \
                     --platform=Sentinel-1 \
                     --time=2020-01-01/2020-01-15 \
                     --bbox=-54,-12,-50,-10 \
                     --username=user \
                     --password=password

Note

Make sure to change --username and --password. You can create an account in SciHub Registration.

You can also search for Sentinel-2 L2A products. Use dataset=S2MSI2A and platform=Sentinel-2.

USGS

To search for Landsat-8 Digital Number in USGS Earth Explorer:

bdc-collector search --provider=USGS \
                     --dataset=LANDSAT_8_C1 \
                     --time=2020-01-01/2020-01-15 \
                     --bbox=-54,-12,-50,-10 \
                     --username=user \
                     --password=password

Note

Make sure to change --username and --password. You can create an account in USGS EROS Registration.

You can also search for others Landsat products:

  • Landsat-4/5 Collection 2 L1, use dataset=landsat_tm_c2_l1
  • Landsat-4/5 Collection 2 L2, use dataset=landsat_tm_c2_l2
  • Landsat-7 Collection 2 L1, use dataset=landsat_etm_c2_l1
  • Landsat-7 Collection 2 L2, use dataset=landsat_etm_c2_l2
  • Landsat-8 Collection 2 L1, use dataset=landsat_ot_c2_l1
  • Landsat-8 Collection 2 L2, use dataset=landsat_ot_c2_l2

For collections like MODIS and Sentinel-2 from USGS, we only support to search for data and download we are still working in progress.

We have created a few BDC Alias for parameters:

  • day_night_indicator='Day': The day indicator to search for scenes in the given region. Default is 'Day'. The supported values are defined by USGS JSON API 'All', 'Day' and 'Night'

Download scenes

The command download has the following parameters:

Search for data set in the given provider.

Options:
    -p, --provider TEXT [required] Provider name to search
    -s, --scene-id TEXT [required] Scene Identifier to download.
    -o, --output   TEXT [required] Save output directory
    -u, --username TEXT            Optional username to download
    -P, --password TEXT            User password
        --help                     Show this message and exit.

Note

Currently, you can only download by scene_id like S2B_MSIL1C_20200223T135109_N0209_R024_T21LZG_20200223T153255.

We will implement way to download from tiles, since some apis (sentinel-sat - SciHub) already support this feature.

USGS

To download Landsat-8 Digital Number from USGS Earth Explorer:

bdc-collector download --provider=USGS \
                       --scene-id=LC08_L1TP_223064_20200831_20200906_01_T1 \
                       --dataset=LANDSAT_8_C1 \
                       --output=. \
                       --username=user \
                       --password=password

SciHub

Warning

The official Copernicus SciHub is being deprecated. Use Dataspace instead.

To download Sentinel-2 from SciHub:

bdc-collector download --provider=SciHub \
                       --scene-id=S2B_MSIL1C_20200223T135109_N0209_R024_T21LZG_20200223T153255 \
                       --output=. \
                       --username=user \
                       --password=password

To download L2A:

bdc-collector download --provider=SciHub \
                       --scene-id=S2B_MSIL2A_20200930T135119_N0214_R024_T21KXA_20200930T175714 \
                       --output=. \
                       --username=user \
                       --password=password

Dataspace

To download Sentinel-2 from Dataspace EcoSystem:

bdc-collector download --provider=Dataspace \
                       --scene-id=S2B_MSIL2A_20200930T135119_N0214_R024_T21KXA_20200930T175714 \
                       --output=. \
                       --dataset SENTINEL-2 \
                       --username=user \
                       --password=password

Google Public Data Sets

Warning

Google Public Data Sets only supports Landsat Collection 1 and Sentinel-2 L1C/L2A). The Landsat Collection 2 is only available in provider USGS (EarthExplorer) If you really want to download Landsat Collection 1, continue in this example.

You can also download both Landsat Digital Number and Sentinel-2 (L1C/L2A) from Google Public Data Sets. In order to do that, you will need to create an Google Service Account Key and export the variable GOOGLE_APPLICATION_CREDENTIALS=path/to/google/your_service_account_key.json.:

export GOOGLE_APPLICATION_CREDENTIALS=path/to/google/your_service_account_key.json

bdc-collector download --provider=Google \
                       --scene-id=LC08_L1TP_223064_20200831_20200906_01_T1 \
                       --output=.

You can download Sentinel-2 products with:

export GOOGLE_APPLICATION_CREDENTIALS=path/to/google/your_service_account_key.json

bdc-collector download --provider=Google \
                       --scene-id=S2B_MSIL1C_20200223T135109_N0209_R024_T21LZG_20200223T153255 \
                       --output=.

ONDA Catalogue

You can also download Sentinel scenes from alternative ONDA DIAS Catalogue.

In order to do that, you must have an account ONDA User Portal Registration.:

bdc-collector download --provider=ONDA \
                       --scene-id=S2B_MSIL1C_20200223T135109_N0209_R024_T21LZG_20200223T153255 \
                       --output=. \
                       --username=user \
                       --password=password

Preparing a new package with BDC-Collectors

In order to attach BDC-Collectors into your application, use the following statements:

from flask import flask
from bdc_collectors.ext import CollectorExtension

app = Flask(__name__)
CollectorExtension(app)

Note

If you would like to connect into database with BDC-Catalog, make sure to follow the steps defined in BDC-Catalog.

Preparing a new provider for BDC-Collectors

The BDC-Collectors follows the Python Entry point specification to discover and load libraries dynamically.

Basically, the BDC-Collectors has the following entry points to deal with dynamic data provider:

  • bdc_db.providers: Entry point to configure the default Catalog Providers. Append new values in your application and make sure to initialize CollectorExtension to make your own providers available.

Note

You can also set bdc_db.scripts if you would like to insert a new SQL for data provider. Check BDC-DB for further details.

These entry points may be defined in the setup.py of your package.

The following code is an example of an entry_points in setup.py file:

entry_points={
    'bdc_collectors.providers': [
        'mycatalog = my_app.mycatalog'
    ]
}