Name		Name	Last commit message	Last commit date
parent directory ..
clustering-gbif-oozie		clustering-gbif-oozie
clustering-gbif		clustering-gbif
ingest-gbif-beam		ingest-gbif-beam
ingest-gbif-fragmenter		ingest-gbif-fragmenter
ingest-gbif-java		ingest-gbif-java
ingestion-integration-tests-hbase		ingestion-integration-tests-hbase
ingestion-integration-tests		ingestion-integration-tests
pre-backbone-release		pre-backbone-release
README.md		README.md
pom.xml		pom.xml

README.md

Pipelines

Module uses Apache Beam as an unified programming model to define and execute data processing pipelines

export-gbif-hbase - The pipeline to export the verbatim data from the GBIF HBase tables and save as ExtendedRecord avro files
ingest-gbif-beam - Main GBIF pipelines for ingestion of biodiversity data
ingest-gbif-fragmenter - Writes raw archive's data to HBase store
ingest-gbif-java - Main GBIF pipelines for ingestion of biodiversity data, Java version