Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 93 additions & 6 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,18 +104,105 @@ example.

### Visualization Workflow

The `visualization` workflow takes a geospatial data file as input and produces
3D web-tiles of the data for visualization in a web-map.
The `visualization` workflow takes geospatial data files as input and produces
cloud optimized tiled outputs for analysis, visualization and archival. This
workflow is designed for large geospatial datasets that require specialized
processing to be displayed efficiently in web-based map applications.

See {class}`ogdc_runner.models.recipe_config.VizWorkflow` for details on
configuration options.

```{warning}
This section of the documentation is incomplete!
**TODO**: more detail / link to viz workflow documentation.
```
#### Overview

The visualization workflow orchestrates a multi-stage pipeline that transforms
geospatial data through several processing steps:

1. **Staging**: Large vector files are sliced into smaller, tiled pieces that
correspond to tiles in a defined Tile Matrix Set (TMS). This step also
handles data standardization, deduplication, and property management.

2. **Rasterization**: Vector tiles are converted to raster formats (GeoTIFFs and
PNGs), with statistics calculated as specified in the configuration.

3. **3D Tile Creation**: Vector data is converted to Cesium 3D tiles format,
enabling efficient web-based visualization of large polygon datasets.

The workflow produces four output formats:

- **GeoPackages** (vector): High-resolution, lossless archival format
- **GeoTIFFs** (raster): Multi-resolution raster data with calculated statistics
- **PNG Web Tiles**: Pre-rendered raster tiles with palettes for quick
visualization
- **Cesium 3D Tiles**: Vector tiles for interactive 3D visualization with
attribute data pop-ups

#### Configuration

In addition to `meta.yaml`, `visualization` workflows require a `config.json`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In addition to `meta.yaml`, `visualization` workflows require a `config.json`
In addition to `meta.yaml`, `visualization` workflows can include a `config.json`

config.json is not required - if not specified, the default is {}.

file that defines:

- Input data source and format
- Tile Matrix Set (TMS) for tiling strategy
- Statistical calculations to perform
- Color palettes for visualization
- Output specifications
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any information on what fields are expected/allowed here? E.g., if I want to change the color palettes, how would I go about doing that? Linking to the appropriate PDG docs here would be great!


The following configuration options are specific to the `visualization` workflow
in `meta.yaml`:

##### `config_file`

The path to the JSON configuration file (default: `"config.json"`).

Example: `"my_custom_config.json"`

##### `batch_size`

The number of tiles to process in parallel (default: `250`). Increasing this
value can improve performance on high-performance computing systems with
sufficient resources.

Example: `500`

#### Core Packages

The visualization workflow is powered by several specialized Python packages:

- **viz-workflow**: The main orchestrator that coordinates configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's link to each of the packages (e.g., https://github.com/PermafrostDiscoveryGateway/viz-workflow/) to make it clear that these are external dependencies of the ogdc.

management and workflow processing.

- **viz-staging**: Prepares vector data by slicing large files into TMS-aligned
tiles, re-projecting data, handling deduplication, and managing file paths.

- **viz-raster**: Converts vector tiles to raster formats (GeoTIFFs for archival
and PNGs for web display), with configurable statistics calculation.

- **viz-3dtiles**: Wraps the py3dtiles library to create Cesium 3D tilesets,
building hierarchical JSON structures and reading shapefiles.

#### Input Requirements

The visualization workflow accepts vector geospatial files as input:

- Shapefiles (.shp)
- GeoPackages (.gpkg)
- GeoJSON (.geojson)

Raster input support (GeoTIFF) is planned for future releases.

#### Use Cases
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!


The visualization workflow is best suited for:

- Large geospatial vector datasets requiring multi-resolution tiling
- Data that needs both web visualization and archival formats
- Datasets with complex attribute information to be explored interactively
- Applications requiring 3D visualization of polygon or point cloud data

For an example of a recipe using the `visualization` workflow, we recommend
taking a look at the
[ogdc-recipes viz-workflow recipe](https://github.com/QGreenland-Net/ogdc-recipes/tree/main/recipes/viz-workflow)
example.

Additional detailed documentation and examples are available in the
[Permafrost Discovery Gateway viz-info repository](https://github.com/PermafrostDiscoveryGateway/viz-info).