WMTS Tile Generator

This project contains a minimal python implementation that can (pre)generate WMTS (Web Map Tile Service) tiles from an existing dynamic WMS (Web Map Service) layer (in this case real-time radar data). These tiles can then be served from a static directory (or object store, like s3) to allow for much better scalability than directly using the WMS server.

Why WMTS?

One of the main drawbacks of WMS is that every image you request needs to be generated on the fly. This requires a fairly complex server to be directly queryable by users. For a public website, the number of these users can be very large and unpredictable. This is a fairly common web development issue, which can be solved by means of horizontal or vertical scaling, but this costs money and time to get right. The WMS server that this solution was developed for, adaguc-server, is not a lightweight server, so finding a cost-effective deployment strategy that can handle unpredictable public traffic is a challenge.

WMTS, like WMS, is one of many ogc standards for displaying images on a map. The main advantage of WMTS is that it works using predefined image tiles and that these tiles are served via web urls without any web request parameters (other than path parameters). This means that a static directory served through any means (e.g. nginx or s3), can serve as a WMTS server.

Specifically, the option to serve these tiles over s3 (or any other similar object store) makes WMTS an exceptional solution to this challenge. Since s3 buckets can be used to serve a static website, an s3 bucket can serve as a WMTS "server". If you combine this with CloudFront for edge location caching (NOTE: Make sure you don't cache the file WMTSCapabilities.xml), you can easilly create a WMTS server that can serve "infinite" users at a constant low cost.

MapProxy

This solution uses MapProxy to convert WMS to WMTS tiles. MapProxy is designed to be used as a proxy, very similar to what this solution does. However, if you use MapProxy as intended, the users are the ones that decide when MapProxy will start generating the cached tiles, which can still lead to spiked load to your WMS server. MapProxy has its own solutions for this (like seeding), but that never fully detaches the user from the backend WMS server. Furthermore this MapProxy server is exposed to the public internet, so it still has the same challenges of scaling (and also security) discussed earlier.

This solution bypasses those issues by using MapProxy as a service to generate the tiles, but not to serve the tiles.

The time dimension

A lot of data that you want to display on a map using WMS has a time dimension. The example shown in this project is real-time radar data. Especially for real-time data, the values in this time dimension that are supported by the WMS layer are constantly changing. This means that the WMTS capabilities XML needs to change as well. This is currently not supported by MapProxy if you run it as intended. This project also solves this issue by constantly querying the WMS getCapabilities endpoint, generating the tiles for the latest timestamps and updating the WMTS capabilities XML. This way it makes the static WMTS protocol slightly more dynamic. This minimal implementation should be easily extended to allow for other dimensions to be handled the same way.

Run locally with Docker

To try this out for yourself, you can simply run the command below.

docker compose up --build

This will start the WMTS Tile Generator service as well as a minimal nginx webserver that exposes the WMTS "server" at localhost:8080. The capabilities xml of this server can be requested at http://localhost:8080/wmts/1.0.0/WMTSCapabilities.xml. The generated tiles are placed in src/wmts_tile_generator/wmts as set in docker-compose.yml. By default, the configured WMS layer for which the tiles are generated is https://geoservices.knmi.nl/adaguc-server?dataset=RADAR& as set in config.py.

Configuration

By default, this service is configured to create a local directory which can be served (e.g. using nginx). It also includes an s3 based operating mode, where it uploads all generated tiles to an s3 bucket and then deletes them locally. This operating mode can be set by setting WMTS_TILE_GENERATOR_SERVER_MODE to s3 and specifying the bucket using WMTS_TILE_GENERATOR_WEBSITE_BUCKET_NAME.

The rest of the supported configuration options can be found in config.py. For configuration options not found here (like tile size, number of zoom levels and dimensions other than time), you would need to edit the code (specifically mapproxy_template.yaml and/or tile_generator.py).

Development

To develop on the python code, you first need to install uv. Then you can run the following command to install all dependencies and the project itself.

uv sync

For vscode users a launch.json is provided for easy debugging.

Linting and formatting

This project was developed using some common linting and formatting tools. To run all of them, run the following lines.

uv run mypy src
uv run pylint src
uv run ruff format src