Commits on Dec 3, 2023

1. warn the user when using sub-optimal PBFs …

   Tilemaker's documentation suggests that users can get PBFs from BBBike:
   https://github.com/systemed/tilemaker/blob/1da4be97dc4ac7f11daf3f417c7ca0a6a34ae47f/docs/VECTOR_TILES.md?plain=1#L25
   
   BBBike uses osmconvert to generate PBFs. Reading through the osmconvert
   code [1], osmconvert will pack up to 31 MB of data into each block.
   
   PBFs from Geofabrik use Osmium, which defaults to packing 8,000 objects
   into each block, resulting in blocks that are more like ~60 KB in size.
   
   More, smaller blocks are better for Tilemaker:
   
   - fewer blocks means less opportunity to use multiple cores
   - larger blocks means higher baseline memory requirement
   
   I emailed BBBike's maintainer. He wants to one day move to Osmium, and
   is understandably not keen on patching osmconvert in the interim.
   
   He did point out that a user can just run `osmium cat` on a PBF from
   BBBike to rejig its innards.
   
   This PR detects when a PBF is suboptimal, warns the user, and provides
   an explanation of how to fix it.
   
   For one of my BBBike PBFs, this results in processing time dropping
   from 90 seconds to 28 seconds. (`osmium cat` itself only takes 17
   seconds, and in any case, only has to be run a single time.)
   
   [1]: http://m.m.i24.cc/osmconvert.c

   

   cldellow committed Dec 3, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 5e04240
   • Browse repository at this point

   Copy the full SHA

   5e04240 View commit details

   Browse the repository at this point in the history