From c3c795b09b0ee05b96c77fb693a271a705ec1471 Mon Sep 17 00:00:00 2001 From: Tawera Manaena Date: Wed, 27 Nov 2024 15:51:53 +1300 Subject: [PATCH] docs: revise developer guide documentation BM-1081 (#3368) ### Motivation As a Basemaps developer, when I run Basemaps locally using the [developer guide][docs/dev-guide], assets (glyphs and sprites) don't load when enabling the labels feature or viewing the Topographic layer. - This pull request updates the [developer guide][docs/dev-guide] documentation to include details about the additional parameters required to serve vector tile imagery with assets. - This pull request also includes some minor cleanup and re-formatting of the existing [developer guide][docs/dev-guide] documentation. ### Modifications #### Additions 1. Adds a [call-out][ref/call-out] to the [**How to run `basemaps` locally**][docs/run-basemaps-locally] page describing how to build Basemaps in _watch mode_. [[commit]][commit/watch-mode] 2. Adds a new page titled **"Bundle the `basemaps` assets archive"** to the [CLI methods][docs/dev-guide] section. The new page describes how to generate the **basemaps** bundled assets archive using the **basemaps/cli** package. [[commit]][commit/assets-usage-wip-1] 3. Adds details to the [Server methods][docs/dev-guide] section's [**Serve `basemaps` using a bundled config file**][docs/bundled-config-file] page describing the value of the assets archive and why you may need to use it. [[commit]][commit/assets-usage-wip-2] 4. Adds details to the new **"Bundle the `basemaps` assets archive"** page describing how to generate the asset files (glyph files and sprite sheets) required to bundle the assets archive. [[commit]][commit/assets-usage] #### Revisions 1. Removes unused internal links from various documentation pages. [[commit]][commit/internal-links] 2. Updates code example commands to reference `build` directories instead of `bin` directories. [[commit]][commit/build-bin] 3. Corrects the description and usage of the `--output` parameter when running the **basemaps/cli** package. [[commit]][commit/output-desc] 4. Fixes minor spelling errors and formatting inconsistencies for various documentation pages. [[commit]][commit/formatting] ### Verification You can view the updated [developer guide][docs/dev-guide] documentation by following these steps. 1. Switch to this branch (`docs/dev-guide-revison`). 2. Run the following command from the root of the repository: ```bash # serve the Basemaps docs locally docker run --rm \ -v $PWD:/docs \ -p 8000:8000 \ ghcr.io/linz/basemaps-mkdocs:v1 serve -a 0.0.0.0:8000 ``` [commit/watch-mode]: https://github.com/linz/basemaps/pull/3368/commits/3df0686ae7f020f74ea5638504e1f1d8ff1e0484 [commit/assets-usage-wip-1]: https://github.com/linz/basemaps/pull/3368/commits/c118d83cd1a299b0916872d96db7b3b75722d4fe#diff-a3ddf2f482248ee60c9134a7a95c6e6de6023c852c9873b9a6d234671abb6fd1 [commit/assets-usage-wip-2]: https://github.com/linz/basemaps/pull/3368/commits/c118d83cd1a299b0916872d96db7b3b75722d4fe#diff-4e24348b01ae8e99d94927c006a6fc7f52a762f519bd9876ccb3126b83ef542a [commit/assets-usage]: https://github.com/linz/basemaps/pull/3368/commits/03b5a3eb2d7cec93a8943985a99dccf6d504c4ee [commit/internal-links]: https://github.com/linz/basemaps/pull/3368/commits/7f7c9a676b0c0f41a449f8ebe13f65499425c459 [commit/build-bin]: https://github.com/linz/basemaps/pull/3368/commits/ec967f0ff4c8138cab23e667d9848070bfba9ce4 [commit/output-desc]: https://github.com/linz/basemaps/pull/3368/commits/b30435d876e67b76625a94c3e9c64896a07602b9 [commit/formatting]: https://github.com/linz/basemaps/pull/3368/commits/1e769da382baa504e94135921cbbb16cb9a5f4d9 [docs/dev-guide]: https://basemaps.linz.govt.nz/docs/developer-guide/ [docs/run-basemaps-locally]: https://basemaps.linz.govt.nz/docs/developer-guide/run-basemaps-locally/ [docs/bundled-config-file]: https://basemaps.linz.govt.nz/docs/developer-guide/server-methods/serve-basemaps-with-bundled-config-file/ [ref/call-out]: https://squidfunk.github.io/mkdocs-material/reference/admonitions/ --- .../bundle-the-basemaps-assets-archive.md | 158 ++++++++++++++++++ .../bundle-the-basemaps-config-file.md | 24 +-- docs/developer-guide/run-basemaps-locally.md | 10 ++ ...serve-basemaps-with-bundled-config-file.md | 115 +++++++++---- ...semaps-with-collection-of-geotiff-files.md | 17 +- 5 files changed, 265 insertions(+), 59 deletions(-) create mode 100644 docs/developer-guide/cli-methods/bundle-the-basemaps-assets-archive.md diff --git a/docs/developer-guide/cli-methods/bundle-the-basemaps-assets-archive.md b/docs/developer-guide/cli-methods/bundle-the-basemaps-assets-archive.md new file mode 100644 index 000000000..a4b8bacd6 --- /dev/null +++ b/docs/developer-guide/cli-methods/bundle-the-basemaps-assets-archive.md @@ -0,0 +1,158 @@ +# Bundle the `basemaps` assets archive + +This guide explains how to generate the `basemaps` bundled assets archive using the **basemaps/cli** package. + +## Get started + +Clone the [**linz/basemaps-config**][bm_config_repo] repository to your machine. + +=== "HTTPS" + + ```bash + git clone https://github.com/linz/basemaps-config.git + ``` + +=== "SSH" + + ```bash + git clone git@github.com:linz/basemaps-config.git + ``` + +=== "GitHub CLI" + + ```bash + gh repo clone linz/basemaps-config + ``` + +!!! abstract "Paths" + + This guide uses variables as shorthand to reference key directories and files. On your machine, note the following paths: + + === "`BM_CONFIG_REPO`" + + The path to the **basemaps-config** repository folder on your machine. + + ```bash + $BM_CONFIG_REPO = {path_to}/basemaps-config + ``` + + === "`BM_CLI_BUILD`" + + The **basemaps/cli** package provides a **bundle-assets** function we can use to generate a bundled assets archive. + + The path to the **build** folder of the **basemaps/cli** package. + + ```bash + $BM_CLI_BUILD = $BM_REPO/packages/cli/build/cli + ``` + + === "`BM_SPRITES_BIN`" + + The **basemaps/sprites** package provides a default function we can use to generate sprite sheets from a collection of sprite images. + + The path to the **bin** folder of the **basemaps/sprites** package. + + ```bash + $BM_SPRITES_BIN = $BM_REPO/packages/sprites/bin + ``` + +## Generate the `basemaps` assets + +### 1. Create local directory + +To prepare for bundling the `basemaps` assets archive, create a local directory on your machine with the following subdirectory structure: + +```md +- assets + - fonts + - sprites +``` + +You can specify a location and directory name of your choice. + +!!! abstract "Path" + + On your machine, consider the following path: + + === "`ASSETS_DIR`" + + The path to the created directory on your machine. + + ```bash + $ASSETS_DIR = {path_to_directory} + ``` + +### 2. Convert fonts into `.pbf` (protobuf-encoded glyphs) files + +To generate the `basemaps` `.pbf` files, follow the instructions described in this [README.md][fonts_readme] file. Alternatively, there is a [repository][pbf_fonts_repo] containing various fonts that have already been converted into the `.pbf` format. + +Move the generated and/or downloaded collections of `.pbf` files **into** the following directory: + +``` +$ASSETS_DIR/assets/fonts +``` + +The contents of your `$ASSETS_DIR/assets/fonts` directory should look similar to the following: + +```md +- $ASSETS_DIR/assets/fonts + - Noto Sans Bold + - 0-255.pbf + - ... + - Open Sans Bold + - 0-255.pbf + - ... +``` + +### 3. Convert sprite files into sprite sheets + +Use the following command to generate the `basemaps` sprite sheets from the collection of topographic sprite files: + +```bash +node $BM_SPRITES_BIN/basemaps-sprites.mjs \ +$BM_CONFIG_REPO/config/sprites/topographic +``` + +The above command will output the resulting sprite sheets to the location from which you executed the command. Move the outputted files **into** the following directory: + +```md +$ASSETS_DIR/assets/sprites +``` + +The contents of your `$ASSETS_DIR/assets/sprites` directory should look similar to the following: + +```md +- $ASSETS_DIR/assets/sprites + - topographic.json + - topographic.png + - topographic@2x.json + - topographic@2x.png +``` + +## Run the `basemaps/cli` package + +### Command + +Use the following command to bundle the `basemaps` assets archive: + +```bash +node $BM_CLI_BUILD/bin.js bundle-assets \ + --assets $ASSETS_DIR \ + --output assets.bundle.tar.co \ +``` + +### Parameters + +=== "`--assets`" + + Specifies the location of the assets folder to use. This folder refers to that which contains the `.pbf` files and sprite sheets on your local machine. + +=== "`--output`" + + Specifies where to save the bundled assets archive, relative to the location from which you execute the command. You can specify a location and filename of your choice. + + + +[bm_config_repo]: https://github.com/linz/basemaps-config +[fonts_readme]: https://github.com/linz/basemaps-config/tree/master/config/fonts +[pbf_fonts_repo]: https://github.com/korywka/fonts.pbf diff --git a/docs/developer-guide/cli-methods/bundle-the-basemaps-config-file.md b/docs/developer-guide/cli-methods/bundle-the-basemaps-config-file.md index 1c397f8f0..80888839d 100644 --- a/docs/developer-guide/cli-methods/bundle-the-basemaps-config-file.md +++ b/docs/developer-guide/cli-methods/bundle-the-basemaps-config-file.md @@ -32,10 +32,10 @@ Clone the [**linz/basemaps-config**][bm_config_repo] repository to your machine. gh repo clone linz/basemaps-config ``` -!!! abstract "Path" +!!! abstract "Paths" This guide uses variables as shorthand to reference key directories and files. On your machine, note the following paths: - + === "`BM_CONFIG_REPO`" The path to the **basemaps-config** repository folder on your machine. @@ -44,14 +44,14 @@ Clone the [**linz/basemaps-config**][bm_config_repo] repository to your machine. $BM_CONFIG_REPO = {path_to}/basemaps-config ``` - === "`BM_CLI_BIN`" + === "`BM_CLI_BUILD`" The **basemaps/cli** package provides a **bundle** function we can use to generate a bundled config file. - The path to the **bin** folder of the **basemaps/cli** package. + The path to the **build** folder of the **basemaps/cli** package. ```bash - $BM_CLI_BIN = $BM_REPO/packages/cli/bin + $BM_CLI_BUILD = $BM_REPO/packages/cli/build/cli ``` ## Run the `basemaps/cli` package @@ -61,9 +61,9 @@ Clone the [**linz/basemaps-config**][bm_config_repo] repository to your machine. Use the following command to bundle the `basemaps` config file: ```bash -node $BM_CLI_BIN/bmc.js bundle \ +node $BM_CLI_BUILD/bin.js bundle \ --config $BM_CONFIG_REPO/config \ - --output $BM_REPO/config/config.bundle.json \ + --output config.bundle.json \ --cache s3://linz-basemaps-staging/basemaps-config/cache/ ``` @@ -75,18 +75,12 @@ node $BM_CLI_BIN/bmc.js bundle \ === "`--output`" - Specifies where to save the bundled config file. You can specify a location of your choice. This guide specifies the `basemaps` repository for convenience. + Specifies where to save the bundled config file, relative to the location from which you execute the command. You can specify a location and filename of your choice. === "`--cache`" Specifies the location of the cache directory. This parameter is optional but recommended to reduce the time needed to construct the config file. - - -[running-basemaps-locally]: ../running-basemaps-locally.md -[configuring-the-basemapsserver-package]: ../running-basemaps-locally.md#2-configuring-the-basemapsserver-package -[path-to-json-config-file]: ../Server%20Methods/json-config-file.md - -[bm_config_repo]: https://github.com/linz/basemaps-config \ No newline at end of file +[bm_config_repo]: https://github.com/linz/basemaps-config diff --git a/docs/developer-guide/run-basemaps-locally.md b/docs/developer-guide/run-basemaps-locally.md index 42bacfc9d..c4704bc57 100644 --- a/docs/developer-guide/run-basemaps-locally.md +++ b/docs/developer-guide/run-basemaps-locally.md @@ -80,6 +80,15 @@ npm run build npm run test ``` +!!! tip + + You can build the `basemaps` packages in [watch mode][tsc_watch] so that they recompile anytime you modify the source code. In a terminal, navigate to `BM_REPO` and run the following command: + + ```bash + # Generate the /build for each package (watch mode) + npm run build -- --watch + ``` + ## Configure the `basemaps/server` package There are two main ways you can configure and run the **basemaps/server** package: @@ -103,3 +112,4 @@ There are two main ways you can configure and run the **basemaps/server** packag [configuration]: https://github.com/linz/basemaps/blob/master/docs/configuration.md [contributing]: https://github.com/linz/basemaps/blob/master/CONTRIBUTING.md [stac]: https://github.com/radiantearth/stac-spec/blob/master/overview.md +[tsc_watch]: https://www.typescriptlang.org/docs/handbook/project-references.html#tsc--b-commandline diff --git a/docs/developer-guide/server-methods/serve-basemaps-with-bundled-config-file.md b/docs/developer-guide/server-methods/serve-basemaps-with-bundled-config-file.md index 901816e69..234c144e4 100644 --- a/docs/developer-guide/server-methods/serve-basemaps-with-bundled-config-file.md +++ b/docs/developer-guide/server-methods/serve-basemaps-with-bundled-config-file.md @@ -4,63 +4,109 @@ This guide shows you how to configure and run the **basemaps/server** package us ## Configure the `basemaps/server` package -The **basemaps/server** package requires a config file to serve the **basemaps** system. +=== "Config file" -You have two options. The first is to use the pre-generated config file stored in the **LINZ AWS S3** bucket. The second is to generate the config file yourself using the **basemaps/cli** package. + The **basemaps/server** package requires a config file to serve the **basemaps** system. -=== "Using the pre-generated config file" + === "Using the pre-generated config file" - !!! abstract "Path" + !!! abstract "Path" - To use the exisiting config file stored in the **LINZ AWS S3** bucket, note of the following path: + To use the exisiting config file stored in the **LINZ AWS S3** bucket, note the following path: - === "`CONFIG_FILE`" + === "`CONFIG_FILE`" - The absolute path to the latest config file stored in the **LINZ AWS S3** bucket. - - ```bash - $CONFIG_FILE = s3://linz-basemaps/config/config-latest.json.gz - ``` + The absolute path to the latest config file stored in the **LINZ AWS S3** bucket. -=== "Generating the config file yourself" + ```bash + $CONFIG_FILE = s3://linz-basemaps/config/config-latest.json.gz + ``` - To generate the config file yourself, follow the [**Bundle the `basemaps` config file**][bundle-the-basemaps-config-file] guide. Then, return to this section. + === "Generating the config file yourself" - !!! abstract "Path" + To generate the config file yourself, follow the [**Bundle the `basemaps` config file**][bundle-the-basemaps-config-file] guide. Then, return to this section. - Once you have generated the config file, make a note of the file's location: + !!! abstract "Path" - === "`CONFIG_FILE`" + Once you have generated the config file, make a note of the file's location: - The path to the generated config file on your machine. - - ```bash - $CONFIG_FILE = {path_to}/config.bundle.json - ``` + === "`CONFIG_FILE`" + + The path to the generated config file on your machine. + + ```bash + $CONFIG_FILE = {path_to}/config.bundle.json + ``` + +=== "Assets archive" + + The **basemaps/server** package also allows you to specify the location of assets (e.g. fonts and sprites) to use when serving the **basemaps** system. + + !!! tip "Note" + + Specifying this location is not required to serve raster imagery. If you wish to serve vector imagery or view the labels overlay, you will need to specify this location. Otherwise, assets referenced by such layers will not load. + + === "Using the pre-generated assets archive" + + !!! abstract "Path" + + To use the exisiting assets archive stored in the **LINZ AWS S3** bucket, note the following path: + + === "`ASSETS_ARCHIVE`" + + The absolute path to the latest assets archive stored in the **LINZ AWS S3** bucket. + + ```bash + $ASSETS_ARCHIVE = s3://linz-basemaps/assets/assets-latest.tar.co + ``` + + === "Generating the assets archive yourself" + + To generate the assets archive yourself, follow the [**Bundle the `basemaps` assets archive**][bundle-the-basemaps-assets-archive] guide. Then, return to this section. + + !!! abstract "Path" + + Once you have generated the assets archive, make a note of the archive's location: + + === "`ASSETS_ARCHIVE`" + + The path to the generated assets archive on your machine. + + ```bash + $ASSETS_ARCHIVE = {path_to}/assets.bundle.tar.co + ``` ## Run the `basemaps/server` package -At this stage, you should have a path to a config file. Either, to that which you are sourcing from the **LINZ AWS S3** bucket, or, have generated using the **basemaps/cli** package. +At this stage, you should have a path to a config file. You may also have a path to an assets archive. Either of which you are sourcing from the **LINZ AWS S3** bucket, or, have generated using the **basemaps/cli** package. !!! abstract "Path" The **basemaps/server** package provides a default function to serve the **basemaps** system. Take note of the following path: - === "`BM_SERVER_BIN`" + === "`BM_SERVER_BUILD`" + + The path to the **build** folder of the **basemaps/server** package. - The path to the **bin** folder of the **basemaps/server** package. - ```bash - $BM_SERVER_BIN = $BM_REPO/packages/server/bin + $BM_SERVER_BUILD = $BM_REPO/packages/server/build ``` ### Command -Use the following command to run the **basemaps** system: +Use one of the following commands to run the **basemaps** system: + +=== "Without assets" + + ```bash + node $BM_SERVER_BUILD/bin.js --config $CONFIG_FILE + ``` -```bash -node $BM_SERVER_BIN/basemaps-server.cjs --config $CONFIG_FILE -``` +=== "With assets" + + ```bash + node $BM_SERVER_BUILD/bin.js --config $CONFIG_FILE --assets $ASSETS_ARCHIVE + ``` ### Parameters @@ -68,8 +114,11 @@ node $BM_SERVER_BIN/basemaps-server.cjs --config $CONFIG_FILE Specifies the location of the bundled config file to use. +=== "`--assets`" + + Specifies the location of the bundled assets archive to use. + -[running-basemaps-locally]: ../running-basemaps-locally.md -[configuring-the-basemapsserver-package]: ../running-basemaps-locally.md#2-configuring-the-basemapsserver-package -[bundle-the-basemaps-config-file]: ../cli-methods/bundle-the-basemaps-config-file.md \ No newline at end of file +[bundle-the-basemaps-config-file]: ../cli-methods/bundle-the-basemaps-config-file.md +[bundle-the-basemaps-assets-archive]: ../cli-methods/bundle-the-basemaps-assets-archive.md diff --git a/docs/developer-guide/server-methods/serve-basemaps-with-collection-of-geotiff-files.md b/docs/developer-guide/server-methods/serve-basemaps-with-collection-of-geotiff-files.md index 521995c28..64c410493 100644 --- a/docs/developer-guide/server-methods/serve-basemaps-with-collection-of-geotiff-files.md +++ b/docs/developer-guide/server-methods/serve-basemaps-with-collection-of-geotiff-files.md @@ -13,7 +13,7 @@ The **basemaps/server** package requires a collection of GeoTIFF files from whic === "`IMAGERY_DIR`" The path to the root folder of your GeoTIFF file collection. - + ```bash $IMAGERY_DIR = {path_to_imagery_directory} ``` @@ -24,12 +24,12 @@ The **basemaps/server** package requires a collection of GeoTIFF files from whic The **basemaps/server** package provides a default function to serve the **basemaps** system. Note the following path: - === "`BM_SERVER_BIN`" + === "`BM_SERVER_BUILD`" + + The path to the **build** folder of the **basemaps/server** package. - The path to the **bin** folder of the **basemaps/server** package. - ```bash - $BM_SERVER_BIN = $BM_REPO/packages/server/bin + $BM_SERVER_BUILD = $BM_REPO/packages/server/build ``` ### Command @@ -37,10 +37,5 @@ The **basemaps/server** package requires a collection of GeoTIFF files from whic Use the following command to run the **Basemaps** system: ```bash -node $BM_SERVER_BIN/basemaps-server.cjs $IMAGERY_DIR +node $BM_SERVER_BUILD/bin.js $IMAGERY_DIR ``` - - - -[running-basemaps-locally]: ../running-basemaps-locally.md -[configuring-the-basemapsserver-package]: ../running-basemaps-locally.md#2-configuring-the-basemapsserver-package \ No newline at end of file