v0.4.0 (Pagination Automation)

0.4.0 - 2021-07-24

Feature

• Added automatic pagination for all document types defined, or all that are returned.
  • this can also be configured with a pageSize per query if you wish to control that.

Thanks:

Thank you @amala-james-cko for making an initial PR and also noticing as well as @erikdroh

Using the feature

The feature is enabled by default, so when requesting a document type set it will automatically page through each set of results for the doc and the move onto the next

The option to set the pageSize per each request is available if there are issues with requesting only 20 pages at a time.

{
  sitemap: {
    linkResolver = Function,
    apiEndpoint = String,
    accessToken = String|null,
    hostname = String,
    fileName = String,
    optionsMapPerDocumentType = Object|Object<Function>,
    defaultEntryOption = Object,
    staticPaths = Array<Object>,
+    pagination = {
+      pageSize: Number,
+    },
    documentTypes = Array,
    sitemapConfig = Object
  }
}

Caveats

The feature may cause an issue I suspect if there are lots of pages, as it currently places it first into an in memory object then into the sitemap stream. So a future release will likely change that flow. But it won't be breaking. :)