Configurable Forge manifest.yml

[tbinna]

Forge app build pipelines typically require a way to configure the manifest.yml file before deploying an app to the Forge platform. In particular, Connect-on-Forge apps will have to configure the Connect app key, Connect remote base URL, or app licensing.

Well-suited solutions for this are configuration languages like Dhall or Apple's new Pkl. These languages allow writing configuration files in Dhall or Pkl syntax and then can convert them to YAML output. The main benefit is that the configuration file can now include dynamic code, such as defining properties based on environment variables or conditionally including parts based on environment variable values.

Dhall is more established than Pkl. Both currently have no Javascript bindings. However, Pkl has a project in the works and is offering node packages with the binaries to integrate Pkl relatively easily into Javascript tooling.

For the above reason, the idea is to provide native Pkl support in Forge apps, meaning instead of maintaining a manifest.yml file, the Forge project would contain a manifest.pkl file. This file will be converted to a manifest.yml file just before deployment to allow admins to inject the desired variables and properties to generate a manifest file for the deployment environment.

There are some challenges that come with this:

Project graph
The project graph integration needs to be updated to work with manifest.pkl files. This means the plugin needs to be able to infer Forge app project dependencies (i.e., Custom UI or UI Kit 2 dependencies) based on the pkg file. The easiest way to do this with the currently available tools might be to write pkl code that does the same as the create-dependencies.ts code does.

Deploy executor
The deploy executor would have to be updated to convert pkl files if a Forge project is using a manifest file in pkl format. This means the user would have to inject the required pkl context. After conversion, the executor would have to remove the manifest.pkl file to avoid it being deployed to the Forge platform.

Dynamic project dependencies
Pkl technically would allow users to specify a manifest file with dynamic module definitions, i.e., a Forge module is included dynamically in the manifest. For starters, the integration should probably fail if that's the case because, without additional information, it would not be possible to decide if the dependency should be created.

[tbinna]

Hey @arnthors, I just came across your question again in issue #20 and wanted to ask if you found a solution. I am also curious what you think about the above.

I am still considering other options, in particular, because pkl is still quite new, and there may be limitations.

[tbinna]

I did some more research on this topic. Allowing dynamic manifest.yml files completely breaks the current approach because the plugin code does not understand Dhall, Pkl, or even a simple templating language. The project graph integration and the package executor rely heavily on the manifest.yml to infer app configuration and dependencies and require those parts to be static.

I can think of two alternative approaches.

1. Add a configuration file, similar to an EsLint or prettier configuration, that defines the knowledge required by the plugin (dependent Nx resource projects, as well as which resources are UI Kit 2 and which are Custom UI). The developer must maintain this configuration and ensure it is in sync with the manifest configuration file (in Pkl, Dahll, etc.).
2. Keep the manifest.yml file as a static file and add an optional manifest file transformation step in the deployment executor. One way to achieve this could be to allow the developer to write a transformation in JSONata.

This certainly needs further investigation but I just wanted to post some thoughts here.

[tbinna]

I have chosen the second approach discussed in the comment above. This is the lowest-hanging fruit that does not add too much unnecessary complications. The feature has been merged and released as part of version 4.2.0, including documentation on how to use it.