Conversation
Co-authored-by: George Jenkins <gvjenkins@gmail.com> Signed-off-by: Scott Rigby <scott@r6by.com>
- Resolve open issues from implementation experience - Document content cache at $HELM_CACHE_HOME/content/ - Add archive-based plugin loading details - Add airgap support considerations (Open Issue helm#5) - Clean up rejected ideas section Signed-off-by: Scott Rigby <scott@r6by.com>
- Fix storage locations table (remove incorrect install cache row) - Add SDK considerations section for controller/platform use cases - Add note about gotemplate plugin handling Template built-in - Clarify helm plugin list --status=all shows separate rows - Note --from-download flag as future enhancement Signed-off-by: Scott Rigby <scott@r6by.com>
…y architecture - Remove "Chart-defined extracted" row from storage locations table - Update explanation: plugins loaded directly from tarball into memory - Update plugin loading precedence: content cache is primary path - Update --from-download example to show extraction (not symlink) - Update implementation checklist to reflect content-addressable cache The Wasm runtime (extism/v1) supports loading plugin binaries directly from bytes via CreatePluginFromData, eliminating the need for disk extraction to a versioned directory. Signed-off-by: Scott Rigby <scott@r6by.com>
- Update Plugin Storage section to clarify no fallback between installed plugins and chart-defined plugins - Add Plugin identity section explaining content hash identification - Add rejected idea helm#6: fallback to installed plugins Chart-defined plugins are identified by their SHA256 digest from Chart.lock, ensuring deterministic builds. An installed plugin with matching name/version cannot substitute because there's no way to verify the content matches the locked digest. Signed-off-by: Scott Rigby <scott@r6by.com>
Update the SDK API Requirements section to reflect the reference implementation PluginRenderer struct: - ContentCachePath (string path) instead of ContentCache interface - PreloadedPlugins as raw bytes, not pre-compiled modules - Add explicit requirements for non-writable filesystems Signed-off-by: Scott Rigby <scott@r6by.com>
- Fix security section: plugins downloaded, not pre-packaged in charts - Replace Pkl with YamlScript as primary render example (Pkl blocked on Java/Wasm support, but kept in file targeting examples) - Move SDK API from open issues to specification (already designed) - Move Airgap Support from open issues to specification (decided on registries.conf) - Add SourceFiles to built-in objects, link Template to open issue - Clarify implementation plan: ref impl complete, upstream pending - Add rejected idea helm#7: pre-packaging plugins within charts - Update reference link to YamlScript Helm discussion Signed-off-by: Scott Rigby <scott@r6by.com>
Signed-off-by: Scott Rigby <scott@r6by.com>
Signed-off-by: Scott Rigby <scott@r6by.com>
Proposes a new chart-defined plugin type `values/v1` that enables plugins to transform chart values before they are coalesced and distributed to subchart dependencies. Use cases: - Map simple user values to complex subchart structures - Apply conditional logic (switch/case) to derive values - Compute values from inputs Execution point in values pipeline: 1-3. Load & merge values 4. values/v1 plugins 5. CoalesceValues() 6. ValidateAgainstSchema() 7. Template rendering Addresses long-standing community requests: - helm/helm#8576 / helm/helm#8580 - map.yaml proposal (2020) - helm/helm#6699 - Passing parent values to subcharts (2020) - helm/helm#12323 - Templated values for subchart dependencies (2023) - helm/helm#13493 - Value CEL mapping (2024) Depends on: Chart-defined plugins HIP (helm#433) Signed-off-by: Scott Rigby <scott@r6by.com>
Proposes a new chart-defined plugin type `values/v1` that enables plugins to transform chart values before they are coalesced and distributed to subchart dependencies. Use cases: - Map simple user values to complex subchart structures - Apply conditional logic (switch/case) to derive values - Compute values from inputs Execution point in values pipeline: 1-3. Load & merge values 4. values/v1 plugins 5. CoalesceValues() 6. ValidateAgainstSchema() 7. Template rendering Addresses long-standing community requests: - helm/helm#8576 / helm/helm#8580 - map.yaml proposal (2020) - helm/helm#6699 - Passing parent values to subcharts (2020) - helm/helm#12323 - Templated values for subchart dependencies (2023) - helm/helm#13493 - Value CEL mapping (2024) Depends on: Chart-defined plugins HIP (helm#433) Signed-off-by: Scott Rigby <scott@r6by.com>
|
|
||
| - Progressive migration from gotemplate to alternative renderers | ||
| - Using gotemplate for some templates while other plugins handle specialized files | ||
| - Plugin composition where one plugin generates files and another processes them |
There was a problem hiding this comment.
I could envision using multiple renderers as stages. like this:
- Deploy manifests for keycloak and postgresql
- Run code that confirms connectivity and health and injects users into keycloak
- Continue deploying manifests of components that depend on keycloak and postgresql
I'm not sure if that would or should be an intended use case, but it could be nice to have that flexibility.
There was a problem hiding this comment.
What you're describing is covered in #373, in general:
- sequencing
- custom wait conditions
This HIP would not help that. Unless there is a future plugin type that individual charts could require - in which case, yes this HIP would enable that.
|
|
||
| #### File Targeting Strategy | ||
|
|
||
| Each `render/v1` plugin declares which files it manages via **glob patterns** in its `plugin.yaml` configuration. At chart load time, Helm matches files to plugins using a "most specific pattern wins" approach: |
There was a problem hiding this comment.
I wonder how this globbing would affect subcharts.
There was a problem hiding this comment.
in this proposal for render/v1 each chart, including subcharts, are sandboxed with their own renderer individually
… plugin types Signed-off-by: Scott Rigby <scott@r6by.com>
Add detailed documentation for how Helm handles unknown plugin types: - Helm validates plugin types at chart load time - Unknown types fail with clear error listing supported types - Error message includes documentation URL (helm.sh/docs/plugins/types) - No action required from chart authors Add links to minimumHelmVersion HIP PR helm#370. Add Rejected Idea helm#8: Requiring chart authors to manually set minimumHelmVersion for plugin types. Auto-detection is preferred. Signed-off-by: Scott Rigby <scott@r6by.com>
gjenkins8
left a comment
There was a problem hiding this comment.
(comments from partial review)
|
|
||
| ## Specification | ||
|
|
||
| Support for Chart-defined plugins will be added starting in Chart API v3. `Chart.yaml` will define a top-level `plugins` key, and `Chart.lock` will capture specific plugin versions and checksums. |
There was a problem hiding this comment.
| Support for Chart-defined plugins will be added starting in Chart API v3. `Chart.yaml` will define a top-level `plugins` key, and `Chart.lock` will capture specific plugin versions and checksums. | |
| Support for Chart-defined plugins will be added starting in [Chart API v3](https://github.com/helm/community/blob/main/hips/hip-0020.md). `Chart.yaml` will define a top-level `plugins` key, and `Chart.lock` will capture specific plugin versions and checksums. |
| apiVersion: v3 | ||
| name: my-chart | ||
| version: 1.0.0 | ||
|
|
||
| # Chart-defined plugins (sequential list order matters for render plugins) | ||
| plugins: | ||
| - name: yamlscript | ||
| type: render/v1 | ||
| repository: oci://ghcr.io/yamlscript/helm-yamlscript | ||
| version: 0.1.0 | ||
| - name: s3 | ||
| type: getter/v1 | ||
| repository: oci://ghcr.io/hypnoglow/helm-s3 | ||
| version: 0.16.3 | ||
|
|
||
| # Renamed from "dependencies" in v2 | ||
| subcharts: | ||
| - name: postgresql | ||
| version: 12.x.x | ||
| repository: https://charts.bitnami.com/bitnami |
There was a problem hiding this comment.
I think remove the sections not relevant to plugins:
| apiVersion: v3 | |
| name: my-chart | |
| version: 1.0.0 | |
| # Chart-defined plugins (sequential list order matters for render plugins) | |
| plugins: | |
| - name: yamlscript | |
| type: render/v1 | |
| repository: oci://ghcr.io/yamlscript/helm-yamlscript | |
| version: 0.1.0 | |
| - name: s3 | |
| type: getter/v1 | |
| repository: oci://ghcr.io/hypnoglow/helm-s3 | |
| version: 0.16.3 | |
| # Renamed from "dependencies" in v2 | |
| subcharts: | |
| - name: postgresql | |
| version: 12.x.x | |
| repository: https://charts.bitnami.com/bitnami | |
| apiVersion: v3 | |
| # Chart-defined plugins (sequential list order matters for render plugins) | |
| plugins: | |
| - name: yamlscript | |
| type: render/v1 | |
| repository: oci://ghcr.io/yamlscript/helm-yamlscript | |
| version: 0.1.0 | |
| - name: s3 | |
| type: getter/v1 | |
| repository: oci://ghcr.io/hypnoglow/helm-s3 | |
| version: 0.16.3 | |
| # Chart-defined plugins (sequential list order matters for render plugins) | ||
| plugins: | ||
| - name: yamlscript | ||
| type: render/v1 | ||
| repository: oci://ghcr.io/yamlscript/helm-yamlscript | ||
| version: 0.1.0 | ||
| - name: s3 | ||
| type: getter/v1 | ||
| repository: oci://ghcr.io/hypnoglow/helm-s3 | ||
| version: 0.16.3 |
There was a problem hiding this comment.
Do we want to group plugin types as a list?
| # Chart-defined plugins (sequential list order matters for render plugins) | |
| plugins: | |
| - name: yamlscript | |
| type: render/v1 | |
| repository: oci://ghcr.io/yamlscript/helm-yamlscript | |
| version: 0.1.0 | |
| - name: s3 | |
| type: getter/v1 | |
| repository: oci://ghcr.io/hypnoglow/helm-s3 | |
| version: 0.16.3 | |
| # Chart-defined plugins (sequential list order matters for render plugins) | |
| plugins: | |
| render/v1: | |
| - name: yamlscript | |
| repository: oci://ghcr.io/yamlscript/helm-yamlscript | |
| version: 0.1.0 | |
| getter/v1: | |
| - name: s3 | |
| repository: oci://ghcr.io/hypnoglow/helm-s3 | |
| version: 0.16.3 |
This is because the ordering of plugins between different types is not relevant.
|
|
||
| The `render/v1` plugin type enables chart authors to use alternative rendering engines beyond gotemplate. | ||
|
|
||
| > **Terminology note**: The plugin type is named `render/v1` (not `template/v1`) because not all rendering engines use templating. For example, Cue-lang uses code evaluation rather than template substitution. |
There was a problem hiding this comment.
I think we can describe this better. Something like?
| > **Terminology note**: The plugin type is named `render/v1` (not `template/v1`) because not all rendering engines use templating. For example, Cue-lang uses code evaluation rather than template substitution. | |
| > **Terminology note**: The plugin type is named `render/v1` (not ie. `template/v1`) to describe the process of generating Kubernetes object manifests from the Chart's source, which more generally is done by code evaluation. For example, Cue-lang uses code evaluation rather than template substitution. |
I'm not sure where the "render" terminology comes from? Perhaps "evaluation" would be better?
|
|
||
| #### Multiple Render Plugins per Chart | ||
|
|
||
| A chart can specify multiple render plugins simultaneously. This enables: |
There was a problem hiding this comment.
| A chart can specify multiple render plugins simultaneously. This enables: | |
| A chart can specify multiple render plugins sequentually. This enables: |
| - Versioned and locked via `Chart.lock` (includes SHA256 digest) | ||
| - Cached as tarballs in content-addressable storage at `$HELM_CACHE_HOME/content/` | ||
| - Loaded directly from tarballs at runtime (no persistent extraction) | ||
| - Shared across multiple charts using the same plugin version |
There was a problem hiding this comment.
| - Shared across multiple charts using the same plugin version | |
| - Shared across multiple charts using the same plugin content hash |
| - Shared across multiple charts using the same plugin version | ||
| - Downloadable ahead of time for airgap scenarios via `helm dependency build` | ||
|
|
||
| **Plugin caching:** Plugin tarballs are stored in Helm's content-addressable cache (`$HELM_CACHE_HOME/content/`) keyed by SHA256 digest, matching how charts are cached. At runtime, plugins are loaded directly from the cached tarball - the `plugin.yaml` and `.wasm` files are read into memory without extracting to disk. |
There was a problem hiding this comment.
| **Plugin caching:** Plugin tarballs are stored in Helm's content-addressable cache (`$HELM_CACHE_HOME/content/`) keyed by SHA256 digest, matching how charts are cached. At runtime, plugins are loaded directly from the cached tarball - the `plugin.yaml` and `.wasm` files are read into memory without extracting to disk. | |
| **Plugin caching:** Plugin tarballs are stored in Helm's content-addressable cache (`$HELM_CACHE_HOME/content/`) keyed by OCI manifest digest, matching how charts are cached. At runtime, plugins are loaded directly from the cached content |
|
|
||
| **Plugin caching:** Plugin tarballs are stored in Helm's content-addressable cache (`$HELM_CACHE_HOME/content/`) keyed by SHA256 digest, matching how charts are cached. At runtime, plugins are loaded directly from the cached tarball - the `plugin.yaml` and `.wasm` files are read into memory without extracting to disk. | ||
|
|
||
| **Plugin versioning:** The content cache supports multiple versions of the same plugin, allowing different charts to depend on different plugin versions without conflicts. |
There was a problem hiding this comment.
Is this trying to say, that as plugin versions are resolved to OCI manifest digests, multiple plugin versions can be supported (as they will have the same manifest digest)?
Seems a little redundant if so?
|
|
||
| **Plugin versioning:** The content cache supports multiple versions of the same plugin, allowing different charts to depend on different plugin versions without conflicts. | ||
|
|
||
| ### Plugin Storage: Installed vs Cached |
There was a problem hiding this comment.
I think the section can simply be removed and replaced by a single sentence: "Chart defined plugins do not interact with CLI plugins"
| Chart-defined plugins are identified by their **content hash** (SHA256 digest), not by name/version. This ensures deterministic builds: | ||
|
|
||
| - `Chart.yaml` specifies name/version for human convenience | ||
| - `helm dependency update` resolves to OCI digest, computes content hash, stores in `Chart.lock` | ||
| - At runtime, Helm loads the plugin by content hash from `Chart.lock` | ||
| - If the hash is not in the cache, the plugin is downloaded |
There was a problem hiding this comment.
This should be clarified / worded differently I think.
(note for myself, I will propose some updates)
This HIP extends HIP-0026 (Wasm plugin system) to enable chart authors to define plugins in
Chart.yamlthat are required for their charts to function.