From 49e911fcef6d2013653db9acd9778b9963f7c680 Mon Sep 17 00:00:00 2001 From: koba1t Date: Thu, 31 Mar 2022 02:35:58 +0900 Subject: [PATCH 01/12] add mini-Kep 'Replacements and Patch value in the structured data' --- .../22-03-value-in-the-structured-data.md | 256 ++++++++++++++++++ 1 file changed, 256 insertions(+) create mode 100644 proposals/22-03-value-in-the-structured-data.md diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md new file mode 100644 index 0000000000..18731501ad --- /dev/null +++ b/proposals/22-03-value-in-the-structured-data.md @@ -0,0 +1,256 @@ + + +# Replacements and Patch value in the structured data. + +**Authors**: +- koba1t + +**Reviewers**: +- natasha41575 +- knverey + +**Status**: implementable + + +## Summary + +This proposal decides the interfaces to change values in the structured data (like json,yaml) inside a Kubernetes objects' field value and implements changing function target a few formats (mainly json). + + +## Motivation + + + +Fields in Kubernetes objects sometimes include values formatted by structured data like json and yaml. +kustomize can strong patch with Kubernetes objects, but kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's yaml field. + +Example, kustomize can't change value `"REPLACE_TARGET_HOSTNAME"` in this yaml file. +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: target-configmap +data: + config.json: |- + {"config": { + "id": "42", + "hostname": "REPLACE_TARGET_HOSTNAME" + }} +``` + +This function has been wanted for a long time. +- https://github.com/kubernetes-sigs/kustomize/issues/680 +- https://github.com/kubernetes-sigs/kustomize/issues/3787 +- https://github.com/kubernetes-sigs/kustomize/issues/4517 + +This function can be an alternative deprecated [vars](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/vars/) function in many use cases. + +**Goals:** + + +1. Provide the way to update values in the structured data like kubernetes objects with many formats. +1. Be able to add replacement able format after. + + +**Non-goals:** + + +1. Do not provide to perfectly alternative for deprecated [vars](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/vars/). + +## Proposal + + + +I propose to add options for replacing the value in structured data to replacements function. My sample implementation is [here](https://github.com/kubernetes-sigs/kustomize/pull/4518). +This idea is add two parameter `format` and `formatPath` to [options](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L67-L80) in replacement [TargetSelector](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L52-L64). The `format` option is used by select to structured data format like "json" or yaml, and The `formatPath` option is "path" to target to change values in structured data with selected format from `format` option. + + +Example. + +```yaml +## source +apiVersion: v1 +kind: ConfigMap +metadata: + name: source-configmap +data: + HOSTNAME: www.example.com +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: target-configmap +data: + config.json: |- + {"config": { + "id": "42", + "hostname": "REPLACE_TARGET_HOSTNAME" + }} +``` + +```yaml +## replacement +replacements: +- source: + kind: ConfigMap + name: source-configmap + fieldPath: data.HOSTNAME + targets: + - select: + kind: ConfigMap + name: target-configmap + fieldPaths: + - data.config\.json + options: + format: 'json' + formatPath: '/config/hostname' +``` + +```yaml +## expected +apiVersion: v1 +kind: ConfigMap +metadata: + name: source-configmap +data: + HOSTNAME: www.example.com +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: target-configmap +data: + config.json: '{"config":{"hostname":"www.example.com","id":"42"}}' +``` + + +### User Stories + + +#### Story 1 + +Scenario summary: As a [end user, extension developer, ...], I want to [...] + + +#### Story 2 + +Scenario summary: As a [end user, extension developer, ...], I want to [...] + + +### Risks and Mitigations + + +### Dependencies + + +### Scalability + + +## Drawbacks + + +## Alternatives + + +## Rollout Plan + + +### Alpha + + +- Will the feature be gated by an "alpha" flag? Which one? +- Will the feature be available in `kubectl kustomize` during alpha? Why or why not? + +### Beta + + +### GA + From f8667010883b420d2caaa60da3d3a655fac063e3 Mon Sep 17 00:00:00 2001 From: koba1t Date: Mon, 4 Apr 2022 04:21:24 +0900 Subject: [PATCH 02/12] improve 'Replacement the value in structured data' --- .../22-03-value-in-the-structured-data.md | 98 ++++++++++++++++--- 1 file changed, 82 insertions(+), 16 deletions(-) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 18731501ad..443c6ad30c 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -45,10 +45,11 @@ This proposal decides the interfaces to change values in the structured data (li If this proposal is an expansion of an existing GitHub issue, link to it here. --> -Fields in Kubernetes objects sometimes include values formatted by structured data like json and yaml. -kustomize can strong patch with Kubernetes objects, but kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's yaml field. +Fields in Kubernetes objects sometimes include values formatted by structured data like json and yaml substrings in a string literal. +kustomize can strong patch with Kubernetes objects, but kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This is a expected behavior, but kustomize will be very helpful if it can change the value structured data like json and yaml substrings in a string literal. + +For example, kustomize can't change the value `"REPLACE_TARGET_HOSTNAME"` in this yaml file straightforwardly. -Example, kustomize can't change value `"REPLACE_TARGET_HOSTNAME"` in this yaml file. ```yaml apiVersion: v1 kind: ConfigMap @@ -85,7 +86,7 @@ What is out of scope for this proposal? Listing non-goals helps to focus discuss and make progress. --> -1. Do not provide to perfectly alternative for deprecated [vars](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/vars/). +1. Do not provide [Unstructured edits, because kustomize eschews parameterization](https://kubectl.docs.kubernetes.io/faq/kustomize/eschewedfeatures/#unstructured-edits). ## Proposal @@ -103,11 +104,85 @@ inline here, if you feel such implementation details are required to adequately If you have a PR, link to it at the top of this section. --> +### Replacement the value in structured data + I propose to add options for replacing the value in structured data to replacements function. My sample implementation is [here](https://github.com/kubernetes-sigs/kustomize/pull/4518). This idea is add two parameter `format` and `formatPath` to [options](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L67-L80) in replacement [TargetSelector](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L52-L64). The `format` option is used by select to structured data format like "json" or yaml, and The `formatPath` option is "path" to target to change values in structured data with selected format from `format` option. +I think these two parameters can't select a specific default value. Therefore kustomize return error message for the user if only one parameter was set. + +#### Example. + +```yaml +## replacement +replacements: +- source: + kind: ConfigMap + name: source-configmap + fieldPath: data.HOSTNAME + targets: + - select: + kind: ConfigMap + name: target-configmap + fieldPaths: + - data.config\.json + options: + format: 'json' + formatPath: '/config/hostname' +``` + + +### Disciplined merge the value in structured data with configMapGenerator + +The proposal is merge two structured data values from substring in a kubernetes object's string literal. + +- https://github.com/kubernetes-sigs/kustomize/issues/680#issuecomment-458834785 + + +## User Stories + + +#### Story 1 + +Scenario summary: Replacement the value inside for structured data(json) in the configMap. + +kustomize patching overlay is very strong to manage common yaml when using many cluster. +But, if you want to set cluster specific change value in the json with configMap data field, you have to replacement whole json file. + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: target-configmap-dev +data: + config.json: |- + {"config": { + "id": "42", + "hostname": "dev.example.com + }} +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: target-configmap-prod +data: + config.json: |- + {"config": { + "id": "42", + "hostname": "prod.example.com" + }} +``` -Example. +So if we can replacement this value in the substring formatted with json, we can easy to overlay this difference. ```yaml ## source @@ -165,16 +240,7 @@ data: config.json: '{"config":{"hostname":"www.example.com","id":"42"}}' ``` - -### User Stories - - -#### Story 1 +#### Story 2 Scenario summary: As a [end user, extension developer, ...], I want to [...] -#### Story 2 +#### Story 3 Scenario summary: As a [end user, extension developer, ...], I want to [...] -Fields in Kubernetes objects sometimes include values formatted by structured data like json and yaml substrings in a string literal. -kustomize can strong patch with Kubernetes objects, but kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This is a expected behavior, but kustomize will be very helpful if it can change the value structured data like json and yaml substrings in a string literal. +kustomize can strong patch to Kubernetes objects (yaml file in most cases) with structured edit. And, It is structured data. +Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. And, kustomize seems it only string literal. +So, kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This function is expected behavior, but kustomize will be very helpful if it can change the value of structured data like json and yaml substrings in a string literal. +Eventually, kustomize doesn't support unstructured edits for string literal. The proposal allows editing of an unstructured string literal as an exception if the string literal is a structured string of a well-known format like json. For example, kustomize can't change the value `"REPLACE_TARGET_HOSTNAME"` in this yaml file straightforwardly. From 9a7014cc14d24a1f4decd49b3fedeade376f8299 Mon Sep 17 00:00:00 2001 From: koba1t Date: Sun, 10 Apr 2022 03:36:46 +0900 Subject: [PATCH 04/12] add proposal 'Disciplined merge the value in structured data with configMapGenerator' --- .../22-03-value-in-the-structured-data.md | 114 ++++++++++++++++-- 1 file changed, 101 insertions(+), 13 deletions(-) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 7c255b61b7..91ea3c756b 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -45,10 +45,10 @@ This proposal decides the interfaces to change values in the structured data (li If this proposal is an expansion of an existing GitHub issue, link to it here. --> -kustomize can strong patch to Kubernetes objects (yaml file in most cases) with structured edit. And, It is structured data. -Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. And, kustomize seems it only string literal. -So, kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This function is expected behavior, but kustomize will be very helpful if it can change the value of structured data like json and yaml substrings in a string literal. -Eventually, kustomize doesn't support unstructured edits for string literal. The proposal allows editing of an unstructured string literal as an exception if the string literal is a structured string of a well-known format like json. +kustomize can strong patch to Kubernetes objects (yaml file in most cases) with structured edit. And, It is structured data.\ +Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. And, kustomize seems it only string literal.\ +So, kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This function is expected behavior, but kustomize will be very helpful if it can change the value of structured data like json and yaml substrings in a string literal.\ +Eventually, kustomize doesn't support unstructured edits for string literal. The proposal allows editing of an unstructured string literal as an exception if the string literal is a structured string of a well-known format like json.\ For example, kustomize can't change the value `"REPLACE_TARGET_HOSTNAME"` in this yaml file straightforwardly. @@ -108,9 +108,9 @@ If you have a PR, link to it at the top of this section. ### Replacement the value in structured data -I propose to add options for replacing the value in structured data to replacements function. My sample implementation is [here](https://github.com/kubernetes-sigs/kustomize/pull/4518). -This idea is add two parameter `format` and `formatPath` to [options](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L67-L80) in replacement [TargetSelector](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L52-L64). The `format` option is used by select to structured data format like "json" or yaml, and The `formatPath` option is "path" to target to change values in structured data with selected format from `format` option. -I think these two parameters can't select a specific default value. Therefore kustomize return error message for the user if only one parameter was set. +I propose to add options for replacing the value in structured data to replacements function. My sample implementation is [here](https://github.com/kubernetes-sigs/kustomize/pull/4518).\ +This idea is add two parameter `format` and `formatPath` to [options](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L67-L80) in replacement [TargetSelector](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L52-L64). The `format` option is used by select to structured data format like "json" or "yaml", and The `formatPath` option is "path" to target to change values in structured data with selected format from `format` option.\ +I think these two parameters can't select a specific default value. Therefore kustomize return error message for the user if only one parameter was set.\ #### Example. @@ -128,17 +128,42 @@ replacements: fieldPaths: - data.config\.json options: - format: 'json' - formatPath: '/config/hostname' + format: 'json' # Setting structured data format. + formatPath: '/config/hostname' # Setting replacements path. ``` +Please check [Story 1](#Story-1). ### Disciplined merge the value in structured data with configMapGenerator -The proposal is merge two structured data values from substring in a kubernetes object's string literal. +This Proposal is add option for [configMapGenerator](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/) to allow merge two string literals when the behavior option is setting to merge and string literals value is structured.\ +This idea is add one parameter for `valueStructuredMergeFormat` to `option`. The `valueStructuredMergeFormat` option is used by select to structured data format like "json" or "yaml". And this function needs to work requires setting `behavior: merge`.\ +This merge operation will be implemented for a part of [Overriding Base ConfigMap Values](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/#overriding-base-configmap-values). It will execute to merge two string literal having same [key](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#define-the-key-to-use-when-creating-a-configmap-from-a-file) name when merging two configMap. -- https://github.com/kubernetes-sigs/kustomize/issues/680#issuecomment-458834785 +#### Example +```yaml +configMapGenerator: +- name: demo-settings + behavior: merge # This function requires `behavior: merge`. + option: + valueStructuredMergeFormat: json # Setting structured data format. + literals: + - config.json: |- + { + "config": { + "hostname": "REPLACE_TARGET_HOSTNAME", + "value": { + "foo": "bar" + } + } + } +``` + +Please check [Story 2](#Story-2). + +#### Appendix +- https://github.com/kubernetes-sigs/kustomize/issues/680#issuecomment-458834785 ## User Stories -kustomize patching overlay is very strong to manage common yaml when using many cluster. +kustomize patching overlay is very strong to manage common yaml when using many cluster.\ But, if you want to set cluster specific change value in the json with configMap data field, you have to replacement whole json file. ```yaml @@ -244,13 +269,76 @@ data: #### Story 2 -Scenario summary: As a [end user, extension developer, ...], I want to [...] +Scenario summary: Merge the two values formatted by structured data(json) with configMapGenerator. + +Many application needs to be set with json format file. And, That file is handled with configMap when the application is running on kubernetes.\ +So, If kustomize configMapGenerator can overlay to one line inside a configMap data value, A json format file will be simple and easy to handle. + +#### Source +```yaml +# base/kustomization.yaml +configMapGenerator: +- name: demo + behavior: merge + literals: + - config.json: |- + { + "config": { + "loglevel": debug, + "parameter": { + "foo": "bar" + } + } + } +``` + +```yaml +# overlay/kustomization.yaml +resources: +- ../base +configMapGenerator: +- name: demo + option: + valueMergeFormat: json + literals: + - config.json: |- + { + "config": { + "hostname": "www.example.com", + "parameter": { + "baz": "qux" + } + } + } +``` + +#### Result +```yaml +apiVersion: v1 +data: + config.json: |- + { + "config": { + "loglevel": debug, + "hostname": "www.example.com", + "parameter": { + "foo": "bar", + "baz": "qux" + } + } + } +kind: ConfigMap +metadata: + name: demo-xxxxxxxxxx # name suffix hash +``` + + #### Story 3 Scenario summary: As a [end user, extension developer, ...], I want to [...] From 50583c4b0e689de8f4a64f52e6c73b00e2e8df27 Mon Sep 17 00:00:00 2001 From: koba1t Date: Thu, 14 Apr 2022 23:58:32 +0900 Subject: [PATCH 05/12] add Story of yaml in configmap and json in Annotations --- .../22-03-value-in-the-structured-data.md | 101 +++++++++++++++++- 1 file changed, 100 insertions(+), 1 deletion(-) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 91ea3c756b..1e68da464e 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -341,13 +341,112 @@ metadata: #### Story 3 -Scenario summary: As a [end user, extension developer, ...], I want to [...] +Scenario summary: Replacement the value inside for yaml in the configMap. +A few applications require to use yaml format config file, and some major cloudnative applications are using that.(ex, Prometheus,AlertManager)\ +A value you want to overlay in yaml is usually into a nested yaml structure. So if you can overlay value inside yaml, you won't need to copy a whole yaml file. + +```yaml +## source +apiVersion: v1 +kind: ConfigMap +metadata: + name: environment-config +data: + env: dev +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: prometheus-config +data: + prometheus.yml: |- + global: + external_labels: + prometheus_env: TARGET_ENVIROMENT + scrape_configs: + - job_name: "prometheus" + static_configs: + - targets: ["localhost:9090"] +``` + +```yaml +## replacement +replacements: +- source: + kind: ConfigMap + name: environment-config + fieldPath: data.env + targets: + - select: + kind: ConfigMap + name: prometheus-config + fieldPaths: + - prometheus\.yml + options: + format: 'yaml' + formatPath: '/global/external_labels/prometheus_env' +``` + +```yaml +## expected +apiVersion: v1 +kind: ConfigMap +metadata: + name: environment-config +data: + env: dev +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: prometheus-config +data: + prometheus.yml: |- + global: + external_labels: + prometheus_env: dev + scrape_configs: + - job_name: "prometheus" + static_configs: + - targets: ["localhost:9090"] +``` + +#### Story 4 + +Scenario summary: Replacement the value inside for json in the annotations. + + +A few times, an application on your cluster requires to set json format config for the *Annotations* on kubernetes yaml resources. We need to overlay in this position value. + +##### Example +```yaml +apiVersion: v1 +kind: Service +metadata: + annotations: + cloud-provider/backend-config: '{"ports": {"appA":"referrence_from_appA"}}' +spec: + ports: + - name: appA + port: 1234 + protocol: TCP + targetPort: 8080 +``` + +- https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#unique_backendconfig_per_service_port +- https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/#listen-ports + + ### Risks and Mitigations -kustomize can strong patch to Kubernetes objects (yaml file in most cases) with structured edit. And, It is structured data.\ -Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. And, kustomize seems it only string literal.\ +kustomize can apply structured edits to Kubernetes objects defined in yaml files.\ +Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. From the kustomize perspective, these "structured" multiline strings are just arbitrary unstructured strings.\ So, kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This function is expected behavior, but kustomize will be very helpful if it can change the value of structured data like json and yaml substrings in a string literal.\ -Eventually, kustomize doesn't support unstructured edits for string literal. The proposal allows editing of an unstructured string literal as an exception if the string literal is a structured string of a well-known format like json.\ +While kustomize won't support unstructured edits in general, theis proposal allows editing of an unstructured string literal as an exception if the string literal is a structured string of a well-known format like json.\ For example, kustomize can't change the value `"REPLACE_TARGET_HOSTNAME"` in this yaml file straightforwardly. From 9adb7535fa28e220f99116b1b815bccbd3535693 Mon Sep 17 00:00:00 2001 From: koba1t Date: Fri, 10 Jun 2022 18:51:02 +0900 Subject: [PATCH 07/12] move behavior: merge in base --- proposals/22-03-value-in-the-structured-data.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 5e634ddad1..95d2daea3d 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -48,7 +48,7 @@ If this proposal is an expansion of an existing GitHub issue, link to it here. kustomize can apply structured edits to Kubernetes objects defined in yaml files.\ Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. From the kustomize perspective, these "structured" multiline strings are just arbitrary unstructured strings.\ So, kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This function is expected behavior, but kustomize will be very helpful if it can change the value of structured data like json and yaml substrings in a string literal.\ -While kustomize won't support unstructured edits in general, theis proposal allows editing of an unstructured string literal as an exception if the string literal is a structured string of a well-known format like json.\ +This proposal allows the user to identify strings literals containing JSON/YAML data so that Kustomize can make structured edits to the data they contain. This allows the requested functionality to be added without violating Kustomize's core principle of supporting structured edits exclusively.\ For example, kustomize can't change the value `"REPLACE_TARGET_HOSTNAME"` in this yaml file straightforwardly. @@ -285,7 +285,6 @@ So, If kustomize configMapGenerator can overlay to one line inside a configMap d # base/kustomization.yaml configMapGenerator: - name: demo - behavior: merge literals: - config.json: |- { @@ -304,6 +303,7 @@ resources: - ../base configMapGenerator: - name: demo + behavior: merge option: valueMergeFormat: json literals: From 3cf12635d47c82a12251a6361f75f6f50931bdff Mon Sep 17 00:00:00 2001 From: koba1t Date: Wed, 15 Jun 2022 05:56:38 +0900 Subject: [PATCH 08/12] scope change to support only JSON/YAML and add support secretGenerator --- proposals/22-03-value-in-the-structured-data.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 95d2daea3d..285d0fb772 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -78,8 +78,7 @@ List the specific goals of the proposal. What is it trying to achieve? How will know that this has succeeded? --> -1. Provide the way to update values in the structured data like kubernetes objects with many formats. -1. Be able to add replacement able format after. +1. Provide the way to update values in the structured data like kubernetes objects with JSON/YAML formats. **Non-goals:** @@ -134,9 +133,9 @@ replacements: Please check [Story 1](#Story-1). -### Disciplined merge the value in structured data with configMapGenerator +### Disciplined merge the value in structured data with configMapGenerator and secretGenerator -This Proposal is add option for [configMapGenerator](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/) to allow merge two string literals when the behavior option is setting to merge and string literals value is structured.\ +This Proposal is add option for [configMapGenerator](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/) and [secretGenerator](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/secretgenerator/) to allow merge two string literals when the behavior option is setting to merge and string literals value is structured.\ This idea is add one parameter for `valueStructuredMergeFormat` to `option`. The `valueStructuredMergeFormat` option is used by select to structured data format like "json" or "yaml". And this function needs to work requires setting `behavior: merge`.\ This merge operation will be implemented for a part of [Overriding Base ConfigMap Values](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/#overriding-base-configmap-values). It will execute to merge two string literal having same [key](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#define-the-key-to-use-when-creating-a-configmap-from-a-file) name when merging two configMap. From 504e805da038c23a9af235f6766028b93f4dc600 Mon Sep 17 00:00:00 2001 From: koba1t Date: Sat, 18 Jun 2022 00:42:33 +0900 Subject: [PATCH 09/12] add non-goal --- proposals/22-03-value-in-the-structured-data.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 285d0fb772..00dbc6e9f8 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -89,6 +89,10 @@ and make progress. 1. Do not provide [Unstructured edits, because kustomize eschews parameterization](https://kubectl.docs.kubernetes.io/faq/kustomize/eschewedfeatures/#unstructured-edits). +1. Do not provide a way for targeting/merging values within strings from the [patches](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/patches/) field. + +1. Do not provide a way for customizing field merge strategies for the embedded data. + ## Proposal Many application needs to be set with json format file. And, That file is handled with configMap when the application is running on kubernetes.\ -So, If kustomize configMapGenerator can overlay to one line inside a configMap data value, A json format file will be simple and easy to handle. +So, If kustomize configMapGenerator can merge for json inside a configMap data value, A json format file will be simple and easy to handle. #### Source ```yaml @@ -308,8 +309,9 @@ resources: configMapGenerator: - name: demo behavior: merge - option: - valueMergeFormat: json + mergeValues: + - key: config.json # Key with a target to merge. + format: json # Setting structured data format MUST be YAML/JSON. literals: - config.json: |- { From a34dd1fc3fd9c96f843de11aa49d7fd267688227 Mon Sep 17 00:00:00 2001 From: koba1t Date: Sun, 17 Dec 2023 05:48:56 +0900 Subject: [PATCH 12/12] fix replacements syntax --- .../22-03-value-in-the-structured-data.md | 25 +++++-------------- 1 file changed, 6 insertions(+), 19 deletions(-) diff --git a/proposals/22-03-value-in-the-structured-data.md b/proposals/22-03-value-in-the-structured-data.md index 2f4b126920..eed7fe6ff6 100644 --- a/proposals/22-03-value-in-the-structured-data.md +++ b/proposals/22-03-value-in-the-structured-data.md @@ -111,9 +111,8 @@ If you have a PR, link to it at the top of this section. ### Replacement the value in structured data -I propose to add options for replacing the value in structured data to replacements function. My sample implementation is [here](https://github.com/kubernetes-sigs/kustomize/pull/4518).\ -This idea is add two parameter `format` and `formatPath` to [options](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L67-L80) in replacement [TargetSelector](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L52-L64). The `format` option is used by select to structured data format like "json" or "yaml", and The `formatPath` option is "path" to target to change values in structured data with selected format from `format` option.\ -I think these two parameters can't select a specific default value. Therefore kustomize return error message for the user if only one parameter was set.\ +I propose to extend the `fieldPath` and `fieldPaths` fields to replace the value in structured data with the replacements function.\ +This idea is extending how to select any field in replacement [TargetSelector](https://github.com/kubernetes-sigs/kustomize/blob/8668691ade05bc17b3c6f44bcd4723735033196e/api/types/replacement.go#L52-L64). If the `source.fieldPath` and `targets.fieldPaths` had extra values after a specific string literal in Yaml, Kustomize tries to parse that string as structure data and tries to drill down using that additional values. #### Example. @@ -129,10 +128,7 @@ replacements: kind: ConfigMap name: target-configmap fieldPaths: - - data.config\.json - options: - format: 'json' # Setting structured data format. - formatPath: '/config/hostname' # Setting replacements path. + - data.config\.json.config.hostname # A path after `config\.json` is pointing one place in the structured data. ``` Please check [Story 1](#Story-1). @@ -249,10 +245,7 @@ replacements: kind: ConfigMap name: target-configmap fieldPaths: - - data.config\.json - options: - format: 'json' - formatPath: '/config/hostname' + - data.config\.json.config.hostname ``` ```yaml @@ -393,10 +386,7 @@ replacements: kind: ConfigMap name: prometheus-config fieldPaths: - - data.prometheus\.yml - options: - format: 'yaml' - formatPath: '/global/external_labels/prometheus_env' + - data.prometheus\.yml.global.external_labels?prometheus_env ``` ```yaml @@ -471,10 +461,7 @@ replacements: kind: ConServicefigMap name: appA-svc fieldPaths: - - metadata.annotations.cloud-provider/backend-config - options: - format: 'json' - formatPath: '/ports/appA' + - metadata.annotations.cloud-provider/backend-config.ports.appA ``` ```yaml