diff --git a/content/blog/new-provider-resource-option-environment-variable-remapping/index.md b/content/blog/new-provider-resource-option-environment-variable-remapping/index.md new file mode 100644 index 000000000000..f7272432a22d --- /dev/null +++ b/content/blog/new-provider-resource-option-environment-variable-remapping/index.md @@ -0,0 +1,161 @@ +--- +title: "New Provider Resource Option: Environment Variable Remapping" +date: 2026-02-12 +draft: false +meta_desc: "Remap provider environment variables to custom keys using the new resource option, pulumi.EnvVarMappings" +meta_image: meta.png +authors: + - guinevere-saenger +tags: + - features + - packages +schema_type: auto + +# Optional: Social media promotional copy (for reference only, does not auto-post) +social: + twitter: + linkedin: +--- + +Pulumi is excited to introduce environment variable remapping as a provider resource option. +From Pulumi version 3.220.0, you can use the new `envVarMappings` resource option to redirect provider environment variables to custom keys. +This is useful when you need multiple Pulumi providers to use different values for the same environment variable. + + + +When configuring a Pulumi provider to authenticate against a cloud provider, there are two main options available. +You can set authentication values as secrets in your Pulumi.yaml config: + +```bash +$ pulumi config set azure-native:clientSecret --secret +``` + +Alternately, you can also use the terminal environment of where you're running your Pulumi commands: + +```bash +$ export ARM_CLIENT_SECRET=1234567 +``` + +Using environment variables in this manner is especially useful in CI environments, or when you'd rather not write that auth token to state, even encrypted. +But there's currently several use cases where this breaks down, due to the hard-coded nature of the environment variables that a given provider expects. + +## Multiple providers or provider instances that expect different authentication values but have the same variable key + +For example, if you are using multiple explicit providers targeting different Azure accounts, you were not able to set these separate configurations via environment variable. + +Instead, users would have to to set these values in the provider config, which may not be desirable for all use cases. +Not only does the provider config write secrets to state (albeit of course encrypted), but it can also result in a noisy diff on an otherwise no-op upgrade when token rotation is used. + +## Remapping environment variables + +For this and similar scenarios, we have a new solution for you: setting mappings of environment variable keys on your provider. +The concept is as follows: + +"For any environment variable that my Pulumi provider expects, I want to be able to tell the provider to use the value of a custom-defined environment variable instead." + +## Example + +Let's say you want to give your provider a specific value for `ARM_CLIENT_SECRET`, one that is different from the `ARM_CLIENT_SECRET` otherwise in use in your shell. +You would first define a new, custom environment variable as follows: + +```bash +$ export CUSTOM_ARM_CLIENT_SECRET=7654321 +``` + +And then you'd tell your provider to setits `ARM_CLIENT_SECRET` to the value of your special `CUSTOM_ARM_CLIENT_SECRET`. + +{{< chooser language "typescript,python,go,csharp,java,yaml" >}} + +{{% choosable language typescript %}} + +```typescript +const provider = new command.Provider("command-provider", {}, { + envVarMappings: { + // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET + "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET", + }, +}); +``` + +{{% /choosable %}} + +{{% choosable language python %}} + +```python +provider = command.Provider("command-provider", + opts=pulumi.ResourceOptions( + env_var_mappings={ + # If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET + "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET", + } + ) +) +``` + +{{% /choosable %}} + +{{% choosable language go %}} + +```go +provider, err := command.NewProvider( + ctx, + "command-provider", + &command.ProviderArgs{}, + pulumi.EnvVarMappings(map[string]string{ + // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET + "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET", + }), +) +``` + +{{% /choosable %}} + +{{% choosable language csharp %}} + +```csharp +var provider = new Command.Provider("command-provider", new Command.ProviderArgs(), new CustomResourceOptions +{ + EnvVarMappings = new Dictionary + { + // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET + { "CUSTOM_ARM_CLIENT_SECRET", "ARM_CLIENT_SECRET" } + } +}); +``` + +{{% /choosable %}} + +{{% choosable language java %}} + +```java +var provider = new Provider("command-provider", ProviderArgs.Empty, CustomResourceOptions.builder() + .envVarMappings(Map.of( + // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET + "CUSTOM_ARM_CLIENT_SECRET", "ARM_CLIENT_SECRET" + )) + .build()); +``` + +{{% /choosable %}} + +{{% choosable language yaml %}} + +```yaml +resources: + command-provider: + type: pulumi:providers:command + options: + envVarMappings: + # If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET + CUSTOM_ARM_CLIENT_SECRET: ARM_CLIENT_SECRET +``` + +{{% /choosable %}} + +{{< /chooser >}} + +You can now customize each environment variable value your provider sees by defining a new environment variable, and then mapping your provider's defined variable to yours. + +For full details, see the [envVarMappings documentation](/docs/iac/concepts/resources/options/envvarmappings/). + +Happy coding! diff --git a/content/blog/new-provider-resource-option-environment-variable-remapping/meta.png b/content/blog/new-provider-resource-option-environment-variable-remapping/meta.png new file mode 100644 index 000000000000..2021885fbc29 Binary files /dev/null and b/content/blog/new-provider-resource-option-environment-variable-remapping/meta.png differ diff --git a/content/docs/iac/concepts/resources/options/_index.md b/content/docs/iac/concepts/resources/options/_index.md index a9ba602bd535..dcfbc380b2c6 100644 --- a/content/docs/iac/concepts/resources/options/_index.md +++ b/content/docs/iac/concepts/resources/options/_index.md @@ -26,6 +26,7 @@ Resource constructors accept the following resource options: - [deletedWith](/docs/concepts/options/deletedwith/): If set, the provider's Delete method will not be called for this resource if the specified resource is being deleted as well. - [replaceWith](/docs/concepts/options/replacewith/): If set, the resource will be replaced if one of the specified resources is replaced. - [dependsOn](/docs/concepts/options/dependson/): specify additional explicit dependencies in addition to the ones in the dependency graph. +- [envVarMappings](/docs/concepts/options/envvarmappings/): remap environment variables to custom keys for provider authentication. - [hideDiffs](/docs/concepts/options/hidediffs/): compact the display of diffs for specified properties in CLI output without affecting update behavior. - [hooks](/docs/concepts/options/hooks/): specify a set of resource hooks that will be executed at specific points in the resource lifecycle. - [ignoreChanges](/docs/concepts/options/ignorechanges/): declare that changes to certain properties should be ignored during a diff. @@ -52,6 +53,7 @@ Not all resource options apply to [component resources](/docs/iac/concepts/compo | [deleteBeforeReplace](/docs/concepts/options/deletebeforereplace/) | No | Components are not replaced by providers | | [deletedWith](/docs/concepts/options/deletedwith/) | Yes | Controls deletion behavior | | [dependsOn](/docs/concepts/options/dependson/) | Yes | Creates explicit dependencies on the component | +| [envVarMappings](/docs/concepts/options/envvarmappings/) | No | Only applies to provider resources | | [hideDiffs](/docs/concepts/options/hidediffs/) | No | Components don't have provider-generated diffs | | [hooks](/docs/concepts/options/hooks/) | No | Components don't have provider lifecycle events | | [ignoreChanges](/docs/concepts/options/ignorechanges/) | No | Components don't have provider-managed inputs. The component implementation may choose to propagate this to children. | diff --git a/content/docs/iac/concepts/resources/options/envvarmappings.md b/content/docs/iac/concepts/resources/options/envvarmappings.md new file mode 100644 index 000000000000..64769b8b6da0 --- /dev/null +++ b/content/docs/iac/concepts/resources/options/envvarmappings.md @@ -0,0 +1,362 @@ +--- +title_tag: "envVarMappings | Resource Options" +meta_desc: The envVarMappings resource option remaps environment variables to custom keys for provider authentication. +title: "envVarMappings" +h1: "Resource option: envVarMappings" +meta_image: /images/docs/meta-images/docs-meta.png +menu: + iac: + identifier: envVarMappings + parent: options-concepts + weight: 6 +aliases: + - /docs/iac/concepts/options/envvarmappings/ + - /docs/intro/concepts/resources/options/envvarmappings/ + - /docs/concepts/options/envvarmappings/ +--- + +The `envVarMappings` resource option allows you to remap environment variables that a provider expects to custom environment variable names. +This is useful when you need to run multiple providers or provider instances that require different values for the same environment variable. + +## When to use envVarMappings + +Use this option when: + +- **Running multiple providers targeting different accounts or regions**: For example, two AWS providers targeting different accounts can each use their own environnment variable-based credentials without conflicting. + +{{% notes type="info" %}} +The `envVarMappings` resource option only applies to provider resources. +It cannot be used on regular resources or component resources. +You must define an explicit provider to use this resource option. +{{% /notes %}} + +## Example + +The following example shows how to remap `CUSTOM_ARM_CLIENT_SECRET` to `ARM_CLIENT_SECRET` so the provider reads from your custom environment variable: + +{{< chooser language "typescript,python,go,csharp,java,yaml" >}} + +{{% choosable language typescript %}} + +```typescript +import * as azure from "@pulumi/azure-native"; + +const provider = new azure.Provider("azure-provider", {}, { + envVarMappings: { + "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET", + }, +}); +``` + +{{% /choosable %}} + +{{% choosable language python %}} + +```python +import pulumi +import pulumi_azure_native as azure + +provider = azure.Provider("azure-provider", + opts=pulumi.ResourceOptions( + env_var_mappings={ + "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET", + } + ) +) +``` + +{{% /choosable %}} + +{{% choosable language go %}} + +```go +provider, err := azure.NewProvider(ctx, "azure-provider", &azure.ProviderArgs{}, + pulumi.EnvVarMappings(map[string]string{ + "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET", + }), +) +``` + +{{% /choosable %}} + +{{% choosable language csharp %}} + +```csharp +using Pulumi; +using Pulumi.AzureNative; + +var provider = new Provider("azure-provider", new ProviderArgs(), new CustomResourceOptions +{ + EnvVarMappings = new Dictionary + { + { "CUSTOM_ARM_CLIENT_SECRET", "ARM_CLIENT_SECRET" } + } +}); +``` + +{{% /choosable %}} + +{{% choosable language java %}} + +```java +import com.pulumi.azurenative.Provider; +import com.pulumi.azurenative.ProviderArgs; +import com.pulumi.resources.CustomResourceOptions; +import java.util.Map; + +var provider = new Provider("azure-provider", ProviderArgs.Empty, + CustomResourceOptions.builder() + .envVarMappings(Map.of( + "CUSTOM_ARM_CLIENT_SECRET", "ARM_CLIENT_SECRET" + )) + .build()); +``` + +{{% /choosable %}} + +{{% choosable language yaml %}} + +```yaml +resources: + azure-provider: + type: pulumi:providers:azure-native + options: + envVarMappings: + CUSTOM_ARM_CLIENT_SECRET: ARM_CLIENT_SECRET +``` + +{{% /choosable %}} + +{{< /chooser >}} + +## Multi-provider example + +A common use case is running two providers targeting different cloud accounts. Here's an example with two AWS providers for production and staging environments: + +{{< chooser language "typescript,python,go,csharp,java,yaml" >}} + +{{% choosable language typescript %}} + +```typescript +import * as aws from "@pulumi/aws"; + +// Production provider reads from PROD_AWS_ACCESS_KEY_ID and PROD_AWS_SECRET_ACCESS_KEY +const prodProvider = new aws.Provider("aws-prod", {}, { + envVarMappings: { + "PROD_AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID", + "PROD_AWS_SECRET_ACCESS_KEY": "AWS_SECRET_ACCESS_KEY", + }, +}); + +// Staging provider reads from STAGING_AWS_ACCESS_KEY_ID and STAGING_AWS_SECRET_ACCESS_KEY +const stagingProvider = new aws.Provider("aws-staging", {}, { + envVarMappings: { + "STAGING_AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID", + "STAGING_AWS_SECRET_ACCESS_KEY": "AWS_SECRET_ACCESS_KEY", + }, +}); + +// Use the providers explicitly +const prodBucket = new aws.s3.Bucket("prod-bucket", {}, { provider: prodProvider }); +const stagingBucket = new aws.s3.Bucket("staging-bucket", {}, { provider: stagingProvider }); +``` + +{{% /choosable %}} + +{{% choosable language python %}} + +```python +import pulumi +import pulumi_aws as aws + +# Production provider reads from PROD_AWS_ACCESS_KEY_ID and PROD_AWS_SECRET_ACCESS_KEY +prod_provider = aws.Provider("aws-prod", + opts=pulumi.ResourceOptions( + env_var_mappings={ + "PROD_AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID", + "PROD_AWS_SECRET_ACCESS_KEY": "AWS_SECRET_ACCESS_KEY", + } + ) +) + +# Staging provider reads from STAGING_AWS_ACCESS_KEY_ID and STAGING_AWS_SECRET_ACCESS_KEY +staging_provider = aws.Provider("aws-staging", + opts=pulumi.ResourceOptions( + env_var_mappings={ + "STAGING_AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID", + "STAGING_AWS_SECRET_ACCESS_KEY": "AWS_SECRET_ACCESS_KEY", + } + ) +) + +# Use the providers explicitly +prod_bucket = aws.s3.Bucket("prod-bucket", opts=pulumi.ResourceOptions(provider=prod_provider)) +staging_bucket = aws.s3.Bucket("staging-bucket", opts=pulumi.ResourceOptions(provider=staging_provider)) +``` + +{{% /choosable %}} + +{{% choosable language go %}} + +```go +// Production provider reads from PROD_AWS_ACCESS_KEY_ID and PROD_AWS_SECRET_ACCESS_KEY +prodProvider, err := aws.NewProvider(ctx, "aws-prod", &aws.ProviderArgs{}, + pulumi.EnvVarMappings(map[string]string{ + "PROD_AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID", + "PROD_AWS_SECRET_ACCESS_KEY": "AWS_SECRET_ACCESS_KEY", + }), +) +if err != nil { + return err +} + +// Staging provider reads from STAGING_AWS_ACCESS_KEY_ID and STAGING_AWS_SECRET_ACCESS_KEY +stagingProvider, err := aws.NewProvider(ctx, "aws-staging", &aws.ProviderArgs{}, + pulumi.EnvVarMappings(map[string]string{ + "STAGING_AWS_ACCESS_KEY_ID": "AWS_ACCESS_KEY_ID", + "STAGING_AWS_SECRET_ACCESS_KEY": "AWS_SECRET_ACCESS_KEY", + }), +) +if err != nil { + return err +} + +// Use the providers explicitly +prodBucket, err := s3.NewBucket(ctx, "prod-bucket", &s3.BucketArgs{}, + pulumi.Provider(prodProvider)) +stagingBucket, err := s3.NewBucket(ctx, "staging-bucket", &s3.BucketArgs{}, + pulumi.Provider(stagingProvider)) +``` + +{{% /choosable %}} + +{{% choosable language csharp %}} + +```csharp +using Pulumi; +using Pulumi.Aws; +using Pulumi.Aws.S3; + +// Production provider reads from PROD_AWS_ACCESS_KEY_ID and PROD_AWS_SECRET_ACCESS_KEY +var prodProvider = new Provider("aws-prod", new ProviderArgs(), new CustomResourceOptions +{ + EnvVarMappings = new Dictionary + { + { "PROD_AWS_ACCESS_KEY_ID", "AWS_ACCESS_KEY_ID" }, + { "PROD_AWS_SECRET_ACCESS_KEY", "AWS_SECRET_ACCESS_KEY" } + } +}); + +// Staging provider reads from STAGING_AWS_ACCESS_KEY_ID and STAGING_AWS_SECRET_ACCESS_KEY +var stagingProvider = new Provider("aws-staging", new ProviderArgs(), new CustomResourceOptions +{ + EnvVarMappings = new Dictionary + { + { "STAGING_AWS_ACCESS_KEY_ID", "AWS_ACCESS_KEY_ID" }, + { "STAGING_AWS_SECRET_ACCESS_KEY", "AWS_SECRET_ACCESS_KEY" } + } +}); + +// Use the providers explicitly +var prodBucket = new Bucket("prod-bucket", new BucketArgs(), new CustomResourceOptions +{ + Provider = prodProvider +}); +var stagingBucket = new Bucket("staging-bucket", new BucketArgs(), new CustomResourceOptions +{ + Provider = stagingProvider +}); +``` + +{{% /choosable %}} + +{{% choosable language java %}} + +```java +import com.pulumi.aws.Provider; +import com.pulumi.aws.ProviderArgs; +import com.pulumi.aws.s3.Bucket; +import com.pulumi.aws.s3.BucketArgs; +import com.pulumi.resources.CustomResourceOptions; +import java.util.Map; + +// Production provider reads from PROD_AWS_ACCESS_KEY_ID and PROD_AWS_SECRET_ACCESS_KEY +var prodProvider = new Provider("aws-prod", ProviderArgs.Empty, + CustomResourceOptions.builder() + .envVarMappings(Map.of( + "PROD_AWS_ACCESS_KEY_ID", "AWS_ACCESS_KEY_ID", + "PROD_AWS_SECRET_ACCESS_KEY", "AWS_SECRET_ACCESS_KEY" + )) + .build()); + +// Staging provider reads from STAGING_AWS_ACCESS_KEY_ID and STAGING_AWS_SECRET_ACCESS_KEY +var stagingProvider = new Provider("aws-staging", ProviderArgs.Empty, + CustomResourceOptions.builder() + .envVarMappings(Map.of( + "STAGING_AWS_ACCESS_KEY_ID", "AWS_ACCESS_KEY_ID", + "STAGING_AWS_SECRET_ACCESS_KEY", "AWS_SECRET_ACCESS_KEY" + )) + .build()); + +// Use the providers explicitly +var prodBucket = new Bucket("prod-bucket", BucketArgs.Empty, + CustomResourceOptions.builder() + .provider(prodProvider) + .build()); +var stagingBucket = new Bucket("staging-bucket", BucketArgs.Empty, + CustomResourceOptions.builder() + .provider(stagingProvider) + .build()); +``` + +{{% /choosable %}} + +{{% choosable language yaml %}} + +```yaml +resources: + # Production provider reads from PROD_AWS_ACCESS_KEY_ID and PROD_AWS_SECRET_ACCESS_KEY + aws-prod: + type: pulumi:providers:aws + options: + envVarMappings: + PROD_AWS_ACCESS_KEY_ID: AWS_ACCESS_KEY_ID + PROD_AWS_SECRET_ACCESS_KEY: AWS_SECRET_ACCESS_KEY + + # Staging provider reads from STAGING_AWS_ACCESS_KEY_ID and STAGING_AWS_SECRET_ACCESS_KEY + aws-staging: + type: pulumi:providers:aws + options: + envVarMappings: + STAGING_AWS_ACCESS_KEY_ID: AWS_ACCESS_KEY_ID + STAGING_AWS_SECRET_ACCESS_KEY: AWS_SECRET_ACCESS_KEY + + # Use the providers explicitly + prod-bucket: + type: aws:s3:Bucket + options: + provider: ${aws-prod} + + staging-bucket: + type: aws:s3:Bucket + options: + provider: ${aws-staging} +``` + +{{% /choosable %}} + +{{< /chooser >}} + +## How it works + +The `envVarMappings` option is a map where: +- **Keys** are custom environment variables you have defined in your environment +- **Values** are the names of environment variables the provider expects + +When the provider initializes, Pulumi checks if your custom environment variable exists. +If it does, the provider sees its value as if it were set under the mapped variable name. + +## Limitations + +- **Provider resources only**: This option only works on provider resources, not on regular resources or components. +- **One-way mapping**: Each custom variable maps to exactly one provider variable. You cannot map multiple custom variables to the same provider variable on a single provider.