Optional Environment Parameters

[tamirkamara]

We're upgrading to v1 and saw that using parameters file directly in actions (like install) is deprecated so we also move to a set too.
Thing is that it doesn't behave the same way, perhaps it's because of the set but might also be a change in v1. Using the following set will fail if PARAM1 environment variable isn't set...
We do this a lot mainly for optional bundle parameters.
Do you have any advice?

schemaType: ParameterSet
schemaVersion: 1.0.1
namespace: ""
name: bundle1
parameters:
  - name: param1
    source:
      env: PARAM1

rpc error: code = Unknown desc = environment variable env is not defined
unable to process provided parameter sets: 1 error occurred:
        * unable to resolve parameter bundle1.param1 from env PARAM1: rpc error: code = Unknown desc = environment variable env is not defined

[carolynvs]

I just double checked against Porter v0.38 and we have always required that environment variables specified in a parameter set exist.

Can you share what you are trying to accomplish and working command in v0 that you are migrating to v1?

[tamirkamara]

This is how we run porter today, especially in dev (but to an extend in prod too): https://github.com/microsoft/AzureTRE/blob/main/Makefile#L194

We reference the parameters file directly in the porter install command, and in v1 that should move to a parameters set as the old way is obsolete.

[carolynvs]

That's right. You can no longer pass a parameter set file (in your case parameters.json) directly to the -p flag, and need to first apply the file with porter parameters apply parameters.json and then reference the parameter set by name porter install -p NAME, where NAME is the name of the parameter set as defined in your parameters.json.

But it sounds like the error you are running into is that the environment variable referenced in parameters.json does not exist? That's what I'm confused about. You were using a parameter set file before with v0, and Porter v0 enforces that the environment variable exists.

Is the problem how to call install and pass the parameter set? Or is the problem that the machine you are running on doesn't have the environment variable defined that is referenced in your parameter set? Because the latter never worked, it's not specific to v1. I'm wondering if you were seeing a different behavior in v1 where you were referencing missing environment variables and somehow getting that to work?

[tamirkamara]

With porter v0 and a parameter set via a file didn't show the behavior of env variables needed to exist. I remember testing it since in v1 (via a set applied) I had to reduce the set to just the mandatory parameters that are always present in the environment. This was a temporary step on my end as our setup will need this capability back (via porter directly or by other means).

[carolynvs]

Porter v0 never supported using undefined environment variables in a parameter set. Below is an example to demonstrate.

~$ porter version
porter v0.38.13 (85b77f62)

~$ cat params.json
{
  "name": "hello-llama",
  "parameters": [
    {
      "name": "name",
      "source": {
        "env": "NOPE"
      }
    }
  ]
}
~$ porter upgrade hello-llama -p ./params.json -r getporter/hello-llama:v0.1.1
Error: unable to process provided parameter sets: [./params.json]: 1 error occurred:
	* unable to resolve parameter hello-llama.name from env NOPE: environment variable env is not defined

We can discuss an enhancement to support attempting to resolve an optional parameter against a source, and what porter should do when it's not found, but it isn't something that ever worked in Porter.

[carolynvs]

I have created #2474 to track adding support for resolving optional params/creds in a set, falling back to the bundle default if they cannot be resolved.