Inject values better than extVars or TLAs

[camh-]

jsonnet has two methods for injecting data/code into a document: external variables (extVars) and top-level arguments (TLAs). extVars have a problem that no defaults can be provided in the absence of an extVar, as the std.extVar function returns an error if the var is not set (and you cannot test for errors). TLAs require the document be a function to be evaluated, rather than just a plain object, and this makes it harder to provide a document as a template object that can be overridden.

Two alternatives come to mind - top-level locals and top-level fields.

top-level local

A top-level local would be the addition of local x = <code>; to the top of a document, where x and code are parameters much like --top-level-code x=<code> is. This seems like it would be pretty simple to do, as given an AST, adding a local binding node to it seems simple. This is already done in jsonnet in a sense with the std library - available to all jsonnet documents without needing an explicit local/import. The downside of this is it doesn't solve the defaults-in-absence problem - a document cannot provide a default for x but allow a top-level local to override it because the order of locals would be wrong.
e.g.

local x = "hello"; // injected with --top-level-local 'x="hello"'
local x = "world"; // default in the document
x

will evaluate to the default "world", not the injected "hello".

Top-level locals could be implemented without changing jsonnet - it just needs to manipulate the AST between parsing and evaluation.

top-level field

A top-level field is similar to a top-level arg (TLA) - it requires that the top-level value be of a specific type (object, like TLA requires a function), and it would add a field to that object, replacing one if it were already present maintaining the field's visibility. This allows a default to be provided and overridden. But this has a couple of downsides:

1. The top-level value needs to be an object - how to inject values when it is an array for example?

2. It seems it might be hard and might need changes to jsonnet itself to implement. TLAs are easier - you just need to evaluate the document and check the type. If it is a function, you evaluate again and pass the arguments. Because the resulting function has not been evaluated at the time the arguments are injected, you get late binding. But if you evaluate the top-level value and check if it is an object, references to the field being overridden would have already been resolved. If you try to inject fields before evaluation, you need to walk the parse tree and possibly pre-evaluate parts. That seems impossible (or madness). For example, how do you inject a field into this expression:

    {a:1} + f("hello") + {b:2}
   

   Where would you inject the field a=3 ? Perhaps it is as simple as injecting all top-level fields as one object by adding + {all top-level fields} to the expression? Hmmm, that may work. Worth experimenting with.

[camh-]

For top-level fields, I would also like to be able to inject nested fields easily.

e.g. if I want to replace helm with pure jsonnet, I would still like the concept of a "values" schema that holds the common configurable items. My imaginary helm replacement would inject a values field into each resource so the resource can refer to $.values.bar to get the value. With a top-level field, I'd want to be able to do --tlf 'values.bar="newbar"' - this would need to be be injected as + {values+: {bar: "newbar"}}. This would mean parsing the field name, but what if the field name was the literal field.bar? Can you have computed fields? (that would be powerful, but maybe not all that useful).

It would seem necessary that after splitting a field spec for nesting, all but the last field would need to be added with +: to prevent removing unspecified existing fields.

[camh-]

I am now thinking of combining this with #23 (implement chained evaluation). That issue describes a syntax for binding a local to the result of the previous evaluation, allowing you to evaluate multiple expressions/files in sequence passing one to the next. I am now thinking using a default name of input and implicitly injecting a local input = <prev>; at the top where <prev> is the evaluation of the previous document.

This injection of values could be done such that CLI flags build up an expression/value that becomes input to the actual file/expression to be interpreted.

jx --input foo=bar --input foo2.bar2=baz file.jsonnet

would be as though file.jsonnet had at the top:

local input = { foo: "bar", foo2: { bar2: "baz } };
<rest of file.jsonnet>

Then specifying these input parameters becomes a matter of defining a syntax for piecemeal specification of individual values on the command line, supporting nested structures. There is no requirement that the input value be an object:

jx --input "hello" file.jsonnet

would be the same as

local input = "hello";
<rest of file.jsonnet>

As input may be a name already used by a file/expression, it should be possible to change the name of that:

jx --input-var foo --input "hello" file.jsonnet

would be the same as

local foo = "hello";
<rest of file.jsonnet>