Expose crosstests.Create

[iwahbe]

Stacked PRs:

• Test int->string type overrides with cross-tests #2502
• Implement runCreateInputCheck in terms of Create #2503
• ->Expose crosstests.Create #2501
• Refactor cross-tests to use (cty.Value, resource.PropertyMap) pairs #2500

---

Expose crosstests.Create

This begins to turn cross-tests into a library with a defined interface. Existing
cross-tests say that they are a prototype level API, and they feel that way. The
prototyping was successful, and cross-tests have proven their worth. We should begin to
move towards a more discover-able API. The first function of which is:

// crosstests/create.go

func Create(
	t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap,
	options ...CreateOption,
)

We can continue to iterate on this API, but we should begin to formalize a boundary
between the implementation of cross-tests and tests written with cross-tests.

[codecov]

Codecov Report

Attention: Patch coverage is 4.47761% with 64 lines in your changes missing coverage. Please review.

Project coverage is 62.50%. Comparing base (23d65d6) to head (563065f).

Files with missing lines	Patch %	Lines
pkg/internal/tests/cross-tests/create.go	0.00%	62 Missing ⚠️
pkg/tests/pulcheck/pulcheck.go	60.00%	2 Missing ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##           iwahbe/stack/1    #2501      +/-   ##
==================================================
- Coverage           62.58%   62.50%   -0.08%     
==================================================
  Files                 380      381       +1     
  Lines               51270    51335      +65     
==================================================
+ Hits                32087    32088       +1     
- Misses              17368    17432      +64     
  Partials             1815     1815              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[VenelinMartinov]

Why do we need to expose both the pulumi and TF inputs in the test framework? I don't think it is the common use case for these to differ - should we instead aim to make it simpler to use this by having one input and handle the conversion in the framework, as before?

I don't think we have that many use cases for the inputs differing outside of type conversions as it this defeats the purpose of the test

[iwahbe]

As I said in #2500 (comment):

We need to test:

• implicit type conversions
• explicit type changes (Testing: Add an integration test for type overrides #2048 is a start)
• max items one fiddlyness (simulating major version upgrades)
• secret injection (if we ever want to handle secrets, or in Configure)
• etc.

Another example: If you want to verify that set order doesn't matter, you would need to do something like:

for _, setPermutation := range permutationsOf(setValues) {
	crosstests.Create(t, schema, tfConfig, resource.PropertyMap{
		"set": resource.NewArrayProperty(setPermutation),
	})
}

I don't think you could reliably do that in tftypes.Value space.

From Create's API, it would be very easy to build in automatic pulumi value inference:

crosstests.Create(t, schema, tfConfig, crosstests.InferValue)

Just as easily, we could make accepting the Pulumi value an option, defaulting to inference:

crosstests.Create(t, schema, tfConfig) // Pulumi value is inferred
crosstests.Create(t, schema, tfConfig, // Pulumi value is explicit
	crosstests.CreatePulumiValue(explicitPulumiValue))

As long as the internals support explicit values (#2500), it's easy to convert to an implicit value. From there, there are many ways we can support implicit values.

[t0yv0]

This is reasonable, thanks:

func Create(
	t T, resource map[string]*schema.Schema, tfConfig cty.Value, puConfig resource.PropertyMap,
	options ...CreateOption,
)

Might need a way to set overrides (info.Schema aka SchemaInfo).

As Venelin points out, some tests may want to omit specifying both tfConfig and puConfig if one can be unambiguously computed from the other.

[t0yv0]

For internal code I wonder sometimes if plain struct options are just easier to work with and discover than functional options. That's a stylistic nit though. It's all good.

[t0yv0]

LGTM, looks like it's getting a little nicer. Always welcome.