Type-safe APIs for existing CRDs?

[mt-inside]

Hi,

I have the same question as here but didn't want to resurrect such an old thread. tl;dr I have a cluster with Istio installed and want to create / manipulate some of its types. (A lot of the examples deal with programmatically deploying new CRDs but I've never seen CRDs managed like that in a real cluster, plus it's not my type).

I've written some code based on the dynamic_* examples and it works - I can get things typed as DynamicObject and print them - they're full of Object and Array types (the type I want to interact with is super complicated). Is this really the best way to do this? I understand that to read inner fields I can use jsonpath but I'd be much better to map()/filter() my way through something more native.

Likewise to make a new one, am I going to need to construct a instance of nested Objects and Arrays - feels really clunky.

I guess two questions:

1. What's the difference between manually making Object({Array({..})}) instances, vs re-specifying the type using the crd-derive (does this even work?) vs the approach here vs examples/crd_derive_custom_schema.rs. Maybe some high-level docs would help on the different approaches? Either way, I don't want to do this if I can avoid it because DRY - I don't want to re-specify someone else's type, and it'll be a load of code for such a complex type.

2. Can we find a better and more native way to do this? I'd love to be able to do something like

let crds: Api<CustomResourceDefinition> = Api::all(client.clone());
let crd = crds.get("DestinationRule").await?;
let crdType = magic(crd);
let drs: Api<crdType> = ...

I realise rust isn't a dynamic language like this, but maybe using index overriding for an API like

let crdType = magic(crd);
let dr = new_dynamic(crdType, o!(subsets = a!("foo", "bar")));
dr["spec"]["subsets"][0] = ...

Or is my best bet just forking k8s_openapi and re-running its auto-generation on the istio schema too? (Seems like a maintenance nightmare, especially when, by definition, the types will be specified in any cluster I interact with)

[shanesveller]

I defer to the maintainers on perceived stability/readiness but you may be able to get a head start using https://github.com/kube-rs/kopium

[mt-inside]

Thanks that's really useful!

[clux]

As it so happens, there is now some high level docs on the different types of typing that we currently support over at https://kube.rs/controllers/object/ from a controller writer POV. Please let me know what you think.

As mentioned above, kopium is indeed the closest thing to this type of magic you describe, but it does not* support generation at run-time, so you're effectively pre-generating the code for all the crds that you wish to support in your app.

There isn't currently any concrete plans for anything more advanced (kopium is evolving at a steady pace), but we are open to good ideas!
you can shoehorn kopium into a CI build step, but it's currently a bit awkward in that it would require cluster access (opened kube-rs/kopium#44 for this).

[mt-inside]

Amazing, thank you for the great answer!

Those docs are great, cleared everything up. I hadn't spotted them at all (via google, github, or docs.rs - probably my fault by just FYI).

Kopium seems like exactly what I need. Agree it'll be a pain in CI - upvoted the issue you linked :) Would be great if we set up a bunch of CI jobs to periodically run it against the schema of common projects and push to repos like k8s_openapi (eg prometheus_openapi, istio_openapi) - that way people can just reach for them as if they're native. What do you think?

As for something more advanced, I still think there's some value in something like python kadet. Connect to a custer, read the schema from a CRD resource, generate a bunch of instances of a struct that could allow calls like my_resource.get("spec").get("subsets").get(0) = .... But if kopium can be easily integrated into CI then those real types will be better in 99% of cases

[nightkr]

Ideally (IMO) it would be possible to use Kopium as a proc macro after kube-rs/kopium#44 is sorted out, transparently integrating into the regular Rust build chain.

As for something more advanced, I still think there's some value in something like python kadet. Connect to a custer, read the schema from a CRD resource, generate a bunch of instances of a struct that could allow calls like my_resource.get("spec").get("subsets").get(0) = .... But if kopium can be easily integrated into CI then those real types will be better in 99% of cases

You should be able to do this already with DynamicObject, since serde_json::Value supports a similar DSL. This should allow you to write my_obj.data["spec"]["subsets"][0] (see impl Index for serde_json::Value) or my_obj.pointer("spec/subsets/0") (see serde_json::Value::pointer).