Feature: Support OCI artifact pulling

[squillace]

The scenario is this. Right now, partly because we were trying to exercise the containerd shim and partly because there wasn't an oras-rs crate we could use from source our app hosts -- whether spin, slight, or future hosts -- can't give us a .toml because either we alter how the runtimes acquire their configs in order to integrate with k8s -- something that doesn't make a lot of sense. Even if we use volume mounts for the config files, you don't want the operational experience to ship separately. Instead, you want one reference to an immutable config of a host and a module. You want the joy of using the image value of the pod spec to point at "the entire thing that runs correctly" -- which includes more than one artifact.

The way we do that without OCI Artifacts is by building a scratch container and dropping both the module and the config for the runtime into it. That's fine, but we need to build a container in order to not use the container. :-) This is entirely a point-in-time thing, as an older friend used to say. No immediate hurries.

BUT: we do need to have a plan to a) support both this method and the OCI Artifact method (oras) and b) understand what that would mean for the yaml experience. We EITHER need to re-use the image key and take both and just do the right thing (check with oras and if not, use docker) OR we do a hard roll to oras and commit, re-using the image key for artifacts only. Alternatively, we could add an artifact key and have the runtime punt unless it was either/or but not both. (Problem with that scenario is that no one would recognize the artifact key as schema valid. :-( )

In any case, there is prior art we might be able to use. https://docs.rs/oras/latest/oras/struct.Client.html never got finished, but Sajay says he'd love to fund that. BUT... also Jacob LeGrone in the CNAB space did https://crates.io/crates/oras so I'll reach out to see where that codebase is. Seems to be private. But if he is willing, we can bring that up to date pretty rapidly.

[squillace]

@devigned and @cpuguy83 whatchya think? We're not doing this now, but the real question is how WOULD be the best way to go about this?

[cpuguy83]

I'm definitely interested in how to make this work with OCI artifacts. This really opens up some interesting cases.

One way we could hack this without OCI artifacts is to cram config in as a "layer" with a special media type.
On the containerd side we can registry a stream processor that understands the layer type and extract it accordingly into what actually becomes the rootfs.

As an example:

{
    "rootfs": {
      "diff_ids": [
        "sha256:<config digest>",
        "sha256:<rootfs digest>",
        ...
      ],
      "type": "layers"
    }
}

Then the media type on the the "config" layer could be something like application/vnd.some.org.spin.v1.config.
We register a stream processor with containerd for application/vnd.some.org.spin.v1.config which will understand what to do with that (e.g. read the config and write it to spin.toml).

[cpuguy83]

That said, just having the config in a normal layer that's separate from the wasm modules (and each wasm module being separate) could really have the same affect.
What to build a new application? Add the right layer digest combinations.

[squillace]

@cpuguy83 yeah it's interesting. I've referenced this in the other thread as well. Usability wise, this would dramatically benefit function dev and deployment as a "simple single immutable graph"; on the other hand, the way we currently expect a cluster operator to understand an installed application will change.

There's another issue I'd love your take on. We're thinking that we're overriding the image key in the podspec to pull in the "graph" or the "container-that-has-the-graph". Is there a smart way to support detection of the approach and use either? (If there were a great thing to do, it would be add an artifact-graph value to the pod spec schema, but hey, baby steps.)

I'm very interested in your

We register a stream processor with containerd for application/vnd.some.org.spin.v1.config which will understand what to do with that (e.g. read the config and write it to spin.toml).

Interesting. So in this case I don't like the yet another media type approach, but at the same time everything else works smoothly here. This feature would add the ability to ship the same host binaries but modify the config separately because it's a different "image". There's some collateral damage to the image platform metadata in the registry again, I think, so I'm loathe to go there when the ultimate objective is OCI Artifact graphs (or some other thing precisely like it). But hey, convince me.

[jsturtevant]

this can be supported with runwasi v0.4.0 and #164 demonstrates this