Use official async-tar

[itowlson]

Fixes #2846.

This built locally for me on Windows. I'm pushing this to see whether it builds in CI, and if the artefacts are usable.

@vdice The dependency is used only (I believe) in the archive layers stuff (#1847). Do we have a test case I can use to hit that?

[vdice]

Thanks for getting this dep back to upstream!

@vdice The dependency is used only (I believe) in the archive layers stuff (#1847). Do we have a test case I can use to hit that?

I thought I did add one but it turns out I was thinking of an e2e test in the containerd-shim-spin repo.

So, the quick way to test is to export SPIN_OCI_ARCHIVE_LAYERS=1 which will force use of archive layers and then go through the flow of spin registry push <ref> and spin up -f <ref> to ensure the published/pulled app works as expected.

[vdice]

PS One other testing note: I believe the app does need at least one static asset with a size over a certain minimum, or else the content is inlined in the oci manifest. update: just tested: size of static asset does not matter, but at least one asset is necessary to test archive layers

[itowlson]

@vdice Okay, I set that environment variable, and did a push from Windows of a test application with one small asset (about a dozen bytes). I was able to spin up from the registry on both Windows and Linux and verify that the application could read the asset. I also downloaded the registry image with a simple OCI client and printed out the layers' media types:

the image has 3 layers
application/vnd.wasm.content.layer.v1+wasm
application/vnd.wasm.content.bundle.v1.tar+gzip
application/vnd.fermyon.spin.application.v1+config

If I remove the environment variable and re-push, the tar+gzip layer disappears, so I'm assuming that my test did indeed exercise the code path and it is doing a reasonable thing (it certainly isn't crashing or anything).

I realise this is a very limited test, but I am going to mark this as ready to review - if there are further tests you'd like me to run then feel free to put the mockers on it.

Thank you ever so much for your guidance on this!

[rylev]

LGTM, but I will note it doesn't fully address the issues laid out in #2846, namely the use of async-tar in general which brings in async-std and associated dependencies.

I'd be happy with merging this PR as is but keeping #2846 open so we can discuss whether removing async-tar entirely (and using tar along with spawn_blocking) would be a better way to go.

[vdice]

LGTM! All apps I tested with work great.

[itowlson]

@rylev I can't find a way to make GitHub unlink the issue but I hear you. I will merge this then reopen your issue.