AWS Public Gallery and/or AMIs

[riain0]

Hello,

back again with more questions.

Context: We are building up a bunch of tooling internally to use Tart and to make it as easy to use as possible for our internal engineers. We use the XCode flavour of the images.

The problem: I've attempted to install the Tart images from GHCR on start up of the EC2 instances but due to the size of them (and maybe some network restrictions somewhere) it fails at fairly random intervals with a Error: The network connection was lost. error. Sometimes at 10%, sometimes 30% sometimes 99%...

My very manual and painful solution: I essentially try and try and try to pull the image manually while SSH'd to the EC2 instance from GHCR then re-push the image to an ECR. Then on start up of the EC2 instance we pull from the ECR so that we only pull once per instance. This seems to work but it makes using several images or keeping images updated incredibly difficult.

An ideal world: Would it be possible to also host the Tart images on the public AWS gallery? Or would it be possible to bake in the images to the AMIs you provide?

Some further context: I've looked into building these AMIs myself, trying to use some sort of pull through cache (not possible with ghcr in AWS currently) and other alternatives but I wanted to ask here first about the above ideas if they may happen and then act accordingly on what the best way forward is.

Edit: Maybe a better question is, is there a way to shrink the image size even smaller, or is the size due to the requirement by Apple EULAs that it must be the entire system?

Thank you!

[fkorotkov]

Hey @riain0, problem with bundling the images is that AMI release cadence is much slower than the release of the images. The idea of duplicating the images to a public ECR registry sounds very interesting! I've briefly looked and it seems for us we will only pay for S3 storage but most importantly egress traffic will be free for us. On your end it will be free within AWS and up to 500/GB/ip/month outside the AWS. Let us look a bit more and we might just start pushing images to ECR in addition to GHCR.

EDIT: we haven't found a way to shrink the images or make them layered. We only tried some trivial ideas like binary diff between images and so on but Xcode is huge. We do have a theory how to implement this but it's an idea for Tart 3.0 later next year.

[riain0]

That makes sense - the overhead of baking AMIs and publishing them is a bit much 😄

Yeah the ECR image mirroring is really appeasing to us as we can add VPC endpoints to ECR in our internal AWS accounts to then use the AWS backbone network to really speed up the downloads.

For the ECR mirror, if you are using AWS + CDK internally it should be fairly trivial and I can help contribute to this / create an RFC so there's solid requirements and agreements, but obviously with billing, AWS account IDs etc you may just want to do this internally.

On the images - yeah that is fair - I understand. I'm not even hugely familiar with XCode really, I'm just a Platform Engineer trying to make some Product and Mobile Engineers very happy! 😄

[riain0]

This is solved by cirruslabs/macos-image-templates#103 🚀

Thanks @fkorotkov!