Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of multi-arch support for extension packages #2320

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

pacostas
Copy link

@pacostas pacostas commented Jan 15, 2025

Summary

This implementation is based on the RFC https://github.com/buildpacks/rfcs/blob/main/text/0128-multiarch-builders-and-package.md

This PR allows the pack extension package to generate multi-platform OCI images and create an image index to combine them.

It also adds the --path flag to point to a directory on the extension that needs to be packaged.

This PR has not been tested in packaging a builder with multi-platform extension.

Output

Before

  • We were not able to create multi-platform extensions
  • We were not able to point to the directory of the extension

After

  • Can generate multi-platform OCI images by using the --target flag
  • Can specify the path of where the extension is located, using the --path flag.

Documentation

  • Should this change be documented?
    • Yes
    • No

Related

Resolves #2183
Resolves #2160

@pacostas pacostas requested review from a team as code owners January 15, 2025 11:53
@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Jan 15, 2025
@github-actions github-actions bot added this to the 0.37.0 milestone Jan 15, 2025
@pacostas pacostas marked this pull request as draft January 16, 2025 12:58
@pacostas pacostas force-pushed the extension-multi-arch branch from dcf618a to 7622d59 Compare January 16, 2025 15:18
Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
…ension .toml file

Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
@pacostas pacostas force-pushed the extension-multi-arch branch from 98cc239 to 3137310 Compare January 16, 2025 15:39
@pacostas pacostas marked this pull request as ready for review January 16, 2025 16:08
@jjbustamante
Copy link
Member

@pacostas Thank you so much for this work!!! I will review it tomorrow!!!

@pacostas
Copy link
Author

You are welcome, feel free to make any suggestions! :)

@pacostas
Copy link
Author

@jjbustamante Hello! With respect to your time, did you have the chance to take a look on the implementation?

@jjbustamante
Copy link
Member

@jjbustamante Hello! With respect to your time, did you have the chance to take a look on the implementation?

@pacostas I am so sorry I haven't reviewed your PR, I will be pulling it and testing it locally, also, we are evaluating the current state of Kaniko because we are not sure how it will affect us. We use Kaniko for our extension implementation.

@pacostas
Copy link
Author

Thank you very much :)

Thank you for the information about kaniko, do you know if kaniko is used somewhere else except extension and if there is a discussion/thread about that issue (to be in the loop)?

@jjbustamante
Copy link
Member

Thank you very much :)

Thank you for the information about kaniko, do you know if kaniko is used somewhere else except extension and if there is a discussion/thread about that issue (to be in the loop)?

We only use Kaniko for extensions, we discussed it during our Working Group meeting on December 12th, but the recording is not on Youtube 😞

Copy link
Member

@jjbustamante jjbustamante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @pacostas
Thanks for working on this! I apologize for the late review, the PR is fine, I added a few nits.

One doubt, do you also want to add support for extensions added to a builder?

…rdcoded values

Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
Signed-off-by: Costas Papastathis <papastathiscr@gmail.com>
@pacostas pacostas force-pushed the extension-multi-arch branch from c002ca3 to 912e3a8 Compare February 25, 2025 12:00
@pacostas
Copy link
Author

Thank you for your review :)

One doubt, do you also want to add support for extensions added to a builder?

good point :) I havent tested it with a builder, as there are a lot of moving parts on the multi-arch side. Do you think we could leave it for now? And I can follow with another PR once things have settled.

@pacostas
Copy link
Author

One test that is failing, I think is not related to this PR. By re-running, I think it will pass.

@jjbustamante
Copy link
Member

One test that is failing, I think is not related to this PR. By re-running, I think it will pass.

Yeah, I need to prune some networks in our windows runner to fix it

@natalieparellano
Copy link
Member

Yeah, I need to prune some networks in our windows runner to fix it

I thought I fixed this! So sad

@jjbustamante
Copy link
Member

@pacostas Thanks for doing the change, the only reason stopping me from merging the change is just because I want to give it a try before merging it! The PR is ready to be merged.

@natalieparellano I am not sure if you have the time, but I thought that maybe we could update the samples repo with the multi-arch version for the extension and test them with this branch. Is it something you can help me? updating the samples repo with multi-arch extensions?

@pacostas
Copy link
Author

@jjbustamante no rush, thank you very much :) BTW I have tested it with this extension.toml, https://github.com/paketo-buildpacks/ubi-nodejs-extension/pull/273/files#diff-21e42dd2276e594017780607a13f07b058d020447a73c11f032c77823a8ed493 I hope the code is useful for testing it with the samples too.

@natalieparellano
Copy link
Member

Is it something you can help me? updating the samples repo with multi-arch extensions?

Sure!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Issue that requests a new feature or improvement.
Projects
None yet
3 participants