Skip to content

Allow locally building multiplatform docker images #6256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 3, 2025
Merged

Allow locally building multiplatform docker images #6256

merged 1 commit into from
Mar 3, 2025

Conversation

swiatekm
Copy link
Contributor

@swiatekm swiatekm commented Dec 9, 2024 •
edited
Loading

What does this PR do?

It allows building cross-platform docker images. We can already cross-build agent binaries, but for docker images we had a check specifically limiting them to the host architecture. However, buildx makes this pretty straightforward. This PR adds a check for buildx platform availability, and uses it if the host arch doesn't match the requested container image arch.

The cross-built container images gets the arch appended to the tag, so when building main for arm64 on a amd64 host, we get the 9.0.0-SNAPSHOT-arm64 tag. The native image gets a tag without the suffix, which is a bit inconsistent, but preserves the current behaviour.

This doesn't change anything for our CI, which builds arm images on arm hosts, but it makes cross-building locally much simpler.

Why is it important?

It's useful sometimes to be able to locally create a cross-platform manifest. I was troubleshooting some issues in our internal clusters, and wanted the SRE team to test a custom image I built, and had to hack around the build system to make it work.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas

How to test this PR locally

DEV=true SNAPSHOT=true PACKAGES=docker DOCKER_VARIANTS=basic PLATFORMS="linux/arm64,linux/amd64" EXTERNAL=true mage package

Should build two docker images.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 9, 2024

This pull request does not have a backport label. Could you fix it @swiatekm? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 9, 2024

backport-v8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Dec 9, 2024
@swiatekm swiatekm added skip-changelog chore Tasks that just need to be done, they are neither bug, nor enhancements labels Dec 9, 2024
@swiatekm swiatekm changed the title wip: allow locally building multiplatform docker images Allow locally building multiplatform docker images Dec 10, 2024
@swiatekm swiatekm added backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify labels Dec 10, 2024
@swiatekm swiatekm marked this pull request as ready for review December 10, 2024 15:44
@swiatekm swiatekm requested a review from a team as a code owner December 10, 2024 15:44
@swiatekm swiatekm requested review from blakerouse, pkoutsovasilis and a team December 10, 2024 15:44
pkoutsovasilis
pkoutsovasilis reviewed Dec 11, 2024
View reviewed changes
pkoutsovasilis
pkoutsovasilis reviewed Dec 11, 2024
View reviewed changes
@swiatekm swiatekm force-pushed the tools/multiplatform-docker branch from 6f9d971 to 8ebc8ca Compare December 12, 2024 15:34
@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@swiatekm
Copy link
Contributor Author

@pkoutsovasilis could you test this version and let me know if it cross-builds correctly? I'm curious if you can build it correctly for both platforms at once. Thanks for the help, really appreciate it!

@swiatekm swiatekm requested a review from cmacknz December 18, 2024 12:11
pkoutsovasilis
pkoutsovasilis approved these changes Dec 31, 2024
View reviewed changes
Copy link
Contributor

@pkoutsovasilis pkoutsovasilis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! sorry the delay but this one slipped my attention

@swiatekm swiatekm added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Dec 31, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@swiatekm swiatekm added backport-active-all Automated backport with mergify to all the active branches and removed backport-8.x Automated backport to the 8.x branch with mergify backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify labels Feb 20, 2025
blakerouse
blakerouse approved these changes Feb 20, 2025
View reviewed changes
Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@swiatekm swiatekm force-pushed the tools/multiplatform-docker branch from 8ebc8ca to 6005568 Compare February 21, 2025 09:58
@swiatekm swiatekm deleted the tools/multiplatform-docker branch March 3, 2025 10:49
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@swiatekm @mergify
Allow locally building multiplatform docker images (#6256)
e5d2544 
(cherry picked from commit 508271b)
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@swiatekm @mergify
Allow locally building multiplatform docker images (#6256)
3a65d2b 
(cherry picked from commit 508271b)
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@swiatekm @mergify
Allow locally building multiplatform docker images (#6256)
67fc433 
(cherry picked from commit 508271b)
This was referenced Mar 3, 2025
Merged
Merged
Merged
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@swiatekm @mergify
Allow locally building multiplatform docker images (#6256)
cfba557 
(cherry picked from commit 508271b)
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@swiatekm @mergify
Allow locally building multiplatform docker images (#6256)
cdd1335 
(cherry picked from commit 508271b)
This was referenced Mar 3, 2025
Merged
Closed
swiatekm added a commit that referenced this pull request Mar 3, 2025
@mergify @swiatekm
Allow locally building multiplatform docker images (#6256) (#7126)
9414b77 
(cherry picked from commit 508271b)

Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
swiatekm added a commit that referenced this pull request Mar 3, 2025
@mergify @swiatekm
Allow locally building multiplatform docker images (#6256) (#7123)
a69450a 
(cherry picked from commit 508271b)

Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
swiatekm added a commit that referenced this pull request Mar 3, 2025
@mergify @swiatekm
Allow locally building multiplatform docker images (#6256) (#7124)
ac50b60 
(cherry picked from commit 508271b)

Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
swiatekm added a commit that referenced this pull request Mar 3, 2025
@mergify @swiatekm
Allow locally building multiplatform docker images (#6256) (#7125)
ac94373 
(cherry picked from commit 508271b)

Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
pchila added a commit that referenced this pull request Mar 3, 2025
@pchila
Revert "Allow locally building multiplatform docker images (#6256)"
610117e 
This reverts commit 508271b.
@pchila pchila mentioned this pull request Mar 3, 2025
Merged
jlind23 pushed a commit that referenced this pull request Mar 3, 2025
@pchila
Revert "Allow locally building multiplatform docker images (#6256)" (#…
339055e 
…7140)

This reverts commit 508271b.
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@pchila @mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
3d22af7 
…7140)

This reverts commit 508271b.

(cherry picked from commit 339055e)
@mergify mergify bot mentioned this pull request Mar 3, 2025
Merged
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@pchila @mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
c6468f7 
…7140)

This reverts commit 508271b.

(cherry picked from commit 339055e)
@mergify mergify bot mentioned this pull request Mar 3, 2025
Merged
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@pchila @mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
222af9f 
…7140)

This reverts commit 508271b.

(cherry picked from commit 339055e)
@mergify mergify bot mentioned this pull request Mar 3, 2025
Merged
mergify bot pushed a commit that referenced this pull request Mar 3, 2025
@pchila @mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
61c058e 
…7140)

This reverts commit 508271b.

(cherry picked from commit 339055e)
@mergify mergify bot mentioned this pull request Mar 3, 2025
Merged
jlind23 pushed a commit that referenced this pull request Mar 3, 2025
@mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
dce2512 
…7140) (#7144)

Unblocking unified release.
jlind23 pushed a commit that referenced this pull request Mar 3, 2025
@mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
bcfff29 
…7140) (#7143)

Unblocking unified release.
jlind23 pushed a commit that referenced this pull request Mar 3, 2025
@mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
6bebea4 
…7140) (#7142)

Unblocking unified release.
jlind23 pushed a commit that referenced this pull request Mar 3, 2025
@mergify
Revert "Allow locally building multiplatform docker images (#6256)" (#…
2767509 
…7140) (#7141)

Unblocking unified release.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dliappis dliappis dliappis approved these changes

@blakerouse blakerouse blakerouse approved these changes

@pkoutsovasilis pkoutsovasilis pkoutsovasilis approved these changes

@cmacknz cmacknz Awaiting requested review from cmacknz

Assignees

@swiatekm swiatekm

Labels
backport-active-all Automated backport with mergify to all the active branches chore Tasks that just need to be done, they are neither bug, nor enhancements skip-changelog Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team Team:Ingest-EngProd
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@swiatekm @elasticmachine @pkoutsovasilis @dliappis @blakerouse