Add Hermetic Swift Toolchain Support with Bzlmod Extension by ma-oli · Pull Request #1630 · bazelbuild/rules_swift

ma-oli · 2025-12-15T18:50:33Z

This PR adds a new Bzlmod extension that downloads and configures standalone Swift toolchains, eliminating the need for local Swift installations. Users can now declaratively specify a Swift version in their MODULE.bazel, and the toolchain will be automatically downloaded from swift.org.

The implementation includes support for Swift 6.2.1 on macOS and all the Linux distributions and platforms currently supported by this version. It also refactors the toolchain configuration to properly support individual Swift tools (swiftc, swift-autolink-extract, swift-symbolgraph-extract) and generalizes the worker substitution logic to work across platforms.

An embedded Swift example (Raspberry Pi 4B blink) is included to demonstrate usage and serve as an integration test.

Usage

swift = use_extension("@rules_swift//swift:extensions.bzl", "swift")
swift.toolchain(name = "swift_toolchain", swift_version = "6.2.1")
use_repo(
    swift,
    "swift_toolchain",
    "swift_toolchain_ubuntu24.04-aarch64",
    "swift_toolchain_xcode",
)

register_toolchains(
    "@swift_toolchain//:cc_toolchain_embedded_ubuntu24.04",
    "@swift_toolchain//:cc_toolchain_embedded_ubuntu24.04-aarch64",
    "@swift_toolchain//:cc_toolchain_embedded_xcode",
    "@swift_toolchain//:swift_toolchain_embedded_ubuntu24.04",
    "@swift_toolchain//:swift_toolchain_embedded_ubuntu24.04-aarch64",
    "@swift_toolchain//:swift_toolchain_embedded_xcode",
    "@swift_toolchain//:swift_toolchain_exec_ubuntu24.04",
    "@swift_toolchain//:swift_toolchain_exec_ubuntu24.04-aarch64",
)

Testing

cd examples/toolchain/embedded
bazelisk build --platforms=//:aarch64 //rpi-4b-blink:Application.bin

aaronsky · 2026-01-02T17:32:07Z

examples/toolchain/embedded/BUILD.bazel

+    name = "llvm-objcopy",
+    actual = select({
+        "@platforms//os:linux": "@swift_toolchain_ubuntu24.04-aarch64//:usr/bin/llvm-objcopy",
+        "@platforms//os:macos": "@swift_toolchain_xcode//:usr/bin/llvm-objcopy",


what do you think about expanding the toolchain template to expose more tools like llvm-objcopy in addition to the tools it already exposes?

The tools themselves are implicitly exposed through the //:files target already.
But if you're referring to aliases like this one, we could but unfortunately we lack a standard distribution constraint setting. Same reasons why we need the user to register toolchains manually in their MODULE.bazel.
If at some point we have a universal linux toolchain, all of this can be simplified.

aaronsky · 2026-01-02T17:33:52Z

examples/toolchain/embedded/MODULE.bazel

+# rules_swift_package_manager currently pulls in rules_go v0.57.0, which has
+# this issue: https://github.com/bazel-contrib/rules_go/issues/4480
+# Which causes bazel mod to error out on bazel 8.5.0
+# When rules_swift_package_manager has upgrade their go version, remove this line.


fixed in cgrindel/rules_swift_package_manager@46409f9

There doesn't seem to have been a release just yet. Do you want to leave it like this or add a git_version_override()?

There should now be a release with this fix

Thank you, yes. This has been removed.

aaronsky · 2026-01-02T17:41:41Z

swift/extensions.bzl


 non_module_deps = module_extension(implementation = _non_module_deps_impl)
+
+def _standalone_toolchain_impl(module_ctx):


nit: if we can get away with declaring this extension reproducible, it'd be good for folks' lockfiles.

i know we talked about toolchain auto-registration and its complexity elsewhere, but surfacing that interest here as well

swift/internal/extensions/standalone_toolchain.bzl

skb-oai · 2026-01-08T19:49:23Z

tools/swift-releases/SwiftReleases.swift

@@ -0,0 +1,274 @@
+import ArgumentParser
+import CryptoKit


not exactly sure how this is used, but if it uses swift-crypto instead of CryptoKit it could work on windows/linux

It's used by the sha256() function. I could use swift-crypto, but that would mean adding another SPM dependency to rules_swift, I'm not sure we want to do this. If we feel strongly about it and we want to run this on other platforms for sure though, I can do the changes. Let me know

aaronsky

LGTM, but I'd love to get @keith or @brentleyjones's insight

adincebic

Huge thanks for the work you put in.

swift/extensions.bzl

adincebic · 2026-01-29T15:31:26Z

tools/swift-releases/SwiftReleases.swift

+
+func fetchReleases() throws -> ReleasesResponse {
+  let url = URL(string: "https://www.swift.org/api/v1/install/releases.json")!
+  let semaphore = DispatchSemaphore(value: 0)


Probably best to convert this to utilize Swift concurrency? My reasons are: way more readable code, way less code and zero need to think about concurrency.

luispadron

Im in favor of merging this, thanks for all the work here! I do think we should figure out the swift-releases question. I'm concerned that we'd need to make more rules_swift releases.

Once we figure that out (could be fine to make more releases to maintain the file, id like some more thoughts there). Also, this would land with the upcoming 4.x release unless we want to cherry-pick it to 3.x as well.

luispadron · 2026-02-02T15:12:44Z

swift/internal/extensions/BUILD.bazel.tpl

+        # https://github.com/swiftlang/swift-package-manager/issues/8648
+        # Note that this is a workaround that got removed in the change below:
+        # https://github.com/swiftlang/swift-package-manager/pull/9246
+        # When the new version of the compiler is released, we should remove this.


Can we specify the version & add a TODO here

luispadron · 2026-02-02T15:18:22Z

swift/internal/extensions/swift_releases.bzl

Can we add a comment on how to update this file somewhere in the file header.

Also, how often do you think we are going to need to update this file + make a rules_swift release? Another approach here might be to make swift_releases some other bzlmod module that is auto-generated/auto-released. Users can then override this and wouldn't require rules_swift releases to update the supported Swift releases

Historically, once every 1-2 months? Swift 6.2 was release in September and we're now at 6.2.4 (about 6 months later).
In a perfect world, the API would expose the checksums and we would be able to retrieve them programmatically and use them. But that's not the case; the checksum is not consistently published for every artifact. Hence the need for this tool.
It would be nice to have these releases auto-published somewhere; I can look into this, but publishing to a new repo not something I will be able to do swiftly because of some technicalities I'd have to deal with around publishing to new repositories.

luispadron · 2026-02-02T15:43:20Z

examples/toolchain/embedded/MODULE.bazel

+# rules_swift_package_manager currently pulls in rules_go v0.57.0, which has
+# this issue: https://github.com/bazel-contrib/rules_go/issues/4480
+# Which causes bazel mod to error out on bazel 8.5.0
+# When rules_swift_package_manager has upgrade their go version, remove this line.


There should now be a release with this fix

luispadron · 2026-02-04T18:42:05Z

swift/internal/extensions/BUILD.bazel.tpl

+    swift_driver = "usr/bin/swiftc",
+    swift_autolink_extract = "usr/bin/swift-autolink-extract",
+    swift_symbolgraph_extract = "usr/bin/swift-symbolgraph-extract",
+    additional_linker_inputs = glob(["usr/lib/swift/**"]),


Do we know what this means for RBE/sandbox? That dir is half a gig when using Xcode

Yes, it's pretty big. It's 2GB in the 6.2.3 Xcode package.
I don't think sandbox will be impacted significantly, because my understanding is that sandbox performance is impacted by the amount of files, not so much by their size. There are about 2000 files in that directory, which is quite a bit but in the grand scheme of things it's in line with what goes in lots of bazel actions in general.
The impact on RBE could be sizable, yes. It'll depend on whether the RBE implementation is able to lazy-load the files it needs during the action (recent versions of Buildbarn do that IIRC); if it doesn't, then it means 2GB will need to be downloaded for the action. The swift front-end is 500MB already; so it would be on top of that.

The problem here that I'm not convinced the alternatives are any better. One obvious way to optimize that would be to select on the platform and only include the directories relevant to that platform. But the platform directories only represent a fraction of the size, and we would probably still need to include the rest. So it would be small gains, while taking the risk of breaking the toolchain. I'm not sure the benefits outweigh the cost.

I'm open to suggestions though.

When using `swift_executable` option to swift_toolchain(), individual swift tools will be called using --driver-mode. This works when switching between swift and swiftc, but tools such as swift-autolink-extract and swift-symbolgraph-extract are separate and aren't recognized by --driver-mode. Therefore, these actions consistently fail in that scenario. As we're trying to add standalone hermetic swift toolchain support, we must have a way to include these tools in actions' input roots, so we're replacing the `swift_executable` toolchain parameter with a `swift_tools` one, which can be used to specify what binary to use for each particular action, as bazel labels. `swift_executable` is still available for backward compatibility, but it will only used if `swift_tools` is not set.

The file_prefix_map feature only works when DEVELOPER_DIR is set, which is only available in an Xcode toolchain. As we're removing the Apple macros in the bazel substitution code, we need to make sure unexpected variables don't show-up in configuration where they're not supported. We're doing so through the introduction of a new private feature called `swift._supports_developer_dir`, that will be enabled only on Xcode toolchains. Individual ActionConfigInfo can then check that feature if they use the __BAZEL_XCODE_DEVELOPER_DIR__ string.

The substitution logic is currently different on MacOS and on Linux. This is a problem as we're trying to add support for a standalone downloadable toolchain, so we're generalizing the logic here through the following changes: * __BAZEL_XCODE_DEVELOPER_DIR__ and __BAZEL_XCODE_SDKROOT__ only make sense in an Xcode context. The rules should make sure they are only used when, respectively, DEVELOPER_DIR and SDKROOT are required, and therefore if these are requested on a standalone toolchain, we would probably want to error out. So we're removing the __APPLE__ macro check, and instead error out if they are called and we are not able to perform the resolution. This is, arguable, a better user experience than silently returning the empty string, as it gives the user the opportunity to adjust toolchain features accordingly. * __BAZEL_SWIFT_TOOLCHAIN_PATH__ should be usable both in Xcode and in a standalone toolchain context. Therefore, we're introducing a new TOOLCHAIN_PATH variable, that will take precedence over the xcrun resolution if it is set. And if it is not, and that xcrun is not available on the system (Linux case), we will also let the user know something went wrong by erroring out. This variable will be set automatically if the user chooses to use swiftc_executable in its swift_toolchain() definition.

This extension can be used to download and declare a precompiled hermetic toolchain (same toolchain as what swiftly would download). It has been tested on both Linux and MacOS. Hermetic toolchains are provided through the `toolchain` extension tag (swift.toolchain()), and allow for selecting the toolchain version. Currently, only 6.2.1 is supported, but more can be added as we go. Example usage: ``` swift = use_extension("@rules_swift//swift:extensions.bzl", "swift") swift.toolchain( name = "swift_toolchain", swift_version = "6.2.1", ) use_repo( swift, "swift_toolchain", "swift_toolchain_ubuntu24.04-aarch64", "swift_toolchain_xcode", ) register_toolchains( "@swift_toolchain//:cc_toolchain_embedded_ubuntu24.04", "@swift_toolchain//:cc_toolchain_embedded_ubuntu24.04-aarch64", "@swift_toolchain//:cc_toolchain_embedded_xcode", "@swift_toolchain//:swift_toolchain_embedded_ubuntu24.04", "@swift_toolchain//:swift_toolchain_embedded_ubuntu24.04-aarch64", "@swift_toolchain//:swift_toolchain_embedded_xcode", "@swift_toolchain//:swift_toolchain_exec_ubuntu24.04", "@swift_toolchain//:swift_toolchain_exec_ubuntu24.04-aarch64", ) ```

This will also serve as a test to make sure swift-embedded works properly, along with the standalone toolchain extension. The example is derived from the swift embedded example project here: https://github.com/swiftlang/swift-embedded-examples To try it, do the following: $ cd examples/toolchain/embedded $ bazelisk build --platforms=//:aarch64 //rpi-4b-blink:Application.bin Note that at this stage, because this embedded code is making use of SPM, and because SPM bazel rules currently tries to run `swift package update` in its repository rules, having a swift executable in the PATH is still a requirement.

Generating the toolchain dictionary currently requires downloading toolchains one by one and calculating their checksum, which is tedious and time consuming. We're adding a utility here that will do that automatically. The tool supports caching downloaded archives and can fetch release information from swift.org's API to generate the checksums dictionary format used by the build system. While we're at it, we're running that utility for the 2 latest releases (6.2.2 and 6.2.3) and adding them to the dict. Because that dictionary is probably gonna keep growing over time, we're also moving it in its own standalone file (swift_releases.bzl).

adincebic · 2026-03-03T15:46:44Z

Im in favor of merging this, thanks for all the work here! I do think we should figure out the swift-releases question. I'm concerned that we'd need to make more rules_swift releases.

Once we figure that out (could be fine to make more releases to maintain the file, id like some more thoughts there). Also, this would land with the upcoming 4.x release unless we want to cherry-pick it to 3.x as well.

If I am not mistaken, some other rules are doing the similar, I think rules_rust though that may have changed.

ma-oli requested review from aaronsky, adincebic, brentleyjones, keith, luispadron and mattrobmattrob as code owners December 15, 2025 18:50

ma-oli force-pushed the add-hermetic-toolchains-support branch from f3857bb to 8c8acca Compare December 15, 2025 19:23

ma-oli mentioned this pull request Dec 15, 2025

Add needed features for hermetic builds on linux. #1627

Open

ma-oli force-pushed the add-hermetic-toolchains-support branch 7 times, most recently from b725865 to e8cb1cc Compare December 16, 2025 03:37

aaronsky reviewed Jan 2, 2026

View reviewed changes

ma-oli force-pushed the add-hermetic-toolchains-support branch 2 times, most recently from c5f0925 to adf15b4 Compare January 6, 2026 17:33

skb-oai reviewed Jan 8, 2026

View reviewed changes

aaronsky approved these changes Jan 9, 2026

View reviewed changes

adincebic reviewed Jan 29, 2026

View reviewed changes

luispadron reviewed Feb 2, 2026

View reviewed changes

luispadron reviewed Feb 4, 2026

View reviewed changes

ma-oli added 6 commits February 9, 2026 09:26

ma-oli force-pushed the add-hermetic-toolchains-support branch from adf15b4 to c7a02a7 Compare March 3, 2026 15:26


		non_module_deps = module_extension(implementation = _non_module_deps_impl)

		def _standalone_toolchain_impl(module_ctx):

Conversation

ma-oli commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Usage

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

skb-oai Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aaronsky left a comment

Choose a reason for hiding this comment

Uh oh!

adincebic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luispadron left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adincebic commented Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ma-oli commented Dec 15, 2025 •

edited

Loading

skb-oai Jan 8, 2026 •

edited

Loading