Implement workspace build separation by poly2it · Pull Request #82 · nix-community/bun2nix

poly2it · 2026-02-11T23:59:00Z

Right now, building a workspace member means pulling in the entire workspace root. Every member shares one derivation, so touching packages/lib rebuilds packages/app too. This PR makes it possible to build each member on its own, with its own derivation and minimal closure.

The new mkDerivation parameters for the extended workspace support are all optional. Existing derivations that don't use them should be unaffected. That said, we recognise you the maintainers may want to reshape the API.

This PR also creates a Rust tool, module-populator, which prepopulates node_modules without running bun. It copies packages straight out of the pre-fetched Nix cache instead. bun install still runs afterward, but only to write Bun's internal metadata. This change was largely motivated by trying to solve an issue we were hitting internally that looks very similar to #71, bun install trying to fetch deps over the network even though the cache already has them. By populating node_modules ourselves before bun install runs, bun no longer needs to resolve anything from the network. This PR resolves those issues for us, but we haven't tested it against the exact reproduction in #71 yet.

A per-member build does roughly this:

The hook copies the workspace-root bun.lock into the member's build dir and strips sibling workspace deps from package.json.
module-populator walks bun.lock, finds each package in the Bun cache using the @@@ marker entries, and copies it into node_modules/.
workspace-promoter rewrites bun.lock to make the target member look like the root package. It collects dependencies from stripped siblings and merges them into the promoted root so transitive resolution still works.
bun install --ignore-scripts runs to reconcile Bun's internal state.
Workspace sibling deps get copied in from their Nix store paths, and a node_modules symlink is placed at the source root so bundlers can resolve imports from sibling code.

This made bun2nix work better for our internal usage in a monorepo at @skeptiva. The workspace-member example roughly emulates our usage.

baileylu121

This is great work, since it's a rather large PR I'm planning to have this be version 2.1.0 once it's ready, bear with me if it takes a couple iterations, but me and I'm sure many others appreciate it a lot.

baileylu121 · 2026-02-12T10:12:50Z

nix/mk-derivation/hook.sh


+  # Copy workspace root lockfile into the build directory if provided.
+  if [ -n "${bunLockFile-}" ]; then
+    echo "Copying workspace lockfile into build directory..."


This (and the latter echo) should probably be a proper setup hook log:

You can see the ones provided by nixpkgs for any setup hook here

baileylu121 · 2026-02-12T10:21:28Z

nix/mk-derivation/hook.sh

+  fi
+
+  echo "Constructing node_modules from cache (no network)..."
+  @modulePopulator@ --cache-dir "$BUN_INSTALL_CACHE_DIR"


Generally, I do prefer the approach of building node_modules ourself, in fact, the original v1 did it this way. The issue with it, however is there are a bunch of hard to find edge cases that crop up from time to time.

I think the best compromise between the two is to catch any exit codes from the module populator and fall back to the current implementation since the union of the two is likely to be able to handle pretty much anything.

Perhaps even a manual configuration flag in mkDerivation would be ideal to allow users to chose between the two.

Maybe we should use module-populator by default, but provide an option like useLegacyCachePopulator or useBunForCachePopulation, which could be automatically suggested in the error message in module-populator fails.

bun2nix attempted to populate the module cache without calling to Bun but failed. Consider filing an issue on [...] and use [option] to enable the fallback populator.

baileylu121 · 2026-02-12T10:24:58Z

nix/mk-derivation/module-populator.nix

+        src = ../../programs;
+
+        cargoLock = {
+          lockFile = ../../programs/Cargo.lock;


Instead of manually specifying the src path twice this should probably just be ${finalAttrs.src}/Cargo.lock.

Same for workspace-promoter

baileylu121 · 2026-02-12T10:34:18Z

nix/mk-derivation.nix

            ];
+          }
+          // lib.optionalAttrs (bunLockFile != null) {
+            bunLockFile = "${bunLockFile}";


If the goal of stringifying this was to force the path to get copied into the store, you can use pkgs.copyPathToStore instead

baileylu121 · 2026-02-12T10:39:44Z

programs/bun2nix/src/error.rs

-Consider updating your local package or contributing to `bun2nix` if this version hasn't been supported yet"
-    )]
-    UnsupportedLockfileVersion(u8),
+    #[error(transparent)]


Seeing as the majority of the custom errors we need have been moved to the library, I'd be totally down with dropping this and using something like color-eyre for the command line utils, but it's something I can follow on with myself if you don't want to include it as part of this PR.

baileylu121 · 2026-02-12T10:51:17Z

programs/module-populator/src/lockfile.rs

+}
+
+/// Parse the metadata object from a package tuple element.
+fn parse_meta(value: &Value) -> Option<PackageMeta> {


This can largely be done with serde's derive macros by just specifying a custom deserialize for the BinField type.

https://serde.rs/impl-deserialize.html

baileylu121 · 2026-02-12T10:52:43Z

programs/module-populator/src/main.rs

+
+    let lock_contents = fs::read_to_string(&cli.lock_file)?;
+
+    println!("Reading lockfile: {}", cli.lock_file.display());


These entries should probably be produced by log, since we already depend on it anyway

baileylu121 · 2026-02-12T10:55:14Z

programs/module-populator/src/main.rs

+
+    for pkg in &packages {
+        match &pkg.kind {
+            PackageKind::Workspace { path } => {


This function feels rather long, this is more of a nitpick/style thing but I'd rather see it broken up into smaller pieces

baileylu121 · 2026-02-12T10:55:50Z

programs/module-populator/src/node_modules.rs

+/// Validate that a bin command name is safe.
+///
+/// Rejects names containing `/`, `\`, null bytes, `.`, `..`, or empty strings.
+fn validate_bin_name(name: &str) {


As with above this should probably say in the function name it does a bunch of assertions or just return a Result

baileylu121 · 2026-02-12T10:57:11Z

programs/module-populator/src/node_modules.rs

+///
+/// Uses `cp -rL` to dereference symlinks and `--no-preserve=mode,ownership`
+/// to ensure files are writable (Nix store files are read-only).
+pub fn copy_package(cache_path: &Path, target_path: &Path) {


If we're going to do a full copy anyway they should probably be reflinks and/or hardlinks just in case the filesystem supports that.

You can see how bun does it here.

We needed to use: nix-community/bun2nix#82 To workaround: nix-community/bun2nix#77 A bit hacky but it works.

poly2it added 6 commits February 7, 2026 13:55

Populate modules without Bun

0589941

Add workspace member isolation support

52fae09

Fix workspace-promoter re-adding stripped deps

3216cb1

Use JSONC format for promoted lockfile

fe27a26

Symlink node_modules at source root for workspace deps

c599df2

Format

dcbbbdd

baileylu121 requested changes Feb 12, 2026

View reviewed changes

Update Bun and bump flake

8aeaabe

arximboldi added a commit to darkmatter/stackpanel that referenced this pull request Mar 2, 2026

Build apps/web as a derivation

ed8d429

We needed to use: nix-community/bun2nix#82 To workaround: nix-community/bun2nix#77 A bit hacky but it works.

arximboldi added a commit to darkmatter/stackpanel that referenced this pull request Mar 2, 2026

Build apps/web as a derivation

3dc05ec

We needed to use: nix-community/bun2nix#82 To workaround: nix-community/bun2nix#77 A bit hacky but it works.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement workspace build separation#82

Implement workspace build separation#82
poly2it wants to merge 7 commits intonix-community:masterfrom
poly2it:module-populator

poly2it commented Feb 11, 2026

Uh oh!

baileylu121 left a comment

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

poly2it Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

baileylu121 Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		let lock_contents = fs::read_to_string(&cli.lock_file)?;

		println!("Reading lockfile: {}", cli.lock_file.display());

Uh oh!

Conversation

poly2it commented Feb 11, 2026

Uh oh!

baileylu121 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants