Calling "node" with the "--preserve-symlinks" option breaks PNPM monorepo symbolic link resolution

[eaf-carter]

encore/tsparser/src/builder/transpiler.rs

Line 114 in 515e734

"--preserve-symlinks".into(),

If the above line is responsible for adding the "--preserve-symlinks" option to the calling of node then it will break and/or conflict with PNPM's dependency management.

PNPM's dependency management

Generally in a monorepo managed by PNPM with multiple packages; packages will share a "virtual store" to resolve their individual dependencies. You can visualize it by the following directory structure.

/my-monorepo
├── node_modules/                # Main node_modules for the workspace
│   ├── .pnpm/                   # pnpm virtual store directory
│   │   ├── package-x@version/     # Virtual store packages with specific versions
│   │   ├── package-y@version/    
│   │   └── ...                  # Other packages
│   └── ...                      # Shared dependencies
├── packages/                    # Contains individual packages
│   ├── package-a/               # First package
│   │   ├── node_modules/        # Local node_modules for package-a
│   │   │   ├── .pnpm/           # Virtual store for package-a
│   │   │   └── package-x@symlink/ # Symlink to package-x in the virtual store
│   │   └── package.json         # package.json for package-a
│   └── ...                      # Other packages
├── pnpm-workspace.yaml          # pnpm workspace configuration file
├── package.json                 # Root package.json for the monorepo
└── pnpm-lock.yaml               # Lock file generated by pnpm

How this normally works

In the structure above /packages/package-a has the following dependency "package-x" which is just a symlink to it in the virtual store.

If package-x had the following code

const pkgy = require('package-y')

It would be resolved since it is located relative to it in the virtual store.

Why --preserve-symlinks breaks this

According to the doc regarding this option

The --preserve-symlinks command-line flag instructs Node.js to use the symlink path for modules as opposed to the real path...

This is an issue because the real path is where the dependencies exist including whatever necessary peer dependencies.

It results in the following issue #1458

This behavior is likely to accommodate traditional system linking done with NPM but is incompatible with PNPM monorepos.

Possible Solution

Do not apply --perserve-symlinks option if pnpm is present in the packageManger field of the package.json

[eandre]

Hmm, that's a bit annoying. I can't recall why we added --preserve-symlinks in the first place. It would be nice to investigate whether it's still needed.

[eaf-carter]

a bit annoying. I can't recall why we add

I feel like it's probably to preserve symlinking done with NPM or Yarn since since with their flat package management it would work fine. In the case of PNPM when the app bundle is generated and executed (locally w/ encore run) within the project directory it needs to be able to reference the real path since it's a directory up in the "virtual store". This gets even wackier when it comes to deploying but that's a whole other can of worms. I've managed to get everything working locally with the "public-hoist-pattern" set to to "*" but that negates the intent behind using something like PNPM.