Skip to content

Eliminate createRequire/require from EXPORT_ES6 output#26384

Open
vogel76 wants to merge 2 commits intoemscripten-core:mainfrom
vogel76:bw_eliminate_require
Open

Eliminate createRequire/require from EXPORT_ES6 output#26384
vogel76 wants to merge 2 commits intoemscripten-core:mainfrom
vogel76:bw_eliminate_require

Conversation

@vogel76
Copy link

@vogel76 vogel76 commented Mar 3, 2026

When building with -sEXPORT_ES6, the generated JavaScript currently uses createRequire(import.meta.url) to polyfill require(), then calls require() for Node.js built-in modules. This breaks bundlers (webpack, Rollup, esbuild, Vite) and Electron's renderer process, forcing users to post-process emscripten output with sed or custom plugins to strip out the offending calls.

This PR replaces all require() calls with await import() when EXPORT_ES6 is enabled, eliminating the need for createRequire entirely.

Why this is safe

  • EXPORT_ES6 requires MODULARIZE (enforced in tools/link.py). MODULARIZE wraps the module body in an async function, so await is valid at the top level of the generated code.
  • The preprocessor already supports this pattern. await import(...) is transformed to an EMSCRIPTEN$AWAIT$IMPORT placeholder during preprocessing (parseTools.mjs:83) and restored during linking (link.py:2136). This mechanism is already used elsewhere in the codebase.
  • Node.js built-in modules expose identical ESM interfaces. (await import('node:fs')).readFileSync works identically to require('node:fs').readFileSync. For CJS npm packages like ws, .default gives the module.exports value.
  • No behavioral change for non-ES6 builds. All changes are guarded by #if EXPORT_ES6 / #else preprocessor conditionals. The existing require() code path is untouched.

Approach

The changes split naturally into two categories:

Shell and runtime files (top-level async context where await works directly):

  • src/shell.js — Remove the createRequire block. Use await import() for worker_threads, fs, path, url, util.
  • src/shell_minimal.js — Same pattern for worker_threads and fs. Replace __dirname with new URL(..., import.meta.url) for wasm file loading.
  • src/runtime_debug.js — Skip local require() for fs/util when EXPORT_ES6; reuse outer-scope variables from shell.js.
  • src/runtime_common.js — Guard perf_hooks require() with EXPORT_ES6 alternative.
  • src/preamble.js — Hoist await import('node:v8') above instantiateSync() for NODE_CODE_CACHING (can't use await inside a sync function).

Library files (synchronous function bodies where await is unavailable):

Library functions execute in synchronous context, so await import() cannot be called inline. Instead, we define library symbols initialized at module top level (async context) and reference them via __deps:

Symbol Module Used by
$nodeOs node:os libatomic.js, libwasm_worker.js
$nodeCrypto node:crypto libwasi.js ($initRandomFill)
$nodeChildProcess node:child_process libcore.js (_emscripten_system)
$nodeWs ws (.default) libsockfs.js (WebSocket connect + listen)
$nodePath node:path libnodepath.js (NODERAWFS)

Shared symbols live in a new src/lib/libnode_imports.js, registered in src/modules.mjs when EXPORT_ES6 is enabled.

Intentionally skipped

  • src/lib/libembind_gen.js — Build-time code running in Node.js directly, not in emitted runtime JS.
  • src/closure-externs/closure-externs.jscreateRequire extern kept; still needed for the non-ES6 path.

What changes in the output

Before (EXPORT_ES6):

import { createRequire } from 'node:module';
var require = createRequire(import.meta.url);
var fs = require('node:fs');

After (EXPORT_ES6):

var fs = await import('node:fs');

The non-ES6 path remains unchanged:

var fs = require('node:fs');

Files modified

13 files modified, 1 new file — 123 insertions, 12 deletions (source files only).

Testing

All existing ESM tests pass:

Test Result
test_esm (default, node, pthreads variants) PASS
test_esm_worker (default, node variants) PASS
test_esm_worker_single_file PASS
test_esm_closure PASS
test_esm_implies_modularize PASS
test_esm_requires_modularize PASS
test_pthread_export_es6 (default, trusted variants) PASS

Additionally verified:

  • emcc -sEXPORT_ES6 -sMODULARIZE output contains zero createRequire or require( occurrences
  • emcc -sEXPORT_ES6 -sMODULARIZE -pthread output contains zero createRequire or require( occurrences
  • Non-ES6 builds still use require() as before (no regression)

Related issues

Related PRs

@vogel76 vogel76 changed the title eliminate require Eliminate createRequire/require() from EXPORT_ES6 output Mar 3, 2026
Copy link
Collaborator

@sbc100 sbc100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, thanks for working on this! Nice work.

if (ENVIRONMENT_IS_NODE) {
#if EXPORT_ES6
return (view) => nodeCrypto.randomFillSync(view);
#else
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep thing simpler could we just always use the nodeCrypto global via __deps? (i.e. make the difference only in how nodeCrypto is globally defined?

#if EXPORT_ES6 && ENVIRONMENT_MAY_BE_NODE
addToLibrary({
$nodeOs: "ENVIRONMENT_IS_NODE ? await import('node:os') : undefined",
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of creating and entirely new file for this one entry maybe just tack this onto libcore.js?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I will consider this change

#if EXPORT_ES6
var fs = await import('node:fs');
#else
var fs = require('node:fs');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can make a helper macro here and do something like:

var fs = {{{ makeNodeImport('node:fs') }}};

Then them be could change if/when we use await import easily later. For example, we maybe want to use await import in all MODULARIZE use cases, not just EXPORT_ES6?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Module['wasm'] = fs.readFileSync(new URL('{{{ TARGET_BASENAME }}}.wasm', import.meta.url));
#endif
#endif
#else
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a lot of extra complexity/duplication. Is there some way to avoid it maybe?

@sbc100 sbc100 requested a review from RReverser March 3, 2026 23:34
@sbc100 sbc100 changed the title Eliminate createRequire/require() from EXPORT_ES6 output Eliminate createRequire/require from EXPORT_ES6 output Mar 4, 2026
@vogel76 vogel76 force-pushed the bw_eliminate_require branch from 0ec991e to 6b74d2a Compare March 4, 2026 23:17
@vogel76
Copy link
Author

vogel76 commented Mar 4, 2026

@sbc100 I hope your remarks have been addressed in added fixup commits. Once you accept it, I will do autosquash rebase to leave only actual commits in clean history.

@vogel76 vogel76 force-pushed the bw_eliminate_require branch from 6b74d2a to 8f585b7 Compare March 4, 2026 23:31
@sbc100
Copy link
Collaborator

sbc100 commented Mar 5, 2026

Is it possible to write a test for this?

When you say "This breaks bundlers", do you know why our current bundler tests in test_browser.py are working? (see test_rollup and test_vite). Would you be able to write a failing test for this breakage?

@sbc100
Copy link
Collaborator

sbc100 commented Mar 5, 2026

Re-commiting, we always squash all changes in the emscripten repo.

If you think this change can be split into to commits then please send a two separate PRs. This is just how we do things in emscripten. It helps keep the tests passing on every commit (for the benefit of bisectors).


var cp = require('node:child_process');
var cp = nodeChildProcess;
var ret = cp.spawnSync(cmdstr, [], {shell:true, stdio:'inherit'});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can just drop the cp variable here and use nodeChildProcess.spawnSync?

src/preamble.js Outdated
#endif

#if NODE_CODE_CACHING && ENVIRONMENT_MAY_BE_NODE && EXPORT_ES6
var nodeV8 = ENVIRONMENT_IS_NODE ? await import('node:v8') : undefined;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use makeNodeImport here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I will deeper review await import usage to encapsulate it.

src/preamble.js Outdated
var v8 = nodeV8;
#else
var v8 = require('node:v8');
#endif
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for this var anymore if you use makeNodeImport I thinik?

I think you for this globalVar its reasonable to call it v8?

#if EXPORT_ES6
global.performance ??= (await import('perf_hooks')).performance;
#else
global.performance ??= require('perf_hooks').performance;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use makeNodeImport here?

var fs = require('node:fs');
var fs = {{{ makeNodeImport('node:fs') }}};
#if WASM == 2
if (globalThis.WebAssembly) Module['wasm'] = fs.readFileSync(__dirname + '/{{{ TARGET_BASENAME }}}.wasm');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this code (which uses __dirname) simply not work with EXPORT_EST today?

If not, this seems like maybe a separate fix that we could land in isolation. e.g. Fix for EXPORT_ES6 + MINIMAL_RUNTIME + ???

@vogel76
Copy link
Author

vogel76 commented Mar 5, 2026

Is it possible to write a test for this?

When you say "This breaks bundlers", do you know why our current bundler tests in test_browser.py are working? (see test_rollup and test_vite). Would you be able to write a failing test for this breakage?

Sure - will add such tests. The sake of such work are maintenance problems of our Hive blockchain interfacing library: wax which compiles blockchain protocol C++ code into wasm and uses it at TS/JS level. We have set of tests related to bundlers and different frameworks where we got problems while loading wasm etc.

https://gitlab.syncad.com/hive/wax/-/tree/develop/examples/ts?ref_type=heads

Bartek Wrona added 2 commits March 5, 2026 14:57
When EXPORT_ES6 is enabled, the generated JS used createRequire() to
polyfill require(), which breaks bundlers (webpack, Rollup, esbuild)
and Electron's renderer process. Since EXPORT_ES6 requires MODULARIZE,
the module body is wrapped in an async function where await is valid.

- shell.js: Remove createRequire block entirely. Use await import()
  for worker_threads, fs, path, url, util. Replace __dirname with
  import.meta.url for path resolution.
- shell_minimal.js: Same pattern for worker_threads and fs. Replace
  __dirname with new URL(..., import.meta.url) for wasm file loading.
- runtime_debug.js: Skip local require() for fs/util when EXPORT_ES6,
  reuse outer-scope variables from shell.js instead.
- runtime_common.js: Guard perf_hooks require() with EXPORT_ES6
  alternative.
- preamble.js: Hoist await import('node:v8') above instantiateSync()
  for NODE_CODE_CACHING since await can't be used inside sync functions.
Library functions run in synchronous context where await is unavailable.
Define top-level library symbols that use await import() at module init
time, then reference them via __deps from synchronous functions.

- Add libnode_imports.js with shared $nodeOs symbol, register in
  modules.mjs when EXPORT_ES6 is enabled.
- libatomic.js, libwasm_worker.js: Use $nodeOs for os.cpus().length
  instead of require('node:os').
- libwasi.js: Define $nodeCrypto for crypto.randomFillSync in
  $initRandomFill. Combine conditional __deps to avoid override.
- libcore.js: Define $nodeChildProcess for _emscripten_system.
- libnodepath.js: Switch $nodePath initializer to await import().
- libsockfs.js: Define $nodeWs ((await import('ws')).default) for
  WebSocket constructor in connect() and Server in listen().
@vogel76 vogel76 force-pushed the bw_eliminate_require branch from 8f585b7 to 6c822be Compare March 5, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants