Make `rustworkx` build and run with pyiodide #1447

IvanIsCoding · 2025-05-10T15:12:51Z

Closes #703

This adds support for Pyodide. We list Pyodie in a brand new tier that is Tier Experimental. Currently, we build with pyoide-build==0.30.2 using the following flags:

export "RUSTFLAGS="-C target-feature=+atomics,+bulk-memory,+mutable-globals -C link-arg=-sSIDE_MODULE=2 -C link-arg=-sWASM_BIGINT -Z emscripten-wasm-eh"
pyodide build

This requires a Rust with nightly-2025-02-01 toolchain and emsdk 4.0.8. I tested with Pyodide 0.28.1a which maps to Python 3.13. Because we depend on so much unstable stuff, this is not ready yet for our CI. I need to figure out a way of adding it.

Related: pyodide/pyodide-recipes#90

coveralls · 2025-05-10T15:26:13Z

Pull Request Test Coverage Report for Build 16921538859

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 94.642%

Totals
Change from base Build 16916606073:	0.0%
Covered Lines:	17805
Relevant Lines:	18813

💛 - Coveralls

IvanIsCoding · 2025-05-10T15:38:39Z

So I tried following: https://pyodide.org/en/stable/usage/building-and-testing-packages.html

I had to install emsdk 4.0.8., pyodide-build==0.30.1, rustc 1.88.0-nightly (00095b3da 2025-04-03)

I got this error: https://pastebin.com/TaT892sU. So yeah, I'd like to keep this build outside of our repository but at least add the code such that when emscripten-forge builds we run.

IvanIsCoding · 2025-05-10T15:39:44Z

@sorin-bolos how did you setup the SDK to build the version from your PR? I might need some help here

IvanIsCoding · 2025-05-10T21:37:59Z

Maybe I should try using the version pinned here: https://github.com/pyodide/pyodide-build/blob/7126c5f1bd56ed7f40ce29cbcbe588267a42434c/pyodide_build/config.py#L240C24-L240C42

IvanIsCoding · 2025-05-10T21:50:32Z

Somehow I got Successfully built /home/ivan/Projects/rustworkx-dev/dist/rustworkx-0.17.0-cp39-abi3-emscripten_4_0_6_wasm32.whl, but only when running with RUSTFLAGS="-v" pyodide build. I will test the wheel.

IvanIsCoding · 2025-05-10T21:57:59Z

Well, I can pip install using the pyodide environment but a simple import rustworkx crashes:

(.venv-pyodide) ➜  tests git:(more-uv-features) python
which: no grealpath in (/home/ivan/Projects/.venv-pyodide/bin:/home/ivan/.local/bin:/usr/share/Modules/bin:/home/ivan/.cargo/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/home/ivan/.dotnet/tools:/home/ivan/.dotnet/tools)
Python 3.13.2 (main, Apr  3 2025, 07:34:46) [Clang 21.0.0git (https:/github.com/llvm/llvm-project 4775e6d9099467df9363e1a3cd on emscripten
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'
>>> import rustworkx
Pyodide has suffered a fatal error. Please report this to the Pyodide maintainers.
The cause of the fatal error was:
Error: Dynamic linking error: cannot resolve symbol invoke_ii
    at stubs.<computed> (/home/ivan/.cache/.pyodide-xbuildenv-0.30.2/0.28.0a1/xbuildenv/pyodide-root/dist/pyodide.asm.js:9:19341)
    at wasm://wasm/0122615e:wasm-function[3874]:0x3ca8c6
    at wasm://wasm/020b656e:wasm-function[4078]:0x2a30bc
    at wasm://wasm/020b656e:wasm-function[4054]:0x2a0f12
    at wasm://wasm/020b656e:wasm-function[4070]:0x2a28c5
    at wasm://wasm/020b656e:wasm-function[2181]:0x1ad753
    at wasm://wasm/020b656e:wasm-function[1119]:0x16345c
    at wasm://wasm/020b656e:wasm-function[1121]:0x163668
    at wasm://wasm/020b656e:wasm-function[1122]:0x1636e6
    at wasm://wasm/020b656e:wasm-function[3519]:0x25a95b {
  pyodide_fatal_error: true
}
Stack (most recent call first):
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1320 in create_module
  File "<frozen importlib._bootstrap>", line 813 in module_from_spec
  File "<frozen importlib._bootstrap>", line 921 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "/home/ivan/Projects/.venv-pyodide/lib/python3.13/site-packages/rustworkx/__init__.py", line 14 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1026 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "<stdin>-0", line 1 in <module>
  File "/lib/python313.zip/_pyrepl/main.py", line 30 in interactive_console
  File "/lib/python313.zip/_pyrepl/__main__.py", line 6 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
Error: Dynamic linking error: cannot resolve symbol invoke_ii
    at stubs.<computed> (/home/ivan/.cache/.pyodide-xbuildenv-0.30.2/0.28.0a1/xbuildenv/pyodide-root/dist/pyodide.asm.js:9:19341)
    at wasm://wasm/0122615e:wasm-function[3874]:0x3ca8c6
    at wasm://wasm/020b656e:wasm-function[4078]:0x2a30bc
    at wasm://wasm/020b656e:wasm-function[4054]:0x2a0f12
    at wasm://wasm/020b656e:wasm-function[4070]:0x2a28c5
    at wasm://wasm/020b656e:wasm-function[2181]:0x1ad753
    at wasm://wasm/020b656e:wasm-function[1119]:0x16345c
    at wasm://wasm/020b656e:wasm-function[1121]:0x163668
    at wasm://wasm/020b656e:wasm-function[1122]:0x1636e6
    at wasm://wasm/020b656e:wasm-function[3519]:0x25a95b {
  pyodide_fatal_error: true
}

IvanIsCoding · 2025-05-10T22:30:33Z

I think setting verbose on the previous comment erased the flags from pyodide build so we are back at step one.

IvanIsCoding · 2025-05-11T03:21:52Z

Ok, we definetely build. Any code that calls Rayon fails, but:

https://gist.github.com/IvanIsCoding/8bb97d428169e073af4afa598e1d2932 has the code of a index.html page that calls Floyd-Warshall
Download index.html from the gist and https://github.com/IvanIsCoding/rustworkx/releases/download/v0.17.0-alpha-pyodide/rustworkx-0.17.0-cp39-abi3-pyodide_2025_0_wasm32.whl, put the files in the same folder
Run python -m http.server --directory . in the same folder to serve a very simple HTTP server
Open http://localhost:8080

It works! The issues inside the pyodide env seem to only affect Node.js, inside a browser it works like a charm:

IvanIsCoding · 2025-05-11T04:32:40Z

After telling the nightly compiler to rebuild std, I built https://github.com/IvanIsCoding/rustworkx/releases/download/v0.17.0-alpha2-pyodide/rustworkx-0.17.0-cp39-abi3-pyodide_2025_0_wasm32.whl which seems to be working.

I will try to see if I figure out how to run our unit tests to confirm things work. But this is shockingly good.

IvanIsCoding · 2025-05-11T15:56:59Z

.cargo/config.toml

 ]
+
+[target.wasm32-unknown-emscripten]
+rustflags = [


Note: it seems like pyodide-build overrides these, so I will open an issue to talk to them and check for alternatives. In the meantime, this is helpful for reference and when we compile outside pyodide-build

It also is useful for rustworkx-core and using rustworkx without python in wasm.

IvanIsCoding · 2025-05-11T16:21:35Z

I have no clue how to include this on CI or even if we should, but there more I test the more things seem to work. I will open an issue on the pyodide-build repository and make a recipe on the pyodide-recipe repository.

mtreinish

Overall this looks fine to me, Just one nit inline. I'd like to try to find time soon to pull this locally and test out wasm and pyodide before approving though just to validate all the new options.

mtreinish · 2025-05-29T11:23:16Z

.cargo/config.toml

 ]
+
+[target.wasm32-unknown-emscripten]
+rustflags = [


It also is useful for rustworkx-core and using rustworkx without python in wasm.

releasenotes/notes/pyodide-support-2172f2313b06f10d.yaml

IvanIsCoding · 2025-05-29T12:45:25Z

Overall this looks fine to me, Just one nit inline. I'd like to try to find time soon to pull this locally and test out wasm and pyodide before approving though just to validate all the new options.

Some minor notes about Rust WASM. There are three Rust targets! We could support more, but here are the caveats:

wasm32-unknown-unknown: easiest target to build for rustworkx-core. impossible to build rustworkx because of Python. cargo doesn't have a default runner for this so it's hard to test.
wasm32-unknown-emscripten: used by Pyodide. emscripten goes from LLVM -> WASM from what I understood, so there needs to be some coordination between emscripten and rustc versions. I higly recommend using Pixi from Add tests for Pyodide build with Pixi and Node.js #1450 to build it
wasm32-wasip1: in theory Python supports this, but I have yet to see someone building and publishing for WASI

After #1450 lands, I think we could test wasm32-unknown-unknown for rustworkx-core with https://rustwasm.github.io/wasm-pack/

mtreinish

I was able to compile a wasm module using this, there are definitely some rough edges still but it's a starting point and explicitly experimental so I think that's fine. I just had one question inline

mtreinish · 2025-08-12T13:57:06Z

.cargo/config.toml

+  "-C", "target-feature=+atomics,+bulk-memory,+mutable-globals",
+  "-C", "link-arg=-sSIDE_MODULE=2",
+  "-C", "link-arg=-sWASM_BIGINT",
+  "-Z", "emscripten-wasm-eh",


This is a nightly option, is it needed, or do we only support building wasm on nightly rust? I was able to build with the latest stable if I exclude this option

They use nightly for pyodide-build, see https://pyodide.org/en/stable/development/abi.html#id2

I think pyodide will be stuck with nightly for a while. That’s why I am so reticent to say we support it. I pinned the nightly version in the Pixi CL exactly for that reason

Well I guess my concern here is that having this in the .cargo/config.toml with a -Z flag prevents us from building rustworkx-core against the emscripten target without pyodide. My preference would be to remove it from the default config and inject the flag when it's needed during a pyodide build.

I will update the PR. Pixi should always get those flags anyway, we use the pyodide-build CLI to retrieve the flags

Pyodide: force only one thread

7bdeaff

IvanIsCoding added 2 commits May 10, 2025 23:24

Add .cargo/config.toml

7ce8078

Unstable feature that makes rayon work

b440e6b

IvanIsCoding added 2 commits May 11, 2025 11:45

Newline

e70b263

Merge remote-tracking branch 'upstream/main' into pyodide-rayon

bf15e3b

IvanIsCoding commented May 11, 2025

View reviewed changes

IvanIsCoding added 2 commits May 11, 2025 12:12

Document pyodide

76f5dff

Add release note for pyodide

4b17893

IvanIsCoding marked this pull request as ready for review May 11, 2025 16:19

IvanIsCoding changed the title ~~[WIP] Make rustworkx build and run with pyiodide~~ Make rustworkx build and run with pyiodide May 11, 2025

IvanIsCoding added the on hold label May 11, 2025

This was referenced May 11, 2025

Add rustworkx alpha recipe pyodide/pyodide-recipes#90

Merged

Wheel built with pyodide-build works on the browser but fails in Python from pyodide venv pyodide/pyodide-build#201

Closed

IvanIsCoding added 3 commits May 11, 2025 16:01

Fix separator

0e3b0ce

Consolidate the bare minimum of flags

3527a6f

Add emscripten-wasm-eh flag

72ceb1a

IvanIsCoding removed the on hold label May 13, 2025

IvanIsCoding requested a review from mtreinish May 13, 2025 04:03

This was referenced May 13, 2025

Add Pixi workflow to verify Pyodide build works #1449

Open

Add tests for Pyodide build with Pixi and Node.js #1450

Open

Merge branch 'main' into pyodide-rayon

826fbe7

IvanIsCoding added this to the 0.17.0 milestone May 17, 2025

IvanIsCoding mentioned this pull request May 17, 2025

Add rustworkx alpha emscripten-forge/recipes#2327

Merged

mtreinish reviewed May 29, 2025

View reviewed changes

IvanIsCoding and others added 2 commits May 29, 2025 08:21

Merge branch 'main' into pyodide-rayon

b6fcd9a

Solve nit

aa34512

IvanIsCoding added 6 commits May 29, 2025 18:23

Merge branch 'main' into pyodide-rayon

1a04b00

Merge branch 'main' into pyodide-rayon

5378a9e

Merge branch 'main' into pyodide-rayon

22af7ad

Merge branch 'main' into pyodide-rayon

d0ebc78

Merge branch 'main' into pyodide-rayon

e2f12bc

Merge branch 'main' into pyodide-rayon

ba93544

This was referenced Jul 4, 2025

Release request #1476

Closed

Prepare 0.17.0 release #1479

Merged

mtreinish approved these changes Aug 12, 2025

View reviewed changes

IvanIsCoding and others added 2 commits August 12, 2025 14:46

Merge branch 'main' into pyodide-rayon

55c7415

Remove flags forcing emscripten builds from requiring nightly

533978e

IvanIsCoding enabled auto-merge August 12, 2025 21:39

IvanIsCoding added this pull request to the merge queue Aug 12, 2025

Merged via the queue into Qiskit:main with commit 4c69907 Aug 12, 2025
36 checks passed

IvanIsCoding deleted the pyodide-rayon branch August 12, 2025 22:15

Make rustworkx build and run with pyiodide #1447

Make rustworkx build and run with pyiodide #1447

Uh oh!

Conversation

IvanIsCoding commented May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 16921538859

Details

💛 - Coveralls

Uh oh!

IvanIsCoding commented May 10, 2025

Uh oh!

IvanIsCoding commented May 10, 2025

Uh oh!

IvanIsCoding commented May 10, 2025

Uh oh!

IvanIsCoding commented May 10, 2025

Uh oh!

IvanIsCoding commented May 10, 2025

Uh oh!

IvanIsCoding commented May 10, 2025

Uh oh!

IvanIsCoding commented May 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

IvanIsCoding commented May 11, 2025

Uh oh!

IvanIsCoding May 11, 2025

Choose a reason for hiding this comment

Uh oh!

mtreinish May 29, 2025

Choose a reason for hiding this comment

Uh oh!

IvanIsCoding commented May 11, 2025

Uh oh!

mtreinish left a comment

Choose a reason for hiding this comment

Uh oh!

mtreinish May 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

IvanIsCoding commented May 29, 2025

Uh oh!

mtreinish left a comment

Choose a reason for hiding this comment

Uh oh!

mtreinish Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

IvanIsCoding Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

mtreinish Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

IvanIsCoding Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Make `rustworkx` build and run with pyiodide #1447

Make `rustworkx` build and run with pyiodide #1447

IvanIsCoding commented May 10, 2025 •

edited

Loading

coveralls commented May 10, 2025 •

edited

Loading

IvanIsCoding commented May 11, 2025 •

edited

Loading