Update SpiderMonkey to ESR128.0 (#474)

* Update docs/version to ESR128

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Update SpiderMonkey

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Update patches

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Apply all patches except streams

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Add bidi patch

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Apply bidi patch

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Streams patch

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Apply streams patch

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Add Macos min sdk 14.2 patch

https://bugzilla.mozilla.org/show_bug.cgi?id=1887728
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Apply Macos min sdk 14.2 patch

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Moving from 0 based column number to 1 based column number

https://bugzilla.mozilla.org/show_bug.cgi?id=1862692
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Draining stopped in jobqueue

https://bugzilla.mozilla.org/show_bug.cgi?id=1620223
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Latin1 external strings

https://bugzilla.mozilla.org/show_bug.cgi?id=1576076
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Remove offthread stuff

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Regen wrappers

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* fix rust code

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Reapply filter

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Fix icu linking errors

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Fix android

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Update ndk to 26 (per SM)

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Fix a test

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Fix wasm test

the content needs to be aligned of 8

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* fix integrity check

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* generalize alignment wrapper

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* fix wasm

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Manually create static lib on windows, strip debug symbols and add only add lib files in artifact

Co-authored-by: Wu Yu Wei <yuweiwu@pm.me>
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Wrapper for NewExternalArrayBuffer

based on: https://hg.mozilla.org/mozilla-central/rev/4cb083a1f85e11336ec37f496b57f181be524c7e
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* fixup 1-origin

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* filter out zlib

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Use zlib from libz-sys

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

* Patch libz-sys

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>

---------

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Co-authored-by: Wu Yu Wei <yuweiwu@pm.me>

.github/workflows/rust.yml

@@ -137,7 +137,7 @@ jobs:
         uses: nttld/setup-ndk@v1
         id: setup-ndk
         with:
-          ndk-version: r25c
+          ndk-version: r26c
       - name: Install Rust
         uses: dtolnay/rust-toolchain@stable
         with:
@@ -181,7 +181,7 @@ jobs:
           python3 ./mozjs-sys/etc/update.py --no-commit mozjs.tar.xz
           # Run `git add` here to force CRLF converted into LF
           # so that we can check diff properly in next run
-          git add --all mozjs
+          git add --all mozjs-sys
       - name: Check patch integrity
         working-directory: ./mozjs-sys
         # Because we've added files in previous run, we need to
@@ -192,8 +192,7 @@ jobs:
   publish-release:
     name: Check version and publish release
     runs-on: ubuntu-latest
-    needs:
-      ["linux", "mac", "windows"]
+    needs: ["linux", "mac", "windows"]
     if: ${{ !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') }}
     steps:
       - uses: actions/checkout@v4
@@ -225,7 +224,16 @@ jobs:
     name: Result
     runs-on: ubuntu-latest
     needs:
-      ["android", "linux", "linux-cross-compile", "mac", "windows", "integrity", "publish-release", "verify-release"]
+      [
+        "android",
+        "linux",
+        "linux-cross-compile",
+        "mac",
+        "windows",
+        "integrity",
+        "publish-release",
+        "verify-release",
+      ]
     if: ${{ always() }}
     steps:
       - name: Mark the job as successful
@@ -234,4 +242,3 @@ jobs:
       - name: Mark the job as unsuccessful
         if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
         run: exit 1
-

Cargo.toml

@@ -14,3 +14,8 @@ bindgen = { version = "0.69", default-features = false, features = [
     "runtime",
     "which-rustfmt",
 ] }
+
+# patch libz-sys for https://github.com/rust-lang/libz-sys/pull/199
+# until released as 1.1.19
+[patch.crates-io]
+libz-sys = { git = "https://github.com/rust-lang/libz-sys.git", rev = "f76b7a24ed0a71ef1b8bdac1895e8535cb2cb9c0" }

README.md

@@ -6,8 +6,8 @@ that are battle-tested in [Servo](https://servo.org/), split in two crates:
 - `mozjs-sys`:  SpiderMonkey and low-level Rust bindings to its C++ API.
 - `mozjs`: Higher-level bindings to the SpiderMonkey API.
 
-Mozjs is currently tracking SpiderMonkey on [ESR-115](https://searchfox.org/mozilla-esr115/source/) branch
-(currently version 115.13).
+Mozjs is currently tracking SpiderMonkey on [ESR-128](https://searchfox.org/mozilla-esr128/source/) branch
+(currently version 128.0).
 
 ## Building from Pre-built Archive
 
@@ -127,8 +127,8 @@ mozjs = { path = "../mozjs/mozjs" }
 
 In order to upgrade to a new version of SpiderMonkey:
 
-1. Find the mozilla-esr115 commit for the desired version of SpiderMonkey, at
-   <https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr115&filter-searchStr=spidermonkey%20pkg>.
+1. Find the mozilla-esr128 commit for the desired version of SpiderMonkey, at
+   <https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr128&filter-searchStr=spidermonkey%20pkg>.
    You are looking for an SM(pkg) tagged with FIREFOX_RELEASE.
    Take a note of the commit number to the left (a hex number such as ac4fbb7aaca0).
 

android-build

@@ -12,7 +12,7 @@ import sys
 from typing import Dict, Optional
 
 
-SUPPORTED_NDK_VERSION = '25'
+SUPPORTED_NDK_VERSION = '26'
 API_LEVEL = '30'
 
 

mozjs-sys/Cargo.toml

@@ -2,7 +2,7 @@
 name = "mozjs_sys"
 description = "System crate for the Mozilla SpiderMonkey JavaScript engine."
 repository.workspace = true
-version = "0.115.13-2"
+version = "0.128.0-0"
 authors = ["Mozilla"]
 links = "mozjs"
 build = "build.rs"
@@ -29,10 +29,13 @@ jitspew = []
 streams = []
 
 [dependencies]
-encoding_c = "0.9.8"
-encoding_c_mem = "0.2.6"
 libc.workspace = true
+# SM depends on them and we provide them using cargo
 libz-sys = "1.1.13"
+encoding_c = "0.9.8"
+encoding_c_mem = "0.2.6"
+# unicode-bidi-ffi = { path = "./mozjs/intl/bidi/rust/unicode-bidi-ffi" }
+icu_capi = "1.4.0" # keep in sync with intl/icu_capi/Cargo.toml
 
 [build-dependencies]
 bindgen.workspace = true

mozjs-sys/build.rs

@@ -84,6 +84,8 @@ fn main() {
 
     if build_from_source {
         fs::create_dir_all(&build_dir).expect("could not create build dir");
+        //TODO: use this and remove `no-rust-unicode-bidi.patch`
+        //cbindgen_bidi(&build_dir);
         build_spidermonkey(&build_dir);
         build_jsapi(&build_dir);
         build_jsapi_bindings(&build_dir);
@@ -218,6 +220,22 @@ fn find_moztools() -> Option<PathBuf> {
     }
 }
 
+/*
+fn cbindgen_bidi(build_dir: &Path) {
+    /// Appends intl/bidi to `root`
+    fn root_to_bidi(root: &Pah) -> PathBuf {
+        root.join("intl").join("bidi")
+    }
+    let mozjs_sys_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
+
+    cbindgen::Builder::new()
+      .with_crate(root_to_bidi(mozjs_sys_dir.join("mozjs")).join("rust").join("unicode-bidi-ffi"))
+      .generate()
+      .expect("Unable to generate bindings")
+      .write_to_file(root_to_bidi(build_dir).join("unicode_bidi_ffi_generated.h"));
+}
+*/
+
 fn build_spidermonkey(build_dir: &Path) {
     let target = env::var("TARGET").unwrap();
     let make;
@@ -287,6 +305,9 @@ fn build_spidermonkey(build_dir: &Path) {
         }
         cmd.env("PKG_CONFIG_PATH", pkg_config_path);
     }
+    if let Ok(include) = std::env::var("DEP_Z_INCLUDE") {
+        cppflags.push(format!("-I{include}").replace("\\", "/"));
+    }
     cppflags.push(get_cc_rs_env_os("CPPFLAGS").unwrap_or_default());
     cmd.env("CPPFLAGS", cppflags);
 
@@ -308,6 +329,19 @@ fn build_spidermonkey(build_dir: &Path) {
         .status()
         .expect(&format!("Failed to run `{:?}`", make));
     assert!(result.success());
+    if target.contains("windows") {
+        let mut make_static = cc::Build::new();
+        make_static.out_dir(build_dir.join("js/src/build"));
+        std::fs::read_to_string(build_dir.join("js/src/build/js_static_lib.list"))
+            .unwrap()
+            .lines()
+            .map(String::from)
+            .for_each(|obj| {
+                make_static.object(obj);
+            });
+        make_static.static_flag(true);
+        make_static.compile("js_static");
+    }
 
     println!(
         "cargo:rustc-link-search=native={}/js/src/build",
@@ -513,7 +547,6 @@ const BLACKLIST_FUNCTIONS: &'static [&'static str] = &[
     "JS::EncodeStencil",
     "JS::FinishDecodeMultiStencilsOffThread",
     "JS::FinishIncrementalEncoding",
-    "JS::FinishOffThreadStencil",
     "JS::FromPropertyDescriptor",
     "JS::GetExceptionCause",
     "JS::GetModulePrivate",
@@ -790,9 +823,35 @@ fn compress_static_lib(build_dir: &Path) -> Result<(), std::io::Error> {
     let mut tar = tar::Builder::new(enc);
 
     if target.contains("windows") {
-        // FIXME We can't figure how to include all symbols into the static file.
-        // So we compress whole build dir as workaround.
-        tar.append_dir_all(".", build_dir)?;
+        let status = Command::new("llvm-strip.exe")
+            .arg("--strip-debug")
+            .arg(build_dir.join("js/src/build/js_static.lib"))
+            .status()
+            .unwrap();
+        assert!(status.success());
+
+        // This is the static library of spidermonkey.
+        tar.append_file(
+            "js/src/build/js_static.lib",
+            &mut File::open(build_dir.join("js/src/build/js_static.lib")).unwrap(),
+        )?;
+        // The bindgen binaries and generated rust files for mozjs.
+        tar.append_file(
+            "jsapi.lib",
+            &mut File::open(build_dir.join("jsapi.lib")).unwrap(),
+        )?;
+        tar.append_file(
+            "jsglue.lib",
+            &mut File::open(build_dir.join("jsglue.lib")).unwrap(),
+        )?;
+        tar.append_file(
+            "jsapi.rs",
+            &mut File::open(build_dir.join("jsapi.rs")).unwrap(),
+        )?;
+        tar.append_file(
+            "gluebindings.rs",
+            &mut File::open(build_dir.join("gluebindings.rs")).unwrap(),
+        )?;
     } else {
         // Strip symbols from the static binary since it could bump up to 1.6GB on Linux.
         // TODO: Maybe we could separate symbols for thos who still want the debug ability.

mozjs-sys/etc/COMMIT

@@ -1 +1 @@
-c275ff04bcba0eaebf89d59a3ea1e802c3eb6b68
+4cff37e891cdf778f142fd8005c71b1dd88898b2

mozjs-sys/etc/filters.txt

@@ -30,7 +30,8 @@
 - /python/mozbuild/mozbuild/test/frontend/data
 - /testing/mozbase/rust
 - /third_party/rust
-- Cargo.toml
+- /Cargo.toml
+- /modules/zlib
 
 # Include the rest.
 + /*

mozjs-sys/etc/get_latest_mozjs.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 
-REPO=mozilla-esr115
+REPO=mozilla-esr128
 
 # get commit and appropriet mozjs tar
 jobs=( $(curl "https://treeherder.mozilla.org/api/project/$REPO/push/?full=true&count=20" | jq '.results[].id') )

mozjs-sys/etc/get_mozjs.sh

@@ -4,7 +4,7 @@ set -o errexit
 set -o nounset
 set -o pipefail
 
-REPO=mozilla-esr115
+REPO=mozilla-esr128
 
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 # get commit and appropriet mozjs tar

mozjs-sys/etc/patches/0005-Do-not-rely-on-internal-.cargo-config.patch

@@ -5,38 +5,46 @@ Subject: [PATCH] Do not rely on internal .cargo/config
 
 
 diff --git a/config/recurse.mk b/config/recurse.mk
-index 4188e9e2136d..21984e7f808a 100644
+index 7956578ac..7ff2ec209 100644
 --- a/config/recurse.mk
 +++ b/config/recurse.mk
-@@ -205,18 +205,6 @@ endif
+@@ -201,24 +201,6 @@ endif
  # Those need to depend on config/export for system wrappers.
  $(addprefix build/unix/stdc++compat/,target host) build/clang-plugin/host: config/export
- 
+
 -# Rust targets, and export targets that run cbindgen need
--# $topobjdir/.cargo/config to be preprocessed first. Ideally, we'd only set it
+-# $topobjdir/.cargo/config.toml to be preprocessed first. Ideally, we'd only set it
 -# as a dependency of the rust targets, but unfortunately, that pushes Make to
 -# execute them much later than we'd like them to be when the file doesn't exist
 -# prior to Make running. So we also set it as a dependency of pre-export, which
 -# ensures it exists before recursing the rust targets and the export targets
 -# that run cbindgen, tricking Make into keeping them early.
--$(rust_targets): $(DEPTH)/.cargo/config
+-# When $topobjdir/.cargo/config exists from an old build, we also remove it because
+-# cargo will prefer to use it rather than config.toml.
+-CARGO_CONFIG_DEPS = $(DEPTH)/.cargo/config.toml
+-ifneq (,$(wildcard $(DEPTH)/.cargo/config))
+-CARGO_CONFIG_DEPS += $(MDDEPDIR)/cargo-config-cleanup.stub
+-endif
+-$(rust_targets): $(CARGO_CONFIG_DEPS)
 -ifndef TEST_MOZBUILD
--pre-export:: $(DEPTH)/.cargo/config
+-recurse_pre-export: $(CARGO_CONFIG_DEPS)
 -endif
 -
- # When building gtest as part of the build (LINK_GTEST_DURING_COMPILE),
- # force the build system to get to it first, so that it can be linked
- # quickly without LTO, allowing the build system to go ahead with
+ $(MDDEPDIR)/cargo-config-cleanup.stub:
+ 	rm $(DEPTH)/.cargo/config
+ 	touch $@
 diff --git a/moz.build b/moz.build
-index 94419df73ada..487b8b4dbabd 100644
+index 9db27fd05..a330fd1aa 100644
 --- a/moz.build
 +++ b/moz.build
-@@ -163,6 +163,4 @@ if CONFIG['MOZ_BUILD_APP']:
+@@ -183,10 +183,6 @@ if CONFIG["MOZ_BUILD_APP"]:
  else:
      include("/toolkit/toolkit.mozbuild")
 
--OBJDIR_PP_FILES[".cargo"] += [".cargo/config.in"]
+-OBJDIR_PP_FILES[".cargo"] += [
+-    CONFIG["MOZ_OVERRIDE_CARGO_CONFIG"] or ".cargo/config.toml.in"
+-]
 -
  DEFINES["top_srcdir"] = TOPSRCDIR
-
+ 
  SPHINX_TREES["contributing"] = "docs/contributing"