proc-macro2 build determinism issue

[nmattia]

This is not directly a rules_rust issue but more of a general Bazel/rustc/proc-macro2 issue. I'm logging it here because this is the first place I came looking. I created a ticket on the proc-macro2 repo though maybe there's something rules_rust can do to avoid the issue more generally.

I noticed that the proc-macro2 build was not deterministic. Building it several times would result in new content every time:

$ for i in $(seq 0 5); do bazel clean && bazel build @crate_index//:proc-macro2 && find bazel-bin/external/crate_index__proc-macro2-1.0.88 -type f -print0 | xargs -0 sha256sum | sha256sum ; done 2>/dev/null
9e4380b3a8f8b21f99ec019d8ec82b194d8a6cc0a41f1bcba2ecd709ff071766  -
85f879d65c675c15ae39eab6433bd84ee6a165b975b9816e2306a2180ddbbd10  -
b1deb8640941ef39dc56d306d2f768c90980b9f3e59644215a2711c44954c04f  -
6a7135a57407b71d0cb0a9582e3d5255e5f1f2c2cfd3a64e340d598bdf43f702  -
36a738c2f0d7c37270cd2590d37a4813d19d249f288f210ea203752803fd9378  -
bfca95a91fe8c64212a8527e212aa16832b342d0ca0f26fbfb48e2698af22b5f  -

This issue is actually caused by the proc-macro2 build script dep-info file which includes absolute paths of the build directory, which is a runner-specific (and invocation-specific) directory path (see the proc-macro2 ticket for more info):

$ cat bazel-bin/external/crate_index__proc-macro2-1.0.88/proc-macro2_bs.out_dir/proc_macro2.d 
/home/ubuntu/.cache/bazel/_bazel_ubuntu/6d065581cce7ad9076e3b8db2b3afaf0/sandbox/linux-sandbox/382/execroot/_main/bazel-out/k8-opt/bin/external/crate_index__proc-macro2-1.0.88/proc-macro2_bs.out_dir/libproc_macro2.rmeta: build/probe.rs

/home/ubuntu/.cache/bazel/_bazel_ubuntu/6d065581cce7ad9076e3b8db2b3afaf0/sandbox/linux-sandbox/382/execroot/_main/bazel-out/k8-opt/bin/external/crate_index__proc-macro2-1.0.88/proc-macro2_bs.out_dir/proc_macro2.d: build/probe.rs

build/probe.rs:

# env-dep:RUSTC_BOOTSTRAP

Cargo allows remapping the paths in the dep-info files but this doesn't work in this case (see the proc-macro2 ticket for more info). A hacky workaround is to patch rules_rust:

diff --git a/cargo/cargo_build_script_runner/bin.rs b/cargo/cargo_build_script_runner/bin.rs
index 8730c809..f8eda5db 100644
--- a/cargo/cargo_build_script_runner/bin.rs
+++ b/cargo/cargo_build_script_runner/bin.rs
@@ -84,7 +84,7 @@ fn run_buildrs() -> Result<(), String> {
     command
         .current_dir(&working_directory)
         .envs(target_env_vars)
-        .env("OUT_DIR", out_dir_abs)
+        .env("OUT_DIR", out_dir_abs.clone())
         .env("CARGO_MANIFEST_DIR", manifest_dir)
         .env("RUSTC", rustc)
         .env("RUST_BACKTRACE", "full");
@@ -195,6 +195,11 @@ fn run_buildrs() -> Result<(), String> {
         .unwrap_or_else(|_| panic!("Unable to write file {:?}", link_flags_file));
     write(&link_search_paths_file, link_search_paths.as_bytes())
         .unwrap_or_else(|_| panic!("Unable to write file {:?}", link_search_paths_file));
+
+    let proc_macro2_d = out_dir_abs.join("proc_macro2.d");
+    if Path::new(&proc_macro2_d).exists() {
+        std::fs::remove_file(proc_macro2_d);
+    }
     Ok(())
 }

Is there a way to generally solve this issue? And is there a cleaner workaround?