Fix last_green w/bazel_worker_api rules_java 8.15 (#1756)

* Fix last_green w/bazel_worker_java rules_java 8.15

Uses the `bazel_worker_java` module and a legacy `WORKSPACE` shim to
replace `@bazel_tools//src/main/protobuf:worker_protocol_java_proto`.
This, and bumping `rules_java` to 8.14.0, fixes breakages in the latest
`last_green` Bazel from commit 00b48b27aa216befc419e540919a1f1df4cda337.

Note, though, that this change bumps `rules_java` to 8.15.0, since it's
now available. This change also updates:

- `gazelle`: 0.44.0 => 0.45.0
- `rules_go`: 0.55.1 => 0.56.1
- `rules_shell`: 0.5.0 => 0.5.1

Also:

- Removes the `--incompatible_autoload_externally` flag from `.bazelrc`.

- Removes the Bazel 9.0.0-pre.20250710.1 CI job added by #1755, since
  it's no longer necessary given the fix from this change.

- Fixes `_get_compiler_srcjar` to return empty `compiler_srcjar_objects`
  instead of returning a valid object. `test_compiler_srcjar_error
  2.12.11` from `dt_patches/dt_patch_test.sh` caught this under
  `WORKSPACE`. Added `test_fail_if_compiler_srcjar_object_is_empty` to
  `test/shell/test_compiler_sources_integrity.sh` to catch the specific
  problem sooner.

- Fixed regex in `target_file_location` from `test_helper.sh` to work
  with Bazel 6.5.0, and fixed unbound variable in `fail` message.

- Added logic to `teardown_suite` in `test_bzlmod_macros.sh` to avoid
  unbound variable breakage when running under Bazel 6.5.0.

- Added logic to skip `test_check_module_bazel_template` under Bazel 6
  in `test_version.sh`.

- Removed restriction to use `protobuf` v29 or lower with Bazel 6 from
  `README.md`.

- Added `scala_compiler_source.*` to the regular expressions for "canonical
  reproducible" messages in `dt_patches/dt_patch_test.sh`. This resolved
  a test breakage when `rules_python` began to emit a "canonical
  reproducible" message. (Also changed `--expunge` to
  `--expunge_async`.)

---

bazelbuild/bazel#26477, which also landed in the 9.0.0-pre.20250714.1
rolling release, broke the `last_green` build:

```
$ USE_BAZEL_VERSION=9.0.0-pre.20250714.1 bazel test //src/... //test/...

ERROR: error loading package '@@bazel_tools//src/main/protobuf':
  Unable to find package for
  @@[unknown repo 'grpc-java' requested from @@bazel_tools]//:java_grpc_library.bzl:

  The repository '@@[unknown repo 'grpc-java' requested from @@bazel_tools]'
  could not be resolved:
  No repository visible as '@grpc-java' from repository '@@bazel_tools'.
```

The failing target, and several others, depended upon
`@bazel_tools//src/main/protobuf:worker_protocol_java_proto`. Nothing
else depends on any other target from that package. So upon @Wyverald's
advice, I attempted to use the `bazel_worker_java` module to replace
this specific dependency:

- bazelbuild/bazel#26579 (comment)

The next failure was due to `rules_scala` overriding `rules_java` to be
8.12.0, whereas `last_green` now requires 8.14.0:

```txt
$ USE_BAZEL_VERSION=last_green bazel build //src/... //test/...

ERROR: src/java/io/bazel/rulesscala/jar/BUILD:3:13:
  in java_library rule //src/java/io/bazel/rulesscala/jar:jar:
  Traceback (most recent call last):
    File ".../external/rules_java+/java/bazel/rules/bazel_java_library.bzl",
      line 26, column 35, in _proxy
        return bazel_java_library_rule(
    File ".../external/rules_java+/java/common/rules/impl/bazel_java_library_impl.bzl",
      line 68, column 43, in bazel_java_library_rule
        target, base_info = basic_java_library(
    File ".../external/rules_java+/java/common/rules/impl/basic_java_library_impl.bzl",
      line 127, column 49, in basic_java_library
        java_info, compilation_info = compile_action(
    File ".../external/rules_java+/java/common/rules/impl/compile_action.bzl",
      line 139, column 46, in compile_action
        java_info = _compile_private_for_builtins(
    File ".../external/rules_java+/java/private/java_common_internal.bzl",
      line 217, column 68, in compile
        get_internal_java_common().create_header_compilation_action(

Error in create_header_compilation_action:
  create_header_compilation_action() missing 2 required positional
  arguments: header_compilation_jar, header_compilation_direct_deps

ERROR: src/java/io/bazel/rulesscala/jar/BUILD:3:13:
  Analysis of target '//src/java/io/bazel/rulesscala/jar:jar' failed
```

Bumping `rules_java` to 8.14.0 in the `single_version_override` fixed
that.

Finally, since there's no legacy `WORKSPACE` API in
bazelbuild/bazel-worker-api, I had to roll my own new
`workspace_compat()` macro. This macro creates
`@bazel_worker_{api,java}` repos from the `bazel-worker-api` archive
using @Wyverald's `http_archive` and `strip_prefix` suggestion
from #1756.

Regarding `--incompatible_autoload_externally`, I'd added it in #1748
when fixing problems with `WORKSPACE` builds under Bazel 8.2.1. However,
it's not really necessary, so I removed it.

* Update Maven artifact versions

- Scalafmt: 3.9.8 => 3.9.9
- `grpc`: 1.73.0 => 1.74.0
- `proto-google-common-protos`: 2.59.2 => 2.60.0
- `sbt-util-interface`: 1.11.3 => 1.11.4

Author: mbland

.bazelci/presubmit.yml

@@ -37,16 +37,6 @@ tasks:
     # Install xmllint
     - sudo apt update && sudo apt install --reinstall libxml2-utils -y
     - "./test_rules_scala.sh || buildkite-agent annotate --style 'warning' \"Optional build with last_green Bazel version failed, [see here](${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}) (It is not mandatory but worth checking)\""
-  # Remove this job once the last_green job passes after a fix lands for
-  # bazelbuild/bazel#26579.
-  test_rules_scala_linux_9_prerelease:
-    name: "./test_rules_scala (Bazel 9 rolling prerelease)"
-    platform: ubuntu2004
-    bazel: 9.0.0-pre.20250710.1
-    shell_commands:
-    # Install xmllint
-    - sudo apt update && sudo apt install --reinstall libxml2-utils -y
-    - "./test_rules_scala.sh"
   test_rules_scala_macos:
     name: "./test_rules_scala"
     platform: macos

.scalafmt.conf

@@ -1,4 +1,4 @@
-version = "3.9.8"
+version = "3.9.9"
 runner.dialect = scala213
 align.openParenCallSite = false
 align.openParenDefnSite = false

MODULE.bazel

@@ -42,7 +42,7 @@ single_version_override(
 bazel_dep(name = "rules_java", version = "7.6.0")
 single_version_override(
     module_name = "rules_java",
-    version = "8.14.0",
+    version = "8.15.0",
 )
 
 bazel_dep(name = "rules_proto", version = "6.0.0")
@@ -66,6 +66,8 @@ single_version_override(
     version = "31.1",
 )
 
+bazel_dep(name = "bazel_worker_java", version = "0.0.6")
+
 scala_protoc = use_extension(
     "//scala/extensions:protoc.bzl",
     "scala_protoc",
@@ -254,11 +256,11 @@ bazel_dep(
 )
 bazel_dep(
     name = "rules_go",
-    version = "0.55.1",
+    version = "0.56.1",
     dev_dependency = True,
     repo_name = "io_bazel_rules_go",  # for com_github_bazelbuild_buildtools
 )
-bazel_dep(name = "gazelle", version = "0.44.0", dev_dependency = True)
+bazel_dep(name = "gazelle", version = "0.45.0", dev_dependency = True)
 
 go_sdk = use_extension(
     "@io_bazel_rules_go//go:extensions.bzl",
@@ -284,7 +286,7 @@ go_deps = use_extension(
 # - https://go.dev/ref/mod#checksum-database
 #
 #   go list -m golang.org/x/tools@latest
-#   curl https://sum.golang.org/lookup/golang.org/x/tools@v0.34.0
+#   curl https://sum.golang.org/lookup/golang.org/x/tools@v0.35.0
 go_deps.module(
     path = "golang.org/x/tools",
     sum = "h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0=",
@@ -302,4 +304,4 @@ use_repo(
 )
 
 bazel_dep(name = "rules_python", version = "1.5.1", dev_dependency = True)
-bazel_dep(name = "rules_shell", version = "0.5.0", dev_dependency = True)
+bazel_dep(name = "rules_shell", version = "0.5.1", dev_dependency = True)

README.md

@@ -1318,36 +1318,12 @@ future compatibility.
 
 __`rules_scala` 7.x officially drops support for Bazel 6.5.0.__ Bzlmod builds
 with Bazel 6.5.0 won't work at all because [Bazel 6.5.0 doesn't support
-'use_repo_rule']( https://bazel.build/versions/6.5.0/rules/lib/globals), which
+'use_repo_rule'](https://bazel.build/versions/6.5.0/rules/lib/globals), which
 ['rules_jvm_external' >= 6.3 requires](
 https://github.com/bazelbuild/rules_scala/issues/1482#issuecomment-2515496234).
 
 At the moment, `WORKSPACE` builds mostly continue to work with Bazel 6.5.0, but
-not out of the box, and may break at any time.
-
-#### Maximum of `protobuf` v29
-
-You _must_ use `protobuf` v29 or earlier. `rules_scala` now uses v30 by default,
-which removes `py_proto_library` and other symbols that Bazel 6.5.0 requires:
-
-```txt
-ERROR: Traceback (most recent call last):
-  File ".../external/bazel_tools/src/main/protobuf/BUILD",
-  line 1, column 46, in <toplevel>
-    load("@com_google_protobuf//:protobuf.bzl", "py_proto_library")
-
-Error: file '@com_google_protobuf//:protobuf.bzl'
-  does not contain symbol 'py_proto_library'
-
-ERROR: .../src/java/io/bazel/rulesscala/worker/BUILD:3:13:
-  no such target '@bazel_tools//src/main/protobuf:worker_protocol_java_proto':
-  target 'worker_protocol_java_proto'
-  not declared in package 'src/main/protobuf'
-  defined by .../external/bazel_tools/src/main/protobuf/BUILD
-  (Tip: use `query "@bazel_tools//src/main/protobuf:*"`
-    to see all the targets in that package)
-  and referenced by '//src/java/io/bazel/rulesscala/worker:worker'
-```
+may break at any time.
 
 #### Configuring the protocol compiler toolchain
 

WORKSPACE

@@ -105,10 +105,10 @@ local_repository(
 
 http_archive(
     name = "io_bazel_rules_go",
-    sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a",
+    sha256 = "89d2050410602142c9acafd01c95baf48b65f8dd16f4771d37c89f82f5e147f2",
     urls = [
-        "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.0.zip",
-        "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip",
+        "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.56.1/rules_go-v0.56.1.zip",
+        "https://github.com/bazel-contrib/rules_go/releases/download/v0.56.1/rules_go-v0.56.1.zip",
     ],
 )
 

deps/latest/MODULE.bazel

@@ -9,7 +9,7 @@ module(
 bazel_dep(name = "bazel_skylib", version = "1.8.1")
 bazel_dep(name = "platforms", version = "1.0.0")
 bazel_dep(name = "protobuf", version = "31.1")
-bazel_dep(name = "rules_java", version = "8.14.0")
+bazel_dep(name = "rules_java", version = "8.15.0")
 bazel_dep(name = "rules_proto", version = "7.1.0")
 
 # https://github.com/bazelbuild/bazel/pull/25681 removed

dt_patches/dt_patch_test.sh

@@ -36,7 +36,7 @@ test_compiler_srcjar() {
   run_in_test_repo \
     test_dt_patches_user_srcjar \
     bazel build "--repo_env=SCALA_VERSION=${SCALA_VERSION}" //... 2>&1 |
-    (! grep "canonical reproducible")
+    (! grep "scala_compiler_source.*canonical reproducible")
 }
 
 test_compiler_srcjar_nonhermetic() {
@@ -46,7 +46,7 @@ test_compiler_srcjar_nonhermetic() {
   run_in_test_repo \
     test_dt_patches_user_srcjar \
     bazel build "--repo_env=SCALA_VERSION=${SCALA_VERSION}" //... 2>&1 |
-    grep 'canonical reproducible'
+    grep 'scala_compiler_source.*canonical reproducible'
 }
 
 test_compiler_srcjar_error() {
@@ -129,7 +129,7 @@ $runner test_compiler_srcjar_error 2.12.13
 # that we grep for will only be outputted the first time (on Bazel >= 6).
 # So we clean the repo first to ensure consistency.
 
-run_in_test_repo 'test_dt_patches_user_srcjar' bazel clean --expunge
+run_in_test_repo 'test_dt_patches_user_srcjar' bazel clean --expunge_async
 
 $runner test_compiler_srcjar 2.12.14
 $runner test_compiler_srcjar 2.12.15

scala/deps.bzl

@@ -2,6 +2,7 @@
 
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe")
+load("//scala/private:macros/workspace_compat.bzl", "workspace_compat")
 
 def rules_scala_dependencies():
     """Instantiates repos needed by rules provided by `rules_scala`."""
@@ -64,3 +65,5 @@ def rules_scala_dependencies():
         strip_prefix = "rules_proto-6.0.2",
         url = "https://github.com/bazelbuild/rules_proto/releases/download/6.0.2/rules_proto-6.0.2.tar.gz",
     )
+
+    workspace_compat()

scala/latest_deps.bzl

@@ -2,6 +2,7 @@
 
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe")
+load("//scala/private:macros/workspace_compat.bzl", "workspace_compat")
 
 def rules_scala_dependencies():
     """Instantiates repos needed by rules provided by `rules_scala`."""
@@ -29,9 +30,9 @@ def rules_scala_dependencies():
         http_archive,
         name = "rules_java",
         urls = [
-            "https://github.com/bazelbuild/rules_java/releases/download/8.14.0/rules_java-8.14.0.tar.gz",
+            "https://github.com/bazelbuild/rules_java/releases/download/8.15.0/rules_java-8.15.0.tar.gz",
         ],
-        sha256 = "bbe7d94360cc9ed4607ec5fd94995fd1ec41e84257020b6f09e64055281ecb12",
+        sha256 = "0a7e8811cac04b553f6c6c0d185046e3c68a4cb774e83f37a7a5fb6a3deee261",
     )
 
     maybe(
@@ -72,7 +73,8 @@ def rules_scala_dependencies():
 
     # Can't upgrade for now because https://github.com/bazel-contrib/rules_python/pull/2760
     # broke Bazel 7 WORKSPACE builds. It's really only a dev dep anyway.
-    # If it's fixed per https://github.com/bazel-contrib/rules_python/issues/3119,
+    # If it's fixed per https://github.com/bazel-contrib/rules_python/issues/3119
+    # (i.e., once https://github.com/bazel-contrib/rules_python/pull/3134 lands),
     # then we can upgrade.
     maybe(
         http_archive,
@@ -85,7 +87,9 @@ def rules_scala_dependencies():
     maybe(
         http_archive,
         name = "rules_shell",
-        sha256 = "b15cc2e698a3c553d773ff4af35eb4b3ce2983c319163707dddd9e70faaa062d",
-        strip_prefix = "rules_shell-0.5.0",
-        url = "https://github.com/bazelbuild/rules_shell/releases/download/v0.5.0/rules_shell-v0.5.0.tar.gz",
+        sha256 = "99bfc7aaefd1ed69613bbd25e24bf7871d68aeafca3a6b79f5f85c0996a41355",
+        strip_prefix = "rules_shell-0.5.1",
+        url = "https://github.com/bazelbuild/rules_shell/releases/download/v0.5.1/rules_shell-v0.5.1.tar.gz",
     )
+
+    workspace_compat()

scala/private/macros/test/MODULE.bzlmod_test

@@ -17,4 +17,4 @@ dev_test_ext = use_extension(
     dev_dependency = True,
 )
 
-bazel_dep(name = "rules_shell", version = "0.4.1")
+bazel_dep(name = "rules_shell", version = "0.5.1")