Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to bzlmod #56

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Migrate to bzlmod #56

wants to merge 8 commits into from

Conversation

hofbi
Copy link
Contributor

@hofbi hofbi commented Jun 13, 2024

Migrate the main project to bzlmod and rename all BUILD files to BUILD.bazel

Closes #53
Closes #54

@hofbi hofbi requested a review from facundominguez as a code owner June 13, 2024 14:26
@facundominguez facundominguez requested a review from avdv June 13, 2024 15:33
@hofbi
Copy link
Contributor Author

hofbi commented Jun 14, 2024

CI probably fails due to some older version of bazel which is used in the nixpkgs, but here I might need your input on which version of packages you want to use.

It was used with skydoc, but we migrated to stardoc which does not produce the
zip file anymore.
@avdv
Copy link
Member

avdv commented Jun 14, 2024

CI probably fails due to some older version of bazel which is used in the nixpkgs, but here I might need your input on which version of packages you want to use.

Thank you, @hofbi. That looks good! I'll update nixpkgs and push my changes in a moment...

We are still using Bazel 7.1 and the lock file is not stable enough to be useful.
Copy link
Member

@avdv avdv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks again @hofbi!

BTW, we have to use Bazel 7.1.0 since that's what is available from nixpkgs right now. So I removed the lock file and added a .bazelversion to make the version explicit.

@hofbi
Copy link
Contributor Author

hofbi commented Jun 14, 2024

Sounds good, thanks.

What do you guys think about using bazelisk to include bazel? This way you can easily control it from the version file and it is the recommended way for installing bazel. But definitely a follow up PR in case we wanna do that.

Another task would also be to convert the tests to bzlmod.

@aherrmann
Copy link
Member

The tests include some Nix components and might depend on some of the patches made to Bazel in nixpkgs.

@hofbi
Copy link
Contributor Author

hofbi commented Jun 14, 2024

The tests include some Nix components and might depend on some of the patches made to Bazel in nixpkgs.

Should we also migrate the tests to bzlmod in this PR?

@avdv
Copy link
Member

avdv commented Jun 14, 2024

Should we also migrate the tests to bzlmod in this PR?

The tests use nixpkgs_python_configure which is part of rules_nixpkgs_python and that is currently not available from the BCR. Same situation for rules_haskell_nixpkgs. I'd say let's leave it at that for now.

@hofbi
Copy link
Contributor Author

hofbi commented Jun 14, 2024

So are we good to merge? I can play arround with a follow up PR for the tests including the other deps as non module dependencies

@avdv
Copy link
Member

avdv commented Jun 14, 2024

So are we good to merge?

CI is still failing. :-(

I can play arround with a follow up PR for the tests including the other deps as non module dependencies

That would be awesome!

@avdv
Copy link
Member

avdv commented Jun 14, 2024

CI run on ubuntu is now green, but macos is failing with:

Repository rule cc_autoconf defined at:
  /private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/cc_configure.bzl:109:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'bazel_tools~cc_configure_extension~local_config_cc':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/cc_configure.bzl", line 91, column 33, in cc_autoconf_impl
		configure_unix_toolchain(repository_ctx, cpu_value, overriden_tools)
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 387, column 46, in configure_unix_toolchain
		overriden_tools["ar"] = _find_generic(repository_ctx, "libtool", "LIBTOOL", overriden_tools)
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 310, column 32, in _find_generic
		auto_configure_fail(msg)
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, column 9, in auto_configure_fail
		fail("\n%sAuto-Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: 
Auto-Configuration Error: Cannot find libtool or LIBTOOL; either correct your path or set the LIBTOOL environment variable
ERROR: <builtin>: fetching cc_autoconf rule //:bazel_tools~cc_configure_extension~local_config_cc: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/cc_configure.bzl", line 91, column 33, in cc_autoconf_impl
		configure_unix_toolchain(repository_ctx, cpu_value, overriden_tools)
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 387, column 46, in configure_unix_toolchain
		overriden_tools["ar"] = _find_generic(repository_ctx, "libtool", "LIBTOOL", overriden_tools)
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 310, column 32, in _find_generic
		auto_configure_fail(msg)
	File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, column 9, in auto_configure_fail
		fail("\n%sAuto-Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: 
Auto-Configuration Error: Cannot find libtool or LIBTOOL; either correct your path or set the LIBTOOL environment variable

@aherrmann
Copy link
Member

bazelbuild/bazel@3c47b47 seems to be relevant to the LIBTOOL issue. I've tried out a Nix provided cc toolchain to avoid this issue here.
With that the MacOS build proceeds further, but fails with

ERROR: /Volumes/External/Users/andreas/clodl/tests/BUILD.bazel:109:15: Action clotestbin-cc-closure.zip failed: (Terminated): bash failed: error executing Action command (from target //:clotestbin-cc-closure) /nix/store/wb7cjcvaa31s5saxi7z24qwhq709hbhz-bash/bin/bash -c ... (remaining 4 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
otool: error: 'dyld[60391]: <no uuid> /private/var/tmp/_bazel_andreas/5a72690ff69d3fb0757d7797a2268561/execroot/io_tweag_clodl_tests/bazel-out/darwin_arm64-opt-exec-ST-467dab1ad9da/bin/external/io_tweag_clodl/loader': No such file or directory
Terminated: 15

@hofbi
Copy link
Contributor Author

hofbi commented Jun 21, 2024

bazelbuild/bazel@3c47b47 seems to be relevant to the LIBTOOL issue. I've tried out a Nix provided cc toolchain to avoid this issue here. With that the MacOS build proceeds further, but fails with

ERROR: /Volumes/External/Users/andreas/clodl/tests/BUILD.bazel:109:15: Action clotestbin-cc-closure.zip failed: (Terminated): bash failed: error executing Action command (from target //:clotestbin-cc-closure) /nix/store/wb7cjcvaa31s5saxi7z24qwhq709hbhz-bash/bin/bash -c ... (remaining 4 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
otool: error: 'dyld[60391]: <no uuid> /private/var/tmp/_bazel_andreas/5a72690ff69d3fb0757d7797a2268561/execroot/io_tweag_clodl_tests/bazel-out/darwin_arm64-opt-exec-ST-467dab1ad9da/bin/external/io_tweag_clodl/loader': No such file or directory
Terminated: 15

I found envoyproxy/envoy-filter-example#121. From there I get the impression that XCode is required to be installed for some reason.

@facundominguez
Copy link
Member

otool: error: 'dyld[60391]: /private/var/tmp/_bazel_andreas/5a72690ff69d3fb0757d7797a2268561/execroot/io_tweag_clodl_tests/bazel-out/darwin_arm64-opt-exec-ST-467dab1ad9da/bin/external/io_tweag_clodl/loader': No such file or directory

I'm guessing loader refers to this helper binary.

@hofbi
Copy link
Contributor Author

hofbi commented Jul 15, 2024

How should we move on with this? Is there something I could/should try or do you have to clarify the next steps internally?

@facundominguez
Copy link
Member

Hello @hofbi. At this point I think some investigation is needed to explain why the OSX build is currently failing to find the loader helper. You are welcome to dive into it.

@hofbi
Copy link
Contributor Author

hofbi commented Jul 15, 2024

Hello @hofbi. At this point I think some investigation is needed to explain why the OSX build is currently failing to find the loader helper. You are welcome to dive into it.

I have no OSX hardware available so can only do debugging in CI, but will try my best

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants