[bug] BazelDeps shared-library packages with chained symlinks fail to install with bazel correctly after 2.15

[moo1210]

Describe the bug

Trying to use conan packages of type shared-library that have chaining symlinks fails with conan 2.15+. This issue appears to have been introduced by #17964.

How to reproduce it

I was able to reproduce this issue with both csm/3.0.4, but I was unable to reproduce it with libsndio/1.9.0. Note their differences in symlinks.

# works
libsndio.so.7 -> libsndio.so.7.2
libsndio.so.7.0 -> libsndio.so.7.2
libsndio.so.7.2
# doesn't work
libcsmapi.so -> libcsmapi.so.3
libcsmapi.so.3 -> libcsmapi.so.3.0.4
libcsmapi.so.3.0.4

Both packages work without issue in Conan 2.14

[franramirez688]

Hi @moo1210

Thanks for reporting this!
Could you provide a minimal use case to reproduce this issue? Also, let me know your Bazel version.
Because I tried it locally and it works fine with the latest Conan version.

[moo1210]

Maybe I've identified two separate problems with shared-library. For my use case, I just downgraded to conan 2.14.0. However in the package I previously named, the same issue seems to occur regardless of version.

But running conan install . and then bazel run //main:demo with either the csm or dpp package fails.
This repro is just the existing bazel sample, with the fmt package replaced along with it's test package from conan center.
conan-19135-repro.zip

# Conan version 2.15.0-2.21.0
error while loading shared libraries: libdpp.so.10.0.35: cannot open shared object file: No such file or directory
# Conan version unknown-2.21.0
error while loading shared libraries: libcsmapi.so.3: cannot open shared object file: No such file or directory

[franramirez688]

Thanks for the repro @moo1210

Yes, I understand what's happening. In a nutshell, if you run the demo application through Conan or manually (loading the conanrun.sh before), it works. For instance, having this conanfile.py as a test:

import os
from conan import ConanFile
from conan.tools.google import Bazel, bazel_layout
from conan.tools.build import can_run


class TestConan(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "BazelToolchain", "BazelDeps"

    def requirements(self):
        self.requires(self.tested_reference_str)

    def build(self):
        bazel = Bazel(self)
        bazel.build(target="//main:demo")

    def layout(self):
        bazel_layout(self)

    def test(self):
        if can_run(self):
            cmd = os.path.join(self.cpp.build.bindir, "main", "demo")
            self.run(cmd, env="conanrun")

And running conan test . csm/3.0.4, it works:

. . .

WARNING: Couldn't auto load rules or symbols, because no dependency on module/repository 'rules_android' found. This will result in a failure if there's a reference to those rules or symbols.
INFO: Analyzed target //main:demo (69 packages loaded, 495 targets configured).
INFO: Found 1 target...
Target //main:demo up-to-date:
  bazel-bin/main/demo
INFO: Elapsed time: 1.539s, Critical Path: 1.05s
INFO: 11 processes: 9 internal, 2 darwin-sandbox.
INFO: Build completed successfully, 11 total actions


======== Testing the package: Executing test ========
csm/3.0.4 (test package): Running test()
csm/3.0.4 (test package): RUN: ./bazel-bin/main/demo

The problem comes when we use the Bazel sandbox, e.g., bazel run, and the executables are using this rpath, for instance:

$ otool -L bazel-bin/main/demo
bazel-bin/main/demo:
	@rpath/libcsmapi.3.dylib (compatibility version 3.0.0, current version 3.0.4)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1900.180.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 3502.1.255)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

Why is it pointing to libcsmapi.3? Because of this:

$ ls -l /Users/xxxxxx/.conan2/p/b/csmebb5aa4b87836/p/lib
total 344
-rwxr-xr-x@ 1 xxxxxx  staff  172992 24 oct 13:13 libcsmapi.3.0.4.dylib
lrwxr-xr-x@ 1  xxxxxx  staff      21 24 oct 13:13 libcsmapi.3.dylib -> libcsmapi.3.0.4.dylib
lrwxr-xr-x@ 1  xxxxxx  staff      17 24 oct 13:13 libcsmapi.dylib -> libcsmapi.3.dylib

$  otool -L /Users/xxxxxx/.conan2/p/b/csmebb5aa4b87836/p/lib/libcsmapi.3.0.4.dylib
/Users/xxxxxx/.conan2/p/b/csmebb5aa4b87836/p/lib/libcsmapi.3.0.4.dylib:
	@rpath/libcsmapi.3.dylib (compatibility version 3.0.0, current version 3.0.4)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1900.180.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)

Then, the Bazel sandbox points to @rpath/libcsmapi.3.dylib, but it does not have any copy of that file there. So, if Conan 2.14 could work with one of those examples, it was merely a coincidence.

As a workaround, you could simply change the csm/BUILD.bazel:

cc_import(
    name = "csmapi_precompiled",
    shared_library = "lib/libcsmapi.3.dylib",
)

And run:

$ bazel clean
$ bazel run //main:demo

Then, it should work fine, as you're telling Bazel to put a copy of that file into the Bazel cache.

What I'm wondering now, whether we should add something like this:

linkopts = [
    "-Wl,-rpath,@loader_path/xxxxxx",
]

And check if it could work using Conan or Bazel directly. I need to look into this 🙏

[moo1210]

That sounds reasonable regarding the csm package and the Bazel sandbox. However, that appears to be a different issue from the one I identified.

So, if Conan 2.14 could work with one of those examples, it was merely a coincidence.

I built Conan from the 2.21.0 tag and reverted cc1a8b3. I can build and run the conan-19135-repro-dpp folder without issue. If I install 2.21.0 from pip, the build fails.

In the cc_import of dpp/BUILD.bazel, the broken build has shared_library as "lib/libdpp.so" whereas the working build has "lib/libdpp.so.10.0.35"

[memsharded]

Hi @moo1210

This has been closed by PR #19193, that will be in next 2.23 (today).
Please don't hesitate to open new tickets for any further question or issue. Thanks for the feedback!