Make sure BBHx knows how to link locally-installed LAPACK

[titodalcanton]

Fixes a recent CI failure noted by @maxtrevor in #4850.

Standard information about the request

This is (kind of) a bug fix. This change affects mostly the Tox tests.

Motivation

Running pytest under Tox requires to install BBHx, which needs to link against LAPACK when building its wheel. LAPACK is typically installed via Conda (specified in tox.ini) and its shared libraries end up in the temporary Conda env created by Tox.

For some reason which I have not figured out, this used to work fine until a few days ago, when the linker started not being able to find liblapacke anymore. As far as I can see, nothing is explicitly telling the linker where to look for the LAPACK shared libraries, so I am not sure how it could work before 🤷.

So this PR explicitly tells the linker to look for shared libraries inside the Conda env, by setting LIBRARY_PATH. I do not particularly like this solution, so better ideas are welcome.

Contents

Adds a few bits to tox.ini in order to explicitly set the env var LIBRARY_PATH to the lib subdirectory of the Conda env prefix. I think assuming liblapack* will be there is safe.

Links to any issues or associated PRs

Fixes a CI failure in #4850.

Testing performed

Ran Tox locally before opening the PR.

• The author of this pull request confirms they will adhere to the code of conduct

[titodalcanton]

This seems to work. An alternative approach could be to change BBHx instead, and tell its extensions to explicitly look for libraries in $CONDA_PREFIX/lib as well. @spxiwh do you have a preference?

[ahnitz]

@titodalcanton @spxiwh I think ideally this would be handled on the BBHx side by providing wheels rather than building from source. I have no objection to merging this if we have a plan to remove this kind of cruft down the line. (and note in the line change so we know when we can remove).

[spxiwh]

I really don't understand why this is happening. Some things I've observed and am confused by:

• When trying to compile the BBHx source codes, the linker is definitely finding liblapack. However, it is not finding liblapacke. This fix demonstrates that both are present, so why the discrepancy? (Is perhaps liblapack installed systemwide in the images used in github, and that's changed somehow?)
• The compiler is including the include path that points to the tox build, but is not getting the library path (which is what this adds). Why?
• What caused this to stop working? Trying older versions of wheel,setuptools etc. doesn't seem to resolve the issue.

Given all that, I'm happy to approve this now, but we should raise an issue to fix this in the future.

I agree with Alex that having BBHx provide wheels is best, but not sure if that's a possibility.

[spxiwh]

Approve, on the proviso that we don't forget that this hack is here .... I think there's another hack here a few lines above this to avoid picking up the latest clang, which ligolw doesn't work with, which would also be good to resolve in the future!

[titodalcanton]

I agree with your observations and questions @spxiwh. When I reproduced the error locally, I was getting missing message both for -llapack and -llapacke.

[spxiwh]

@titodalcanton When I built BBHx using pip wheel . from a conda env without lapack installed, I also saw errors for both libraries. When I installed lapack I did not see the error and the wheel built successfully .... So I could not reproduce this error (especially I couldn't reproduce a case where only liblapacke was missing).

[titodalcanton]

Ok I will merge this to unblock a bunch of other PRs, and open an issue to remind us about all the cruft in tox.ini…