Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sparrow Build Failure due to unlinked yaml-cpp #25

Open
njh80 opened this issue Jan 11, 2024 · 4 comments
Open

Sparrow Build Failure due to unlinked yaml-cpp #25

njh80 opened this issue Jan 11, 2024 · 4 comments

Comments

@njh80
Copy link

njh80 commented Jan 11, 2024

Dear Rehier Group,

Thanks for helping earlier with Utilities. I've been debugging and reached as far as the Sparrow install. The puffin bootstrap has installed Core, Utilities, Readuct and Database but when trying to install Sparrow 5.0.0 it is unable to find yaml-cpp.

yaml-cpp is install as:
$ ls /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/libyaml-cpp.so
libyaml-cpp.so libyaml-cpp.so.0.7 libyaml-cpp.so.0.7.0

I've tried adding the full path to the -RPATH as it suggests by including the cmake flag:
-DCMAKE_EXE_LINKER_FLAGS="-Wl,-rpath,/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/"

Any ideas as to what I've done wrong would be very helpful

OS is RedHat CentOS7. Output log is (after installing other packages via python3 -m puffin -c puffin.yaml bootstrap:

Preparing Sparrow...

-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc
-- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++
-- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Scine::UtilsOS found locally at /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/cmake/ScineUtilsOS
Cloning into 'cereal-src'...
Note: switching to 'v1.2.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

git switch -c

Or undo this operation with:

git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 51cbda5 Merge branch 'develop' for release 1.2.2
CMake Warning (dev) at build/cereal-src/CMakeLists.txt:2 (project):
Policy CMP0048 is not set: project() command manages VERSION variables.
Run "cmake --help-policy CMP0048" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

The following variable(s) would be set to empty:

PROJECT_VERSION
PROJECT_VERSION_MAJOR
PROJECT_VERSION_MINOR
PROJECT_VERSION_PATCH

This warning is for project developers. Use -Wno-dev to suppress it.

-- Cereal was not found in your PATH, so it was downloaded.
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found Boost: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/cmake/Boost-1.74.0/BoostConfig.cmake (found version "1.74.0") found components: program_options filesystem
-- Found PythonInterp: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/python3 (found version "3.8.18")
-- Cereal found locally at cereal_DIR-NOTFOUND
-- Targets to copy for python bindings: Sparrow;Scine::UtilsOS
-- Found PY_sphinx: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/sphinx
-- Configuring done
-- Generating done
-- Build files have been written to: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/sparrow/build
Scanning dependencies of target SparrowApp
[ 1%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/CalculationHandler.cpp.o
[ 2%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/CommandLineOptions.cpp.o
[ 2%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/SparrowInitializer.cpp.o
[ 3%] Building CXX object src/Sparrow/CMakeFiles/SparrowApp.dir/App/main.cpp.o
[ 4%] Linking CXX executable sparrow
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libyaml-cpp.so.0.7, needed by /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so, not found (try using -rpath or -rpath-link)
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::StartedScalar()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to vtable for YAML::BadConversion'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::insert_map_pair(YAML::detail::node&, YAML::detail::node&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::ostream_wrapper::write(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::memory::create_node()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::Write(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::set_null()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to vtable for YAML::InvalidNode'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to typeinfo for YAML::InvalidNode' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::set_scalar(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::end()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::~Emitter()'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::memory_holder::merge(YAML::detail::memory_holder&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::size() const'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::empty_scalar[abi:cxx11]()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::good() const'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::InvalidNode::~InvalidNode()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to typeinfo for YAML::BadSubscript'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::PrepareNode(YAML::EmitterNodeType::value)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to typeinfo for YAML::BadConversion'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::SetMapFormat(YAML::EMITTER_MANIP)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::convert_to_map(std::shared_ptrYAML::detail::memory_holder const&)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::PrepareIntegralStream(std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&) const' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::Write(bool)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::begin()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::SetSeqFormat(YAML::EMITTER_MANIP)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::BadConversion::~BadConversion()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::c_str() const'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to vtable for YAML::RepresentationException' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::Emitter()'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::BadSubscript::~BadSubscript()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to vtable for YAML::Exception'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::detail::node_data::mark_defined()' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::Emitter::SetLocalValue(YAML::EMITTER_MANIP)'
/home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to YAML::convert<bool>::decode(YAML::Node const&, bool&)' /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/../lib/gcc/x86_64-conda-linux-gnu/11.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/libutilsos.so: undefined reference to vtable for YAML::BadSubscript'
collect2: error: ld returned 1 exit status
make[2]: *** [src/Sparrow/sparrow] Error 1
make[1]: *** [src/Sparrow/CMakeFiles/SparrowApp.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/main.py", line 148, in
main()
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/main.py", line 134, in main
main_bootstrap(config)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/main.py", line 73, in main_bootstrap
bootstrap(config)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/bootstrap.py", line 85, in bootstrap
program.install(program_build_dir, install_dir, config["resources"]["cores"])
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/programs/sparrow.py", line 22, in install
self.scine_module_install(repo_dir, install_dir, ncores, add_lib=True, add_bin=True)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/programs/program.py", line 162, in scine_module_install
subprocess.run(["make", "-j" + str(ncores), "install"], env=env, check=True)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['make', '-j1', 'install']' returned non-zero exit status 2.
@steinmig
Copy link

Based on your output in qcscine/utilities#7 which was:

 -- Found package yaml-cpp at /home/njh80/modulesSource/miniforge3/envs/chemoton/share/cmake/yaml-cpp

you have a similar problem as with the Core dependency. Only that here the problem is not a different version, but that Sparrow cannot find the dependency with which Utils has been built.
It should work if you add the directory where libyaml.so is located (probably /home/njh80/modulesSource/miniforge3/envs/chemoton/lib to your LD_LIBRARY_PATH, but that could get you back to problems like the one in qcscine/utilities#7

Due to the dependencies between our modules, it is a bit difficult to mix in dependencies between different locations without setting up the environment carefully. In general, we recommend: only install boost, mongo-c, and mongo-cxx separately and then our bootstrap should be able to download and compile all dependencies and configure everything into one install directory.

So my hotfix recommendation would be: remove the yaml-cpp location from all your paths, delete the utils build directory (/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/utils) and start the bootstrap again. If the cmake of utils shows the message:

-- yaml-cpp was not found in your PATH, so it was downloaded.

and then first compiles the yaml-cpp library (noticeable by having more warnings printed), then you should have it properly installed within the puffin bootstrap and Sparrow should link fine.

Simply adding the yaml location to the LD_LIBRARY_PATH would be the faster solution if it works straightaway, because you don't need to compile utils again, but you need to be careful to always add this to your path if you want to use our software and this dependency will not be added to the puffin.sh file generated after the bootstrap

@njh80
Copy link
Author

njh80 commented Jan 12, 2024

Thanks again @steinmig! Looks like the LD_LIBRARY_PATH did the trick here. I'm not sure how yaml-cpp was installed into the virtual environment originally, maybe Conda installed it with gcc?

I managed to get a bit further with the bootstrapping up to Molassembler, but then there was a different problem. It's probably a similar problem but I thought I'd just add it to this thread (I can setup a fresh query if preferred).

This time Nauty isn't downloading. I've got a copy of nauty (2.8.6) elsewhere but it is not in the PATH, LD_LIBRARY_PATH, or CPATH. The identical error occurs even when I include my nauty on the PATH (I assume it is looking for 2.7.0).

Preparing Molassembler...

-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc
-- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++
-- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Scine::UtilsOS found locally at /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/cmake/ScineUtilsOS
-- Enabling GCC specific warning flags
-- Found MKL: /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include
-- Found Intel MKL libraries:
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_avx2.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_vml_avx2.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_core.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_intel_lp64.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_gnu_thread.so
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found MKL for use with Eigen3
CMake Error at dev/cmake/ColorMessages.cmake:39 (message):
Could not download file: "Unsupported protocol" Please download the
file at https://scine.ethz.ch/static/download/nauty27r1.tar.gz and place it
in your build tree at
/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/nauty27r1.tar.gz
Call Stack (most recent call first):
dev/cmake/DownloadFileHelper.cmake:18 (cmessage)
cmake/ImportNauty.cmake:23 (download_file)
CMakeLists.txt:77 (import_nauty)

-- Configuring incomplete, errors occurred!

I can do wget to the https fine in the folder but every time I rebootstrap Puffin I need to clear out the existing software to avoid the error:

File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_puffin/programs/program.py", line 121, in scine_module_install
repository.remotes.origin.pull()
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/git/remote.py", line 1068, in pull
res = self._get_fetch_info_from_stderr(proc, progress, kill_after_timeout=kill_after_timeout)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/git/remote.py", line 858, in _get_fetch_info_from_stderr
proc.wait(stderr=stderr_text)
File "/home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/git/cmd.py", line 657, in wait
raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
cmdline: git pull -v -- origin

Maybe editing the bootstrap.py so that it can clear and continue interrupted processes might help? For example:

core = Utils(config.programs()["core"])
if not os.path.isfile(install_dir + "/lib/libcore.so"):
shutil.rmtree(core_build_dir, ignore_errors=True)
core.install(core_build_dir, install_dir, config["resources"]["cores"])

@steinmig
Copy link

According to https://stackoverflow.com/questions/29816529/unsupported-protocol-while-download-tar-gz-package and similar search results, the problem is that your cmake installation does not support SSL.

A possible quick fix would be:

  • Download the file and place it as instructed
  • Instead of starting the bootstrap, go to /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/ and execute:
    • cmake ..
    • make -j4 install
      This might compile and install Molassembler together with the library correctly. Then you could try to relaunch the boostrap, which should hopefully detect that Molassembler is configured and installed, or detects now the installed nauty library and does not require the download. The later might require you to set:
      export LD_LIBRARY_PATH=/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib:/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib64:$LD_LIBRARY_PATH

Again, this is just a hot fix and might complicate things, if you are updating in the future. The cleaner version is to load/install a cmake version with SSL support

@njh80
Copy link
Author

njh80 commented Jan 19, 2024

Hi @steinmig, sorry for not getting back sooner. I've been having a go and essentially the way bootstrap of Puffin is coded it doesn't try to check if modules are already there first. It just starts the build in the same folder and evidently errors when it realises the things are already there, so that would need a change to the puffin bootstrap method.

I've got around this by editing the puffin module - I'd recommend it as a general fix. However, the module still causes some trouble. I've also had to manually pip install . in each python module because the puffin pip call was installing in the wrong place.

The CMake version should have SSL support but I didn't install this and I'm finding an install of a newer CMake a tad difficult.

Any ideas why the created in situ modules aren't happy? My understanding is that the file ImportNauty.cmake is meant to build a file nautyTargets.cmake but that this is being ignored.
CMakeCache.txt
ImportNauty.txt

-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc
-- Check for working C compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++
-- Check for working CXX compiler: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Scine::UtilsOS found locally at /rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/install/lib/cmake/ScineUtilsOS
-- Enabling GCC specific warning flags
-- Found MKL: /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include
-- Found Intel MKL libraries:
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/include
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_avx2.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_vml_avx2.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_core.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_intel_lp64.so
-- /usr/local/Cluster-Apps/intel/2020.2/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64/libmkl_gnu_thread.so
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found MKL for use with Eigen3
CMake Error at build/nauty/nautyConfig.cmake:1 (include):
include could not find load file:

/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/nauty/nautyTargets.cmake

Call Stack (most recent call first):
cmake/ImportNauty.cmake:12 (find_package)
CMakeLists.txt:77 (import_nauty)

CMake Error at build/RingDecomposerLib/RingDecomposerLibConfig.cmake:1 (include):
include could not find load file:

/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/RingDecomposerLib/RingDecomposerLibTargets.cmake

Call Stack (most recent call first):
cmake/ImportRingDecomposerLib.cmake:11 (find_package)
CMakeLists.txt:83 (import_rdl)

-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- pybind11 v2.10.4
-- Found PythonInterp: /home/njh80/modulesSource/miniforge3/envs/chemoton/bin/python3 (found suitable version "3.8.18", minimum required is "3.6")
-- Found PythonLibs: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/libpython3.8.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Pybind11 was not found in your PATH, so it was downloaded.
-- Found PY_pytest: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/pytest
-- Found PY_scine_utilities: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/scine_utilities/scine_utilities.cpython-38-x86_64-linux-gnu.so
-- Could NOT find PY_pybind11_stubgen (missing: PY_PYBIND11_STUBGEN)
-- Not generating typechecking stubs for python package as pybind11-stubgen was not found
-- Found PY_pip: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/pip
-- Found PY_sphinx: /home/njh80/modulesSource/miniforge3/envs/chemoton/lib/python3.8/site-packages/sphinx
-- Configuring incomplete, errors occurred!
See also "/rds-d7/user/njh80/hpc-work/chemoton-conda/puffin/software/build/molassembler/build/CMakeFiles/CMakeOutput.log".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@steinmig @njh80