SOS! Please help me how to install libmesh with DataTransferKit (dtk) successfully!

[haxushu]

Goal

Run the example program Solution Transfer: Transfer Solution From One Mesh to Another successfully.

Plan

1. Install Trilinos with dtk enabled
2. Install libmesh with Trilinos

Done

1. Clone trilinos release 12.4.1
2. Clone DataTransferKit release 2.0.0 to Trilinos's directory (release newer than 2.0.0 doesn't contain several header files such as DTK_MeshContainer.hpp)
3. Configure Trilinos and Dtk using the following script (mkdir build && cd build)

# Clear previous configure
rm -rf CMakeCache.txt
rm -rf CMakeFiles

cmake .. \
    -DCMAKE_INSTALL_PREFIX=$HOME/soft/Trilinos_diy    \
    -D CMAKE_BUILD_TYPE=Release \
    -DTrilinos_ENABLE_Thyra:BOOL=ON \
    -DTPL_ENABLE_MOAB=OFF \
    -DTPL_ENABLE_Libmesh=OFF \
    -DLibmesh_LIBRARY_DIRS='/home/xushu/soft/libmesh_diy_nodtk/lib' \
    -DLibmesh_INCLUDE_DIRS='/home/xushu/soft/libmesh_diy_nodtk/include' \
    -D TPL_ENABLE_MPI=ON \
    -D TPL_ENABLE_BLAS=ON \
    -D TPL_ENABLE_LAPACK=ON \
    -D TPL_BLAS_LIBRARIES:STRING="/usr/lib/x86_64-linux-gnu/libblas.so.3" \
    -D TPL_LAPACK_LIBRARIES:STRING="/usr/lib/x86_64-linux-gnu/liblapack.so.3" \
    -D TPL_ENABLE_Boost=ON \
    -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION=ON \
    -D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
    -D Trilinos_EXTRA_REPOSITORIES="DataTransferKit" \
    -D Trilinos_ENABLE_DataTransferKit=ON \
    -D DataTransferKit_ENABLE_DBC=ON \
    -D DataTransferKit_ENABLE_TESTS=OFF \
    -D DataTransferKit_ENABLE_EXAMPLES=ON \

4. Build and install Trilinos. However, Makefile.export.trilnos is not found in $HOME/soft/Trilinos_diy. I copy the Makefile in the build directory to the install directory and rename it Makefile.export.trilnos, since I know the libmesh configure needs this file to check trilinos.
5. Configure libmesh with ../configure --with-trilinos=$HOME/soft/Trilinos_diy and build.

Problem

1. Everything is ok until CXXLD fparser_parse-opt causes collect2: error: ld returned 1 exit status:

/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `Epetra_Import::~Epetra_Import()'
/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `DataTransferKit::CommIndexer::l2g(int) const'
/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `typeinfo for DataTransferKit::EntityIterator'
/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `Epetra_MultiVector::Norm2(double*) const'
/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `Epetra_MultiVector::ReciprocalMultiply(double, Epetra_MultiVector const&, Epetra_MultiVector const&, double)'
/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `CellTopologyData const* shards::getCellTopologyData<shards::Pyramid<14u> >()'
/usr/bin/ld: ./.libs/libmesh_opt.so: undefined reference to `Tpetra::DistObject<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::doImport(Tpetra::SrcDistObject const&, Tpetra::Import<int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const&, Tpetra::CombineMode, bool)'

there are still many lines but they are omitted for brief, the whole log link is here.

2. I make -n to show the command and try to manully link it to the trilinos lib:

 ./libtool --silent --tag=CXX   --mode=link mpicxx -std=gnu++17 -O2 -felide-constructors -funroll-loops -fstrict-aliasing -Wdisabled-optimization    -fopenmp -no-undefined -o fparser_parse-opt src/apps/fparser_parse_opt-fparser_parse.o libmesh_opt.la  -Wl,-rpath,/home/xushu/soft/Trilinos_diy_12_2/lib -Wl,-rpath,/home/xushu/soft/petsc-3.16.6/arch-linux-c-debug/lib -L/home/xushu/soft/petsc-3.16.6/arch-linux-c-debug/lib -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9 -lz -lpetsc -lflapack -lfblas -lpthread -lmpichfort -lmpich -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -ldl 

with the following added

-L$HOME/soft/Trilinos_diy/lib
-lclassic_dtk                -lintrepid          -lstk_mesh_fixtures      -lteuchoscomm           -ltpetra -lIogn         -ldtk_basicgeometryadapters  -lio_info_lib       -lstk_ngp_test           -lteuchoscore           -ltpetraclassic -lIogs         -ldtk_c_api                  -lkokkosalgorithms  -lstk_topology           -lteuchoskokkoscomm     -ltpetraclassiclinalg -lIohb         -ldtk_fortran_api            -lkokkoscontainers  -lstk_unit_test_utils    -lteuchoskokkoscompat   -ltpetraclassicnodeapi -lIonit        -ldtk_interface              -lkokkoscore        -lstk_util_command_line  -lteuchosnumerics       -ltpetraext -lIoss         -ldtk_intrepidadapters       -lkokkoskernels     -lstk_util_diag          -lteuchosparameterlist  -ltpetrainout -lIotm         -ldtk_operators              -lrtop              -lstk_util_env           -lteuchosparser         -ltriutils -lIotr         -ldtk_stkmeshadapters        -lsacado            -lstk_util_parallel      -lteuchosremainder -lIovs         -ldtk_utils                  -lshards            -lstk_util_registry      -lthyracore -lbelos        -lepetra                     -lstk_io            -lstk_util_util          -lthyraepetra -lbelosepetra  -lepetraext                  -lstk_io_util       -lstratimikos            -lthyraepetraext -lbelostpetra  -lgtest                      -lstk_mesh_base     -lstratimikosbelos       -lthyratpetra

Unfortunately, I still failed and record the log.

HELP

I currently need Solution Transfer urgently. Could someone help me how to deal with this or show me how to install libmesh with dtk? Thanks a lot!

[jwpeterson]

Hi,

Our Trilinos support is definitely a bit out of date. Looking at m4/trilinos.m4, it looks like the last time we seriously tried to support it was Trilinos 11 (?) and you are trying to use Trilinos 12 so that could be one cause of your issues. Based on the linking errors you are getting, I think you are on the right track in trying to add more libraries on the link line, but unfortunately Trilinos is a large library and difficult to link against... I would try and see if there is some type of trilinos-config script that gets installed with Trilinos that you could run and have it tell you the correct link line to use. Other than that I don't really have any good suggestions.

--
John

[haxushu]

John,

Thanks for your suggestions! You save me from the linking stage: ). Now I have installed libmesh with Trilinos 10 + dtk2.0.0 and run the Solution Transfer program. However, the same problem with issue 2130 happens: the transfered solution isn't reliable. It seems to have last for a long time and may be hard to deal with.

In fact, what I want to do is to solve a PDE with two variables on respective adaptive meshes. So I think I need to transfer one variable from one mesh to the other, then solve the equation system and finally transfer that back. Are there any classes or methods in libmesh supporting me to achieve that?

Thanks a lot!

Shu