IMPORTANT NOTE: THE CMAKE SYSTEM IS ACTIVELY BEING DEVELOPED
This means, if you are reading these instructions from a non-release branch/tag, they could be out-of-date. If you need the latest Xyce capabilities, it is recommended to use the Autotools build, as documented on the Xyce Building Guide.
At the time of this writing, the CMake system will build only against the develop branch of Trilinos. We are targeting the Trilinos 14 release, which will have a rewritten CMake system, already on Trilinos develop, that required changes in the Xyce CMake system.
This guide describes the basic process for compiling and installing a Xyce binary using the CMake build system. It is easiest to view these instructions with full formatting on the Xyce GitHub website. (For instructions on building Xyce with the Autotools system, see the Xyce Building Guide.)
If you do not want to build from source, binary installers for Windows, Mac and Red Hat Linux are made available for every release of Xyce on the Xyce website.
Xyce can be built in two variants: serial or with MPI parallelism. The variant is determined by whether Trilinos is built in serial or with MPI. While the parallel and serial variants can co-exist on a system, they must be installed in different directories (for both Trilinos and Xyce).
If you have not yet done so, clone the Xyce repository from GitHub, or download one of the release tarballs either from GitHub or the Xyce website.
The main challenge in building Xyce involves properly obtaining and/or building the third-party libraries (TPLs), particularly Trilinos. We are in the process of developing a CMake "superbuild" capability that can automatically download and build many of the TPLs. The Using the Superbuild section covers that approach. The Standard Build Approach section covers the more traditional method. For either method, a certain minimal set of dependencies are required, which is covered first.
Note that the install/uninstall commands may require the use of sudo on
Unix-like systems.
If many flags are being given to CMake on the command line (for Trilinos or Xyce), you might want to create a configuration script. See the Configuration Scripts section for more detail.
Some systems require modifications to the following instructions. These are covered in the System-Specific Modifications section for the following systems:
Most of the software needed to build Xyce is available via package management systems. (See the Modifications section for MacOS and Windows systems.)
You will need to obtain the following tools if they aren't already loaded on your system:
- C/C++ compiler suite — C++11-compatible (e.g., gcc 4.9 or later, Clang 3.3 or
later)
(These could be in separate packages on your system.) - Fortran compiler (e.g., gfortran)
- Some package managers bundle Fortran with the C and C++ compilers.
- Trilinos has Fortran code, but leveraging it is technically optional. Not using the Fortran code, though, could result in slightly lower performance.
- Xyce, itself, does not use Fortran.
- Build system (e.g., Make, Ninja, Jom)
- CMake (3.17 or later)
- Bison (3.0.4 or later)
- flex (2.5.34 or later)
You will also need the following libraries:
On some systems, some of the tools and/or libraries also have "developer" library packages associated with the primary packages. Those will also need to be installed.
The standard build approach requires all the third-party software to be in place prior to invoking CMake to configure Xyce. Be sure you have followed the instructions in the Obtaining the Dependencies section, above. Then proceed through the following tasks, which are broken into three steps:
- Obtaining the TPLs (except Trilinos)
- Building Trilinos
- Building Xyce
We encourage you to understand the steps below; but, after obtaining the TPLs, the TL;DR steps for a serial build on a Unix-like system are:
cd <your-build-directory>
mkdir trilinos-build
cd trilinos-build
cmake \
-D CMAKE_INSTALL_PREFIX=/path/to/Trilinos_install \
-C path/to/Xyce/cmake/trilinos/trilinos-config.cmake \
path/to/Trilinos
cmake --build . -j 2 -t install
cd ..
mkdir xyce-build
cd xyce-build
cmake \
-D CMAKE_INSTALL_PREFIX=/path/to/install \
-D Trilinos_ROOT=/path/to/Trilinos_install \
path/to/Xyce
cmake --build . -j 2 -t installThe minimal third-party libraries needed for Xyce are:
- SuiteSparse (2.1.1 or later)
- FFTW (3.x)
For parallel builds you will also need Open MPI (MPICH should also work, but the Xyce team does not regularly build with that library).
The above are available in most package managers. Again, on some systems, the libraries might have "developer" packages that will need to be installed in addition to the primary packages. Anything not available via a package manager will need to be built from source.
Building SuiteSparse is not difficult. However, the only part of SuiteSparse
used by Xyce is AMD. As an alternative building process, we have provided a
"CMakeLists.txt" file in the Xyce/cmake/trilinos/AMD/ directory. Using CMake,
the file will allow you to compile and install only the AMD library. See the
comment block at the top of the file for
instructions on its use.
While Trilinos is available in some package managers, the particular installation may not have all the features required by Xyce. If you find Trilinos is available, you can try skipping to the Xyce section to see if Xyce configures and builds. If not, continue with these instructions.
To build Trilinos on Windows, see the Windows section under System-Specific Modifications.
The following process will produce a serial Trilinos installation that will contain only the libraries needed by Xyce. For the parallel version, be sure to understand the serial build process prior to reading the Building Trilinos with MPI Parallelism section, below.
As mentioned, since the CMake build system is being updated, we require a
version of Trilinos near their development head. At the time of the Xyce 7.6
release, the specific SHA we test is
b91cc3dcd9c94fd57cb6d4b7b7baf49292aa1df5. Earlier versions will likely not
work with Xyce's CMake build system.
To obtain a working Trilinos build, you can clone it from the Trilinos GitHub
Page and checkout b91cc3dcd9.
Alternatively, you can use this direct
link.
If you use a git clone, you can obtain just the b91cc3dcd9 files (and a
faster download) by running:
git clone --shallow-since 2022-09-15 --branch develop https://github.com/trilinos/Trilinos.git
(cd Trilinos ; git checkout b91cc3dcd9)After checking out the specific SHA, above, you will get a warning from git about being in a detached HEAD state. This is ok as we are just building the code.
In order to build Trilinos, first create a "build" directory in a convenient
location. The name does not matter, and can be something simple like
trilinos_build. On Unix-like systems, the default Trilinos installation
location is /usr/local. This can be changed by adding the following flag to
the CMake invocation.
-D CMAKE_INSTALL_PREFIX=</path/to/Trilinos_install> \As with Xyce, both a parallel and serial build of Trilinos can exist on the
same system, but they must be in different directories. We recommend specifying
unique sub-directories in /usr/local, such as /usr/local/trilinos_serial.
If you install Trilinos in a temporary location, you will need to rebuild it
when building a new version of Xyce. The recommended version of Trilinos does
not change often, so it can save time to have an installed Trilinos library.
If you have compilers or libraries in non-standard locations, see the Other Trilinos Options section, below.
A CMake "initial cache" file, called trilinos-config.cmake, is included in
the Xyce repository in the cmake/trilinos directory. The file contains a
typical set of options for a Xyce-oriented serial Trilinos build.
To configure Trilinos (using the default /usr/local install location), enter
the build directory and run:
cmake -C path/to/Xyce/cmake/trilinos/trilinos-config.cmake path/to/TrilinosOnce the configuration step has completed, run the following in the build directory to build and install Trilinos:
cmake --build . -j 2 -t installThe "-j 2" designates the number of processors to be used for compiling Trilinos. Choose an appropriate number for your system.
After the Xyce installation is successful, and if you do not plan on building Trilinos again, you may delete the Trilinos build directory (see the Uninstalling Xyce section first, however).
The following are various flags that might be needed when building Trilinos. These can be added to the command line, or put in a configuration script.
CMake will use the first compiler set it finds on your system. You can specify the compilers by adding the following flags to the CMake invocation:
-D CMAKE_C_COMPILER=<C-compiler> \
-D CMAKE_CXX_COMPILER=<C++-compiler> \
-D CMAKE_Fortran_COMPILER=<Fortran-compiler> \You may need to use a full path if they are not visible in your default paths.
If no Fortran compiler is available you can use:
-DTrilinos_ENABLE_Fortran=OFF \
to disable any Fortran-dependent code in Trilinos.
On some systems it may be necessary to add the following to your CMake invocation if the installed C compiler treats implicitly defined functions as errors, such as Apple's clang.
-DCMAKE_C_FLAGS="-Wno-error=implicit-function-declaration" \
Finally, if the third-party libraries (AMD, BLAS and LAPACK) are not visible in your default paths, use the following flags to help CMake find the libraries:
-D AMD_LIBRARY_DIRS=/path/to/AMD/lib \
-D AMD_INCLUDE_DIRS=/path/to/AMD/include \
-D BLAS_LIBRARY_DIRS=/path/to/BLAS/lib \
-D LAPACK_LIBRARY_DIRS=/path/to/LAPACK/lib \See the Trilinos Build Reference for more details on the Trilinos build options.
To enable MPI parallelism in Xyce, Trilinos must be built with MPI enabled. An
"initial cache" file, called trilinos-config-MPI.cmake, is provided for MPI
builds. In addition to using the MPI-oriented cache file, the MPI compilers
must be explicitly specified to CMake. The following CMake invocation should
work on most systems:
cmake \
-C path/to/Xyce/cmake/trilinos/trilinos-config-MPI.cmake \
-D CMAKE_C_COMPILER=mpicc \
-D CMAKE_CXX_COMPILER=mpicxx \
-D CMAKE_Fortran_COMPILER=mpifort \
path/to/TrilinosNote that the exact compiler names may be different on your system. Also, the
above invocation will install Trilinos in /usr/local.
As with the serial build, once the configuration step is done, run the following in the build directory to compile and install Trilinos:
cmake --build . -j 2 -t installOnce Trilinos is installed, you can build and install Xyce. By default, Xyce
will be installed in the /usr/local/ directory. To specify a different
installation location, add the following flag to the CMake invocation:
-D CMAKE_INSTALL_PREFIX=</path/to/install>Again, if you plan to have both a parallel and serial build of Xyce on your
system, they must be in different directories. We recommend specifying unique
sub-directories in /usr/local, such as /usr/local/xyce_serial.
If Trilinos is not located in your path, add the following flag to the CMake invocation:
-D Trilinos_ROOT=/path/to/Trilinos_installCreate a build directory for Xyce (such as xyce_build) and go into that
directory. Then run CMake using:
cmake [flags] path/to/XyceThen, to build and install Xyce, run:
cmake --build . -j 2 -t installThe "-j 2" designates the number of processors to be used for compiling Xyce. Choose an appropriate number for your system.
Xyce has a Verilog-A model compiler capability, which uses the "Xyce/ADMS" compiler tool. See the Xyce/ADMS Users Guide for more information on the capability and for instructions on using Xyce/ADMS.
To enable the feature with CMake, install ADMS prior to building Xyce. Then, to enable the capability in the Xyce build, add the following flag to the Xyce CMake invocation:
-D Xyce_PLUGIN_SUPPORT=ONThe CMake support for the plugin capability is still being developed. As such, there are some differences from the website:
- The "toys" example is installed in
/path/to/install/share/examples/toys. - The
buildxyceplugin.shscript requires the absolute path to the.vafiles. - The name of the plugin file can vary by system (e.g., on a Mac, the "toys" library will be called, "libtoys.dylib", not "toys.so").
AT THE TIME OF THIS WRITING, THE SUPERBUILD DOES NOT WORK. IT WILL BE ADDRESSED AS PART OF THE CMake REWRITE
While easy, this approach has not been thoroughly tested, so should be considered a "beta" capability. Also, it currently installs everything into an "install" directory in the build directory, which may not be ideal for most users. Therefore, we encourage people to use the standard build approach, which involves only a few more steps.
Assuming the dependencies have been installed, CMake will automatically build the following components:
- ADMS
- FFTW
- SuiteSparse
- Trilinos
Then CMake will compile a serial build of Xyce that enables the (optional) Xyce/ADMS model plugin capability. Note that, since CMake is building several packages, the process could take a long time.
To perform a superbuild, follow this procedure:
- Create a "build" directory somewhere on your system (the location and directory name are not important)
- From the build directory, run the following command:
cmake -D Xyce_USE_SUPERBUILD=ON path/to/Xyce
- Then run
The "-j 2" designates the number of processors to be used for compiling. Choose an appropriate number for your system.
cmake --build . -j 2
The Xyce CMake does not create an uninstall script. However, on installation it does produce an "install manifest" file, which lists every installed file with their full path. To remove a Xyce installation from a Unix-like system, simply run:
xargs rm < install_manifest.txtNote that the above method is immediate and permanent. You may also want to
uninstall Trilinos. As with Xyce, an install_manifest.txt file is produced
when Trilinos is installed.
If you do not want to keep the build directories, simply copy the
install_manifest.txt file(s) to a safe location (the file name can be
changed). Then Xyce and/or Trilinos can be uninstalled at any time using the
xarg command, above (be sure to specify the correct filename).
If you wish to test the Xyce installation, run the Xyce Regression Suite. See the Running the Xyce Regression Suite documentation on the Xyce home page. Note that the test suite is controlled with Perl and Bash scripts, and some tests require Python with Scipy and Numpy.
If many flags are being applied as part of a CMake invocation, you might want to create a configuration script. This is essentially a shell script with the CMake invocation. The following commands are for Unix-like systems. Analogous scripts can be created for Windows systems.
As an example, to configure an MPI-enabled Trilinos build, one might create a file containing:
#!/bin/sh
cmake \
-C path/to/Xyce/cmake/trilinos/trilinos-config.cmake \
-D CMAKE_C_COMPILER=mpicc \
-D CMAKE_CXX_COMPILER=mpicxx \
-D CMAKE_Fortran_COMPILER=mpifort \
-D CMAKE_INSTALL_PREFIX=$HOME/install/Trilinos/mpi/ \
path/to/TrilinosMore options can be added as needed. The file can be made executable by running the following on the command line:
chmod a+x <filename>Running the configuration script in the build directory is the same as running the CMake configure line directly.
The instructions above are applicable to most operating systems. Modifications for specific systems are added below as we become aware of them.
Compiling Xyce on Windows is not a small task at the moment, primarily because Windows does not have the equivalent of a package manager. Internally, we use the Intel compiler suite with the Intel Math Kernel Library (MKL). The MKL provides the BLAS, LAPACK and FFT capabilities (removing the need for FFTW). At the beginning of 2021, Intel rebranded their tool chains as the oneAPI Toolkits, and makes them available for free. The oneAPI Base Toolkit is sufficient for building Xyce and Trilinos. Running the setvars.bat script that installs with oneAPI can help ensure that the necessary environment variables and paths are set for building.
Note that the oneAPI toolkit requires Microsoft C++ BuildTools to be installed first to provide required system libraries. The Microsoft Build Tools will also supply CMake and the NMAKE build tool.
When building Trilinos on Windows, cloning and configuring take a few extra
steps. To clone on Windows, it may be necessary to to do a sparse checkout;
for a brief period of time in Trilinos, there were a few files of the form
aux.* which caused an issue in Windows. To work around this issue,
git clone ^
--branch develop ^
--single-branch ^
--no-checkout ^
--sparse ^
--config core.protectNTFS=false ^
--shallow-since 2022-09-15 ^
https://github.com/trilinos/Trilinos.git ^
source/Trilinos
pushd source\Trilinos
git sparse-checkout init
git sparse-checkout set --no-cone "/*" "!/packages/muelu/research"
git checkout b91cc3dcd9
popd
Trilinos does not test their code on Windows. The initial configuration file
for Trilinos in path/to/Xyce/cmake/trilinos/trilinos-config.cmake may
require a few extra options:
cmake ^
-C path\to\Xyce\cmake\trilinos\trilinos-config.cmake ^
-D HAVE_TEUCHOS_LAPACKLARND=OFF ^
-D Trilinos_ENABLE_Stokhos=OFF ^
-D Trilinos_ENABLE_Sacado=OFF ^
-D Trilinos_ENABLE_Amesos2=OFF ^
...
path\to\Trilinos
On Windows, Bison and flex are available via the WinFlexBison package. For SuiteSparse, see the Building SuiteSparse section above, under Obtaining the TPLs. Once all the TPLs are in place, continue with the standard Trilinos and Xyce build processes.
To compile Xyce on Windows, there are additional CMake options that must be added in generating the build configuration.
-D Xyce_USE_FFT=TRUE
-D Xyce_USE_INTEL_FFT=TRUEThe Xyce regression suite must be run in a Unix-like environment with Perl. Therefore, to use it in Windows, you will need to install Cygwin. It might also be possible to use the Windows Subsystem for Linux (WSL), but we have no experience with it.
Cygwin installs a minimal set of packages by default, so all of the dependencies will have to be explicitly added, including the AMD library. As of this writing the required packages are:
- git (optional)
- gcc-core
- gcc-g++
- gcc-fortran
- make (or ninja)
- cmake
- bison
- flex
- liblapack0, liblapack-devel
- libsuitesparseconfig-devel, libamd-devel
- fftw, libfftw3-devel
Cygwin also needs additional flags to be applied when compiling Trilinos and Xyce. For Trilinos, add the following to the CMake invocation:
-D CMAKE_CXX_FLAGS="-D_BSD_SOURCE -D_GNU_SOURCE" \
-D CMAKE_C_FLAGS="-D_BSD_SOURCE -D_GNU_SOURCE" \
-D AMD_INCLUDE_DIRS="/usr/include/suitesparse" \For Xyce, add the following to the CMake invocation:
-D CMAKE_CXX_FLAGS="-D_BSD_SOURCE -D_GNU_SOURCE" \
-D CMAKE_C_FLAGS="-D_BSD_SOURCE -D_GNU_SOURCE" \Ubuntu releases starting with 17.10 and through the 19.x series (including
18.04), have a broken version of Open MPI in their package repositories. Open
MPI is functional in Ubuntu 20.04. (In the problem releases, the version of
Open MPI in the repositories is compiled with the --enable-heterogeneous
option, which breaks MPI's standard compliance and causes Xyce to fail in some
situations.)
If you are running a version of Ubuntu that has this issue, the only workaround
is to uninstall the OpenMPI package and build OpenMPI from source, without the
--enable-heterogeneous option. Comment 11 of the Launchpad bug
report
contains instructions for how to rebuild and install Open MPI using the Debian
package building system.
If you wish to check whether your system's install of OpenMPI has this issue, you can run a small test program from the Xyce source tree. To test your OpenMPI package, copy the file, Xyce/src/test/MPITest/testBUG967.c, into temporary directory. Then compile and run it using the following commands:
mpicc -o testBUG967 testBUG967.c
mpirun -np 2 ./testBUG967If the run produces any output with the word BAD, your OpenMPI install is broken and cannot be used. You must instead use the workaround described above.
MacOS does not have a native package manager, but there are several third-party package managers available. We have had success with MacPorts and Homebrew.
BLAS and LAPACK are provided by Xcode and should give better performance than the BLAS and LAPACK libraries provided by the package managers. Therefore, you do not need to install those libraries, explicitly.
We have seen a rare failure when using the Apple Clang compilers and not using a Fortran compiler. If the Trilinos build fails while compiling AztecOO, then you will either need to use different C/C++ compilers, or install gfortran with a package manager (it is usually supplied with the gcc package).
Xyce/ADMS on MacOS
The buildxyceplugin.sh script relies on the behavior of Gnu "readlink", which
is very different than BSD readlink (MacOS is based on BSD Unix). You can
obtain Gnu readlink by installing the "coreutils" package from MacPorts or
Homebrew.