INSTALL-NOTES

****************************************************************************
*  Copyright 1997-2017                                                     *
*  Lawrence Livermore National Security, LLC.                            *
*  All rights reserved.                                                    *
****************************************************************************
****************************************************************************
*                                                                          *
*     Structured Adaptive Mesh Refinement Applications Infrastructure      *
*                   http://www.llnl.gov/CASC/SAMRAI                        *
*                                                                          *
****************************************************************************
*    For installation help, see INSTALL-NOTES.                             *
*    For release notes, see RELEASE-NOTES.                                 *
*    For copyright information, see COPYRIGHT and LICENSE.          *
*    For code and user documentation, see files in /docs directory         *
*    For help and information about updates and releases, please send      *
*        email to samrai@llnl.gov                                          *
*    For reporting bugs and suggestions, internal CASC users please use    *
*        the CASC bug tracking system at http://www-casc-llnl.gov/bugs/    *
*        and external users please send email to samrai-bugs@llnl.gov      *
*    For more information on SAMRAI see http://www.llnl.gov/CASC/SAMRAI/   *
****************************************************************************
****************************************************************************

Overview
--------

This document describes the SAMRAI installation process.

Documentation for the library can be found in the docs directory after
you untar the distribution.  See the docs/main.html web page for pointers
to other documents.

Directory Structure
-------------------

We recommend the following directory structure:

	SAMRAI-vX.Y.Z/
		SAMRAI/
		ARCH1-CXX1-OPTDEBUG1/
			config/ lib/
		ARCH2-CXX2-OPTDEBUG2/
			config/ lib/
		ARCH3-CXX3-OPTDEBUG3/
			config/ lib/

where SAMRAI/ is the SAMRAI source distribution and the other
subdirectories contain library archives and configuration files
for a particular architecture, compiler, and set of optimization
flags.  For example,

	SAMRAI-v3.3.3/
		SAMRAI/
		solaris-g++-debug/
			config/ lib/
		solaris-KCC-opt/
			config/ lib/
		rs6000-g++-optdebug/
			config/ lib/

The subdirectories config/, lib/, etc.,
are created automatically by the SAMRAI installation procedure
(configure).

(1) Unpacking the Distribution
------------------------------

The SAMRAI source code is generally distributed as a gzipped tar file.
SAMRAI version numbers are of the form X.Y.Z, where X is the major release
number, Y is the minor release number, and Z is the bug fix patch number.
To unpack the distribution as described above, do:

        (a) gunzip SAMRAI-vX.Y.Z.tar.gz
	(b) mkdir SAMRAI-vX.Y.Z (substitue appropriate version numbers)
	(b) cd SAMRAI-vX.Y.Z
        (e) tar xvf ../SAMRAI-vX.Y.Z.tar

You may need to use the GNU tar (often named gtar) due to name length
problems with some of the native tar implementations.

Note that this assumes that the SAMRAI tar file is in the same directory
as SAMRAI-v.X.Y.Z.  If this is not the case, you will need to change the
path in the zcat to point to the SAMRAI tar file.

After these steps, the directories should look as follows:

	SAMRAI-vX.Y.Z/
		SAMRAI/

(2) Configuring Make Options
----------------------------

SAMRAI uses the GNU autoconfiguration toolkit.  The ``configure'' shell
script attempts to guess correct values for various system-dependent features
needed during the compilation.  These values are used to generate Makefiles
in the SAMRAI subdirectories. 

The object files for SAMRAI files are typically compiled into a separate
directory from the SAMRAI source.  This approach facilitates building separate
SAMRAI installations in the same directory structure (e.g., for an SGI or Sun
that share a file system).  This object directory can be placed at the same
level as the SAMRAI distribution:

	SAMRAI-vX.Y.Z/
		SAMRAI/
		objs/

where the directory name ``objs'' is arbitrary.  To configure SAMRAI for your
architecture, do the following:

	(a) mkdir objs
	(b) cd objs
	(c) sh ../SAMRAI/configure [options]

where [options] are a set of optional flags to select different configuration
options.   Use "../SAMRAI/configure --help" to get a list of all possible
options and their explanations.  LLNL users may wish to use the
"config.suggest" tool which provides a set of configure options that work on
CASC and LC systems.  

    LLNL Users:  
	(a) sh ../SAMRAI/config.suggest         -- lists architectures on which
						   we have configured.
	(b) sh ../SAMRAI/config.suggest [arch]  -- prints configure options

        (c) ../config.suggest [arch] | xargs ../configure -- use options

See the section "Configure Options" below for more details on supported 
configuration options.  

(3) Editing Options after Configuration
---------------------------------------

The configuration script will create a number of files in the ``objs'' object
directory, including config, include, lib, and source directories.  The source
directory contains Makefiles that will be used to compile the SAMRAI code.  The
config and include subdirectories contain the following file paths:

		config/Makefile.config
		include/SAMRAI/SAMRAI_config.h

These two files contain the configuration information required for the SAMRAI
build.  You may edit these files to fix options that were incorrectly set by
the SAMRAI configuration script.  The Makefile.config file is included by every
SAMRAI Makefile and contains definitions such as compilers and flags.  The
SAMRAI_config.h file is a configuration file included by all C++ source files.
It #defines certain macros (e.g., HAVE_MPI) that reflect the architecture and
options specified to ``configure'' at configuration time.

(4) Making the Libraries
------------------------

Change directory into ``objs'' and GNUmake (standard Unix make will
NOT work) to build the libraries.  You may make the entire library or
only those sections for a specified problem dimension.

	(a) cd objs
	(b) gmake library

Get a cup of coffee.  Go watch ``The Simpsons.''  Compiling all of the
SAMRAI distribution will take from twenty minutes to well over an hour, unless
you have a more modern operating system and can take advantage of the
parallelism in gmake.  Then building SAMRAI will build in significantly shorter
amount of time.

You may also re-make the archive via ``gmake archive'' or remove all object
and library files via ``gmake clean''.

(4) Making the tools
------------------------

The optional tools may be built using:

    gmake tools

This includes the restart-redistribute utility to convert a SAMRAI
restart file to run on N processors to one compatible with running on
M processors.


(6) Installing the Libraries
----------------------------

After the libraries have been created, install the libraries and configuration
files using:

	(a) gmake install

This will move files into the subdirectory specified by PREFIX in the
configuration command.  The default behavior will place the installed
code in a directory of the form ARCH-CXX-DEBUG (see above).  

IMPORTANT NOTE:  When linking to an installed version of the library,
it is not necessary to define SAMRAI or OBJECT in the user's
Makefile because these are defined by the installed version of 
Makefile.config file which should be included in the user's Makefile.
However, for development work that links in the libraries from
the ``objs'' directory, the user makefile must define SAMRAI to be the
top of the SAMRAI source tree and OBJECT to be the ``objs'' directory.

(6) Cleaning Up
---------------

After installing the libraries, you may remove the source and 
object directory ``objs'' in which SAMRAI was compiled.  However, 
it is recommended that you do not remove the SAMRAI source
directory, since it contains source files for the debugger.


Configure Options:
-------------------

Use the help message from the configure script:

        ./configure --help 

to list all available configuration options.  

The default configuration builds support for SAMRAI datatypes based on
bool, char, int, float, double, and dcomplex.

Substitute the locations of the the various packages you wish to
include.  SAMRAI optionally uses a number of specialized high-level
libraries, such as HDF5, PETSc, Sundials, etc.  By default, these are
disabled.  To use any of the libraries, you must know where they are
installed on your particular system and use the appropriate configure
flag to specify the location.

For many, particularly users at LLNL, it may be easier to just use one of 
the configurations tested by the development team.  To do this, use the 
output of the ``config.suggest'' script as the argument to configure.
This script translates a simple index string into configure options
that are known to work (at the time of release).  To get help on using
the "config.suggest" script, run it without any arguments.  The script
"config.suggest" simply gives a suggestion.  You are ultimately
responsible for looking at the options, understanding them and modifying
them to suit your needs.  The script "config.suggest" does not show all
possible configure options--use "configure --help" for that.

	sh ../SAMRAI/config.suggest         -- lists architectures supported
	sh ../SAMRAI/config.suggest [arch]  -- prints configure options
        ../config.suggest [arch] | xargs ../configure -- use options

The default settings used by config.suggest refer to locations on the
LLNL systems where we have found them.  However, please note that these
locations ARE subject to change and it is not feasible for us to keep up
with those changes here.  If you are at LLNL, please check with the
custodians of the libraries you would like to use.  They may provide
precompiled installations you can use.

Please note that the SAMRAI-PETSc vector interface was developed using
PETSc version 2.3.3.  In order to use the PETSc library, version 2.3.3
MUST be installed on your system.  SAMRAI is incompatible with prior
(and possibly more recent) PETSc releases.  There is a tight coupling
between SAMRAI and PETSc so other versions will potentially have
interface changes that will break things.  In addition, note that
PETSc uses the blas routines, so remember to use the blas option (see
the --help message) if you will be linking to PETSc.

For compilers FORTRAN, C, and C++, the list of default compilers is given in
the order in which they are tried.  The first compiler to successfully create 
and run a simple test is chosen.  If a different compiler is needed, it must be 
specified on the configuration line using `--with-COMPILER[=compiler]`.

The defaults for the command
        sh ../SAMRAI/configure 
 are:
        --enable-debug
        --disable-opt
        --enable-vpath (except under IRIX64)
        --disable-xargs
        --with-MPI
        --with-CC     (CC gcc cc)
        --with-CXX    (mpKCC KCC g++)
        --with-F77    (f77 g77 xlf f2c)
        

To change default settings, use the flags described above. For example:

	sh ../SAMRAI/configure --with-CXX=g++

	sh ../SAMRAI/configure --with-CXX=/usr/bin/CC --without-MPI

	sh ../SAMRAI/configure --disable-debug --enable-opt

	sh ../SAMRAI/configure --with-petsc=/usr/casc/petsc

To link with the proper MPI libraries, make sure that your PATH environment
variable includes the location of the C compiler script needed to correctly
link with MPI (e.g., mpicc for MPICH).

Note that some machines pre-define certain environment variables that will
over-ride those in the script.  For example, IBM machines are prone to defining
ARCH to be rs6k, whereas the configure script would have returned ARCH as
rs6000.  These pre-defined environment variables may cause problems with the
configuration script.

Versions of libraries/tools used by the development team
--------------------------------------------------------

The following are the versions of libraries and tools we have used in
the development of this release.  This list should not be considered
as a requirement list (with the exception of PETSc), it is intended to
provide information on versions we believe should be working.  Users
who encounter problems compiling and linking and want to see if they
are using a version significantly older or newer.  The versions listed
are used in our regression test suite.

NOTE: Because our visualization and restart capabilities are both dependent
on HDF5, the configure script will stop if you do not specify an HDF5
installation in the configure step.  You may still build the library
without it by explicitly specifying "--without-hdf5" in the configure
arguments.

Tool 		Version(s)
--------------  --------------------------------------  
gcc		4.5.0, 4.6.0

icc		12.0.191, 12.1.293

xlc	        v9

pathscale	Does not currently work.  Core dumps and produces incorrect results.

PGI             Does not currently work.

Boost		1.45.0, 1.46.1

HDF5		1.8.4, 1.8.5

MPICH2          1.3.2    (MPICH1 is no longer supported and 
	                  has bugs which will hang SAMRAI)

OpenMPI	        1.5

PETSc 	        3.2    (this version is required if PETSc is used)

Sundials        2.3.0

hypre           2.6.0b

VisIt		1.8.0

doxygen	        1.6.1

graphviz	2.26


Visualization Tools:
--------------------

SAMRAI uses the VisIt visualization system.  This replaces the
vizamrai utility in previous releases.  VisIt binaries, source, and
documentation are available at http://www.llnl.gov/visit.  You must
compile with HDF in order to create VisIt output files.  See the
SAMRAI/test/applications/Euler program for an example of how to use
the VisItDataWriter.

Location users can find VisIt in /usr/apps/visit on CASC systems and
/usr/gapps/ on LC systems.

Performance Analysis Tools:
---------------------------

SAMRAI maintains links to the Tau performance analysis tool.  Tau
(Tuning and Analysis Utilities) is a freely available tool out of the
University of Oregon (http://www.cs.uoregon.edu/research/paracomp/tau)
that is available on all CASC and LC platforms.

To use Tau, you must use the --with-tau configure option.

Links to Tau are maintained through the SAMRAI timers.  Hence, you
must manually instrument your code with SAMRAI timers to use it. Tau
provides a profile view of times on each processor so it is useful to
identify load imbalances and distribution of computational workload.

Tau is available on a variety of machines and is invoked using the 
``--with-tau=<tau-makefile>'' where the location of the Tau makefile 
maintained on LLNL systems is as follows: 

   i386-linux (casc linux):
      (for standard timers only - no hardware counters or mem checking)
         --with-tau=/usr/casc/samrai/tau/tau-2.13.5/i386_linux/lib/Makefile.tau-linuxtimers-mpi

To view the profile files that Tau generates upon running the code, use
the "jracy" tool located in <tau-dir>/bin/jracy, where <tau-dir> corresponds
to the above tau directory locations (i.e. everything before /lib).

See the document SAMRAI/docs/userdocs/TimerInstrumentation.pdf for
more information about the use of Tau.

Documentation
--------------------

SAMRAI uses the Doxygen system for building code documentation.  The
Graphviz is also utilized for building the inheritance/relationship
diagrams within the documentation.

Please note that older version of Doxygen will hang when building the
documentation. The SAMRAI team is using the versions indicated above;
if you are using older versions it may not work.

Doxygen is available at:  http://www.stack.nl/~dimitri/doxygen
Graphviz is available at: http://www.graphviz.org/

****************************************************************************
                            Platform notes
****************************************************************************

uP and IBM Power Systems
------------------------

On the Livermore Computing IBM Power Systems the configure script
needs to be run with the "nopoe" command in order to correctly
determine the compiler settings.  For example:

nopoe  ../SAMRAI/configure `eval ../SAMRAI/config.suggest up_xlc` 

The regression test suite needs to be run with the SHELL set to bash
(or some other non-IBM sh shell).

gmake SHELL=/usr/local/bin/bash checktest

The default sh (/bin/sh) does not work with some of the scripting
features being used.

Apple Mac OS X
--------------

Note that this port has very little support by the LLNL SAMRAI team; we
don't use Mac's.

SAMRAI was compiled successfully on Leopard.  The GNU 4.3 compilers
and HDF that are configured using the DarwinPorts port were used and
Xcode 3.0 installed for MPI.  Note that the X code toolkit from Appple
does not include fortran which is why an alternative compiler chain
is used.

For reference the following is the output of the "port installed" command 
to show what development environemnt was used for the Mac.

   > port installed
     The following ports are currently installed:
       emacs @22.2_0 (active)
       expat @2.0.1_0 (active)
       gawk @3.1.6_0 (active)
       gcc43 @4.3.0_1 (active)
       gettext @0.17_2
       gettext @0.17_3 (active)
       gmp @4.2.2_1 (active)
       hdf5 @1.6.7_3 (active)
       libiconv @1.12_0 (active)
       mpfr @2.3.0_4+darwin_i386
       mpfr @2.3.1_0+darwin_i386 (active)
       ncurses @5.6_0 (active)
       ncursesw @5.6_0+darwin_9
       ncursesw @5.6_1 (active)
       zlib @1.2.3_1 (active)
  
XCode version is:

      > xcodebuild -version
      Component versions: DevToolsCore-921.0; DevToolsSupport-893.0

Configure was run using:

          > eval ../SAMRAI/configure `../SAMRAI/config.suggest macosx_gnu`

We also tested using the MPI compiler frontend wrapper with (note that
the default XCode compilers must be overridden):

   export CXX=g++-mp-4.3
   export CC=gcc-mp-4.3
   export F77=gfortran-mp-4.3
   eval ../SAMRAI/configure \
         --with-CXX=mpicxx \
         --with-CC=mpicc \
	 --with-F77=mpif77 \
	 --with-hdf5=/opt/local \
	 --with-parallel-run-bin=/usr/bin/mpirun


****************************************************************************
****************************************************************************
*  Looking for list of configure options?  We have encapsulated this in    *
*  the new ``config.suggest'' tool.                                        *
*                                                                          *
*  SEE DISCUSSION UNDER "CONFIGURE OPTIONS" AT LINE 208 ABOVE.             *
****************************************************************************