INSTALL

======================================================================
GUIDE FOR INSTALLING ``nsim``/``nmag`` all FROM SOURCES (big tar ball)
======================================================================

.. contents:: Table of Contents Installing Nmag all from big tarball

Installation requirements
-------------------------

Building ``nmag all`` will require approximatively 1.4 GB of harddisk space.
After compilation and issuing ``make clean``, ~550 MB will be
used. (The reason for this large footprint is that the ``nmag``
archive tries to make very few assumptions on pre-installed software
and hence has to bring along all the packages it depends on. This is
particularly relevant for computing clusters.)

Depending on hardware characteristics, compilation will usually take
between 15 minutes and 2 hours.

The following software packages have to be available on the build system:

- A sane build environment:

  - The GNU compiler collection: ``gcc``, ``g++``, ...;
  - ``m4`` (scripting language used by configure scripts that are generated by ``autoconf``);
  - other GNU standard tools, such as ``make``, ``tar``, ``gzip``, ``bzip2``, ...;

- Libraries for fast linear algebra: ``blas``, ``lapack``;
- ``gawk`` (GNU awk);
- ``readline`` libraries and headers (for ``python``, ``ipython``, ``nsim_i``).
- zlib / libz (package zlib1g-dev on Ubuntu/Debian)


On Ubuntu 11.10 you can install these requirements using::

  apt-get install g++ libblas-dev libreadline-dev \
    make m4 gawk zlib1g-dev readline-common liblapack-dev


Download
--------

Download the tarball (with a name such as nmag-0.2.0.tar.gz) from http://nmag.soton.ac.uk. (try `this directory <http://nmag.soton.ac.uk/nmag/0.2/download/all>`__ to see available tar balls.


Installation
------------

Untar the archive::

  $ tar xvzf nmag-0.2.0.tar.gz

At this point you should know that, once the package has been compiled,
you won't be able to move the nmag-0.2.0 directory to another location.
Doing so would make the package unusable, since absolute paths are hard-coded
inside the executables.
Therefore be sure to unpack the package into an appropriate directory.
Enter the directory just created by the tar command (which should be
named "nmag-0.2.0")::

  $ cd nmag-0.2.0

and run ``make``::

  $ make

This command will build ``nsim`` and a number of other required
packages.

All the software is installed inside the directory ``nmag-0.2.0``,
so root privileges are not required (see `Setting up the environment`_).

If all goes well, nsim can be started once the 'make' command has completed::

  $ ./nsim/bin/nsim

If this produces a prompt::

  >>>

then ``nsim`` was compiled successfully from source. (However, the libreadline-dev libraries were missing, and thus the prompt is not an Interactive Python prompt.)

If::

  $ ./nsim/bin/nsim

produces a prompt like::

  In [1]: 

then libreadline-dev has been found during the compilation, and the prompt resembles the IPython prompt.

(The interpreter can be left, as usual, by pressing Control+d,
which will send an "End of File" command on its input stream.)


Setting up the environment
~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to run ``nsim`` (and related programs such as ``ncol``,
``nmagpp``, etc.), the shell has to know the path where to find
these utilities. While it is possible to start ``nsim`` as
``./nsim/bin/nsim``, it is much more convenient in the long run
to adjust the shell's $PATH environment variable appropriately.

For the widespread ``bash`` shell, this can be done with these
commands::

  $ cd nmag-0.2.0 # enter the main installation directory
  $ export PATH=`pwd`/nsim/bin:$PATH

After these adjustments, ``nsim`` can be started with the command
``nsim`` without further path adjustments, regardless of the current
directory.

However, as this ``export`` command has to be issued within
very new shell, it may make sense to add it to the shell startup
command file ``.bashrc``.

Executing the command ``make hints`` in the ``nsim`` build directory
will give information on how to adjust ``.bashrc``::

  $ cd nmag-0.2.0
  $ make hints

will produce output similar to ::

  To be able to launch nsim more easily, append the following line
  to the end of your shell configuration file (for BASH users
  this file is .bashrc and can be found on your home directory ~/.bashrc):
  export PATH="/home/me/nmag-0.2.0/nsim/bin:$PATH"

Note that there are other shells which behave differently from bash,
such as in particular cshell or tcshell. For these in particular, the
command to set environment variables is ``setenv`` rather than
``export``.


Testing the software
--------------------

After successful completion, we recommend to run a suite of unit and
regression tests. If these pass (i.e. they do not fail), then
everything seems to work.

The test can be run by calling::

  $ cd nmag-0.2.0
  $ make check

This should take less than one minute to complete. If the output finishes with ::

  ======== tests finished: k passed, l skipped in XX.XX seconds ==========

(where k and l are integers), then all seems to be
well. Congratulations. While you can attempt to run the tests below,
this should not be necessary.

The l tests that have been skipped, take a bit more computation time (of the
order of 1 hour). To test them as well, one can use::

  $ cd nmag-0.2.0
  $ make checkslow


If, after completion of either of these checks, the last line reads::

  ======== tests finished: k passed, m failed, l skipped in XX.XX seconds ========

then m tests have failed. You will also get further debug output
following this summary line. 

Other available test targets are ``make checkmpi`` and ``make
checkhlib``. Please read the corresponding sections in the manual
before running those  -- they will fail unless the right environment has
been set up.


Common test failure
-------------------

The following messages are typical for the output of ``make check`` if the zlib library was not present *during the compilation of nmag*. (This relates to the pytables python module using hdf5 which in turn relies on zlib to read and write compressed data.)

::

  regression/nmesh/import/gmsh1_0/test_gmsh1_0_import.py[2] .F
  ../../../nsim/mpi/test_mpi_hello.py[2] ss
  ../../../unit/nmeshmirror/test_nmeshmirror.py[1] .
  ../../../unit/pycaml/test_refcount.py[1] .
  ../../../unit/pycaml/test_trial.py[2] ..
  ../../nmag/bigbar/mpi/test_slow_mpi_bigbar.py[2] ss
  ../../nmag/bigbar/nmag4/test_slow_bigbar.py[2] ss
  ../../nmag/bugs/twohmx/test_hlib_twohmx.py[1] s
  ../../nmag/demag_twomat/test_slow_bug_ticket175.py[1] s
  ../../nmag/nmagprobe/test_all.py[1] F
  ../../nmag/pymodules/test_python_modules_installed.py[1] .
  ../../nmag/restart/test_restart.py[1] .
  ../../nmesh/import/gmsh1_0/test_gmsh1_0_import.py[2] .F
  ../../../nsim/mpi/test_mpi_hello.py[2] ss
  ../../../../unit/nmeshmirror/test_nmeshmirror.py[1] .
  ../../../unit/pycaml/test_refcount.py[1] .
  ../../../unit/pycaml/test_trial.py[2] ..

  <snip>

  % (complib, default_complib), FiltersWarning )
  HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: H5Dio.c line 174 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: H5Dio.c line 448 in H5D_read(): can't read data
    major: Dataset
    minor: Read failed
  #002: H5Dchunk.c line 1729 in H5D_chunk_read(): unable to read raw data chunk
    major: Low-level I/O
    minor: Read failed
  #003: H5Dchunk.c line 2760 in H5D_chunk_lock(): data pipeline read failed
    major: Data filters
    minor: Filter operation failed
  #004: H5Z.c line 1094 in H5Z_pipeline(): required filter is not registered
    major: Data filters
    minor: Read failed
  Closing remaining open files: /home/test/local/nmag-0.2.0/nsim/tests/mesh/s3n1ellipsoid.nmesh.h5... done
  make[3]: *** [run_dat.h5] Error 1
  make[2]: *** [all] Error 2

  <snip>
 
          #check that file name exists;
          if not os.path.exists(filename):
  E           raise NmeshUserError,"file '%s' does not exist" % filename
  >           NmeshUserError: file 'sphere2.nmesh' does not exist
  
  [/home/test/local/nmag-0.2.0/nsim/interface/nmeshlib/lib1.py:853]
  - - - - - - - -  test_gmsh_1_0_import_gzipped: recorded stdout - - - - - - - - -
         root:2011-12-07 11:24:05,242 nmesh_exceptions.py    8 CRITICAL Exception type: NmeshUserError 
         root:2011-12-07 11:24:05,243 nmesh_exceptions.py    9 CRITICAL Exception msg : file 'sphere2.nmesh' does not exist 
  
  ________________________________________________________________________________
  ======= tests finished: 52 passed, 4 failed, 16 skipped in 18.17 seconds =======
         root:2011-12-07 11:24:05,664   shell.py   85 WARNING SystemExit with exit_status=1
  make[1]: *** [check] Error 1
  make[1]: Leaving directory `/home/test/local/nmag-0.2.0/nsim/tests'




.. comment:
  
  
  Testing the software
  --------------------
  
  After successful compilation, we suggest to run a small test to ensure
  ``nsim`` is working properly.
  `gnuplot <http://www.gnuplot.info>`__ is required for producing the final
  graph with the results. ``gnuplot`` is a plotting program;
  many distributions provide an easy way to install it.
  The script runs a basic simulation example, which should bring up
  most unresolved problems::
  
    $ cd nmag-0.2.0
    $ cd nsim/tests/regression/nmag/bigbar
    $ make check
  
  The simulation should take 5-10 minutes, depending on the machine's
  hardware. In the end, it will produce a graph that can be viewed with
  the command::
  
    $ gv results/results.eps
  
  This graph shows the time evolution of the three components of the
  mean magnetization for a Permalloy bar of size 30 nm x 30 nm x 100
  nm. There should be good agreement between the ``nsim`` data and those
  produced using ``OOMMF`` and ``Magpar`` for the same problem.


Removing the software
---------------------

When compilation has finished, unnecessary files can be removed with
the command ``make clean``. The command ``make uninstall`` will remove
all the files produced in the compilation phase and essentially leave
the bare source tree. (The name ``uninstall`` may seem strange at
first, as this way of building ``nsim`` does not install anything in
system directories such as ``/usr/lib``, but this is justified as we
recursively call ``make uninstall`` on all the packages ``nsim``
depends on which will be installed into a ``nmag-0.2.0`` subdirectory.)

To completely clean nmag from the system, it is sufficient to simply
remove the ``nmag-0.2.0`` directory::

  $ rm -rf nmag-0.2.0

The important point to remember is that the ``nmag``/``nsim``
installation lives entirely within this directory and does not touch
anything outside it. (The directory /tmp may be an exception, though.)



Package and platform specific comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The packages which are more likely to be missing are ``gawk`` and
``readline``.

``gawk`` (GNU awk) is a very common program. Most operating system
distributions provide means to fetch and install a ``gawk`` package,
which often already comes pre-installed. (We need gawk to compile one
of the libraries [Petsc] that we depend on.)

``readline`` is commonly installed on linux systems, but often without
its development files. If the directory ``/usr/include/readline`` (or
``/usr/local/include/readline``) exists, readline development headers
should be available. Otherwise, it is likely that the readline library
is installed without its development files.  On Debian-based Linux
distributions such as Ubuntu, one can issue the command
``aptitude install libreadline-dev``
(resp. ``apt-get install libreadline-dev``) to install the
missing package.

Troubleshooting issues such as ending up with a ``nsim_i`` executable
which does not show a coloured prompt and misses features such as tab
autocompletion are discussed in the section FAQ_.

The GNU compiler collection, as well as ``m4``, ``tar``, ``gzip``, ``bzip2``
should already be present if the machine already has been used for
some software development.

These are widespread developer tools and --- as for ``gawk`` --- the
operating system distribution should provide an easy way to install
them.

The same holds for the ``lapack`` and the ``blas`` linear algebra
libraries.

Debian/Ubuntu
+++++++++++++

A fresh installation on debian (etch) requires the following extra
packages to be installed:

``m4`` ``gawk`` ``zlib1g-dev`` ``g++`` ``libblas-dev`` 
``liblapack-dev`` ``readline-common`` ``libreadline-dev`` ``make``

for example through executing::

  apt-get install readline-dev g++ libblas-dev libreadline-dev \
    make m4 gawk zlib1g-dev readline-common liblapack-dev


Mac OS X
++++++++

On Mac OS X, fink_ can be used to install the additional tools needed by
``nsim`` (such as ``gawk``, for example).


Platforms that compile Nmag from source
---------------------------------------

The Nmag 0.1 software has been compiled successfully (and without intervention)
on the following systems:

   - on Windows XP SP3 (Intel Core 2) through andLinux based on Ubuntu 9.04;

   - IntelMac (=CoreDuo CPU in MacBook) Mac OS X (10.4) system;

   - IntelMac (Intel Core 2 in iMac) Mac OS X (10.4.11) system;

   - PowerPC Mac (MacMini, iMac) Mac OS X Tiger (10.4) system;

   - MacBook Pro (Intel), Mac OS X Leopard (10.5) system;

   - AMD Athlon 64 (running in 32bit mode) with Debian Linux (Etch);

   - AMD Athlon 64 (running in 32bit mode) with Debian Linux (Lenny);

   - Opteron machine running 64bit linux;

   - Pentium 3 machine running 32bit linux;

   - Pentium M machine running 32bit Ubuntu Linux (Gutsy);

   - AthlonXP 2200+ running Ubuntu 7.04 (Feisty), and Ubuntu 7.10
     (Gutsy), both 32 bit

   - Athlon64 X2 4200+ Dual Core running Ubuntu 7.10 64bit

   - IBM x440 system (with 8x1.9Ghz Intel Xeon processors) running
     Red Hat Enterprise Linux AS release 3 (Taroon Update 2)

   - Debian 32 and 64 bit with Pentium 4, E7200, and Q9300 processors 

   - AMD64 Athlon64 3200, OpenSuSE 11.0, 64bit, Kernel 2.6.25.20

   - Dual Xeon quad core processors, 2.4 GHz, 24GB RAM, Red Hat Enterprise Linux Desktop 5.3

   - Fedora 12

Platforms that needed a little help:

   - Aspire One Netbook running the (Fedora based) Linux Linpus Lite
     on intel Atom processor. The user reports that the 'diffutils'
     package on the Linpus is broken and stopped the installation. He
     downloaded the original diffutils from the GNU webpage, installed
     this. Afterwards, Nmag compiled automatically from source (Dec 2008).


The Nmag 0.2 software has been compiled successfully (and without intervention)
on the following systems:

   - Ubuntu 11.10 (Intel 32 and 64 bit)

   - Ubuntu 11.04

   - Mac OS X 10.6 (Snow Leopard)

   - Mac OS X 10.7 (Lion)



.. _FAQ:

FAQ (Frequently Asked Questions)
--------------------------------

.. contents::
  :local:


What is all this talk about ``nsim`` --- Isn't the package named ``nmag``?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``nsim`` is a general purpose multiphysics simulation
package. ``nmag`` is a particular application based on it,
providing micromagnetism.

The major effort is to install ``nsim``. Once this is done
``nmag`` will ''just work''. It is for this reason, that there are
many references to ``nsim`` in this document.

What machines/operating systems does nsim compile on?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This installation tool was designed to work on Linux platforms.  It
does not (yet) work on MS Windows, but it has been successfully
used to compile ``nsim`` on Mac (both PowerPC and Intel
architectures) and may work under other flavours of unix OSes.

See `Platforms that compile nmag from source`_ for a list of systems on
which a successful compilation has been reported.

Why do I not have autocompletion and colours in ``nsim_i``?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You have just installed ``nsim``.  And ``nsim_i`` works, but the
``nsim_i`` output is not coloured and the auto-completition is not
available either. 

Probably ``nsim``, was compiled on a system where the readline library
(or its developer files) was not installed.  ``nsim_i`` needs this
library for features such as command auto-completition and coloured
text.

One way to resolve this is to install the libreadline developer
files (which should in particular provide ``readline.h``),
as explained in the section `Installation requirements`_
and then recompile from scratch.

This is a short-cut to recompile just the python interpreter
(i.e. the readline-dependent part)::

  $ cd nmag-0.2.0
  $ rm -rf .deps_python_* python/
  $ make .deps_python_install
  $ make --dry-run | grep touch | sh

The last line is necessary to update the dependency files:
normally rebuilding ``python`` would cause all the other libraries
(mpich, petsc, etc.) to be rebuilt as well, the last line
is a trick to avoid this!

How can I recompile the ``nsim`` package, *together* with all the required libraries*?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Normally there should be no need to recompile the ``nsim`` package.
If for some reasons you want to do so, you can issue the following
commands::

  $ cd nmag-0.2.0
  $ make uninstall
  $ make

Note that ``make uninstall`` will remove all the files produced by a
previous compilation of ``nsim``. So, all the programs and libraries
required by ``nsim`` will be rebuilt by these commands (this can take
some time).

Can I free some (unused) disk space after compiling nsim from source?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You have compiled ``nsim`` from source and it occupies more about 1.4 GB
of space on your hard-drive. Some directories which were necessary for
the compilation can be removed after the compilation has finished. The
command is::

      $ cd nmag-0.2.0
      $ make clean

This will free about 0.9 GB.

Binary files in nmag-0.2.0/bin not working
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The installation of ``nsim`` produces a number of executables in the
``nmag-0.2.0/bin`` directory. When you try to run some of these, they do
not necessarily work properly. Do you need to be concerned that there
is something wrong with the installation?

No, probably everything is alright.  The software is not installed
in the traditional unix directories (``/usr/bin``, ``/usr/lib``,
etc.) therefore the operating system needs to be instructed where
to find executables, libraries, etc.

``nsim`` does that by calling the shell script
``nmag-0.2.0/exports.bash``.

If (for some strange reason) you want to execute the binary files
under ``.../nmag-0.2.0/bin``, these definitions have to be pre-loaded
into the shell. For example, to start the Objective Caml interpreter
that is compiled as part of the ``nsim`` installation from source, you
could use::

  $ cd nmag-0.2.0
  $ . exports.bash
  $ ./bin/ocaml

This may not be necessary, but is strongly suggested.
(Note that this is specific to the bash shell.)

I want to recompile only ``nsim``, how can I do this?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This should not be necessary for normal users. If you are a developer
and you are extending the ``nsim`` OCaml core, then this is what you
need::

  $ cd nmag-0.2.0
  $ . exports.bash
  $ cd nsim # enter the nsim source directory
  $ make

These commands will only compile ``nsim``, without reconfiguring it.
If you want to reconfigure and recompile ``nsim`` use the following
commands::

  $ cd nmag-0.2.0
  $ rm nsim/config/configuration.inc
  $ make
  
Note that the ``make`` command is used from the main ``nmag-0.2.0``
directory, not ``nmag-0.2.0/nsim``.  These commands will configure
``nsim`` by running ``nmag-0.2.0/config/configure.py`` and passing to it
the proper command line options to specify the paths of the required
libraries.

I get an HDF5-DIAG error message when trying to run an example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The error message reads::

  HDF5-DIAG: Error detected in HDF5 library version: 1.6.5 thread 0.
  Back trace follows.

This error has been seen on a (misconfigured) linux system and is
caused by the hdf5 library believing to link to ``libz`` but
actually linking to a library that has the same name but different
content. In particular, it seems that Matlab provides it's own
``libz`` library (which has nothing to do with the standard
``libz``) and this library was earlier in the search path than the
actual ``libz``.

A workaround is to copy the right ``libz`` (a file like
``/usr/lib/libz.so.*``) to the ``nmag-0.2.0/lib`` directory. It
seems prudent to also copy the symbolic links to this
library, for example with::

  $ cp /usr/lib/libz.so.* nmag-0.2.0/lib


Known issues
------------

On some Suse systems the installation failed, due to a weakness
of the petsc configuration process: petsc seems unable to link against
the lapack library. The problem was solved by adding an appropriate
flag to the ``LDFLAGS`` environment variable::

  $ cd nmag-0.2.0
  $ export LDFLAGS=-lgfortran
  $ make

This is the same procedure as explained in the section Installation_
with just one more line to set the LDFLAGS variable.

.. _fink: http://finkproject.org/
.. _`nsim developers`: nmag@soton.ac.uk