Source code pushed to the master branch is expected to run cleanly through the hypre regression tests, so developers should do an appropriate degree of testing ahead of time. The amount of testing needed will depend on the nature of the code changes. Small changes may require little or no testing, while larger additions may require running the full suite of regression tests in hypre. This page gives some information on how to do this.

## Basics about the AUTOTEST directory

The `AUTOTEST` directory contains a number of Bourne shell test scripts.  Each test can be run directly, but this is only recommended for getting help information, for example:

    check-int.sh -help

To run an actual test, prepend it with `test.sh` as follows:

    test.sh check-int.sh ../src

This will ensure that stderr and stdout are redirected to files. To see if a test passes, just check that the size of the resulting error file is zero:

    ls -l *.err

If the error file is not empty, it will contain information about the errors produced. Sometimes this will simply be pointers to other error files. In that case, just follow the trail and you will eventually get to the underlying errors.

## Starting from the top

The main test script is `machine-tux.sh`. As the name indicates, this contains all of the tests that are run on our tux machines every night. It can be run manually as follows, but note that it may not run correctly on all developer platforms because of differences in hardware, operating systems, compiler versions, etc. (inform the project leader if the script does not run correctly on your machine):

    test.sh machine-tux.sh ../src

This is probably not the first test script to run, but it gives a top-down view of what eventually needs to run, so it is useful to inspect what is inside. Manually running some of the sub-tests in `machine-tux.sh` is a useful way to do some intermediate testing. The main sub-test called is `basictest.sh`. This high-level script is a way to easily test a specific build and set of runs. For example, the following line will configure with debugging (the `-co:` option), make the library and test drivers using multiple threads (the `-mo:` option), run the Struct and IJ tests (the optional `-ro:` option), and run the default example tests (the optional `-eo:` option):

    test.sh basictest.sh ../src -co: --enable-debug -mo: -j test -ro: -struct -ij -eo:

To clean up the output produced from testing, use `cleantest.sh` (check the `-help` option for usage info).

Before running AUTOTEST scripts, it is often good to do some manual testing with the `src/test/runtest.sh` script. For example, after configuring and compiling the test drivers, executing the following in the `src/test` directory will run one particular Struct PFMG test and all of the IJ tests:

    runtest.sh TEST_struct/pfmgvcycle.sh TEST_ij/*.sh

To check the results, look at the error files as before:

    ls -l TEST*/*.err

Cleaning up the output can be done with the `src/test/cleantest.sh` script.

## Running on LC machines with GPU
### lassen and tioga
To pass the tests, run `runtest.sh` with `-save lassen` on lassen. It is also recommended to add `-mpibind` to make the runs faster. For the main test script `machine-lassen.sh`, it is important to make sure to run on **exactly 4** nodes. Consider using the script [lassen.sh](https://github.com/hypre-space/hypre/wiki/files/lassen.sh) to submit the job. The regression tests on tioga require **2 nodes**. Run with submitting [tioga.sh](https://github.com/hypre-space/hypre/wiki/files/tioga.sh)