Test documentation service

The documentation for each test is maintained with the source of the test in the GitHub repository. There are two ways to display test documentation: within the Conformance Tool and on a static web page. This note describes the static web page view.

Test kits

An instance of toolkit can contain any number of distinct test kits. A test kit is simply a collection of tests.

Toolkit ships with an internal test kit that is always available.

There can be a collection of test kits stored in the External Cache. Within each environment there is a default test kit. An environment can also contain test session specific test kits.

The collection of test kits are arranged in a well defined order that makes up a search path similar to the command PATH in Linux or the Classpath in Java.

The last test kit in the search path is the internal one that ships with toolkit.

Prior to that in the search path is the default area of the currently selected environment.

Prior to that in the search path is the area named for the current test session (within the current environment).

So, assuming that all these optional test kits are populated, there are three test kits in the search path.

Scanning the test kits

Here is the process for scanning the test kits and producing the output.

For each test kit, scan the actorcollections directory and get the actor id and name for each actor. Actor ids containing an underscore in the name identify options. No underscore means the is is for an actor.

This is used to produce the top level Actor Types listing.

For each actor type, list the options.

The content listed so far is assembled onto a single web page, the index page. It includes a top-level listing for each test. By top-level listing I mean a display of its id and title. The full documentation for the individual tests is not generated until requested by clicking on a link. Each test, as listed, is a link to this contents.

Per-test documentation

The following details are displayed for a test.

Test: test id
Description: the test's readme.txt file, first line
Profile: by scanning the test's collections.txt file
Description: the test's readme.txt file, content starting with line 1. This is added raw from the file. Obviously this displays better if in HTML.
** Listing of sections**: taken from the index.idx file - gives section ids in order of execution

For each section:

Section name: same as section directory name
Section documentation: this a raw dump of the section's readme.txt file with minimal conversion of double blank lines to HTML paragraph tags.
Within in each section is the steps. For each step...

The transaction id is extracted from the testplan step. The actor id is extracted from the transaction id. The tables show the translation from the ids to displayable strings.

If the transaction id or actor id are not listed below, it is assumed that this is a validation-only step (no transaction generated). Transactions and validations are identified in the display by headings of their type.

If the transaction (validation) has a Goal element, the contents (text) of that element are added to the display.

The transaction and actor types are displayed. If the transaction is a Stored Query then the query file is scanned, the query id extracted and its name is looked up and displayed.

The expected status is displayed along with expected error code and message if appropriate.

If the transaction includes an assertion section then that content is formatted and displayed.

Actor map

        XCQ : 'Responding Gateway',
        XCR : 'Responding Gateway',
        IGR : 'Initiating Gateway',
        ImagingDocSetIigRetrieve : 'Imaging Initiating Gateway',
        StoredQuery : 'XDS',
        Register : 'XDS',
        Mu : 'XDS',
        ImagingDocSetRetrieve : 'Imaging Document Source',
        XDRProvideAndRegister : 'Document Recipient'

Transaction map

        XCQ : 'Cross Community Query',
        XCR : 'Cross Community Retrieve',
        IGR : 'Stored Query',
        ImagingDocSetIigRetrieve : 'RAD-69',
        StoredQuery : 'Stored Query',
        Register : 'Register',
        Mu : 'Metadata Update',
        ImagingDocSetRetrieve : 'Imaging Document Set Retrieve',
        XDRProvideAndRegister : 'Provide and Register'

Documentation formatting

The tool scans the test kits, the internal-to-toolkit test kit, the shared test kit in the external cache, and the private test kit in the external cache (private/shared from the point of view of an individual test session) as a collection of browsable web pages. The content, in real time, is pulled from the test definitions. That way it is always up to date with the service.

The service pulls content from the test-specific readme.txt, the section-specific readme.txt, and the Goals part of the testplan files and generates the formatted HTML output. BTW, the readme.TXT files can be text/plain or text/html. It looks nicer if they are text/html.