Add docs on project structure and modular design

README.md

@@ -3,7 +3,73 @@
 
 Test suite for validation of openEO back-ends against the openEO API and related specifications.
 
-## Install/Setup
+
+
+## Project structure and modular design
+
+The openEO test suite was developed in separate work packages,
+which is roughly reflected in the project structure.
+The separate modules allow to run the test suite in a more fine-grained way,
+focussing on a specific API aspect to test or verify
+(e.g. just collection metadata validation or individual process testing).
+
+
+> **Note**
+> In the following overview includes some invocation examples that are given as basic reference.
+> Make sure to also check the more [detailed documentation of the test run options](#run-options) further in the docs.
+
+
+- **WP1 General test suite framework** (lead implementation partner: VITO)
+  - Main location: [`src/openeo_test_suite/lib`](./src/openeo_test_suite/lib)
+  - Provides various reusable utilities and helpers to power the openEO test suite,
+    and defines a pytest plugin to properly hook into the various phases of the pytest framework.
+  - This module also defines some "internal" tests that are just meant to test these utilities in isolation,
+    but it is not part of the openEO test suite itself:
+    ```bash
+    pytest src/openeo_test_suite/lib/internal-tests
+    ```
+- **WP2 Validation of collection metadata** (lead implementation partner: EURAC)
+  - Main location: [`src/openeo_test_suite/tests/collections`](./src/openeo_test_suite/tests/collections)
+  - Defines tests to validate openEO collection metadata against specs like
+    the [openEO API](https://openeo.org/) and [STAC](https://stacspec.org/en).
+  - Usage example of just running these tests against a desired openEO backend URL:
+    ```bash
+    pytest src/openeo_test_suite/tests/collections \
+      -U https://openeo.example \
+      --html=reports/collection-metadata.html
+    ```
+- **WP3 Validation of process metadata** (lead implementation partner: EODC)
+  - Main location: [`src/openeo_test_suite/tests/processes/metadata`](./src/openeo_test_suite/tests/processes/metadata)
+  - TODO: [Open-EO/openeo-test-suite#19](https://github.com/Open-EO/openeo-test-suite/issues/19)
+- **WP4 General openEO API compliance validation** (lead implementation partner: EODC)
+  - TODO: [Open-EO/openeo-test-suite#20](https://github.com/Open-EO/openeo-test-suite/issues/20)
+- **WP5 Individual process testing** (lead implementation partner: M. Mohr)
+  - Main location: [`src/openeo_test_suite/tests/processes/processing`](./src/openeo_test_suite/tests/processes/processing)
+  - Provides tests to validate individual openEO processes,
+    based on the expected input-output examples
+    defined in the [openeo-processes](https://github.com/Open-EO/openeo-processes) project
+    under [Open-EO/openeo-processes#468](https://github.com/Open-EO/openeo-processes/pull/468)
+  - Very basic usage example of just running these tests:
+    ```bash
+    pytest src/openeo_test_suite/tests/processes/processing \
+      --html=reports/individual-processes.html
+    ```
+    Note that this invocation will not actually execute anything,
+    see the [runner info](#individual-process-testing-runner)
+    and the [more extensive usage examples](#individual-process-testing-examples) for more information and functional examples.
+- **WP6 Full process graph execution** (lead implementation partner: EURAC)
+  - Main location: [`src/openeo_test_suite/tests/workflows`](./src/openeo_test_suite/tests/workflows)
+  - Provides tests to run full processes graphs and evaluate the results.
+  - Usage example of just running these tests against a desired openEO backend URL:
+    ```bash
+    pytest src/openeo_test_suite/tests/workflows \
+      -U https://openeo.example \
+      --s2-collection SENTINEL2_L2A \
+      --html=reports/workflows.html
+    ```
+
+
+## Installation and setup
 
 Clone this repository and some git submodules with additional assets/data, e.g.:
 
@@ -43,7 +109,7 @@ activate the virtual environment in your shell with:
 source venv/bin/activate
 ```
 
-### Install dependencies
+### Install `openeo-test-suite` package (with dependencies)
 
 Install the project and its dependencies in your virtual environment with:
 
@@ -56,10 +122,10 @@ which makes sure that code changes will be reflected immediately in your virtual
 without the need of (re)installing the project.
 
 
-#### Additional dependencies related to runners for individual process testing
+### Additional optional dependencies related to runners for individual process testing <a name="runner-dependencies"></a>
 
 The individual process testing module of the test suite allows to pick
-a specific process "runner" (see further for more documentation).
+a specific process "runner" (see [further](#individual-process-testing-runner) for more documentation).
 Some of these runners require additional optional dependencies to be installed in your virtual environment,
 which can be done by providing an appropriate "extra" identifier in the `pip install` command:
 
@@ -73,13 +139,13 @@ which can be done by providing an appropriate "extra" identifier in the `pip ins
     ```
 
 Note that it might be not possible to install both "dask" and "vito" extras
-in the same environment because of conflicting dependency constraints.
+seamlessly in the same environment because of conflicting dependency constraints.
 
 
-## Running the test suite
+## Test suite run options <a name="run-options"></a>
 
-The test suite at least requires an openEO backend URL to run against.
-It can be specified through a `pytest` command line option
+Most modules of the test suite at least require an openEO backend URL to run against.
+It can be specified through a `pytest` command line option:
 
     pytest --openeo-backend-url=openeo.example
 
@@ -88,12 +154,6 @@ It can be specified through a `pytest` command line option
 
 If no explicit protocol is specified, `https://` will be assumed automatically.
 
-
-## Additional run options
-
-In addition to the `--openeo-backend-url` option, there are several other options
-to control some aspects of the test suite run.
-
 ### Process selection
 
 Various tests depend on the availability of certain openEO processes.
@@ -116,10 +176,10 @@ If both are specified, the union of both will be considered.
   Enabling this option will consider experimental processes and tests.
 
 
-### Runner for individual process testing
+### Runner for individual process testing <a name="individual-process-testing-runner"></a>
 
-One module of the test suite is dedicated to **individual process testing**,
-where each process is tested individually with a given set of inputs and expected outputs.
+The goal of the **individual process testing** module of the test suite
+is testing each openEO process individually with one or more pairs of input and expected output.
 Because there are a lot of these tests (order of thousands),
 it is very time-consuming to run these through the standard, HTTP based openEO REST API.
 As a countermeasure, the test suite ships with several experimental **runners**
@@ -143,20 +203,21 @@ with currently one of the following options:
     These tests will be marked as skipped.
 - `dask`: Executes the tests directly via the [openEO Dask implementation](https://github.com/Open-EO/openeo-processes-dask) (as used by EODC, EURAC, and others)
   - Requires [openeo_processes_dask](https://github.com/Open-EO/openeo-processes-dask) package being installed in test environment.
-    See installation instructions above for more practical info.
+    See [installation instructions](#runner-dependencies) above for more practical info.
   - Covers all implemented processes.
 - `vito`: Executes the tests directly via the
   [openEO Python Driver implementation](https://github.com/Open-EO/openeo-python-driver) (as used by CDSE, VITO/Terrascope, and others).
   - Requires [openeo_driver](https://github.com/Open-EO/openeo-python-driver) package being installed in test environment.
-    See installation instructions above for more practical info.
+    See [installation instructions](#runner-dependencies) above for more practical info.
   - Only covers a subset of processes due to the underlying architecture of the back-end implementation.
     In particular, it only covers the pure Python code paths, but not the PySpark related aspects.
 
 See [openeo_test_suite/lib/process_runner](https://github.com/Open-EO/openeo-test-suite/tree/main/src/openeo_test_suite/lib/process_runner)
 for more details about these runners and inspiration to implement your own runner.
 
 
-#### Usage examples of individual process testing with runner option
+
+#### Usage examples of individual process testing with runner option <a name="individual-process-testing-examples"></a>
 
 The individual process tests can be run by specifying the `src/openeo_test_suite/tests/processes/processing` as test path.
 Some use examples with different options discussed above: