Docs tidy up pass

docs/source/conf.py

@@ -84,5 +84,6 @@
 # autodoc_pydantic_model_show_validator_summary = False
 autodoc_pydantic_model_member_order = "bysource"
 intersphinx_mapping = {
-    "dgbowl_schemas": ("https://dgbowl.github.io/dgbowl-schemas/main", None)
+    "dgbowl_schemas": ("https://dgbowl.github.io/dgbowl-schemas/main", None),
+    "xarray": ("https://docs.xarray.dev/en/stable", None),
 }

docs/source/index.rst

@@ -18,14 +18,20 @@
 - the job and job queue configuration utility, :mod:`tomato.ketchup`;
 - a set of device drivers, see the :ref:`driver library`.
 
-This project has received funding from the European Union’s Horizon 2020 research
-and innovation programme under grant agreement No 957189. The project is part of
-BATTERY 2030+, the large-scale European research initiative for inventing the
-sustainable batteries of the future.
+This project has received funding from:
+
+- the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 957189,
+- the Emmy Noether Programme of the DFG, under project No. 490703766
+- the Priority Programme 2419 HyCAM of the DFG, under project No. 523873650
+
+The project is part of BATTERY 2030+, the large-scale European research initiative for inventing the sustainable batteries of the future.
 
 .. codeauthor::
     Peter Kraus,
-    Loris Ercole
+    Alexandre Gbocho,
+    Graham Kimbell,
+    Loris Ercole,
+
 
 .. toctree::
    :maxdepth: 1
@@ -42,6 +48,10 @@ sustainable batteries of the future.
    :maxdepth: 1
    :caption: tomato driver library
 
+   tomato-example-counter <https://github.com/dgbowl/tomato-example-counter>
+   tomato-psutil <https://github.com/dgbowl/tomato-psutil>
+   tomato-bronkhorst <https://github.com/dgbowl/tomato-bronkhorst/>
+   tomato-jumo <https://github.com/dgbowl/tomato-jumo>
 
 .. toctree::
    :maxdepth: 1

docs/source/installation.rst

@@ -9,19 +9,15 @@ and can be installed using:
 
     pip install tomato[docs,testing]
 
-We strongly recommend installing **tomato** into a separate ``conda`` or ``venv``
-environment.
+We strongly recommend installing **tomato** into a separate ``conda`` or ``venv`` environment.
 
 .. note::
 
-    The optional targets ``[docs]`` and ``[testing]`` will install packages required
-    for building this documentation and running the test-suite, respectively.
+    The optional targets ``[docs]`` and ``[testing]`` will install packages required for building this documentation and running the test-suite, respectively.
 
 Testing the installation
 ````````````````````````
-To run the test-suite, you need to first install **tomato** using the above command,
-and then you need to clone the git repository, and launch ``pytest`` from within the
-created ``tomato`` folder:
+To run the test-suite, you need to first install **tomato** using the above command, and then you need to clone the git repository, and launch ``pytest`` from within the created ``tomato`` folder:
 
 .. code::
 

docs/source/quickstart.rst

@@ -10,9 +10,7 @@ First time set-up
 
     This section assumes that **tomato** has been successfully :ref:`installed<installation>`.
 
-To prepare **tomato** for its first execution, a *settings file* containing the basic
-configuration of the driver needs to be placed in **tomato's** *appdir* folder. By
-default, the *appdir* path is:
+To prepare **tomato** before its first run, a *settings file* containing the basic configuration for the :mod:`tomato.daemon` needs to be placed in **tomato's** *appdir* folder. By default, the *appdir* path is:
 
 - ``$env:localappdata\dgbowl\tomato\<version>`` on Windows,
 - ``$HOME/.config/tomato/<version>`` on Linux.
@@ -26,20 +24,17 @@ The easiest way to do create this file is using the provided ``tomato init`` com
     kraus@dorje:/home/kraus/$ tomato init
     Success: wrote default settings into /home/kraus/.config/tomato/1.0a1/settings.toml
 
-Where *appdir* is ``/home/kraus/.config/tomato/1.0a1/``. The *appdir* can be specified
-using the ``--appdir`` argument to **tomato**.
+Where *appdir* is ``/home/kraus/.config/tomato/1.0a1/``. A custom *appdir* can be specified using the ``--appdir`` argument to **tomato**.
 
 
 **tomato** program flowchart
 ````````````````````````````
 **tomato** currently contains two command line user-facing utilities/executables:
 
-- :mod:`~tomato.tomato`, responsible for management of the state daemon process, and
+- :mod:`~tomato.tomato`, responsible for management of the daemon process, and
 - :mod:`~tomato.ketchup`, responsible for job submission and management.
 
-These two executables then internally spawn other processes, including the state daemon
-``tomato-daemon``, a ``tomato-job`` process for every running job, and a ``tomato-driver``
-process for managing all devices of a certain driver type.
+These two executables then internally spawn other processes, including the state daemon process ``tomato-daemon``, a ``tomato-job`` process for every running job, and a ``tomato-driver`` process for each driver type, managing all devices of a that type.
 
 .. mermaid::
 
@@ -75,11 +70,11 @@ process for managing all devices of a certain driver type.
 `````````````````````````````
 The following concepts are used in **tomato**:
 
-- *devices*, which represent separately addressable and optionally multichannel instruments in the lab,
+- *devices*, which represent separately addressable (and optionally multichannel) instruments in the lab,
 - *components*, which represent the individual channels of each *device*, describing the role of this *component* and its associated capabilities,
-- *pipelines*, which represent the real world organisation of device *components* into independent experimental set-ups, optionally containing *components* from one or more *device*,
-- *jobs*, which are processes that carry out a *payload* (i.e. a set of experimental instructions) on a *pipeline*, and
-- *drivers*, which are processes managing all *devices* of a certain driver type and their *component*:
+- *pipelines*, which represent the real world organisation of device *components* into independent experimental set-ups, containing *components* from one or more *device*,
+- *jobs*, which are processes that carry out a *payload* (i.e. a set of experimental instructions) on a *pipeline* by instructing its *components* to perform individual *tasks*, and
+- *drivers*, which are separate processes, each managing all defined *devices* of a certain driver type as well as their *components*:
 
 .. mermaid::
 
@@ -147,9 +142,7 @@ The following concepts are used in **tomato**:
 
 Settings file
 `````````````
-The *settings file* contains the basic information required to start the ``tomato-daemon``.
-This is a ``settings.toml`` file, which has to be located in the *appdir*. The default
-file generated by ``tomato init`` looks similar to the below example:
+The *settings file* contains the basic information required to start the ``tomato-daemon``. The filename is hard-coded to be a ``settings.toml`` file, which has to be located in the *appdir*. The default file generated by ``tomato init`` looks similar to the below example:
 
 .. code-block:: toml
    :linenos:
@@ -166,32 +159,25 @@ file generated by ``tomato init`` looks similar to the below example:
     example_counter.testpar = 1234
 
 
-In addition to the *appdir*, a second path, *datadir*, is used to specify the location of
-the data created by **tomato**. The default *datadir* is:
+In addition to the *appdir*, a second path, *datadir*, is used to specify the location of the data created by **tomato**. The default *datadir* is:
 
 - ``$env:localappdata\dgbowl\tomato\<version>`` on Windows,
 - ``$HOME/.local/share/tomato/<version>`` on Linux.
 
 In the default *settings file* shown above, the following entries are specified:
 
-- ``jobs.storage`` which is the directory where the data and logs of **tomato** jobs
-  will be stored,
-- ``devices.config`` which points to a ``yaml``-formatted :ref:`devices file <devfile>`,
-  defining the hardware configuration of the devices managed by **tomato**.
+- ``jobs.storage`` which is the directory where the data and logs of **tomato** jobs will be stored,
+- ``devices.config`` which points to a ``yaml``-formatted :ref:`devices file <devfile>`, defining the hardware configuration of the devices managed by **tomato**.
 
-Additional, *driver*-specific settings may be provided in the ``[drivers]`` section,
-following the example of the ``drivers.example_counter.testpar`` entry.
+Additional, *driver*-specific settings may be provided in the ``[drivers]`` section, following the example of the ``drivers.example_counter.testpar`` entry. These *driver*-specific settings are passed to each *driver* when its process is launched and the :class:`DriverInterface` is initialised, and can therefore contain paths to various libraries or other files necessary for the *driver* to function.
 
 .. _devfile:
 
 Devices file
 ````````````
-This ``yaml``-formatted file contains information about each *device*, corresponding to
-an individual piece of hardware managed by **tomato**, as well as information about the
-organisation of such *devices* into *pipelines*.
+This ``yaml``-formatted file contains information about each *device*, corresponding to an individual piece of hardware managed by **tomato**, as well as information about the organisation of the individually-addressable *components* of those *devices* into *pipelines*.
 
-When the devices file is not present, the default file shipped with **tomato** will be
-used instead:
+When the devices file is not present, the default file shipped with **tomato** will be used instead:
 
 **devices** section
 *******************
@@ -207,12 +193,9 @@ The ``devices`` section of the default *devices file* is shown below:
         channels: ["1"]
         pollrate: 1
 
-Here, we define a single device using the :mod:`~tomato.drivers.example_counter` driver.
-The definition includes the ``address`` of the device (:class:`str` type) as well as an
-enumeration of individually-addressable channels the device has (:class:`list[str]`).
+Here, we define a single device using the :mod:`~tomato.drivers.example_counter` driver. The definition includes the ``address`` of the device (:class:`str` type) as well as an enumeration of individually-addressable channels the device has (:class:`list[str]`).
 
-For example, the devices shown in the :ref:`concepts flowchart <concepts>` above would
-be defined as:
+For example, the devices shown in the :ref:`concepts flowchart <concepts>` above would be defined as:
 
 .. code-block:: yaml
    :linenos:
@@ -237,9 +220,7 @@ be defined as:
 
 .. note::
 
-    The ``pollrate`` here is an internal setting for **tomato**. It is the frequency
-    with which the job interacting with the device driver asks the driver for data,
-    **not** the frequency with which the driver asks the hardware for data!
+    The ``pollrate`` here is an internal setting for **tomato**. It is the frequency with which the *job* interacting with the device *driver* asks the *driver* for data, **not** the frequency with which the *driver* asks the hardware for data! This latter "sampling" frequency can be defined in each individual *task* submitted as a *payload* of a *job*.
 
 **pipelines** section
 *********************
@@ -256,10 +237,7 @@ The default ``pipelines`` section looks as follows:
             device: dev-counter
             channel: "1"
 
-Here, a single *pipeline* called ``pip-counter`` is defined to contain the one available
-channel of the ``dev-counter`` device (defined on line 5) shown further above. For multi
-channel devices, it is also possible to define a *pipeline* for each channel automatically.
-For example, with the following definition:
+Here, a single *pipeline* called ``pip-counter`` is defined to contain the one available channel of the ``dev-counter`` device (defined on line 5) shown further above. For multi channel devices, it is also possible to define a *pipeline* for each channel automatically, e.g. using the following definition:
 
 .. code-block:: yaml
    :linenos:
@@ -272,13 +250,9 @@ For example, with the following definition:
             device: dev-counter
             channel: each
 
-a set of pipelines would be created using each of the available channels in
-``dev-counter``, replacing the ``*`` in line 2 using the channel. Therefore, one
-*pipeline*  with name ``pip-counter-1`` would be created. However, if the ``dev-counter``
-contained more than one channel, a *pipeline* would be generated for each channel.
+Here, a set of pipelines would be created using each of the available channels in ``dev-counter``, replacing the ``*`` in line 2 using the channel. Therefore, one *pipeline*  with name ``pip-counter-1`` would be created. However, if the ``dev-counter`` were to contain more than one channel, a *pipeline* would be generated for each channel.
 
-Finally, a definition for the pipelines shown in the :ref:`concepts flowchart <concepts>`
-above can be defined as:
+Finally, a definition for the pipelines shown in the :ref:`concepts flowchart <concepts>` above can be defined as:
 
 .. code-block:: yaml
    :linenos:
@@ -310,8 +284,19 @@ above can be defined as:
 
 Payload file
 ````````````
-The *payload* file contains all information required to enter a *job* into the queue and
-allow its assignment onto a *pipeline*. The overall schema of the *payload* is defined
-in the :mod:`dgbowl_schemas.tomato` module, and is parsed using :func:`dgbowl_schemas.tomato.to_payload`:
+The *payload* file contains all information required to enter a *job* into the queue and allow its assignment onto a *pipeline*. The overall schema of the *payload* is defined in the :mod:`dgbowl_schemas.tomato` module, and is parsed using :func:`dgbowl_schemas.tomato.to_payload`:
 
-.. autopydantic_model:: dgbowl_schemas.tomato.payload_1_0.Payload
+.. autopydantic_model:: dgbowl_schemas.tomato.payload.Payload
+
+
+.. note::
+
+    Of particular importance here is the specification of the individual :class:`Tasks`. Some general :class:`Task` parameters are abstracted by **tomato**, such as the ``max_duration``, ``sampling_interval`` and ``polling_interval``. The other are *driver* or *component* specific, and can be specified using the ``technique_params`` :class:`dict`.
+
+.. autopydantic_model:: dgbowl_schemas.tomato.payload.Task
+
+.. warning::
+
+    Currently, **tomato** performs no validation of any ``technique_params`` at *job* submission. It also does not check whether the ``sampling_interval`` is realistic. This means your *jobs* may crash when the ``technique_params`` are out of bounds when the :class:`Task` is being executed.
+
+    However, such validation is planned for ``tomato-2.0``, see https://github.com/dgbowl/tomato/issues/99.