Skip to content

Commit

Permalink
make overly awkward fom a section
Browse files Browse the repository at this point in the history
  • Loading branch information
adswa committed Nov 1, 2023
1 parent a36da9b commit 76ac99e
Showing 1 changed file with 52 additions and 51 deletions.
103 changes: 52 additions & 51 deletions docs/basics/101-133-containersrun.rst
Original file line number Diff line number Diff line change
Expand Up @@ -203,62 +203,13 @@ name to give to the container, and a path or url to a container Image:
This command downloaded the container from Singularity Hub, added it to
the ``midterm_project`` dataset, and recorded basic information on the
container under its name "midterm-software" in the dataset's configuration at
``.datalad/config``. You can find out more about them in a dedicated :ref:`find-out-more on these additional configurations <fom-containerconfig>`.

.. find-out-more:: What changes in .datalad/config when one adds a container?
:name: fom-containerconfig
:float:

.. runrecord:: _examples/DL-101-133-102
:language: console
:workdir: dl-101/DataLad-101/midterm_project

$ cat .datalad/config

This recorded the Image's origin on Singularity-Hub, the location of the
Image in the dataset under ``.datalad/environments/<NAME>/image``, and it
specifies the way in which the container should be used: The line

.. code-block:: bash
cmdexec = singularity exec {img} {cmd}
can be read as: "If this container is used, take the ``cmd`` (what you wrap in a
:dlcmd:`containers-run` command) and plug it into a
:shcmd:`singularity exec` command. The mode of calling Singularity,
namely ``exec``, means that the command will be executed inside of the container.

You can configure this call format by modifying it in the config file, or calling :dlcmd:`containers-add` with the option ``--call-fmt <alternative format>``.
This can be useful to, for example, automatically bind-mount the current working directory in the container.
In the alternative call format, the placeholders ``{img}``, ``{cmd}``, and ``{img_dspath}`` (a relative path to the dataset containing the image) are available.
In all other cases with variables that use curly brackets, you need to escape them with another curly bracket.
Here is an example call format that bind-mounts the current working directory (and thus the dataset) automatically:

.. code-block:: bash
datalad containers-add --call-fmt 'singularity exec -B {{pwd}} --cleanenv {img} {cmd}'
Note that the Image is saved under ``.datalad/environments`` and the
configuration is done in ``.datalad/config`` -- as these files are version
controlled and shared with together with a dataset, your software
container and the information where it can be re-obtained from are linked
to your dataset.

This is how the ``containers-add`` command is recorded in your history:

.. runrecord:: _examples/DL-101-133-103
:language: console
:workdir: dl-101/DataLad-101/midterm_project
:cast: 10_yoda
:notes: The software container got added to your datasets history

$ git log -n 1 -p
``.datalad/config``. You can find out more about them in an upcoming dedicated section on these additional configurations.

Such configurations can, among other things, be important to ensure correct container invocation on specific systems or across systems.
One example is *bind-mounting* directories into containers, i.e., making a specific directory and its contents available inside a container.
Different containerization software (versions) or configurations of those determine *default bind-mounts* on a given system.
Thus, depending on the system and the location of the dataset on this system, a shared dataset may be automatically bind-mounted or not.
To ensure that the dataset is correctly bind-mounted on all systems, let's add a call-format specification with a bind-mount to the current working directory following the information in the :ref:`find-out-more on additional container configurations <fom-containerconfig>`.
To ensure that the dataset is correctly bind-mounted on all systems, let's add a call-format specification with a bind-mount to the current working directory.

.. runrecord:: _examples/DL-101-133-104
:language: console
Expand Down Expand Up @@ -375,6 +326,56 @@ only benefit your future self, but also whomever you share your dataset with, as
the information about the container is shared together with the dataset. How cool
is that?


What changes in .datalad/config when one adds a container?


.. runrecord:: _examples/DL-101-133-102
:language: console
:workdir: dl-101/DataLad-101/midterm_project

$ cat .datalad/config

This recorded the Image's origin on Singularity-Hub, the location of the
Image in the dataset under ``.datalad/environments/<NAME>/image``, and it
specifies the way in which the container should be used: The line

.. code-block:: bash
cmdexec = singularity exec {img} {cmd}
can be read as: "If this container is used, take the ``cmd`` (what you wrap in a
:dlcmd:`containers-run` command) and plug it into a
:shcmd:`singularity exec` command. The mode of calling Singularity,
namely ``exec``, means that the command will be executed inside of the container.

You can configure this call format by modifying it in the config file, or calling :dlcmd:`containers-add` with the option ``--call-fmt <alternative format>``.
This can be useful to, for example, automatically bind-mount the current working directory in the container.
In the alternative call format, the placeholders ``{img}``, ``{cmd}``, and ``{img_dspath}`` (a relative path to the dataset containing the image) are available.
In all other cases with variables that use curly brackets, you need to escape them with another curly bracket.
Here is an example call format that bind-mounts the current working directory (and thus the dataset) automatically:

.. code-block:: bash
datalad containers-add --call-fmt 'singularity exec -B {{pwd}} --cleanenv {img} {cmd}'
Note that the Image is saved under ``.datalad/environments`` and the
configuration is done in ``.datalad/config`` -- as these files are version
controlled and shared with together with a dataset, your software
container and the information where it can be re-obtained from are linked
to your dataset.

This is how the ``containers-add`` command is recorded in your history:

.. runrecord:: _examples/DL-101-133-103
:language: console
:workdir: dl-101/DataLad-101/midterm_project
:cast: 10_yoda
:notes: The software container got added to your datasets history

$ git log -n 1 -p


.. only:: adminmode

Add a tag at the section end.
Expand Down

0 comments on commit 76ac99e

Please sign in to comment.