03_running_nodes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Author: Lester Hedges<br>\n",
    "Email:&nbsp;&nbsp; lester.hedges@bristol.ac.uk\n",
    "\n",
    "___Jupyter Recap___:\n",
    "* Press Shift+Enter to execute a cell and move to the cell below.\n",
    "* Press Ctrl+Enter to execute a cell and remain in that cell.\n",
    "* Run a shell command on the underlying operating system by prefixing the command with an exclamation mark, !\n",
    "* Remember that the flow is in the order that you execute cells, which is not necessarily linear in the notebook. Keep track of the numbers in brackets to the left of the cell!\n",
    "\n",
    "\n",
    "# Running nodes\n",
    "\n",
    "The previous notebook showed you how to write a node to perform minimisation of a molecular system within an interactive Jupyter notebook. This notebook introduces you to some of the other ways of running BioSimSpace nodes, showing how the same script can be used in several different ways. \n",
    "\n",
    "## Running nodes on the command-line\n",
    "\n",
    "The typical way of interacting with BioSimSpace is by running a workflow component, or _node_, from the command-line. A node is just a normal Python script that is run using the `python` interpreter. Let's use the molecular minimisation example from the previous notebook, which we've provided as a Python script called `minimisation.py` within `nodes` directory. (This is the just the previous notebook, downloaded as a regular Python script.)\n",
    "\n",
    "From the command-line, we can query the node to see what it does and get information about the inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python nodes/minimisation.py --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous notebook, input was achieved via a graphical user interface where the user could configure options and upload files. On the command-line, inputs must be set as command-line arguments. From the information provided in the node itself, i.e. the description, the definition of inputs and outputs, BioSimSpace has autogenerated a nicely formatted [argparse](https://docs.python.org/3/library/argparse.html) help message that describes how the node works. The information shows all of the inputs and outputs, let's us know which inputs are optional, and specifies any default values or constraints.\n",
    "\n",
    "Note that it's possible to pass options to the node in various ways, e.g. directly on the command-line, using a [YAML](https://en.wikipedia.org/wiki/YAML) configuration file, or even using environment variables. This provides a lot of flexibility in the way in which BioSimSpace nodes can be run. For now we'll just pass arguments on the command-line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try running the node without any arguments and seeing what the output is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python nodes/minimisation.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thankfully we've provided some files for you. As before, these are found in the `input` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls input/amber"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(The files define a solvated alanine dipeptide system in [AMBER](http://ambermd.org) format.)\n",
    "\n",
    "Let's now run the minimisation node using these files as input. In the interests of time, let's also reduce the number of steps to 1000. The files can be passed to the script in various ways. All of the following are allowed:\n",
    "\n",
    "```bash\n",
    "!python nodes/minimisation.py --steps=1000 --files=\"input/ala.crd, input/ala.top\"\n",
    "!python nodes/minimisation.py --steps=1000 --files input/ala.crd input/ala.top\n",
    "!python nodes/minimisation.py --steps=1000 --files input/*\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python nodes/minimisation.py --steps=1000 --files input/amber/*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the process has finished running (the asterisk to the left of the cell will disappear) we should find that the minimised molecular system has been written to the working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls minimised.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the files have been written in the same format as the original molecular system, i.e. AMBER.\n",
    "\n",
    "We have also provided some GROMACS format input files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls input/gromacs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now run the node using these files as input. This is a larger system so the minimisation will take a little longer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python nodes/minimisation.py --steps=1000 --files input/gromacs/*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There should now be two additional GROMACS format output files in the working directory. (Remember that they won't appear until the cell above finishes running.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls minimised.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running nodes from within BioSimSpace\n",
    "\n",
    "BioSimSpace also provides functionality for running nodes internally. This allows you to call a node from within a script, thereby using existing nodes as building blocks for more complicated workflows. To see what nodes are available we can use the `list` function from the `BSS.Node` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import BioSimSpace as BSS\n",
    "\n",
    "BSS.Node.setNodeDirectory(\"nodes\")\n",
    "BSS.Node.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get information about a particular node, we simply pass its name to the help function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "BSS.Node.help(\"minimisation\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To execute a node we use the `run` function. Let's see how this works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(BSS.Node.run)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function takes a dictionary of input values and returns another dictionary containing the outputs. Let's generate a valid input dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input = {\"files\": [\"input/amber/ala.crd\", \"input/amber/ala.top\"], \"steps\": 1000}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now run the `minimisation` node, passing the dictionary from above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = BSS.Node.run(\"minimisation\", input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's print the output dictionary to see the result of running the node:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That completes this section of the tutorial. We hope you have a better understanding of the aims of the BioSimSpace project and have the knowledge needed to write your own interoperable workflow nodes, which will be needed for the exercies below.\n",
    "\n",
    "\n",
    "# Exercises\n",
    "\n",
    "In the following excercises you will write several workflow nodes. Make sure to use the built in help if you are unsure of what to do. In particular, the following might be useful:\n",
    "\n",
    "* BioSimSpace packages can query themselves to find out what functionality is available. For example, `BSS.IO.fileFormats()` returns a list of supported file formats and `BSS.Process.packages()` returns a list of the supported molecular dynamics packages. These can be useful for setting allowed values for input requirements.\n",
    "* You can create protocols and processes indirectly using the `BSS.Protocol.createProtocol` and `BSS.Process.createProcess` functions. These are useful for generating objects from the input to a node, e.g. a string specifying the chosen protocol.\n",
    "* `help(BSS.Gateway)`: Get information about the `Gateway` package and the different input `Requirement` types that are available.\n",
    "* `help(BSS.Types)`: Get information about the `Types` package. These can be used to define default values for input requirements (of matching type). Remember that the `Units` package provides a convenient shortcut for declaring `Types`, e.g. `10*BSS.Units.Length.nanometer` is equivalent to `BSS.Types.Length(10, \"nanometer\")`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "10 * BSS.Units.Length.nanometer == BSS.Types.Length(10, \"nanometer\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Write a node to convert between file formats. This should take a set of molecular files as input and convert them to a single file in another format. If you get stuck, an answer can be found [here](nodes/conversion.ipynb).\n",
    "\n",
    "2. Write a node to genrate input files for different molecular dynamics protocols and packages As input, this node should take a set of molecular files, the name of a protocol, and the name of a supported molecular dynamics package. As output it should return a list of the input files that are generated by BioSimSpace. Don't worry about setting options for the individual protocols, just return the inputs for the defaults. If you get stuck, an answer can be found [here](nodes/input_files.ipynb).\n",
    "\n",
    "3. Write a node to equilibrate a molecular system. Consider supporting some of the options that are available in `BSS.Protocol.Equilibration`. At a minimum, the node should take a set of molecular files as input and return a different set of files representing the equilibrated system as output. If you get stuck, an answer can be found [here](nodes/equilibrationipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}