Updated tutorial 3 docs

docs/_static/tutorials/fetch.py

@@ -0,0 +1,49 @@
+from shutil import copy
+
+
+def extract(filename_in, filename_out, start=0, end=None, heading=None):
+    with open(filename_in) as fd:
+        flines = fd.readlines()
+
+    # If heading is provided, find the line number of the heading
+    if heading:
+        for i, line in enumerate(flines):
+            if heading in line:
+                start = i
+                break
+        else:
+            raise ValueError(f'Heading {heading} not found in file {filename_in}')
+
+        indent = len(flines[start]) - len(flines[start].lstrip())
+
+        for i in range(start + 1, len(flines)):
+            line = flines[i]
+            if not line.startswith(' ' * (indent + 1)):
+                end = i
+                break
+
+        if end is None:
+            raise ValueError(f'Could not find the end of the {heading} block')
+
+    flines = flines[start:end]
+
+    # Find the shortest leading whitespace and strip this from all lines (as otherwise the markdown will be rendered as a code block)
+    min_indent = min(len(line) - len(line.lstrip()) for line in flines)
+    flines = [line[min_indent:] for line in flines]
+
+    # Manual conversion of double spaces to /
+    flines = [line[:-2] + '\ \n' if (line.endswith('  \n')
+                                     and not line.strip().startswith('-')) else line for line in flines]
+
+    with open(filename_out, 'w') as fd:
+        fd.writelines(flines)
+
+
+if __name__ == '__main__':
+
+    # Tutorial 3
+    extract('../../../tutorials/tutorial_3/01-ki/zno.md',
+            'tutorial_3/md_excerpts/zno_wannierize_section.md', heading='Wannierize')
+    extract('../../../tutorials/tutorial_3/01-ki/zno.md', 'tutorial_3/md_excerpts/zno_w2kc.md', -4, -3)
+    extract('../../../tutorials/tutorial_3/01-ki/zno.md', 'tutorial_3/md_excerpts/zno_ham.md', -3, -2)
+    copy('../../../tutorials/tutorial_3/01-ki/01-koopmans-dfpt/Koopmans_DFPT_bandstructure.png', 'tutorial_3/')

docs/_static/tutorials/tutorial_3/md_excerpts/zno_ham.md

@@ -0,0 +1 @@
+- ✅ `03-kc_ham` completed  

docs/_static/tutorials/tutorial_3/md_excerpts/zno_w2kc.md

@@ -0,0 +1 @@
+- ✅ `02-wann2kc` completed  

docs/_static/tutorials/tutorial_3/md_excerpts/zno_wannierize_section.md

@@ -0,0 +1,28 @@
+- **Wannierize**
+  - ✅ `01-scf` completed  
+  - ✅ `02-nscf` completed  
+  - **Wannierize Block 1**
+    - ✅ `01-wannier90_preproc` completed  
+    - ✅ `02-pw2wannier90` completed  
+    - ✅ `03-wannier90` completed  
+  - **Wannierize Block 2**
+    - ✅ `01-wannier90_preproc` completed  
+    - ✅ `02-pw2wannier90` completed  
+    - ✅ `03-wannier90` completed  
+  - **Wannierize Block 3**
+    - ✅ `01-wannier90_preproc` completed  
+    - ✅ `02-pw2wannier90` completed  
+    - ✅ `03-wannier90` completed  
+  - **Wannierize Block 4**
+    - ✅ `01-wannier90_preproc` completed  
+    - ✅ `02-pw2wannier90` completed  
+    - ✅ `03-wannier90` completed  
+  - **Wannierize Block 5**
+    - ✅ `01-wannier90_preproc` completed  
+    - ✅ `02-pw2wannier90` completed  
+    - ✅ `03-wannier90` completed  
+  - ✅ `08-merge_occ_wannier_hamiltonian` completed  
+  - ✅ `09-merge_occ_wannier_u` completed  
+  - ✅ `10-merge_occ_wannier_centers` completed  
+  - ✅ `11-bands` completed  
+  - ✅ `12-projwfc` completed  

docs/tutorials/tutorial_3.rst

@@ -3,20 +3,20 @@
 Tutorial 3: the band structure of ZnO (calculated with explicit k-points)
 =========================================================================
 
-In this tutorial we will calculate the band structure of bulk zinc oxide using the k-space formulation of Koopmans. The input file for this tutorial can be downloaded :download:`here <../../tutorials/tutorial_3/zno.json>`.
+In this tutorial we will calculate the band structure of bulk zinc oxide using the k-space formulation of Koopmans. The input file for this tutorial can be downloaded :download:`here <../../tutorials/tutorial_3/01-ki/zno.json>`.
 
 Calculating the Koopmans band structure
 ---------------------------------------
 
 The input file
 ^^^^^^^^^^^^^^
 
-First, let us inspect the input file:
+First, let us inspect the ``workflow`` block in the input file:
 
-.. literalinclude:: ../../tutorials/tutorial_3/zno.json
-  :lines: 1-9
-  :lineno-start: 1
-  :emphasize-lines: 4,6,8-9
+.. literalinclude:: ../../tutorials/tutorial_3/01-ki/zno.json
+  :lines: 2-15
+  :lineno-start: 2
+  :emphasize-lines: 3,5,7-8
 
 Here we tell the code to calculate the KI bandstructure using the DFPT primitive cell approach. We will not actually calculate the screening parameters in this tutorial (because this calculation takes a bit of time) so we have set ``calculate_alpha`` to ``False`` and we have provided some pre-computed screening parameters in the ``alpha_guess`` field.
 
@@ -25,28 +25,34 @@ The rest of the file contains the atomic coordinates, k-point configuration, and
 Running the calculation
 ^^^^^^^^^^^^^^^^^^^^^^^
 
-Running ``koopmans zno.json`` should produce an output with several sections: after the header there is the Wannierization
+Running ``koopmans zno.json`` should produce an output with each of the steps of the workflow. In particular, there is the Wannierization:
 
-.. literalinclude:: ../../tutorials/tutorial_3/zno.out
-  :lines: 16-36
-  :lineno-start: 16
-  :language: text
+----
+
+.. include:: ../_static/tutorials/tutorial_3/md_excerpts/zno_wannierize_section.md
+  :parser: myst_parser.sphinx_
+
+----
 
 which is very similar to what we saw for silicon, except now we have several blocks for the occupied manifold (discussed below). Then we have 
 
-.. literalinclude:: ../../tutorials/tutorial_3/zno.out
-  :lines: 37-40
-  :lineno-start: 37
-  :language: text
+----
+
+.. include:: ../_static/tutorials/tutorial_3/md_excerpts/zno_w2kc.md
+  :parser: myst_parser.sphinx_
+
+----
 
 where the ``Wannier90`` files are converted into a format readable by the ``kcw.x`` code.
 
 If we had instructed the code to calculate the alpha parameters, this would be followed by an extra block where these are calculated. But since we have already provided these, the workflow progresses immediately to the final step
 
-.. literalinclude:: ../../tutorials/tutorial_3/zno.out
-  :lines: 42-46
-  :lineno-start: 42
-  :language: text
+----
+
+.. include:: ../_static/tutorials/tutorial_3/md_excerpts/zno_ham.md
+  :parser: myst_parser.sphinx_
+
+----
 
 where the Koopmans Hamiltonian is constructed, and the band structure computed.
 
@@ -55,31 +61,31 @@ Plotting the results
 
 Running the workflow will have produced several directories containing various input and output ``Quantum ESPRESSO`` files. It will also have generated ``png`` band structure plots, including this one of the Koopmans band structure:
 
-.. figure:: ../../tutorials/tutorial_3/zno_koopmansdfpt_bandstructure.png
+.. figure:: ../_static/tutorials/tutorial_3/Koopmans_DFPT_bandstructure.png
   :width: 600
   :align: center
   :alt: the autogenerated band structure plot for ZnO
 
   The autogenerated band structure plot for ZnO
 
-However, suppose we want to make a nicer, more comprehensive plot comparing the LDA and Koopmans band structures. To achieve this, we will load all of the information from the ``zno.kwf`` file. This will provide us with a ``SinglepointWorkflow`` object which contains all of the calculations and their associated results. For example, we can access the calculations corresponding to the Koopmans and LDA band structures as follows:
+However, suppose we want to make a nicer, more comprehensive plot comparing the LDA and Koopmans band structures. To achieve this, we will load all of the information from the ``zno.pkl`` file. This will provide us with a ``SinglepointWorkflow`` object which contains all of the calculations and their associated results. For example, we can access the calculations corresponding to the Koopmans and LDA band structures as follows:
 
-.. literalinclude:: ../../tutorials/tutorial_3/plot_bandstructures.py
+.. literalinclude:: ../../tutorials/tutorial_3/01-ki/plot_bandstructures.py
   :lines: 5-14
 
 and then the band structures are in the ``results`` dictionary of these calculators:
 
-.. literalinclude:: ../../tutorials/tutorial_3/plot_bandstructures.py
+.. literalinclude:: ../../tutorials/tutorial_3/01-ki/plot_bandstructures.py
   :lines: 16-22
 
 These band structures are ``BandStructure`` objects `(documented here) <https://wiki.fysik.dtu.dk/ase/ase/dft/kpoints.html?highlight=bandstructure#ase.spectrum.band_structure.BandStructure>`_. Among other things, this class implements a ``plot`` function that allows us to plot them on the same set of axes:
 
-.. literalinclude:: ../../tutorials/tutorial_3/plot_bandstructures.py
+.. literalinclude:: ../../tutorials/tutorial_3/01-ki/plot_bandstructures.py
   :lines: 24-26
 
-With a few further aesthetic tweaks (download the full script :download:`here <../../tutorials/tutorial_3/plot_bandstructures.py>`) we obtain the following plot:
+With a few further aesthetic tweaks (download the full script :download:`here <../../tutorials/tutorial_3/01-ki/plot_bandstructures.py>`) we obtain the following plot:
 
-.. figure:: ../../tutorials/tutorial_3/zno_bandstructures.png
+.. figure:: ../../tutorials/tutorial_3/01-ki/zno_bandstructures.png
   :width: 600
   :align: center
 
@@ -102,7 +108,7 @@ The first step to any calculation on a periodic system is to obtain a good set o
 
 In the above calculation we gave you some Wannier projections to use:
 
-.. literalinclude:: ../../tutorials/tutorial_3/zno.json
+.. literalinclude:: ../../tutorials/tutorial_3/01-ki/zno.json
   :lines: 47-55
   :lineno-start: 47
 
@@ -112,9 +118,15 @@ But what if you need to come up with your own Wannier projections? In this bonus
 Configuring the Wannierization
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To determine a good choice for the Wannier projections, we can first calculate a projected density of states (pDOS). Take your input file and change the ``task`` from ``singlepoint`` to ``dft_bands``, and then run ``koopmans zno.json``. This will run the DFT bandstructure workflow, which will produce a directory called ``dft_bands`` that contains various files, including a ``png`` of the bandstructure and pDOS. If you look at this file, you will see that the DFT band structure of ZnO consists of several sets of bands, each well-separated in energy space. As the pDOS shows us, the filled bands correspond to zinc 3s, zinc 3p, oxygen 2s, and then zinc 3d hybridized with oxygen 2p. Meanwhile, the lowest empty bands correspond to Zn 4s bands.
+To determine a good choice for the Wannier projections, we can first calculate a projected density of states (pDOS). Take your input file and change the ``task`` from ``singlepoint`` to ``dft_bands``, and then run ``koopmans zno.json`` (for the sake of tidiness, this input file can be found in the ``02-dft_bands`` directory). This will run the DFT bandstructure workflow, which will produce various files, including a ``png`` of the bandstructure and pDOS. If you look at this file, you will see that the DFT band structure of ZnO consists of several sets of bands, each well-separated in energy space:
 
-A sensible choice for the occupied projectors is therefore
+.. figure:: ../../tutorials/tutorial_3/02-dft_bands/zno_bandstructure.png
+  :width: 600
+  :align: center
+
+  The pDOS of ZnO
+
+As the pDOS shows us, the filled bands correspond to zinc 3s, zinc 3p, oxygen 2s, and then zinc 3d hybridized with oxygen 2p. Meanwhile, the lowest empty bands correspond to Zn 4s bands. A sensible choice for the occupied projectors is therefore
 
 .. code:: json
 
@@ -127,7 +139,7 @@ A sensible choice for the occupied projectors is therefore
             {"site": "O", "ang_mtm": "l=1"}]
        ]  
   
-Here we will use of the block-Wannierization functionality to wannierize each block of bands separately. If we didn't do this, then the Wannierization procedure might mix orbitals from different blocks of bands (the algorithm minimizes the spatial spread without regard to the energies of the orbitals it is mixing). This sort of mixing between orbitals of very different energies is generally detrimental to the Wannierization and the resulting Koopmans band structure.
+Here we will use of the block-Wannierization functionality to wannierize each block of bands separately. If we didn't do this, then the Wannierization procedure might mix orbitals from different blocks of bands (the algorithm minimizes the spatial spread without regard to the energies of the orbitals it is mixing). This sort of mixing between orbitals of very different energies is generally detrimental to the resulting Koopmans band structure.
 
 For the empty bands we want to obtain two bands corresponding to the Zn 4s orbitals. These must be disentangled from the rest of the empty bands, which is achieved via the following ``Wannier90`` keywords.
 
@@ -139,13 +151,21 @@ For the empty bands we want to obtain two bands corresponding to the Zn 4s orbit
 
 To determine good values for these keywords, we clearly need a more zoomed-in band structure than the default. We can obtain this via the ``*.fig.pkl`` files that ``koopmans`` generates. Here is a short code snippet that replots the band structure over a narrower energy range
 
-.. literalinclude:: ../../tutorials/tutorial_3/replot_dft_bandstructure.py
+.. literalinclude:: ../../tutorials/tutorial_3/02-dft_bands/replot_dft_bandstructure.py
+
+which produces the new figure
+
+.. figure:: ../../tutorials/tutorial_3/02-dft_bands/zno_bandstructure_rescaled.png
+  :width: 600
+  :align: center
+
+  The zoomed-in band structure of ZnO
 
 Based on this figure, choose values for these two keywords and add them to your input file in the ``w90`` block. Append the Zn 4s projections to the list of projections, too.
 
 .. note::
 
-  ``dis_froz_max`` and ``dis_win_max`` should **not** be provided relative to the valence band edge. Meanwhile the band structure plots have set the valence band edge to zero. Make sure to account for this by shifting the values of ``dis_froz_max`` and ``dis_win_max`` by 9.3 eV (the valence band edge energy; you can get this value yourself via ``grep 'highest occupied level' dft_bands/scf.pwo``)
+  ``dis_froz_max`` and ``dis_win_max`` should **not** be provided relative to the valence band edge. Meanwhile the band structure plots have set the valence band edge to zero. Make sure to account for this by shifting the values of ``dis_froz_max`` and ``dis_win_max`` by 9.3 eV (the valence band edge energy; you can get this value yourself via ``grep 'highest occupied level' 01-scf/scf.pwo``)
 
 .. note::
 
@@ -154,7 +174,7 @@ Based on this figure, choose values for these two keywords and add them to your
 Testing the Wannierization
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To test your wannierization, you can now switch to the ``wannierize`` task and once again run ``koopmans zno.json``. This will generate a ``wannier`` directory as well as a band structure plot, this time with the interpolated band structure plotted on top of the explicitly evaluated band structure. Ideally, the interpolated band structure should lie on top of the explicit one. Play around with the values of ``dis_froz_max`` and ``dis_win_max`` until you are happy with the resulting band structure.
+To test your wannierization, you can now switch to the ``wannierize`` task and once again run ``koopmans zno.json`` (or move to ``03-wannierize``). This will generate various directories containing calculations, and ultimately band structure plot, this time with the interpolated band structure plotted on top of the explicitly evaluated band structure. Ideally, the interpolated band structure should lie on top of the explicit one. Play around with the values of ``dis_froz_max`` and ``dis_win_max`` until you are happy with the resulting band structure.
 
 .. hint::
 

src/koopmans/workflows/_workflow.py

@@ -1605,8 +1605,7 @@ def plot_bandstructure(self,
 
         # Saving the figure to file (as png and also in editable form)
         workflow_name = self.name
-        for s in ['workflow', 'mock', 'benchgen', 'stumbling', 'check']:
-            workflow_name = workflow_name.replace(s, '')
+        workflow_name = workflow_name.replace('workflow', '').replace(' ', '_')
         filename = filename if filename is not None else f'{workflow_name}_bandstructure'
         legends = [ax.get_legend() for ax in axes if ax.get_legend() is not None]
         utils.savefig(fname=filename + '.png', bbox_extra_artists=legends, bbox_inches='tight')