diff --git a/README.md b/README.md
index 485576e..f990105 100644
--- a/README.md
+++ b/README.md
@@ -27,6 +27,7 @@ Comprehensive benchmarking of protein-ligand structure generation methods
- [Tutorials](#tutorials)
- [How to prepare PoseBench data](#how-to-prepare-posebench-data)
- [Available inference methods](#available-inference-methods)
+- [How to run a sweep of benchmarking experiments](#how-to-run-a-sweep-of-benchmarking-experiments)
- [How to run inference with individual methods](#how-to-run-inference-with-individual-methods)
- [How to run inference with a method ensemble](#how-to-run-inference-with-a-method-ensemble)
- [How to create comparative plots of inference results](#how-to-create-comparative-plots-of-inference-results)
@@ -325,6 +326,26 @@ python3 posebench/data/components/protein_apo_to_holo_alignment.py dataset=astex
+## How to run a sweep of benchmarking experiments
+
+
+
+Build inference scripts for one's desired sweep
+
+```bash
+python3 scripts/build_inference_script.py sweep=true export_hpc_headers=true
+```
+
+Submit the inference scripts for job scheduling
+
+```bash
+sbatch scripts/inference/*_inference_*.sh
+```
+
+**NOTE**: See the config file `configs/scripts/build_inference_script.yaml` for more details.
+
+
+
## How to run inference with individual methods
@@ -337,7 +358,7 @@ Prepare CSV input files
python3 posebench/data/diffdock_input_preparation.py dataset=posebusters_benchmark
python3 posebench/data/diffdock_input_preparation.py dataset=astex_diverse
python3 posebench/data/diffdock_input_preparation.py dataset=dockgen
-python3 posebench/data/diffdock_input_preparation.py dataset=casp15 input_data_dir="$PWD"/data/casp15_set/targets input_protein_structure_dir="$PWD"/data/casp15_set/casp15_holo_aligned_predicted_structures
+python3 posebench/data/diffdock_input_preparation.py dataset=casp15 input_data_dir=data/casp15_set/targets input_protein_structure_dir=data/casp15_set/casp15_holo_aligned_predicted_structures
```
Run inference on each dataset
@@ -356,16 +377,14 @@ python3 posebench/models/diffdock_inference.py dataset=casp15 batch_size=1 repea
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=diffdock dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=diffdock dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=diffdock dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=diffdock dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=diffdock dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=diffdock dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Analyze inference results for each dataset
```bash
@@ -414,16 +433,14 @@ python3 posebench/models/fabind_inference.py dataset=dockgen repeat_index=1
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=fabind dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=fabind dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=fabind dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=fabind dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=fabind dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=fabind dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Analyze inference results for each dataset
```bash
@@ -443,7 +460,7 @@ Prepare CSV input files
python3 posebench/data/dynamicbind_input_preparation.py dataset=posebusters_benchmark
python3 posebench/data/dynamicbind_input_preparation.py dataset=astex_diverse
python3 posebench/data/dynamicbind_input_preparation.py dataset=dockgen
-python3 posebench/data/dynamicbind_input_preparation.py dataset=casp15 input_data_dir="$PWD"/data/casp15_set/targets
+python3 posebench/data/dynamicbind_input_preparation.py dataset=casp15 input_data_dir=data/casp15_set/targets
```
Run inference on each dataset
@@ -455,23 +472,21 @@ python3 posebench/models/dynamicbind_inference.py dataset=astex_diverse repeat_i
...
python3 posebench/models/dynamicbind_inference.py dataset=dockgen repeat_index=1
...
-python3 posebench/models/dynamicbind_inference.py dataset=casp15 batch_size=1 input_data_dir="$PWD"/data/casp15_set/casp15_holo_aligned_predicted_structures repeat_index=1
+python3 posebench/models/dynamicbind_inference.py dataset=casp15 batch_size=1 input_data_dir=data/casp15_set/casp15_holo_aligned_predicted_structures repeat_index=1
...
```
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=dynamicbind dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=dynamicbind dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=dynamicbind dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=dynamicbind dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=dynamicbind dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=dynamicbind dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Analyze inference results for each dataset
```bash
@@ -504,7 +519,7 @@ Prepare CSV input files
python3 posebench/data/neuralplexer_input_preparation.py dataset=posebusters_benchmark
python3 posebench/data/neuralplexer_input_preparation.py dataset=astex_diverse
python3 posebench/data/neuralplexer_input_preparation.py dataset=dockgen
-python3 posebench/data/neuralplexer_input_preparation.py dataset=casp15 input_data_dir="$PWD"/data/casp15_set/targets input_receptor_structure_dir="$PWD"/data/casp15_set/casp15_holo_aligned_predicted_structures
+python3 posebench/data/neuralplexer_input_preparation.py dataset=casp15 input_data_dir=data/casp15_set/targets input_receptor_structure_dir=data/casp15_set/casp15_holo_aligned_predicted_structures
```
Run inference on each dataset
@@ -516,23 +531,21 @@ python3 posebench/models/neuralplexer_inference.py dataset=astex_diverse repeat_
...
python3 posebench/models/neuralplexer_inference.py dataset=dockgen repeat_index=1
...
-python3 posebench/models/neuralplexer_inference.py dataset=casp15 repeat_index=1
+python3 posebench/models/neuralplexer_inference.py dataset=casp15 chunk_size=4 repeat_index=1
...
```
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=neuralplexer dataset=posebusters_benchmark num_processes=1 remove_initial_protein_hydrogens=true assign_partial_charges_manually=true cache_files=false repeat_index=1
+python3 posebench/models/inference_relaxation.py method=neuralplexer dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=neuralplexer dataset=astex_diverse num_processes=1 remove_initial_protein_hydrogens=true assign_partial_charges_manually=true cache_files=false repeat_index=1
+python3 posebench/models/inference_relaxation.py method=neuralplexer dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=neuralplexer dataset=dockgen num_processes=1 remove_initial_protein_hydrogens=true assign_partial_charges_manually=true cache_files=false repeat_index=1
+python3 posebench/models/inference_relaxation.py method=neuralplexer dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Align predicted protein-ligand structures to ground-truth complex structures
```bash
@@ -576,7 +589,7 @@ Prepare CSV input files
python3 posebench/data/rfaa_input_preparation.py dataset=posebusters_benchmark
python3 posebench/data/rfaa_input_preparation.py dataset=astex_diverse
python3 posebench/data/rfaa_input_preparation.py dataset=dockgen
-python3 posebench/data/rfaa_input_preparation.py dataset=casp15 input_data_dir="$PWD"/data/casp15_set/targets
+python3 posebench/data/rfaa_input_preparation.py dataset=casp15 input_data_dir=data/casp15_set/targets
```
Run inference on each dataset
@@ -602,13 +615,11 @@ python3 posebench/data/rfaa_output_extraction.py dataset=casp15
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=rfaa dataset=posebusters_benchmark num_processes=1 remove_initial_protein_hydrogens=true
-python3 posebench/models/inference_relaxation.py method=rfaa dataset=astex_diverse num_processes=1 remove_initial_protein_hydrogens=true
-python3 posebench/models/inference_relaxation.py method=rfaa dataset=dockgen num_processes=1 remove_initial_protein_hydrogens=true
+python3 posebench/models/inference_relaxation.py method=rfaa dataset=posebusters_benchmark remove_initial_protein_hydrogens=true
+python3 posebench/models/inference_relaxation.py method=rfaa dataset=astex_diverse remove_initial_protein_hydrogens=true
+python3 posebench/models/inference_relaxation.py method=rfaa dataset=dockgen remove_initial_protein_hydrogens=true
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Align predicted protein-ligand structures to ground-truth complex structures
```bash
@@ -634,7 +645,7 @@ python3 posebench/models/ensemble_generation.py ensemble_methods=\[rfaa\] input_
# NOTE: the suffixes for both `output_dir` and `ensemble_benchmarking_repeat_index` should be modified to e.g., 2, 3, ...
...
# now score the CASP15-compliant submissions using the official CASP scoring pipeline
-python3 posebench/analysis/inference_analysis_casp.py method=rfaa dataset=casp15 targets="[T1124, T1127v2, T1146, T1152, T1158v1, T1158v2, T1158v3, T1158v4, T1186, T1187, T1188]" repeat_index=1
+python3 posebench/analysis/inference_analysis_casp.py method=rfaa dataset=casp15 targets='[T1124, T1127v2, T1146, T1152, T1158v1, T1158v2, T1158v3, T1158v4, T1186, T1187, T1188]' repeat_index=1
...
```
@@ -678,16 +689,14 @@ mkdir -p forks/Vina/inference/vina_diffdock_casp15_outputs_1 && cp -r data/test_
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=vina vina_binding_site_method=diffdock dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=vina vina_binding_site_method=diffdock dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=vina vina_binding_site_method=diffdock dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=vina vina_binding_site_method=diffdock dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
-python3 posebench/models/inference_relaxation.py method=vina vina_binding_site_method=diffdock dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1 repeat_index=1
+python3 posebench/models/inference_relaxation.py method=vina vina_binding_site_method=diffdock dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true repeat_index=1
...
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Analyze inference results for each dataset
```bash
@@ -719,20 +728,21 @@ Gather all template ligands generated by `TULIP` via its dedicated [GitHub repos
```bash
python3 posebench/data/tulip_output_extraction.py dataset=posebusters_benchmark
python3 posebench/data/tulip_output_extraction.py dataset=astex_diverse
+python3 posebench/data/tulip_output_extraction.py dataset=dockgen
python3 posebench/data/tulip_output_extraction.py dataset=casp15
```
Relax the generated ligand structures inside of their respective protein pockets
```bash
-python3 posebench/models/inference_relaxation.py method=tulip dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1
+python3 posebench/models/inference_relaxation.py method=tulip dataset=posebusters_benchmark remove_initial_protein_hydrogens=true assign_partial_charges_manually=true
+...
+python3 posebench/models/inference_relaxation.py method=tulip dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true
...
-python3 posebench/models/inference_relaxation.py method=tulip dataset=astex_diverse remove_initial_protein_hydrogens=true assign_partial_charges_manually=true num_processes=1
+python3 posebench/models/inference_relaxation.py method=tulip dataset=dockgen remove_initial_protein_hydrogens=true assign_partial_charges_manually=true
...
```
-**NOTE**: Increase `num_processes` according to your available CPU/GPU resources to improve throughput
-
Analyze inference results for each dataset
```bash
@@ -740,6 +750,8 @@ python3 posebench/analysis/inference_analysis.py method=tulip dataset=posebuster
...
python3 posebench/analysis/inference_analysis.py method=tulip dataset=astex_diverse
...
+python3 posebench/analysis/inference_analysis.py method=tulip dataset=dockgen
+...
```
Analyze inference results for the CASP15 dataset
@@ -789,8 +801,8 @@ python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cas
python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cases/astex_diverse/ensemble_inputs.csv output_dir=data/test_cases/astex_diverse/top_consensus_ensemble_predictions_1 max_method_predictions=40 export_top_n=1 export_file_format=null skip_existing=true relax_method_ligands_post_ranking=true resume=true cuda_device_index=0 ensemble_methods='[diffdock, dynamicbind, neuralplexer, rfaa, tulip, vina]' ensemble_benchmarking=true ensemble_benchmarking_dataset=astex_diverse ensemble_ranking_method=consensus ensemble_benchmarking_repeat_index=1
...
# benchmark using the DockGen dataset e.g., after generating 40 complexes per target with each method
-python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cases/dockgen/ensemble_inputs.csv output_dir=data/test_cases/dockgen/top_consensus_ensemble_predictions_1 max_method_predictions=40 export_top_n=1 export_file_format=null skip_existing=true relax_method_ligands_post_ranking=false resume=true cuda_device_index=0 ensemble_methods='[diffdock, dynamicbind, neuralplexer, rfaa, vina]' ensemble_benchmarking=true ensemble_benchmarking_dataset=dockgen ensemble_ranking_method=consensus ensemble_benchmarking_repeat_index=1
-python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cases/dockgen/ensemble_inputs.csv output_dir=data/test_cases/dockgen/top_consensus_ensemble_predictions_1 max_method_predictions=40 export_top_n=1 export_file_format=null skip_existing=true relax_method_ligands_post_ranking=true resume=true cuda_device_index=0 ensemble_methods='[diffdock, dynamicbind, neuralplexer, rfaa, vina]' ensemble_benchmarking=true ensemble_benchmarking_dataset=dockgen ensemble_ranking_method=consensus ensemble_benchmarking_repeat_index=1
+python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cases/dockgen/ensemble_inputs.csv output_dir=data/test_cases/dockgen/top_consensus_ensemble_predictions_1 max_method_predictions=40 export_top_n=1 export_file_format=null skip_existing=true relax_method_ligands_post_ranking=false resume=true cuda_device_index=0 ensemble_methods='[diffdock, dynamicbind, neuralplexer, rfaa, tulip, vina]' ensemble_benchmarking=true ensemble_benchmarking_dataset=dockgen ensemble_ranking_method=consensus ensemble_benchmarking_repeat_index=1
+python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cases/dockgen/ensemble_inputs.csv output_dir=data/test_cases/dockgen/top_consensus_ensemble_predictions_1 max_method_predictions=40 export_top_n=1 export_file_format=null skip_existing=true relax_method_ligands_post_ranking=true resume=true cuda_device_index=0 ensemble_methods='[diffdock, dynamicbind, neuralplexer, rfaa, tulip, vina]' ensemble_benchmarking=true ensemble_benchmarking_dataset=dockgen ensemble_ranking_method=consensus ensemble_benchmarking_repeat_index=1
...
# benchmark using the CASP15 dataset e.g., after generating 40 complexes per target with each method
python3 posebench/models/ensemble_generation.py input_csv_filepath=data/test_cases/casp15/ensemble_inputs.csv output_dir=data/test_cases/casp15/top_consensus_ensemble_predictions_1 combine_casp_output_files=true max_method_predictions=40 export_top_n=5 export_file_format=casp15 skip_existing=true relax_method_ligands_post_ranking=false resume=true cuda_device_index=0 ensemble_methods='[diffdock, dynamicbind, neuralplexer, rfaa, tulip, vina]' ensemble_benchmarking=true ensemble_benchmarking_dataset=casp15 ensemble_ranking_method=consensus ensemble_benchmarking_repeat_index=1
@@ -806,7 +818,7 @@ python3 posebench/analysis/inference_analysis.py method=ensemble dataset=astex_d
python3 posebench/analysis/inference_analysis.py method=ensemble dataset=dockgen repeat_index=1
...
# analyze benchmarking results for the CASP15 dataset
-python3 posebench/analysis/inference_analysis_casp.py method=ensemble dataset=casp15 ensemble_ranking_method=consensus repeat_index=1
+python3 posebench/analysis/inference_analysis_casp.py method=ensemble dataset=casp15 repeat_index=1
...
```
diff --git a/docs/source/acknowledgements.rst b/docs/source/acknowledgements.rst
index af2b6aa..6f5faa7 100644
--- a/docs/source/acknowledgements.rst
+++ b/docs/source/acknowledgements.rst
@@ -2,5 +2,5 @@ Acknowledgements
================
.. mdinclude:: ../../README.md
- :start-line: 875
- :end-line: 892
+ :start-line: 887
+ :end-line: 904
diff --git a/docs/source/available_methods.rst b/docs/source/available_methods.rst
index 0ebb58d..5441165 100644
--- a/docs/source/available_methods.rst
+++ b/docs/source/available_methods.rst
@@ -2,8 +2,8 @@ Available inference methods
================
.. mdinclude:: ../../README.md
- :start-line: 286
- :end-line: 323
+ :start-line: 287
+ :end-line: 324
.. note::
Have a new method to add? Please let us know by creating a pull request. We would be happy to work with you to integrate new methodology into this benchmark!
diff --git a/docs/source/bonus.rst b/docs/source/bonus.rst
index e0896f8..73c3257 100644
--- a/docs/source/bonus.rst
+++ b/docs/source/bonus.rst
@@ -2,8 +2,8 @@ Bonus
================
.. mdinclude:: ../../README.md
- :start-line: 910
- :end-line: 912
+ :start-line: 922
+ :end-line: 924
.. image:: ./_static/WorkBench.jpeg
:alt: My brain after building PoseBench
diff --git a/docs/source/citing_this_work.rst b/docs/source/citing_this_work.rst
index cca6931..26cb2e1 100644
--- a/docs/source/citing_this_work.rst
+++ b/docs/source/citing_this_work.rst
@@ -2,5 +2,5 @@ Citing this work
================
.. mdinclude:: ../../README.md
- :start-line: 894
- :end-line: 906
+ :start-line: 906
+ :end-line: 918
diff --git a/docs/source/comparative_plots.rst b/docs/source/comparative_plots.rst
index c4cd431..58fb04a 100644
--- a/docs/source/comparative_plots.rst
+++ b/docs/source/comparative_plots.rst
@@ -2,5 +2,5 @@ How to create comparative plots of inference results
================
.. mdinclude:: ../../README.md
- :start-line: 822
- :end-line: 831
+ :start-line: 834
+ :end-line: 843
diff --git a/docs/source/data_preparation.rst b/docs/source/data_preparation.rst
index 10eecd2..bc54df2 100644
--- a/docs/source/data_preparation.rst
+++ b/docs/source/data_preparation.rst
@@ -2,5 +2,5 @@ How to prepare `PoseBench` data
================
.. mdinclude:: ../../README.md
- :start-line: 147
- :end-line: 280
+ :start-line: 148
+ :end-line: 281
diff --git a/docs/source/ensemble_inference.rst b/docs/source/ensemble_inference.rst
index 11f20e4..fde71c3 100644
--- a/docs/source/ensemble_inference.rst
+++ b/docs/source/ensemble_inference.rst
@@ -2,8 +2,8 @@ How to run inference with a method ensemble
================
.. mdinclude:: ../../README.md
- :start-line: 763
- :end-line: 814
+ :start-line: 775
+ :end-line: 828
.. note::
In addition to having `consensus` as an available value for `ensemble_ranking_method`, one can also set `ensemble_ranking_method=ff` to have the method ensemble's top-ranked predictions selected using the criterion of "minimum (molecular dynamics) force field energy" (albeit while incurring a very large runtime complexity).
diff --git a/docs/source/for_developers.rst b/docs/source/for_developers.rst
index ecc12d3..195e22c 100644
--- a/docs/source/for_developers.rst
+++ b/docs/source/for_developers.rst
@@ -2,5 +2,5 @@ For developers
================
.. mdinclude:: ../../README.md
- :start-line: 837
- :end-line: 871
+ :start-line: 849
+ :end-line: 883
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 874b012..ca81fad 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -29,6 +29,7 @@ Welcome to PoseBench's documentation!
tutorials
data_preparation
available_methods
+ sweep_inference
method_inference
ensemble_inference
comparative_plots
diff --git a/docs/source/installation.rst b/docs/source/installation.rst
index 3c00cf9..d0786be 100644
--- a/docs/source/installation.rst
+++ b/docs/source/installation.rst
@@ -2,5 +2,5 @@ Installation
================
.. mdinclude:: ../../README.md
- :start-line: 41
- :end-line: 129
+ :start-line: 42
+ :end-line: 130
diff --git a/docs/source/method_inference.rst b/docs/source/method_inference.rst
index 2bfcb18..09a268e 100644
--- a/docs/source/method_inference.rst
+++ b/docs/source/method_inference.rst
@@ -2,5 +2,5 @@ How to run inference with individual methods
================
.. mdinclude:: ../../README.md
- :start-line: 331
- :end-line: 757
+ :start-line: 352
+ :end-line: 769
diff --git a/docs/source/sweep_inference.rst b/docs/source/sweep_inference.rst
new file mode 100644
index 0000000..3924214
--- /dev/null
+++ b/docs/source/sweep_inference.rst
@@ -0,0 +1,6 @@
+How to run a sweep of benchmarking experiments
+================
+
+.. mdinclude:: ../../README.md
+ :start-line: 332
+ :end-line: 346
diff --git a/docs/source/tutorials.rst b/docs/source/tutorials.rst
index cdaab80..090eb38 100644
--- a/docs/source/tutorials.rst
+++ b/docs/source/tutorials.rst
@@ -2,5 +2,5 @@ Tutorials
================
.. mdinclude:: ../../README.md
- :start-line: 135
- :end-line: 141
+ :start-line: 136
+ :end-line: 142