Documentation update and added a full test

.github/workflows/ci.yml

@@ -75,10 +75,10 @@ jobs:
       - name: Check out pipeline code
         uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
 
-      - name: Set up Nextflow
+      - name: Install Nextflow
         uses: nf-core/setup-nextflow@v2
         with:
-          version: "${{ matrix.NXF_VER }}"
+          version: "${{ matrix.NXT_VER }}"
 
       - name: Set up Apptainer
         if: matrix.profile == 'singularity'

CITATIONS.md

@@ -10,13 +10,41 @@
 
 ## Pipeline tools
 
-- [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)
+- [BaSiCPy](https://basicpy.readthedocs.io/en/stable/)
 
-> Andrews, S. (2010). FastQC: A Quality Control Tool for High Throughput Sequence Data [Online].
+  > Peng, T., Thorn, K., Schroeder, T., Wang, L., Theis, F. J., Marr, C., Navab, N. A BaSiC Tool for Background and Shading Correction of Optical Microscopy Images. Nature Communication 2017 June 08; 8(1):14836. doi: [10.1038/ncomms14836](http://doi.org/10.1038/ncomms14836).
+
+- [ASHLAR](https://labsyspharm.github.io/ashlar/)
+
+  > Muhlich, J. L., Chen, Y., Yapp, C., Russell, D., Santagata, S., Sorger, P. K. Stitching and registering highly multiplexed whole-slide images of tissues and tumors using ASHLAR. Bioinformatics 2022 October; 38(19):4613–4621. doi: [10.1093/bioinformatics/btac544](https://doi.org/10.1093/bioinformatics/btac544).
+
+- [Backsub](https://github.com/SchapiroLabor/Background_subtraction)
+
+  > Schapiro, D., Sokolov, A., Yapp, C. et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nat Methods 2022; 19:311–315. doi: [10.1038/s41592-021-01308-y](https://doi.org/10.1038/s41592-021-01308-y)
+
+- [Coreograph](https://github.com/HMS-IDAC/UNetCoreograph)
+
+  > Schapiro, D., Sokolov, A., Yapp, C. et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nat Methods 2022; 19:311–315. doi: [10.1038/s41592-021-01308-y](https://doi.org/10.1038/s41592-021-01308-y)
+
+- [Cellpose](https://cellpose.readthedocs.io/en/latest/index.html)
+
+  > Pachitariu, M., Stringer, C. Cellpose 2.0: how to train your own model. Nat Methods 2022; 19:1634–1641. doi: [10.1038/s41592-022-01663-4](https://doi.org/10.1038/s41592-022-01663-4)
+
+- [Mesmer](https://deepcell.readthedocs.io/en/master/)
+
+  > Greenwald, N.F., Miller, G., Moen, E. et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat Biotechnol 2022; 40:555–565. doi: [10.1038/s41587-021-01094-0](https://doi.org/10.1038/s41587-021-01094-0)
+
+- [MCQuant](https://github.com/labsyspharm/quantification)
+
+  > Schapiro, D., Sokolov, A., Yapp, C. et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nat Methods 2022; 19:311–315. doi: [10.1038/s41592-021-01308-y](https://doi.org/10.1038/s41592-021-01308-y)
+
+- [SciMap](https://scimap.xyz/)
+
+  > Nirmal et al. SCIMAP: A Python Toolkit for Integrated Spatial Analysis of Multiplexed Imaging Data. Journal of Open Source Software 2024; 9(97):6604, doi: [10.21105/joss.06604](https://doi.org/10.21105/joss.06604)
 
 - [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
 
-> Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.
+  > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.
 
 ## Software packaging/containerisation tools
 
@@ -39,3 +67,7 @@
 - [Singularity](https://pubmed.ncbi.nlm.nih.gov/28494014/)
 
   > Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLoS One. 2017 May 11;12(5):e0177459. doi: 10.1371/journal.pone.0177459. eCollection 2017. PubMed PMID: 28494014; PubMed Central PMCID: PMC5426675.
+
+## Test Data
+
+> Schapiro, D., Sokolov, A., Yapp, C. et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nat Methods 2022; 19:311–315. doi: [10.1038/s41592-021-01308-y](https://doi.org/10.1038/s41592-021-01308-y)

README.md

@@ -26,66 +26,76 @@
 
 If you want to run the original MCMICRO pipeline outside of nf-core, please see <https://mcmicro.org/>.
 
-<!-- TODO nf-core:
-   Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the
-   major pipeline sections and the types of output it produces. You're giving an overview to someone new
-   to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction
--->
+The nf-core/mcmicro pipeline is an end-to-end processing pipeline that transforms multi-channel whole-slide images into single-cell data. It takes samplesheet and markersheet files as input and perfoms registration, segmentation and quantification. Multiple segmentation modules are available and can be run in parallel. The pipeline can also optionally perform background and shading correction, background subtraction, and supports tissue microarrays. It returns a pre-segmentation image file, a segmentation mask image, and a cell x feature array spreadsheet.
 
-<!-- TODO nf-core: Include a figure that guides the user through the major workflow steps. Many nf-core
-     workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples.   -->
-<!-- TODO nf-core: Fill in short bullet-pointed list of the default steps in the pipeline -->
+![nf-core/mcmicro metro diagram](assets/mcmicro_metro.png)
 
-<!-- ## Usage
+The MC in MCMICRO stands for 'Multiple Choice' and that refers to the fact that there will be multiple module options available for many steps in the pipeline. The currently supported options are provided in parentheses for required steps.
+
+1. Registration ([Ashlar](https://nf-co.re/modules/ashlar/))
+2. Segmentation ([Cellpose](https://nf-co.re/modules/cellpose/), [Mesmer](https://nf-co.re/modules/deepcell_mesmer/))
+3. Quantification ([MCQuant](https://nf-co.re/modules/mcquant/))
+
+## Usage
 
 > [!NOTE]
 > If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.
 
-<!-- TODO nf-core: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
-     Explain what rows and columns represent. For instance (please edit as appropriate):
-
-First, prepare a samplesheet with your input data that looks as follows:
+First, prepare a samplesheet and markersheet with your input data that looks as follows:
 
 `samplesheet.csv`:
 
 ```csv
-sample,fastq_1,fastq_2
-CONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz
+sample,cycle_number,channel_count,image_tiles
+TEST1,1,4,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle1.ome.tif
+
 ```
 
-Each row represents a fastq file (single-end) or a pair of fastq files (paired end).
+Each row of the samplesheet represents a single cycle `ome.tiff` microscopy image file. The first column `sample` is the sample identifier. It can be repeated in multiple rows for samples with multiple cycles. The second column `cycle_number` is a number corresponding to the cycle number of the image. The third column `channel_count` is a count of the number of channels in the image. The fourth column `image_tiles` is a path or URL to the input image file.
 
--->
+`markersheet.csv`:
 
-<!-- Now, you can run the pipeline using: -->
+```csv
+channel_number,cycle_number,marker_name
+1,1,DNA 1
+2,1,Na/K ATPase
+3,1,CD3
+4,1,CD45RO
+```
 
-<!-- TODO nf-core: update the following command to include all required parameters for a minimal example -->
+Each row of the markersheet represents a single channel in the associated sample image. The first column `channel_number` is an identifier for the respective channel. The second column `cycle_number` corresponds to the cycle number of the image and it must match the `cycle_number` in the supplied samplesheet. The third column `marker_name` is the name of the marker for the given channel and cycle.
 
-<!-- ```bash
+> [!WARNING]
+> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_; see [docs](https://nf-co.re/docs/usage/getting_started/configuration#custom-configuration-files).
+
+Now, you can run the pipeline using:
+
+```bash
 nextflow run nf-core/mcmicro \
    -profile <docker/singularity/.../institute> \
-   --input samplesheet.csv \
+   --input_cycle samplesheet.csv \
+   --marker_sheet markers.csv \
    --outdir <OUTDIR>
-``` -->
-
-> [!WARNING]
-> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_; see [docs](https://nf-co.re/docs/usage/getting_started/configuration#custom-configuration-files).
+```
 
-<!-- For more details and further functionality, please refer to the [usage documentation](https://nf-co.re/mcmicro/usage) and the [parameter documentation](https://nf-co.re/mcmicro/parameters). -->
+For more details and further functionality, please refer to the [usage documentation](https://nf-co.re/mcmicro/usage) and the [parameter documentation](https://nf-co.re/mcmicro/parameters).
 
-<!-- ## Pipeline output
+## Pipeline output
 
 To see the results of an example test run with a full size dataset refer to the [results](https://nf-co.re/mcmicro/results) tab on the nf-core website pipeline page.
 For more details about the output files and reports, please refer to the
-[output documentation](https://nf-co.re/mcmicro/output). -->
+[output documentation](https://nf-co.re/mcmicro/output).
 
-<!-- ## Credits
+## Credits
 
-nf-core/mcmicro was originally written by TBD.
+nf-core/mcmicro was written by Rob Young and Jeremy Muhlich from the Laboratory of Systems Pharmacology at Harvard Medical School. It is based on version 1.0 of mcmicro (labsyspharm/mcmicro) which was also developed there.
 
-We thank the following people for their extensive assistance in the development of this pipeline: -->
+We thank the following people for their assistance in the development of this pipeline:
 
-<!-- TODO nf-core: If applicable, make list of people who have also contributed -->
+- [heylf](https://github.com/heylf)
+- [Florian Wuennemann](https://github.com/FloWuenne)
+- [Phil Ewels](https://github.com/ewels)
+- [Adam Taylor](https://github.com/adamjtaylor)
 
 ## Contributions and Support
 
@@ -98,8 +108,6 @@ For further information or help, don't hesitate to get in touch on the [Slack `#
 <!-- TODO nf-core: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. -->
 <!-- If you use nf-core/mcmicro for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->
 
-<!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
-
 If you use nf-core/mcmicro for your analysis, please cite it using the following article: [Schapiro et al. 2022 Nat. Methods](https://www.nature.com/articles/s41592-021-01308-y)
 
 An extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.

assets/markers-test_full.csv

@@ -0,0 +1,9 @@
+channel_number,cycle_number,marker_name
+1,1,DNA_6
+2,1,ELANE
+3,1,CD57
+4,1,CD45
+5,2,DNA_7
+6,2,ELANE7
+7,2,CD577
+8,2,CD457

assets/methods_description_template.yml

@@ -3,8 +3,6 @@ description: "Suggested text and references to use when describing pipeline usag
 section_name: "nf-core/mcmicro Methods Description"
 section_href: "https://github.com/nf-core/mcmicro"
 plot_type: "html"
-## TODO nf-core: Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
-## You inject any metadata in the Nextflow '${workflow}' object
 data: |
   <h4>Methods</h4>
   <p>Data was processed using nf-core/mcmicro v${workflow.manifest.version} ${doi_text} of the nf-core collection of workflows (<a href="https://doi.org/10.1038/s41587-020-0439-x">Ewels <em>et al.</em>, 2020</a>), utilising reproducible software environments from the Bioconda (<a href="https://doi.org/10.1038/s41592-018-0046-7">Grüning <em>et al.</em>, 2018</a>) and Biocontainers (<a href="https://doi.org/10.1093/bioinformatics/btx192">da Veiga Leprevost <em>et al.</em>, 2017</a>) projects.</p>

assets/samplesheet-test_full.csv

@@ -0,0 +1,5 @@
+sample,cycle_number,channel_count,image_tiles
+TEST1,1,4,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle1.ome.tif
+TEST1,2,4,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle2.ome.tif
+TEST2,1,4,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle2.ome.tif
+TEST2,2,4,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/imaging/ome-tiff/cycif-tonsil-cycle3.ome.tif

conf/test.config

@@ -37,4 +37,8 @@ process {
         memory = "6.GB"
         cpus = 2
     }
+    withName: ".*:CELLPOSE"{
+        memory = "12.GB"
+        cpus   = 4
+    }
 }

conf/test_full.config

@@ -17,8 +17,29 @@ params {
     // Input data for full size test
     // TODO nf-core: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
     // TODO nf-core: Give any required params for the test so that command line flags are not needed
-    input = params.pipelines_testdata_base_path + 'viralrecon/samplesheet/samplesheet_full_illumina_amplicon.csv'
 
-    // Genome references
-    genome = 'R64-1-1'
+    max_cpus   = 2
+    max_memory = '6.GB'
+    max_time   = '6.h'
+
+    // Input data
+    input_cycle  = "${projectDir}/assets/samplesheet-test_full.csv"
+    marker_sheet = "${projectDir}/assets/markers-test_full.csv"
+
+    // TODO: Add samplesheet and markersheet to testdata repo then switch to this format:
+    //input_cycle  = params.pipelines_testdata_base_path + 'mcmicro/samplesheet/???.csv'
+
+    illumination = "basicpy"
+    segmentation = "mesmer,cellpose"
+}
+
+process {
+    withName: ".*:DEEPCELL_MESMER" {
+        memory = "6.GB"
+        cpus = 2
+    }
+    withName: 'CELLPOSE'{
+        memory = "12.GB"
+        cpus   = 4
+    }
 }