chore: merge pull request #7 from nf-core/dev

release v1.0.2
nf-core · Jul 11, 2023 · 87ea6e2 · 87ea6e2
2 parents 0ca38c3 + 3fe2406
commit 87ea6e2
Show file tree

Hide file tree

Showing 27 changed files with 241 additions and 659 deletions.
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -116,4 +116,3 @@ To get started:
 Devcontainer specs:
 
 - [DevContainer config](.devcontainer/devcontainer.json)
-- [Dockerfile](.devcontainer/Dockerfile)
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -42,7 +42,7 @@ body:
     attributes:
       label: System information
       description: |
-        * Nextflow version _(eg. 22.10.1)_
+        * Nextflow version _(eg. 23.04.0)_
         * Hardware _(eg. HPC, Desktop, Cloud)_
         * Executor _(eg. slurm, local, awsbatch)_
         * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_

diff --git a/.github/workflows/awsfulltest.yml b/.github/workflows/awsfulltest.yml
@@ -14,14 +14,17 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Launch build-reference workflow via tower
-        uses: seqeralabs/action-tower-launch@v1
+        uses: seqeralabs/action-tower-launch@v2
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
           access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
+          revision: ${{ github.sha }}
           workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/marsseq/work-${{ github.sha }}
           parameters: |
             {
+              "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
+              "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/marsseq/results-${{ github.sha }}"
               "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/marsseq/results-test-${{ github.sha }}",
               "build_references": true,
               "velocity": true
@@ -45,8 +48,11 @@ jobs:
               "genomes_base": "s3://${{ secrets.AWS_S3_BUCKET }}/marsseq/results-test-${{ github.sha }}/references",
               "velocity": true
             }
-          profiles: test_full,aws_tower
+          profiles: test_full
+
       - uses: actions/upload-artifact@v3
         with:
           name: Tower debug log file
-          path: tower_action_*.log
+          path: |
+            tower_action_*.log
+            tower_action_*.json
diff --git a/.github/workflows/awstest.yml b/.github/workflows/awstest.yml
@@ -12,11 +12,12 @@ jobs:
     steps:
       # Launch workflow using Tower CLI tool action
       - name: Launch build-reference workflow via tower
-        uses: seqeralabs/action-tower-launch@v1
+        uses: seqeralabs/action-tower-launch@v2
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
           access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
+          revision: ${{ github.sha }}
           workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/marsseq/work-${{ github.sha }}
           parameters: |
             {
@@ -43,8 +44,11 @@ jobs:
               "genomes_base": "s3://${{ secrets.AWS_S3_BUCKET }}/marsseq/results-test-${{ github.sha }}/references",
               "velocity": true
             }
-          profiles: test,aws_tower
+          profiles: test
+
       - uses: actions/upload-artifact@v3
         with:
           name: Tower debug log file
-          path: tower_action_*.log
+          path: |
+            tower_action_*.log
+            tower_action_*.json
diff --git a/.gitpod.yml b/.gitpod.yml
@@ -1,4 +1,9 @@
 image: nfcore/gitpod:latest
+tasks:
+  - name: Update Nextflow and setup pre-commit
+    command: |
+      pre-commit install --install-hooks
+      nextflow self-update
 
 vscode:
   extensions: # based on nf-core.nf-core-extensionpack

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,13 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## v1.0.2 - [11-07-2023]
+
+### `Dependencies`
+
+- [#6](https://github.com/nf-core/marsseq/pull/6) sync with template 2.8
+- [#4](https://github.com/nf-core/marsseq/issues/4) - Fix AWS testing with s3 bucket
+
 ## v1.0.1 - [26-06-2023]
 
 ### `Fixed`

diff --git a/CITATIONS.md b/CITATIONS.md
@@ -29,6 +29,7 @@
   > Benjamin Kaminow, Dinar Yunusov, Alexander Dobin; bioRxiv 2021.05.05.442755; doi:
 
 - [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
+
   > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.
 
 ## Software packaging/containerisation tools
@@ -47,5 +48,8 @@
 
 - [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241)
 
+  > Merkel, D. (2014). Docker: lightweight linux containers for consistent development and deployment. Linux Journal, 2014(239), 2. doi: 10.5555/2600239.2600241.
+
 - [Singularity](https://pubmed.ncbi.nlm.nih.gov/28494014/)
+
   > Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLoS One. 2017 May 11;12(5):e0177459. doi: 10.1371/journal.pone.0177459. eCollection 2017. PubMed PMID: 28494014; PubMed Central PMCID: PMC5426675.
diff --git a/README.md b/README.md
@@ -36,7 +36,7 @@ Next, you have to generate genome references to incorporate ERCC spike-ins. Refe
 ```bash
 nextflow run nf-core/marsseq \
   -profile <docker/singularity/.../institute> \
-  --genome <mm10,mm9,GRCh38> \
+  --genome <mm10,mm9,GRCh38_v43> \
   --build_references \
   --input samplsheet.csv \
   --outdir <OUTDIR>
@@ -47,7 +47,7 @@ Now, you can run the pipeline using:
 ```bash
 nextflow run nf-core/marsseq \
   -profile <docker/singularity/.../institute> \
-  --genome <mm10,mm9,GRCh38> \
+  --genome <mm10,mm9,GRCh38_v43> \
   --input samplesheet.csv \
   --outdir <OUTDIR>
 ```
@@ -57,11 +57,11 @@ nextflow run nf-core/marsseq \
 > provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_;
 > see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).
 
-For more details, please refer to the [usage documentation](https://nf-co.re/marsseq/usage) and the [parameter documentation](https://nf-co.re/marsseq/parameters).
+For more details and further functionality, please refer to the [usage documentation](https://nf-co.re/marsseq/usage) and the [parameter documentation](https://nf-co.re/marsseq/parameters).
 
 ## Pipeline output
 
-To see the the results of a test run with a full size dataset refer to the [results](https://nf-co.re/marsseq/results) tab on the nf-core website pipeline page.
+To see the results of an example test run with a full size dataset refer to the [results](https://nf-co.re/marsseq/results) tab on the nf-core website pipeline page.
 For more details about the output files and reports, please refer to the
 [output documentation](https://nf-co.re/marsseq/output).
 

diff --git a/assets/methods_description_template.yml b/assets/methods_description_template.yml
@@ -5,13 +5,17 @@ section_href: "https://github.com/nf-core/marsseq"
 plot_type: "html"
 data: |
   <h4>Methods</h4>
-  <p>Data was processed using nf-core/marsseq v${workflow.manifest.version} ${doi_text} of the nf-core collection of workflows (<a href="https://doi.org/10.1038/s41587-020-0439-x">Ewels <em>et al.</em>, 2020</a>).</p>
+  <p>Data was processed using nf-core/marsseq v${workflow.manifest.version} ${doi_text} of the nf-core collection of workflows (<a href="https://doi.org/10.1038/s41587-020-0439-x">Ewels <em>et al.</em>, 2020</a>), utilising reproducible software environments from the Bioconda (<a href="https://doi.org/10.1038/s41592-018-0046-7">Grüning <em>et al.</em>, 2018</a>) and Biocontainers (<a href="https://doi.org/10.1093/bioinformatics/btx192">da Veiga Leprevost <em>et al.</em>, 2017</a>) projects.</p>
   <p>The pipeline was executed with Nextflow v${workflow.nextflow.version} (<a href="https://doi.org/10.1038/nbt.3820">Di Tommaso <em>et al.</em>, 2017</a>) with the following command:</p>
   <pre><code>${workflow.commandLine}</code></pre>
+  <p>${tool_citations}</p>
   <h4>References</h4>
   <ul>
-    <li>Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017). Nextflow enables reproducible computational workflows. Nature Biotechnology, 35(4), 316-319. <a href="https://doi.org/10.1038/nbt.3820">https://doi.org/10.1038/nbt.3820</a></li>
-    <li>Ewels, P. A., Peltzer, A., Fillinger, S., Patel, H., Alneberg, J., Wilm, A., Garcia, M. U., Di Tommaso, P., & Nahnsen, S. (2020). The nf-core framework for community-curated bioinformatics pipelines. Nature Biotechnology, 38(3), 276-278. <a href="https://doi.org/10.1038/s41587-020-0439-x">https://doi.org/10.1038/s41587-020-0439-x</a></li>
+    <li>Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017). Nextflow enables reproducible computational workflows. Nature Biotechnology, 35(4), 316-319. doi: <a href="https://doi.org/10.1038/nbt.3820">10.1038/nbt.3820</a></li>
+    <li>Ewels, P. A., Peltzer, A., Fillinger, S., Patel, H., Alneberg, J., Wilm, A., Garcia, M. U., Di Tommaso, P., & Nahnsen, S. (2020). The nf-core framework for community-curated bioinformatics pipelines. Nature Biotechnology, 38(3), 276-278. doi: <a href="https://doi.org/10.1038/s41587-020-0439-x">10.1038/s41587-020-0439-x</a></li>
+    <li>Grüning, B., Dale, R., Sjödin, A., Chapman, B. A., Rowe, J., Tomkins-Tinch, C. H., Valieris, R., Köster, J., & Bioconda Team. (2018). Bioconda: sustainable and comprehensive software distribution for the life sciences. Nature Methods, 15(7), 475–476. doi: <a href="https://doi.org/10.1038/s41592-018-0046-7">10.1038/s41592-018-0046-7</a></li>
+    <li>da Veiga Leprevost, F., Grüning, B. A., Alves Aflitos, S., Röst, H. L., Uszkoreit, J., Barsnes, H., Vaudel, M., Moreno, P., Gatto, L., Weber, J., Bai, M., Jimenez, R. C., Sachsenberg, T., Pfeuffer, J., Vera Alvarez, R., Griss, J., Nesvizhskii, A. I., & Perez-Riverol, Y. (2017). BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics (Oxford, England), 33(16), 2580–2582. doi: <a href="https://doi.org/10.1093/bioinformatics/btx192">10.1093/bioinformatics/btx192</a></li>
+    ${tool_bibliography}
   </ul>
   <div class="alert alert-info">
     <h5>Notes:</h5>

diff --git a/assets/multiqc_config.yml b/assets/multiqc_config.yml
@@ -1,7 +1,7 @@
 report_comment: >
-  This report has been generated by the <a href="https://github.com/nf-core/marsseq" target="_blank">nf-core/marsseq</a>
+  This report has been generated by the <a href="https://github.com/nf-core/marsseq/1.0.2" target="_blank">nf-core/marsseq</a>
   analysis pipeline. For information about how to interpret these results, please see the
-  <a href="https://nf-co.re/marsseq" target="_blank">documentation</a>.
+  <a href="https://nf-co.re/marsseq/1.0.2/output" target="_blank">documentation</a>.
 report_section_order:
   "nf-core-marsseq-methods-description":
     order: -1000

diff --git a/assets/nf-core-marsseq_logo_light.png b/assets/nf-core-marsseq_logo_light.png
diff --git a/assets/schema_input.json b/assets/schema_input.json
@@ -7,10 +7,10 @@
     "items": {
         "type": "object",
         "properties": {
-            "sample": {
+            "batch": {
                 "type": "string",
                 "pattern": "^\\S+$",
-                "errorMessage": "Sample name must be provided and cannot contain spaces"
+                "errorMessage": "Batch name must be provided and cannot contain spaces"
             },
             "fastq_1": {
                 "type": "string",
@@ -29,8 +29,23 @@
                         "maxLength": 0
                     }
                 ]
+            },
+            "amp_batches": {
+                "type": "string",
+                "pattern": "^\\S+.xlsx$",
+                "errorMessage": "Amplification batches has to be in xlsx format"
+            },
+            "seq_batches": {
+                "type": "string",
+                "pattern": "^\\S+.xlsx$",
+                "errorMessage": "Sequencing batches has to be in xlsx format"
+            },
+            "well_cells": {
+                "type": "string",
+                "pattern": "^\\S+.xlsx$",
+                "errorMessage": "Well cells has to be in xlsx format"
             }
         },
-        "required": ["sample", "fastq_1"]
+        "required": ["batch", "fastq_1", "fastq_2", "amp_batches", "seq_batches", "well_cells"]
     }
 }
diff --git a/assets/slackreport.json b/assets/slackreport.json
@@ -3,7 +3,7 @@
         {
             "fallback": "Plain-text summary of the attachment.",
             "color": "<% if (success) { %>good<% } else { %>danger<%} %>",
-            "author_name": "sanger-tol/readmapping v${version} - ${runName}",
+            "author_name": "nf-core/marsseq v${version} - ${runName}",
             "author_icon": "https://www.nextflow.io/docs/latest/_static/favicon.ico",
             "text": "<% if (success) { %>Pipeline completed successfully!<% } else { %>Pipeline completed with errors<% } %>",
             "fields": [

diff --git a/conf/genomes.config b/conf/genomes.config
@@ -13,12 +13,12 @@ params {
     // GENCODE references (https://www.gencodegenes.org/)
     genomes {
         'mm10_v32' {
-            bowtie2     = "${params.genomes_base}/mm10/bowtie2/"
-            fasta       = "${params.genomes_base}/mm10/GRCm39.fa"
+            bowtie2     = "${params.genomes_base}/mm10_v32/bowtie2/"
+            fasta       = "${params.genomes_base}/mm10_v32/GRCm39.fa"
             fasta_url   = "http://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M32/GRCm39.primary_assembly.genome.fa.gz"
-            gtf         = "${params.genomes_base}/mm10/GRCm39.gtf"
+            gtf         = "${params.genomes_base}/mm10_v32/GRCm39.gtf"
             gtf_url     = "http://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M32/gencode.vM32.annotation.gtf.gz"
-            star        = "${params.genomes_base}/mm10/star/"
+            star        = "${params.genomes_base}/mm10_v32/star/"
         }
         'mm10' {
             bowtie2     = "${params.genomes_base}/mm10/bowtie2/"
@@ -36,11 +36,13 @@ params {
             gtf_url     = "https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M1/gencode.vM1.annotation.gtf.gz"
             star        = "${params.genomes_base}/mm9/star/"
         }
-        'GRCh38' {
-            fasta       = "${params.genomes_base}/mm10/GRCh38.fa"
-            gtf         = "${params.genomes_base}/mm10/GRCh38.gtf"
-            bowtie2     = "${params.genomes_base}/mm10/Bowtie2Index/"
-            star        = "${params.genomes_base}/mm10/STARIndex/"
+        'GRCh38_v43' {
+            bowtie2     = "${params.genomes_base}/GRCh38_v43/bowtie2/"
+            fasta       = "${params.genomes_base}/GRCh38_v43/GRCh38.fa"
+            fasta_url   = "http://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_43/GRCh38.primary_assembly.genome.fa.gz"
+            gtf         = "${params.genomes_base}/GRCh38_v43/GRCh38.gtf"
+            gtf_url     = "http://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_43/gencode.v43.annotation.gtf.gz"
+            star        = "${params.genomes_base}/GRCh38_v43/star/"
         }
     }
 }
diff --git a/docs/usage.md b/docs/usage.md
@@ -59,7 +59,7 @@ An [example samplesheet](../assets/samplesheet.csv) has been provided with the p
 The typical command for running the pipeline is as follows:
 
 ```bash
-nextflow run nf-core/marsseq --input samplesheet.csv --outdir <OUTDIR> --genome GRCh37 -profile docker
+nextflow run nf-core/marsseq --input ./samplesheet.csv --outdir ./results --genome GRCh37 -profile docker
 ```
 
 This will launch the pipeline with the `docker` configuration profile. See below for more information about profiles.
@@ -78,7 +78,8 @@ If you wish to repeatedly use the same parameters for multiple runs, rather than
 Pipeline settings can be provided in a `yaml` or `json` file via `-params-file <file>`.
 
 > ⚠️ Do not use `-c <file>` to specify parameters as this will result in errors. Custom config files specified with `-c` must only be used for [tuning process resource specifications](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources), other infrastructural tweaks (such as output directories), or module arguments (args).
-> The above pipeline run specified with a params file in yaml format:
+
+The above pipeline run specified with a params file in yaml format:
 
 ```bash
 nextflow run nf-core/marsseq -profile docker -params-file params.yaml
@@ -90,7 +91,6 @@ with `params.yaml` containing:
 input: './samplesheet.csv'
 outdir: './results/'
 genome: 'GRCh37'
-input: 'data'
 <...>
 ```