Merge pull request #370 from drpatelh/fixes

Fix #368
nf-core · Mar 23, 2023 · 5daf5a8 · 5daf5a8
2 parents 6b23e8c + e5d5293
commit 5daf5a8
Show file tree

Hide file tree

Showing 17 changed files with 85 additions and 58 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,6 +11,8 @@ Special thanks to the following for their code contributions to the release:
 
 - [Friederike Hanssen](https://github.com/FriederikeHanssen)
 - [Hugo Tavares](https://github.com/tavareshugo)
+- [James Fellows Yates](https://github.com/jfy133)
+- [Jessica Wu](https://github.com/wutron)
 - [Matthew Wells](https://github.com/mattheww95)
 - [Maxime Garcia](https://github.com/maxulysse)
 - [Phil Ewels](https://github.com/ewels)
@@ -30,6 +32,7 @@ Thank you to everyone else that has contributed by reporting bugs, enhancements
 - [[#348](https://github.com/nf-core/viralrecon/issues/348)] - Document full parameters of iVar consensus
 - [[#349](https://github.com/nf-core/viralrecon/issues/349)] - ERROR in Script plasmidID
 - [[#356](https://github.com/nf-core/viralrecon/issues/356)] - Add NEB SARS-CoV-2 primers
+- [[#368](https://github.com/nf-core/viralrecon/issues/368)] - Incorrect depth from ivar variants reported in variants long table
 - Updated pipeline template to [nf-core/tools 2.7.2](https://github.com/nf-core/tools/releases/tag/2.7.2)
 - Add `tower.yml` for Report rendering in Nextflow Tower
 - Use `--skip_plasmidid` by default

diff --git a/README.md b/README.md
@@ -102,6 +102,8 @@ A number of improvements were made to the pipeline recently, mainly with regard
 
 4. Start running your own analysis!
 
+   > - Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those provided by the `-c` Nextflow option can be used to provide any configuration except for parameters; see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).
+
    - Typical command for Illumina shotgun analysis:
 
      ```bash

diff --git a/bin/ivar_variants_to_vcf.py b/bin/ivar_variants_to_vcf.py
@@ -126,7 +126,7 @@ def parse_ivar_line(line):
     QUAL = "."
 
     ## Determine FILTER field
-    INFO = f"DP={line[11]}"
+    INFO = f"DP={int(float(line[11]))}"
     pass_test = line[13]
 
     return (

diff --git a/conf/modules_illumina.config b/conf/modules_illumina.config
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Config file for defining DSL2 per module options and publishing paths
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Available keys to override module options:
         ext.args   = Additional arguments appended to command in module.
         ext.args2  = Second set of arguments appended to command in module (multi-tool modules).

diff --git a/conf/modules_nanopore.config b/conf/modules_nanopore.config
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Config file for defining DSL2 per module options and publishing paths
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Available keys to override module options:
         ext.args   = Additional arguments appended to command in module.
         ext.args2  = Second set of arguments appended to command in module (multi-tool modules).

diff --git a/conf/test_full_nanopore.config b/conf/test_full_nanopore.config
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Nextflow config file for running full-size tests
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Defines input files and everything required to run a full size pipeline test.
 
     Use as follows:

diff --git a/conf/test_full_sispa.config b/conf/test_full_sispa.config
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Nextflow config file for running full-size tests
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Defines input files and everything required to run a full size pipeline test.
 
     Use as follows:

diff --git a/conf/test_nanopore.config b/conf/test_nanopore.config
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Nextflow config file for running minimal tests
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Defines input files and everything required to run a fast and simple pipeline test.
 
     Use as follows:

diff --git a/conf/test_sispa.config b/conf/test_sispa.config
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Nextflow config file for running minimal tests
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Defines input files and everything required to run a fast and simple pipeline test.
 
     Use as follows:

diff --git a/docs/usage.md b/docs/usage.md
@@ -4,6 +4,10 @@
 
 > _Documentation of pipeline parameters is generated automatically from the pipeline schema and can no longer be found in markdown files._
 
+## Pipeline parameters
+
+Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those provided by the `-c` Nextflow option can be used to provide any configuration except for parameters; see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).
+
 ## Samplesheet format
 
 ### Illumina

diff --git a/lib/NfcoreTemplate.groovy b/lib/NfcoreTemplate.groovy
@@ -32,6 +32,21 @@ class NfcoreTemplate {
         }
     }
 
+    //
+    //  Warn if using custom configs to provide pipeline parameters
+    //
+    public static void warnParamsProvidedInConfig(workflow, log) {
+        if (workflow.configFiles.size() > 1) {
+            log.warn "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
+                "  Multiple config files detected!\n" +
+                "  Please provide pipeline parameters via the CLI or Nextflow '-params-file' option.\n" +
+                "  Custom config files including those provided by the '-c' Nextflow option can be\n" +
+                "  used to provide any configuration except for parameters.\n\n" +
+                "  Docs: https://nf-co.re/usage/configuration#custom-configuration-files\n" +
+                "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+        }
+    }
+
     //
     // Generate version string
     //

diff --git a/lib/Utils.groovy b/lib/Utils.groovy
@@ -33,15 +33,15 @@ class Utils {
         }
 
         if (channels_missing | channel_priority_violation) {
-            log.warn "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
+            log.warn "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                 "  There is a problem with your Conda configuration!\n\n" +
                 "  You will need to set-up the conda-forge and bioconda channels correctly.\n" +
                 "  Please refer to https://bioconda.github.io/\n" +
                 "  The observed channel order is \n" +
                 "  ${channels}\n" +
                 "  but the following channel order is required:\n" +
                 "  ${required_channels_in_order}\n" +
-                "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+                "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
         }
     }
 }
diff --git a/lib/WorkflowCommons.groovy b/lib/WorkflowCommons.groovy
@@ -9,11 +9,11 @@ class WorkflowCommons {
     //
     private static void genomeExistsError(params, log) {
         if (params.genomes && params.genome && !params.genomes.containsKey(params.genome)) {
-            log.error "=============================================================================\n" +
+            log.error "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                 "  Genome '${params.genome}' not found in any config files provided to the pipeline.\n" +
                 "  Currently, the available genome keys are:\n" +
                 "  ${params.genomes.keySet().join(", ")}\n" +
-                "============================================================================="
+                "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
             System.exit(1)
         }
     }
@@ -74,14 +74,14 @@ class WorkflowCommons {
             )
         }
         if (total != (left + right)) {
-            log.warn "=============================================================================\n" +
+            log.warn "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                 "  Please check the name field (column 4) in the file supplied via --primer_bed.\n\n" +
                 "  All of the values in that column do not end with those supplied by:\n" +
                 "      --primer_left_suffix : $primer_left_suffix\n" +
                 "      --primer_right_suffix: $primer_right_suffix\n\n" +
                 "  This information is required to collapse the primer intervals into amplicons\n" +
                 "  for the coverage plots generated by the pipeline.\n" +
-                "==================================================================================="
+                "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
         }
     }
 
@@ -121,13 +121,13 @@ class WorkflowCommons {
         def intersect = bed_contigs.intersect(fai_contigs)
         if (intersect.size() != bed_contigs.size()) {
             def diff = bed_contigs.minus(intersect).sort()
-            log.error "=============================================================================\n" +
+            log.error "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                 "  Contigs in primer BED file do not match those in the reference genome:\n\n" +
                 "  ${diff.join('\n  ')}\n\n" +
                 "  Please check:\n" +
                 "    - Primer BED file supplied with --primer_bed\n" +
                 "    - Genome FASTA file supplied with --fasta\n" +
-                "============================================================================="
+                "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
             System.exit(1)
         }
     }

diff --git a/lib/WorkflowIllumina.groovy b/lib/WorkflowIllumina.groovy
@@ -75,11 +75,11 @@ class WorkflowIllumina {
                 if (line.contains('>')) {
                     count++
                     if (count > 1) {
-                        log.warn "=============================================================================\n" +
+                        log.warn "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                             "  This pipeline does not officially support multi-fasta genome files!\n\n" +
                             "  The parameters and processes are tailored for viral genome analysis.\n" +
                             "  Please amend the '--fasta' parameter.\n" +
-                            "==================================================================================="
+                            "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
                         break
                     }
                 }
@@ -118,12 +118,12 @@ class WorkflowIllumina {
                 if (name.contains(name_prefix)) {
                     count++
                     if (count > 1) {
-                        log.warn "=============================================================================\n" +
+                        log.warn "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                             "  Found '${name_prefix}' in the name field of the primer BED file!\n" +
                             "  This suggests that you have used the SWIFT/SNAP protocol to prep your samples.\n" +
                             "  If so, please set '--ivar_trim_offset 5' as suggested in the issue below:\n" +
                             "  https://github.com/nf-core/viralrecon/issues/170\n" +
-                            "==================================================================================="
+                            "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
                         break
                     }
                 }

diff --git a/lib/WorkflowMain.groovy b/lib/WorkflowMain.groovy
@@ -62,6 +62,9 @@ class WorkflowMain {
         // Print parameter summary log to screen
         log.info paramsSummaryLog(workflow, params, log)
 
+        // Warn about using custom configs to provide pipeline parameters
+        NfcoreTemplate.warnParamsProvidedInConfig(workflow, log)
+
         // Validate workflow parameters via the JSON schema
         if (params.validate_params) {
             NfcoreSchema.validateParameters(workflow, params, log)
@@ -105,7 +108,7 @@ class WorkflowMain {
                             "   - https://github.com/nf-core/configs/blob/master/conf/pipeline/viralrecon/genomes.config\n\n" +
                             " If you would still like to blame us please come and find us on nf-core Slack:\n" +
                             "   - https://nf-co.re/viralrecon#contributions-and-support\n" +
-                            "============================================================================="
+                            "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
         if (params.genomes && params.genome && params.genomes.containsKey(params.genome)) {
             def genome_map = params.genomes[ params.genome ]
             if (primer_set) {
@@ -117,7 +120,7 @@ class WorkflowMain {
                         if (genome_map.containsKey(primer_set_version)) {
                             genome_map = genome_map[ primer_set_version ]
                         } else {
-                            log.error "=============================================================================\n" +
+                            log.error "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                                 " --primer_set_version '${primer_set_version}' not found!\n\n" +
                                 " Currently, the available primer set version keys are: ${genome_map.keySet().join(", ")}\n\n" +
                                 " Please check:\n" +
@@ -128,7 +131,7 @@ class WorkflowMain {
                             System.exit(1)
                         }
                     } else {
-                        log.error "=============================================================================\n" +
+                        log.error "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                             " --primer_set '${primer_set}' not found!\n\n" +
                             " Currently, the available primer set keys are: ${genome_map.keySet().join(", ")}\n\n" +
                             " Please check:\n" +
@@ -138,7 +141,7 @@ class WorkflowMain {
                         System.exit(1)
                     }
                 } else {
-                    log.error "=============================================================================\n" +
+                    log.error "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n" +
                         " Genome '${params.genome}' does not contain any primer sets!\n\n" +
                         " Please check:\n" +
                         "   - The value provided to --genome (currently '${params.genome}')\n" +

diff --git a/workflows/illumina.nf b/workflows/illumina.nf
@@ -1,7 +1,7 @@
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     VALIDATE INPUTS
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
 def valid_params = [
@@ -34,9 +34,9 @@ def variant_caller = params.variant_caller
 if (!variant_caller) { variant_caller = params.protocol == 'amplicon' ? 'ivar' : 'bcftools' }
 
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     CONFIG FILES
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
 ch_multiqc_config        = file("$projectDir/assets/multiqc_config_illumina.yml", checkIfExists: true)
@@ -47,9 +47,9 @@ ch_blast_outfmt6_header     = file("$projectDir/assets/headers/blast_outfmt6_hea
 ch_ivar_variants_header_mqc = file("$projectDir/assets/headers/ivar_variants_header_mqc.txt", checkIfExists: true)
 
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     IMPORT LOCAL MODULES/SUBWORKFLOWS
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
 //
@@ -77,9 +77,9 @@ include { BAM_TRIM_PRIMERS_IVAR   } from '../subworkflows/local/bam_trim_primers
 include { FASTQ_TRIM_FASTP_FASTQC } from '../subworkflows/local/fastq_trim_fastp_fastqc'
 
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     IMPORT NF-CORE MODULES/SUBWORKFLOWS
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
 //
@@ -97,12 +97,12 @@ include { MOSDEPTH as MOSDEPTH_AMPLICON } from '../modules/nf-core/mosdepth/main
 // SUBWORKFLOW: Consisting entirely of nf-core/modules
 //
 include { FASTQ_ALIGN_BOWTIE2       } from '../subworkflows/nf-core/fastq_align_bowtie2/main'
-include { BAM_MARKDUPLICATES_PICARD } from '../subworkflows/nf-core/bam_markduplicates_picard'
+include { BAM_MARKDUPLICATES_PICARD } from '../subworkflows/nf-core/bam_markduplicates_picard/main'
 
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     RUN MAIN WORKFLOW
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
 // Info required for completion email and summary
@@ -645,9 +645,9 @@ workflow ILLUMINA {
 }
 
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     COMPLETION EMAIL AND SUMMARY
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
 workflow.onComplete {
@@ -658,7 +658,7 @@ workflow.onComplete {
 }
 
 /*
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     THE END
-========================================================================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */