1.10.1

This release was a big internal refactor and didn't feel like enough visible changes were present to release it as 1.11, so it's named 1.10.1 instead

Internal

• some of the simpler file validations moved to the command-line parsing part of harpy #159
• [hpc] has much less redundant code

New Features

• All workflows with an --extra-params option now have some program-specific argument validation #158
• --snakemake now has validations
• --hpc now has validations
• [align ema] made read fragment density optimization off by default and is now exposed as a command-line argument to toggle on

Other changes

• [hpc] now checks if the executor plugin is installed and only prints the notice if it isn't
• [stitchparams] and [popgroup] have slightly nicer printing

Full Changelog: 1.10...1.10.1

1.10

New Features

• assembly and metassembly workflows
• --contigs option for align ... sv ... and phase workflows
• calculations for molecular coverage too
• non-singleton metrics added to alignment reports

Internal

• remove pandas dependency b/c no longer using Paramspace()
  • impute parameter file now gets transcribed into the config.yaml
• new and better validations
• some validations have progressbar and are parallelized
  • progressbar respects --quiet
• config.yaml files are written using the yaml stdlib for consistency
• workflow summaries have more robust logic

Breaking changes

• parameter file for impute has a new name column that will name relevant outputs for a given parameter set
  • this affects the output directories now, which are named according to this name value
• config.yaml file restructured a bit, mainly some params have better snake-cased names
  • skipreports is now skip under a new reports hierarchy

Pull Requests

• Add Metassembly by @pdimens in #144
• add assembly by @pdimens in #146
• Molecule coverage by @pdimens in #149
• Hotfixes by @pdimens in #150
• 151 provide harpy a list of primary autosomessex chroms for plottingreports for chromosome level assemblies by @pdimens in #152
• add quast reports to assembly by @pdimens in #155
• add singleton/non stats by @pdimens in #156

Full Changelog: 1.9...1.10

1.9

Breaking changes

• --prefix for variant simulations is now just sim to reduce redundancy of defaults
• --heterozygosity for variant simulations now outputs a diploid genome by default
  • previous behavior can be achieved with --only-vcf option
  • see tutorial here

1.8.2

Changes

• ploidy bounds on snp workflows
• snp docstring improved to guide users towards freebayes if polyploid
• preflight fastq parsing speedup

PRs and issues

• improve snp text by @pdimens in #139
• #138

Full Changelog: 1.8.1...1.8.2

1.8.1

fixed

• [simulate snpindel] random indel count was propogated in the workflow as the snp count
• [simulate variants...] --heterozygosity moved to configs in the docstring

Full Changelog: 1.8.0...1.8.1

1.8.0

Breaking Changes

• none

Fixes

• makewindows bug fixed that caused it to fail if the contig was smaller than the window
• fixed a bug in the vcf parsing that caused false positives for contigs without biallelic loci #133
• Harpy and progress bar hangs when there was nothing for the workflow to do

Changes

• snakemake logs in /logs/snakemake are compressed after seeing how big they can get #132
• original snakemake logs deleted upon completion of workflow
• even more robust progress bar
• empty log files and log directories are removed after harpy finishes #136
• snp normalization is back for harpy snp (and it works better than the original)

added

• --threads to harpy resume #134
• bam indexing restored to being an iterative job with wildcards

internal

• removed deprecated lines of code
• added initial parts for metassembly, but it's hidden and nowhere near ready
• snp and sv conda environments consolidated into variants.yaml

Closed PR's

• log changes and fixes by @pdimens in #135

Full Changelog: 1.7.0...1.8.0

1.7.0

What's Changed

• fix reports by @pdimens in #128
• add and style progress spinners by @pdimens in #129
• tidy up impute by @pdimens in #130

Full Changelog: 1.6.1...1.7.0

1.6.1

fixed

• [align] invalid barcodes are properly outputted with bx_stats.py
• [align] individual reports properly calculate valid/invalid (also provide some more info)
• "downloading dependencies" text no long has a strange flutter

improved

• snakefile errors are printed to the terminal instead of being vague
• onstart text now colored, aligned, and follow consistent formatting
• [simulations] [sv] onstart text clearer
• [align strobe] allows up to 2 supplmentary reads in the output
• all errors now have succinct names
• fasta-based errors have improved phrasing

Full Changelog: 1.6...1.6.1

1.6

It's been a challenge to get here, but Harpy finally has progress bars!!! Yes, the days of seeing a wall of Snakemake output are gone. The standard snakemake output will still be in the recently-added (1.5) log file in output/logs/snakemake/. You will now see dynamic progress bars for each step. As a result of this change, when things go wrong, a very succinct error is shown in the console.

New

• progress bars instead of snakemake output

Changed behavior

• --quiet now prevents the progress bars (rather than adding --quiet all to the snakemake call)
• onstart text also lists the snakemake logfile
• keyboard interrupts (CTRL+C) exit harpy gracefully now

Internals

• messages have been removed from the snakemake rules, so their output can be used by the new progress bars
• several rules have been renamed (progress bar reasons)
• onstart, onsuccess, onerror from snakefiles was moved into the new launch_snakemake() function (progress bar reasons)
• snakemake log file creation also moved out of rules
• overall, the snakemake rules are "cleaner" now

Pull Requests

• add a progress bar by @pdimens in #125
• advanced progressbar by @pdimens in #126

Full Changelog: 1.5...1.6

New progress bars

New workflow errors

New keyboard interrupt

1.5

Added

• --keep-unmapped options to align workflows
• --min-quality added to sv naibr

Changes (not breaking)

• STITCH aggregate report is now a storyboard
• snakemake calls now include --show-failed-logs
• snakemake log is now named workflow.runX.DATE.snakelog
  • workflow is the kind of harpy command you invoked (e.g. sv_naibr)
  • runX where X is the attempt number for that output directory
  • DATE given in MONTH_DAY_YEAR format

Changes (breaking)

• --quality-filter is changed to --min-quality to be consistent across workflows
• --molecule-distance has a short-name of -d now (used to be -m)
• some other argument shortnames were changed to better reflect their natural usage

Bugs fixed

• snp workflows had conflicting -x shortnames for ploidy and extra params
• alignment report correctly reports intervals in the colored boxes on the second page
• some reports have updated text

Full Changelog: 1.4.2...1.5