forked from samtools/bcftools
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathNEWS
435 lines (287 loc) · 16 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
## Release a.b
* `csq`
- significant reduction of memory usage in the local -l mode for VCFs
with thousands of samples and 20% reduction in the non-local
haplotype-aware mode.
- fixes a small memory leak and formatting issue in FORMAT/BCSQ at
sites with many consequences
* `+dosage`: fix some serious bugs
* VariantKey support
## Release 1.9 (18th July 2018)
* `annotate`
- REF and ALT columns can be now transferred from the annotation file.
- fixed bug when setting vector_end values.
* `consensus`
- new -M option to control output at missing genotypes
- variants immediately following insersions should not be skipped. Note
however, that the current fix requires normalized VCF and may still
falsely skip variants adjacent to multiallelic indels.
- bug fixed in -H selection handling
* `convert`
- the --tsv2vcf option now makes the missing genotypes diploid, "./." instead of "."
- the behavior of -i/-e with --gvcf2vcf changed. Previously only sites with
FILTER set to "PASS" or "." were expanded and the -i/-e options dropped
sites completely. The new behavior is to let the -i/-e options control which
records will be expanded. In order to drop records completely, one can stream
through "bcftools view" first.
* `csq`
- since the real consequence of start/splice events are not known, the aminoacid
positions at subsequent variants should stay unchanged
- add `--force` option to skip malformatted transcripts in GFFs with out-of-phase
CDS exons.
* `+dosage`: output all alleles and all their dosages at multiallelic sites
* `+fixref`: fix serious bug in -m top conversion
* `-i/-e` filtering expressions:
- add two-tailed binomial test
- add functions N_PASS() and F_PASS()
- add support for lists of samples in filtering expressions, with many
samples it was impractical to list them all on the command line. Samples
can be now in a file as, e.g., GT[@samples.txt]="het"
- allow multiple perl functions in the expressions and some bug fixes
- fix a parsing problem, '@' was not removed from '@filename' expressions
* `mpileup`: fixed bug where, if samples were renamed using the `-G` (`--read-groups`)
option, some samples could be omitted from the output file.
* `norm`: update INFO/END when normalizing indels
* `+split`: new -S option to subset samples and to use custom file names instead of the defaults
* `+smpl-stats`: new plugin
* `+trio-stats`: new plugin
* Fixed build problems with non-functional configure script produced on some platforms
## Release 1.8 (April 2018)
* `-i, -e` filtering: Support for custom perl scripts
* `+contrast`: New plugin to annotate genotype differences between groups of samples
* `+fixploidy`: New options for simpler ploidy usage
* `+setGT`: Target genotypes can be set to phased by giving `--new-gt p`
* `run-roh.pl`: Allow to pass options directly to `bcftools roh`
* Number of bug fixes
## Release 1.7 (February 2018)
* `-i, -e` filtering: Major revamp, improved filtering by FORMAT fields
and missing values. New GT=ref,alt,mis etc keywords, check the documenation
for details.
* `query`: Only matching expression are printed when both the -f and -i/-e
expressions contain genotype fields. Note that this changes the original
behavior. Previously all samples were output when one matching sample was
found. This functionality can be achieved by pre-filtering with view and then
streaming to query. Compare
bcftools query -f'[%CHROM:%POS %SAMPLE %GT\n]' -i'GT="alt"' file.bcf
and
bcftools view -i'GT="alt"' file.bcf -Ou | bcftools query -f'[%CHROM:%POS %SAMPLE %GT\n]'
* `annotate`: New -k, --keep-sites option
* `consensus`: Fix --iupac-codes output
* `csq`: Homs always considered phased and other fixes
* `norm`: Make `-c none` work and remove `query -c`
* `roh`: Fix errors in the RG output
* `stats`: Allow IUPAC ambiguity codes in the reference file; report the number of missing genotypes
* `+fill-tags`: Add ExcHet annotation
* `+setGt`: Fix bug in binom.test calculation, previously it worked only for nAlt<nRef!
* `+split`: New plugin to split a multi-sample file into single-sample files in one go
* Improve python3 compatibility in plotting scripts
## Release 1.6 (September 2017)
* New `sort` command.
* New options added to the `consensus` command. Note that the `-i, --iupac`
option has been renamed to `-I, --iupac`, in favor of the standard
`-i, --include`.
* Filtering expressions (`-i/-e`): support for `GT=<type>` expressions and
for lists and ranges (#639) - see the man page for details.
* `csq`: relax some GFF3 parsing restrictions to enable using Ensembl
GFF3 files for plants (#667)
* `stats`: add further documentation to output stats files (#316) and
include haploid counts in per-sample output (#671).
* `plot-vcfstats`: further fixes for Python3 (@nsoranzo, #645, #666).
* `query` bugfix (#632)
* `+setGT` plugin: new option to set genotypes based on a two-tailed binomial
distribution test. Also, allow combining `-i/-e` with `-t q`.
* `mpileup`: fix typo (#636)
* `convert --gvcf2vcf` bugfix (#641)
* `+mendelian`: recognize some mendelian inconsistencies that were
being missed (@oronnavon, #660), also add support for multiallelic
sites and sex chromosomes.
## Release 1.5 (June 2017)
* Added autoconf support to bcftools. See `INSTALL` for more details.
* `norm`: Make norm case insensitive (#601). Trim the reference allele (#602).
* `mpileup`: fix for misreported indel depths for reads containing adjacent
indels (3c1205c1).
* `plot-vcfstats`: Open stats file in text mode, not binary (#618).
* `fixref` plugin: Allow multiallelic sites in the `-i, --use-id reference`.
Also flip genotypes, not just REF/ALT!
* `merge`: fix gVCF merge bug when last record on a chromosome opened a
gVCF block (#616)
* New options added to the ROH plotting script.
* `consensus`: Properly flush chain info (#606, thanks to @krooijers).
* New `+prune` plugin for pruning sites by LD (R2) or maximum number of
records within a window.
* New N_MISSING, F_MISSING (number and fraction misssing) filtering
expressions.
* Fix HMM initilization in `roh` when snapshots are used in multiple
chromosome VCF.
* Fix buffer overflow (#607) in `filter`.
## Release 1.4.1 (8 May 2017)
* `roh`: Fixed malfunctioning options `-m, --genetic-map` and `-M, --rec-rate`,
and newly allowed their combination. Added a convenience wrapper `misc/run-roh.pl`
and an interactive script for visualizing the calls `misc/plot-roh.py`.
* `csq`: More control over warning messages (#585).
* Portability improvements (#587). Still work to be done on this front.
* Add support for breakends to `view`, `norm`, `query` and filtering (#592).
* `plot-vcfstats`: Fix for python 2/3 compatibility (#593).
* New `-l, --list` option for `+af-dist` plugin.
* New `-i, --use-id` option for `+fix-ref` plugin.
* Add `--include/--exclude` options to `+guess-ploidy` plugin.
* New `+check-sparsity` plugin.
* Miscellaneous bugfixes for #575, #584, #588, #599, #535.
## Release 1.4 (13 March 2017)
Two new commands - `mpileup` and `csq`:
* The `mpileup` command has been imported from samtools to bcftools. The
reasoning behind this is that bcftools calling is intimately tied to mpileup
and any changes to one, often requires changes to the other. Only the
genotype likelihood (BCF output) part of mpileup has moved to bcftools,
while the textual pileup output remains in samtools. The BCF output option
in `samtools mpileup` will likely be removed in a release or two or when
changes to `bcftools call` are incompatible with the old mpileup output.
The basic mpileup functionality remains unchanged as do most of the command
line options, but there are some differences and new features that one
should be aware of:
- The option `samtools mpileup -t, --output-tags` changed to `bcftools
mpileup -a, --annotate` to avoid conflict with the `-t, --targets`
option common across other bcftools commands.
- `-O, --output-BP` and `-s, --output-MQ` are no longer used as they are
only for textual pipelup output, which is not included in `bcftools
mpileup`. `-O` short option reassigned to `--output-type` and `-s`
reassigned to `--samples` for consistency with other bcftools commands.
- `-g, --BCF`, `-v, --VCF`, and ` -u, --uncompressed` options from
`samtools mpileup` are no longer used, being replaced by the
`-O, --output-type` option common to other bcftools commands.
- The `-f, --fasta-ref` option is now required by default to help avoid user
errors. Can be diabled using `--no-reference`.
- The option `-d, --depth .. max per-file depth` now behaves as expected
and according to the documentation, and prints a meaningful diagnostics.
- The `-S, --samples-file` can be used to rename samples on the fly. See man
page for details.
- The `-G, --read-groups` functionality has been extended to allow
reassignment, grouping and exclusion of readgroups. See man page for
details.
- The `-l, --positions` replaced by the `-t, --targets` and
`-T, --targets-file` options to be consistent with other bcftools
commands.
- gVCF output is supported. Per-sample gVCFs created by mpileup can be
merged using `bcftools merge --gvcf`.
- Can generate mpileup output on multiple (indexed) regions using the
`-r, --regions` and `-R, --regions-file` options. In samtools, one
was restricted to a single region with the `-r, --region` option.
- Several speedups thanks to @jkbonfield (cf3a55a).
* `csq`: New command for haplotype-aware variant consequence calling.
See man page and [paper](https://www.ncbi.nlm.nih.gov/pubmed/28205675).
Updates, improvements and bugfixes for many other commands:
* `annotate`: `--collapse` option added. `--mark-sites` now works with
VCF files rather than just tab-delimited files. Now possible to annotate
a subset of samples from tab file, not just VCF file (#469). Bugfixes (#428).
* `call`: New option `-F, --prior-freqs` to take advantage of prior knowledge
of population allele frequencies. Improved calculation of the QUAL score
particularly for REF sites (#449, 7c56870). `PLs>=256` allowed in
`call -m`. Bugfixes (#436).
* `concat --naive` now works with vcf.gz in addition to bcf files.
* `consensus`: handle variants overlapping region boundaries (#400).
* `convert`: gvcf2vcf support for mpileup and GATK. new `--sex` option to
assign sex to be used in certain output types (#500). Large speedup of
`--hapsample` and `--haplegendsample` (e8e369b) especially with `--threads`
option enabled. Bugfixes (#460).
* `cnv`: improvements to output (be8b378).
* `filter`: bugfixes (#406).
* `gtcheck`: improved cross-check mode (#441).
* `index` can now specify the path to the output index file. Also, gains the
`--threads` option.
* `merge`: Large overhaul of `merge` command including support for merging
gVCF files created by `bcftools mpileup --gvcf` with the new `-g, --gvcf`
option. New options `-F` to control filter logic and `-0` to set missing
data to REF. Resolved a number of longstanding issues (#296, #361, #401,
#408, #412).
* `norm`: Bugfixes (#385,#452,#439), more informative error messages (#364).
* `query`: `%END` plus `%POS0`, `%END0` (0-indexed) support - allows easy BED
format output (#479). `%TBCSQ` for use with the new `csq` command. Bugfixes
(#488,#489).
* `plugin`: A number of new plugins:
- `GTsubset` (thanks to @dlaehnemann)
- `ad-bias`
- `af-dist`
- `fill-from-fasta`
- `fixref`
- `guess-ploidy` (deprecates `vcf2sex` plugin)
- `isecGT`
- `trio-switch-rate`
and changes to existing plugins:
- `tag2tag`: Added `gp-to-gt`, `pl-to-gl` and `--threshold` options and
bugfixes (#475).
- `ad-bias`: New `-d` option for minimum depth.
- `impute-info`: Bugfix (49a9eaf).
- `fill-tags`: Added ability to aggregate tags for sample subgroups, thanks
to @mh11. (#503). HWE tag added as an option.
- `mendelian`: Bugfix (#566).
* `reheader`: allow muiltispace delimiters in `--samples` option.
* `roh`: Now possible to process multiple samples at once. This allows
considerable speedups for files with thousands of samples where the cost of
HMM is neglibible compared to I/O and decompressing. In order to fit tens of
thousands samples in memory, a sliding HMM can be used (new `--buffer-size`
option). Viterbi training now uses Baum-Welch algorithm, and works much
better. Support for gVCFs or FORMAT/PL tags. Added `-o, output` and
`-O, --output-type` options to control output of sites or regions
(compression optional). Many bugs fixed - do not segfault on missing PL
values anymore, a typo in genetic map calculation resulted in a slowdown and
incorrect results.
* `stats`: Bugfixes (16414e6), new options `-af-bins` and `-af-tags` to control
allele frequency binning of output. Per-sample genotype concordance tables
added (#477).
* `view -a, --trim-alt-alleles` various bugfixes for missing data and more
informative errors should now be given on failure to pinpoint problems.
General changes:
* Timestamps are now added to header lines summarising the command (#467).
* Use of the `--threads` options should be faster across the board thanks to
changes in HTSlib meaning meaning threads are now shared by the compression
and decompression calls.
* Changes to genotype filtering with `-i, --include` and `-e, --exclude` (#454).
## Noteworthy changes in release 1.3.1 (22 April 2016)
* The `concat` command has a new `--naive` option for faster operations on
large BCFs (PR #359).
* `GTisec`: new plugin courtesy of David Laehnemann (@dlaehnemann) to count
genotype intersections across all possible sample subsets in a VCF file.
* Numerous VCF parsing fixes.
* Build fix: _peakfit.c_ now builds correctly with GSL v2 (#378).
* Various bug fixes and improvements to the `annotate` (#365), `call` (#366),
`index` (#367), `norm` (#368, #385), `reheader` (#356), and `roh` (#328)
commands, and to the `fill-tags` (#345) and `tag2tag` (#394) plugins.
* Clarified documentation of `view` filter options, and of the
`--regions-file` and `--targets-file` options (#357, #411).
## Noteworthy changes in release 1.3 (15 December 2016)
* `bcftools call` has new options `--ploidy` and `--ploidy-file` to make
handling sample ploidy easier. See man page for details.
* `stats`: `-i`/`-e` short options changed to `-I`/`-E` to be consistent with
the filtering `-i`/`-e` (`--include`/`--exclude`) options used in other
tools.
* general `--threads` option to control the number of output compression
threads used when outputting compressed VCF or BCF.
* `cnv` and `polysomy`: new commands for detecting CNVs, aneuploidy, and
contamination from SNP genotyping data.
* various new options, plugins, and bug fixes, including #84, #201, #204,
#205, #208, #211, #222, #225, #242, #243, #249, #282, #285, #289, #302,
#311, #318, #336, and #338.
## Noteworthy changes in release 1.2 (2 February 2016)
* new `bcftools consensus` command
* new `bcftools annotate` plugins: fixploidy, vcf2sex, tag2tag
* more features in `bcftools convert` command, amongst others new
`--hapsample` function (thanks to Warren Kretzschmar @wkretzsch)
* support for complements in `bcftools annotate --remove`
* support for `-i`/`-e` filtering expressions in `bcftools isec`
* improved error reporting
* `bcftools call`
- the default prior increased from `-P 1e-3` to `-P 1.1e-3`, some clear
calls were missed with default settings previously
- support for the new symbolic allele `<*>`
- support for `-f GQ`
- bug fixes, such as: proper trimming of DPR tag with `-c`; the `-A` switch
does not add back records removed by `-v` and the behaviour has been made
consistent with `-c` and `-m`
* many bug fixes and improvements, such as
- bug in filtering, FMT & INFO vs INFO & FMT
- fixes in `bcftools merge`
- filter update AN/AC with `-S`
- isec outputs matching records for both VCFs in the Venn mode
- annotate considers alleles when working with `Number=A,R` tags
- new `--set-id` feature for annotate
- `convert` can be used similarly to `view`