Fastaq

Manipulate FASTA and FASTQ files

Contents

• Introduction
• Installation
  • Using pip3
  • From source
  • Running the tests
• Usage
  • Examples
  • Available commands
  • For developers
• License
• Feedback/Issues

Introduction

Python3 script to manipulate FASTA and FASTQ (and other format) files, plus API for developers

Installation

There are a number of ways to install Fastaq and details are provided below. If you encounter an issue when installing Fastaq please contact your local system administrator. If you encounter a bug please log it here or email us at path-help@sanger.ac.uk.

Pip install

Install from PyPi

pip3 install pyfastaq

Or pip install the latest development version directly from this repo.

pip3 install git+https://github.com/sanger-pathogens/Fastaq.git

From source

If you want to edit the codebase, clone this repo and install in editable mode.

# Clone and install from this repository:
git clone https://github.com/sanger-pathogens/Fastaq.git && cd Fastaq && pip install -e ".[tests]"

Running the tests

The test can be run from the top level directory:

pytest tests

Runtime dependencies

These must be available in your path at run time:

• samtools 0.1.19
• gzip
• gunzip

Usage

The installation will put a single script called fastaq in your path. The usage is:

fastaq <command> [options]

Key points:

• To list the available commands and brief descriptions, just run fastaq
• Use fastaq command -h or fastaq command --help to get a longer description and the usage of that command.
• The type of input file is automatically detected. Currently supported: FASTA, FASTQ, GFF3, EMBL, GBK, Phylip.
• fastaq only manipulates sequences (and quality scores if present), so annotation is ignored where present in the input.
• Input and output files can be gzipped. An input file is assumed to be gzipped if its name ends with .gz. To gzip an output file, just name it with .gz at the end.
• You can use a minus sign for a filename to use stdin or stdout, so commands can be piped together. See the example below.

Examples

Reverse complement all sequences in a file:

fastaq reverse_complement in.fastq out.fastq

Reverse complement all sequences in a gzipped file, then translate each sequence:

fastaq reverse_complement in.fastq.gz - | fastaq translate - out.fasta

Available commands

Command	Description
acgtn_only	Replace every non acgtnACGTN with an N
add_indels	Deletes or inserts bases at given position(s)
caf_to_fastq	Converts a CAF file to FASTQ format
capillary_to_pairs	Converts file of capillary reads to paired and unpaired files
chunker	Splits sequences into equal sized chunks
count_sequences	Counts the sequences in input file
deinterleave	Splits interleaved paired file into two separate files
enumerate_names	Renames sequences in a file, calling them 1,2,3... etc
expand_nucleotides	Makes every combination of degenerate nucleotides
fasta_to_fastq	Convert FASTA and .qual to FASTQ
filter	Filter sequences to get a subset of them
get_ids	Get the ID of each sequence
get_seq_flanking_gaps	Gets the sequences flanking gaps
interleave	Interleaves two files, output is alternating between fwd/rev reads
make_random_contigs	Make contigs of random sequence
merge	Converts multi sequence file to a single sequence
replace_bases	Replaces all occurrences of one letter with another
reverse_complement	Reverse complement all sequences
scaffolds_to_contigs	Creates a file of contigs from a file of scaffolds
search_for_seq	Find all exact matches to a string (and its reverse complement)
sequence_trim	Trim exact matches to a given string off the start of every sequence
sort_by_name	Sorts sequences in lexographical (name) order
sort_by_size	Sorts sequences in length order
split_by_base_count	Split multi sequence file into separate files
strip_illumina_suffix	Strips /1 or /2 off the end of every read name
to_fake_qual	Make fake quality scores file
to_fasta	Converts a variety of input formats to nicely formatted FASTA format
to_mira_xml	Create an xml file from a file of reads, for use with Mira assembler
to_orfs_gff	Writes a GFF file of open reading frames
to_perfect_reads	Make perfect paired reads from reference
to_random_subset	Make a random sample of sequences (and optionally mates as well)
to_tiling_bam	Make a BAM file of reads uniformly spread across the input reference
to_unique_by_id	Remove duplicate sequences, based on their names. Keep longest seqs
translate	Translate all sequences in input nucleotide sequences
trim_Ns_at_end	Trims all Ns at the start/end of all sequences
trim_contigs	Trims a set number of bases off the end of every contig
trim_ends	Trim fixed number of bases of start and/or end of every sequence
version	Print version number and exit

For developers

Here is a template for counting the sequences in a FASTA or FASTQ file:

from pyfastaq import sequences
seq_reader = sequences.file_reader(infile)
count = 0
for seq in seq_reader:
    count += 1
print(count)

Hopefully you get the idea and there are plenty of examples in tasks.py. Detection of the input file type and whether gzipped or not is automatic. See help(sequences) for the various methods already defined in the classes Fasta and Fastq.

License

Fastaq is free software, licensed under GPLv3.

Feedback/Issues

Please report any issues to the issues page or email path-help@sanger.ac.uk.