Manipulate FASTA and FASTQ files
Python3 script to manipulate FASTA and FASTQ (and other format) files, plus API for developers
There are a number of ways to install Fastaq and details are provided below. If you encounter an issue when installing Fastaq please contact your local system administrator. If you encounter a bug please log it here or email us at path-help@sanger.ac.uk.
Install from PyPi
pip3 install pyfastaqOr pip install the latest development version directly from this repo.
pip3 install git+https://github.com/sanger-pathogens/Fastaq.gitIf you want to edit the codebase, clone this repo and install in editable mode.
# Clone and install from this repository:
git clone https://github.com/sanger-pathogens/Fastaq.git && cd Fastaq && pip install -e ".[tests]"The test can be run from the top level directory:
pytest tests
These must be available in your path at run time:
- samtools 0.1.19
- gzip
- gunzip
The installation will put a single script called fastaq in your path.
The usage is:
fastaq <command> [options]
Key points:
- To list the available commands and brief descriptions, just run
fastaq - Use
fastaq command -horfastaq command --helpto get a longer description and the usage of that command. - The type of input file is automatically detected. Currently supported:
FASTA,FASTQ,GFF3,EMBL,GBK,Phylip. fastaqonly manipulates sequences (and quality scores if present), so annotation is ignored where present in the input.- Input and output files can be gzipped. An input file is assumed to be gzipped if its name ends with .gz. To gzip an output file, just name it with .gz at the end.
- You can use a minus sign for a filename to use stdin or stdout, so commands can be piped together. See the example below.
Reverse complement all sequences in a file:
fastaq reverse_complement in.fastq out.fastq
Reverse complement all sequences in a gzipped file, then translate each sequence:
fastaq reverse_complement in.fastq.gz - | fastaq translate - out.fasta
| Command | Description |
|---|---|
| acgtn_only | Replace every non acgtnACGTN with an N |
| add_indels | Deletes or inserts bases at given position(s) |
| caf_to_fastq | Converts a CAF file to FASTQ format |
| capillary_to_pairs | Converts file of capillary reads to paired and unpaired files |
| chunker | Splits sequences into equal sized chunks |
| count_sequences | Counts the sequences in input file |
| deinterleave | Splits interleaved paired file into two separate files |
| enumerate_names | Renames sequences in a file, calling them 1,2,3... etc |
| expand_nucleotides | Makes every combination of degenerate nucleotides |
| fasta_to_fastq | Convert FASTA and .qual to FASTQ |
| filter | Filter sequences to get a subset of them |
| get_ids | Get the ID of each sequence |
| get_seq_flanking_gaps | Gets the sequences flanking gaps |
| interleave | Interleaves two files, output is alternating between fwd/rev reads |
| make_random_contigs | Make contigs of random sequence |
| merge | Converts multi sequence file to a single sequence |
| replace_bases | Replaces all occurrences of one letter with another |
| reverse_complement | Reverse complement all sequences |
| scaffolds_to_contigs | Creates a file of contigs from a file of scaffolds |
| search_for_seq | Find all exact matches to a string (and its reverse complement) |
| sequence_trim | Trim exact matches to a given string off the start of every sequence |
| sort_by_name | Sorts sequences in lexographical (name) order |
| sort_by_size | Sorts sequences in length order |
| split_by_base_count | Split multi sequence file into separate files |
| strip_illumina_suffix | Strips /1 or /2 off the end of every read name |
| to_fake_qual | Make fake quality scores file |
| to_fasta | Converts a variety of input formats to nicely formatted FASTA format |
| to_mira_xml | Create an xml file from a file of reads, for use with Mira assembler |
| to_orfs_gff | Writes a GFF file of open reading frames |
| to_perfect_reads | Make perfect paired reads from reference |
| to_random_subset | Make a random sample of sequences (and optionally mates as well) |
| to_tiling_bam | Make a BAM file of reads uniformly spread across the input reference |
| to_unique_by_id | Remove duplicate sequences, based on their names. Keep longest seqs |
| translate | Translate all sequences in input nucleotide sequences |
| trim_Ns_at_end | Trims all Ns at the start/end of all sequences |
| trim_contigs | Trims a set number of bases off the end of every contig |
| trim_ends | Trim fixed number of bases of start and/or end of every sequence |
| version | Print version number and exit |
Here is a template for counting the sequences in a FASTA or FASTQ file:
from pyfastaq import sequences
seq_reader = sequences.file_reader(infile)
count = 0
for seq in seq_reader:
count += 1
print(count)Hopefully you get the idea and there are plenty of examples in tasks.py. Detection of the input file type and whether gzipped or not is automatic. See help(sequences) for the various methods already defined in the classes Fasta and Fastq.
Fastaq is free software, licensed under GPLv3.
Please report any issues to the issues page or email path-help@sanger.ac.uk.