A collection of tools for evaluating RNA seq mapping. The tools were used to benchmark some RNA mapping tools on real and synthetic third generation sequencing datasets. Detailed description of synthetic dataset preparation can be found at RNAseq_benchmark/Data_preparation.md. Description of the evaluation process for real and simulated datasets can be found at RNAseq_benchmark/RNAseq_benchmark.md.
Recently a new version of tools has been published, with multiple bugs fixed and several new options. The main new funcionality of the RNAseqEval tool is proposing new transcripts/annotation. Details on new transcript discovery algorithm can be found at doc/Transcript_discovery.md.
Since the RNAseqEval repository contains python scripts, it does not require an installation. Only requirement is to clone RNAseqEval repository together with the samscripts submodule:
git clone --recursive https://github.com/kkrizanovic/RNAseqEval.git
The other way is to clone the main repository first and then update the submodule:
git clone https://github.com/kkrizanovic/RNAseqEval.git
cd RNAseqEval
git submodule update --init --recursive
The repository contains Python scripts only and requires a Python 2 interpreter. We have tested it on Ubuntu server 14.04 and 16.04 with Python 2.7.12 However, we believe that it should also work with older versions of Python and on other operating systems.
Run generate_transcriptome.py to generate a trancriptome from a reference and a set of annotations.
How to run:
generate_transcriptome.py annotations.gtf reference.fasta transcriptome.fasta
Generated transcriptome can be used to simulate RNA reads using DNA simulators such as PBSIM.
Run RNAseqEval.py for general evaulation of mappings in sam file against reference and optionally annotations. This script is intended to evaluate real dataset mapping. Run RNAseqEval.py without any arguments to print options.
Usage:
/home/kkrizanovic/src/RNAseqEval/RNAseqEval.py eval-mapping <reference FASTA file> <input SAM file> options
options:"
-a <file> : a reference annotation (GFF/GTF/BED) file
-o (--output) <file> : output file to which the report will be written
-ex (--expression) : calculate and output gene expression
--no_check_strand : when matching alignments to annotations, do not take strand
into account, only chromosome and position
--no_per_base_stats : do not calculate per-base stats, such are percentage of
matches, mismatches, inserts and deletes
-sqn (--save_query_names) : save query names for alignments that managed to hit an exon
and for contiguous alignments. Query names are saved to files
with filenames determined from output file.
For this option output and annotation files must be specified
-ai (--alowed_inaccuracy) : A maximumn distance in bases between an annotation and an alignment
where the alignment is still considered correct (default 5)
-mo (--min_overlap) : A minimum overlap between an annotation and an alignment that is considered valid
(default 5)
--old_bma_calc : Calculate best matching annotation only based on maximizing the number of bases an alignment
on an annotation. The number of bases outside an annotation is not take into account in this case
--leave_chrom_names : do not preprocess chromosome names in SAM, reference and annotation files
Chromosome names are by default preprocessed to make them match the format "chr[designation]"
--calc_new_annotations: calculate potential new annotations, if a sufficient number of alignments (default 3)
better fits a combination of exons then any existing annotation, that combination
of exons is suggested as a new annotation
Detailed description of running modes, options and output can be found at doc/RNAseqEval.md.
Run Process_pbsim_data.py to evaluate mappings of data generated by the PBSIM simulator. It requires the simulation data to be organized in a specific way.
How to run:
Process_pbsim_data.py process simulation_root_folder mappings.sam annotations.gtf
Detailed description of options, output and required simulation data organization can be found at doc/Process_pbsim_data.md.
Folder example_dataset contains an example dataset, simulated by applying PBSIM o a transcriptome. Example dataset was generated usign reference and annotations for Drosophilla Melanogaster chromosome 4. The folder contains everything necessary to test out our evaluation tools. More on example dataset can be fouund at example_dataset/example.md.
This work has been supported in part by Croatian Science Fundation under projects UIP-11-2013-7353 and IP-01-2018-5886.