v1.0.0rc1 release

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.gitignore

@@ -100,8 +100,11 @@ venv.bak/
 # mkdocs documentation
 /site
 
-# data
-tests
-
 # mypy
 .mypy_cache/
+site/
+
+# Results and data folder
+data
+docs/demo/results
+tmp

CODE_OF_CONDUCT.md

@@ -0,0 +1 @@
+docs/code_of_conduct.md

CONTRIBUTING.md

@@ -0,0 +1 @@
+docs/contributing.md

LICENCE

@@ -0,0 +1 @@
+docs/licence.md

README.md

@@ -1,51 +1,14 @@
-# Nanocompore v1.0.0b6
+![Nanocompore](docs/pictures/Nanocompore_logo.png)
 
 ---
 
-**Software package that identifies differences in nanopore raw signal (SNPs and base modifications) between 2 samples**
+[![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://lbesson.mit-license.org/)
+[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)
 
 ---
 
-# Installation
+**Nanocompore identifies differences in ONT nanopore sequencing raw signal corresponding to RNA modifications by comparing 2 samples**
 
-Ideally, before installation, create a clean python3 virtual environment to deploy the package, using virtualenvwrapper for example ([see tutorial](http://www.simononsoftware.com/virtualenv-tutorial-part-2/)).
+Nanocompore compares 2 ONT nanopore direct RNA sequencing datasets from different experimental conditions expected to have a significant impact on RNA modifications. It is recommended to have at least 2 replicates per condition. For example one can use a control condition with a significantly reduced number of modifications such as a cell line for which a modification writing enzyme was knocked-down or knocked-out. Alternatively, on a smaller scale transcripts of interests could be synthesized in-vitro.
 
-## Dependencies
-
-Nanocompore relies on a few robustly well maintained third party python libraries (numpy, tqdm, pyfaidx, matplotlib, seaborn and pandas).
-The correct versions of packages are installed together with the software when using pip.
-
-## Installation with pip
-
-* To install the package
-
-    ```pip3 install git+https://github.com/a-slide/nanocompore.git```
-
-* To update the package:
-
-    ```pip3 install git+https://github.com/a-slide/nanocompore.git --upgrade```
-
-# Usage
-
-Nanocompore compares 2 nanopore sequencing datasets from 2 experimental conditions expected to have an impact on DNA/RNA modifications. In particular the software was optimized to work with direct RNA sequencing data.
-
-To use nanocompore raw nanopore sequencing data have to be prepared using a basecaller (Albacore or MInKNOW. Ask ONT), your favourite long read aligner such as [Minimap2](https://github.com/lh3/minimap2) and resquiggled using [Nanopolish](https://github.com/jts/nanopolish). Finally eventalign data have to be collapsed per kmer using [NanopolishComp Eventalign_collapse](https://github.com/a-slide/NanopolishComp)
-
-The preparation of data and the package usage are detailed in the [usage jupyter notebook](https://github.com/a-slide/nanocompore/blob/master/tests/nanocompore_usage.ipynb)
-
-Example of pvalue analysis using Mann_Whitney paired test:
-
-![pvalues](pictures/pvalues.png)
-
-Example of signal shift between 2 samples:
-
-![signal_shift](pictures/signal_shift.png)
-
-
-# Note to power-users and developers
-
-Please be aware that **nanocompore** is an experimental package that is still under development. It was tested under Linux Ubuntu 16.04 LTS and in an HPC cluster environment running under Red Hat Enterprise 7.1.
-
-You are welcome to contribute by requesting additional functionalities, reporting bugs or by forking and submitting pull requests
-
-Thank you
+Full documentation is available at http://nanocompore.rna.rocks

docs/alternative.md

@@ -0,0 +1,6 @@
+# Alternative and complementary packages
+
+Here is a non-exhaustive list of alternative/complementary packages that could also be used to analyse RNA methylation from nanopore sequencing datasets:
+
+* [Tombo](https://github.com/nanoporetech/tombo)
+* ...

docs/code_of_conduct.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at aleg@ebi.ac.uk. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

docs/contributing.md

@@ -0,0 +1,81 @@
+# Contributing to nanocompore
+
+First of all, thanks for considering contributing to `nanocompore`! 👍 It's people like you that make it rewarding for us to work on `nanocompore`.
+
+`nanocompore` is an open source project, maintained by publicly funded academic researchers.
+
+[repo]: https://github.com/tleonardi/nanocompore
+[issues]: https://github.com/tleonardi/nanocompore/issues
+[new_issue]: https://github.com/tleonardi/nanocompore/issues/new
+[code_of_conduct]: https://github.com/tleonardi/nanocompore/blob/master/docs/code_of_conduct.md
+
+## Code of conduct
+
+Please note that this project is released with a [Contributor Code of Conduct][code_of_conduct.md]. By participating in this project you agree to abide by its terms.
+
+## How you can contribute
+
+There are several ways you can contribute to this project. If you want to know more about why and how to contribute to open source projects like this one, see this [Open Source Guide](https://opensource.guide/how-to-contribute/).
+
+### Share the love ❤️
+
+Think `nanocompore` is useful? Let others discover it, by telling them in person, via Twitter or a blog post.
+
+Using `nanocompore` for a paper you are writing? Please cite it.
+
+### Ask a question ⁉️
+
+Using `nanocompore` and got stuck? Browse the documentation to see if you can find a solution.
+
+Still stuck? Post your question as an [issue on GitHub][new_issue].
+
+While we cannot offer user support, we'll try to do our best to address it, as questions often lead to better documentation or the discovery of bugs.
+
+
+### Propose an idea 💡
+
+Have an idea for a new `nanocompore` feature? Take a look at the [issue list][issues] to see if it isn't included or suggested yet. If not, suggest your idea as an [issue on GitHub][new_issue]. While we can't promise to implement your idea, it helps to:
+
+* Explain in detail how it would work.
+* Keep the scope as narrow as possible.
+
+See below if you want to contribute code for your idea as well.
+
+### Report a bug 🐛
+
+Using `nanocompore` and discovered a bug? That's annoying! Don't let others have the same experience and report it as an [issue on GitHub][new_issue] so we can fix it. A good bug report makes it easier for us to do so, so please include:
+
+* Your operating system name and version (e.g. Mac OS 10.13.6).
+* Any details about your local setup that might be helpful in troubleshooting.
+* Detailed steps to reproduce the bug.
+
+### Improve the documentation 📖
+
+Noticed a typo on the website? Think a function could use a better example? Good documentation makes all the difference, so your help to improve it is very welcome!
+
+1. Fork [nanocompore][repo] and clone it to your computer. To learn more about this process, see [this guide](https://guides.github.com/activities/forking/).
+2. Edit the README.md file and submit a pull request. We will review your changes and include the fix in the next release.
+
+### Contribute code 📝
+
+Care to fix bugs or implement new functionality for `nanocompore`? Awesome! 👏 Have a look at the [issue list][issues] and leave a comment on the things you want to work on. See also the development guidelines below.
+
+## Development guidelines
+
+We try to follow the [GitHub flow](https://guides.github.com/introduction/flow/) for development and the [PEP 8](https://www.python.org/dev/peps/pep-0008/) style Guide for Python Code.
+
+1. Fork [nanocompore][repo] and clone it to your computer. To learn more about this process, see [this guide](https://guides.github.com/activities/forking/).
+
+2. If you have forked and cloned the project before and it has been a while since you worked on it, [pull changes from the original repo](https://help.github.com/articles/merging-an-upstream-repository-into-your-fork/) to your clone by using `git pull upstream master`.
+
+3. Make your changes and test the modified code.
+
+4. Commit and push your changes.
+
+5. Submit a [pull request](https://guides.github.com/activities/forking/#making-a-pull-request).
+
+
+
+---
+
+This file was adapted from a template created by [peterdesmet](https://gist.github.com/peterdesmet/e90a1b0dc17af6c12daf6e8b2f044e7c).

docs/data_preparation.md

@@ -0,0 +1,50 @@
+# Data preparation
+
+Before using nanocompore, sequencing data have to be basecalled (Albacore or Guppy), aligned on a transcriptome reference and resquiggled with Nanopolish.
+
+To simplify the data preprocessing we wrote a Nextflow pipeline which automatises all these steps as well as extra quality control steps: https://github.com/tleonardi/nanocompore_pipeline
+
+### Reads basecalling
+
+Firstly, raw fast5 reads have to be basecalled with a recent version of ONT basecaller. Basecalled fast5 files are not required for the rest of the analysis, only the raw fast5 and the basecalled fastq.
+
+Example with [Guppy v2.3.5](https://community.nanoporetech.com/downloads)
+
+```bash
+guppy_basecaller -i {raw_fast5_dir} -s {dest_dir} --flowcell {flowcell_id} --kit {Kit_id} -r --calib_detect --enable_trimming true --trim_strategy rna --reverse_sequence true
+```
+
+Then the output fastq files should be concatenated in a single one.
+
+```bash
+cat {dir_to guppy output}/*.fastq > {basecalled_fastq}
+```
+
+### Transcriptome alignment
+
+Basecalled reads have to be aligned to a reference. For dRNA-Seq, reads should be aligned to a reference **transcriptome (not genome)** in a non-spliced fashion. For example, one can download reference transcriptome fasta files directly from Gencode for [human](ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_29/gencode.v29.transcripts.fa.gz) and [mouse](ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M20/gencode.vM20.transcripts.fa.gz).
+
+ Bam files have to be filtered to remove any reads that would be unmapped, secondary and supplementary as well as reads mapped on the reverse strand (SAM flag 2324). We also recommend to discard reads with a very low alignment score (MAPQ<10). Finally, reads have then to be sorted and indexed.
+
+Example with [Minimap2 v2.16](https://github.com/lh3/minimap2)
+
+```bash
+minimap2 -ax map-ont -L {transcriptome_fasta} {basecalled_fastq} | samtools view -bh -F 2324 -q 10 | samtools sort -O bam > {aligned_reads_bam}
+
+samtools index {aligned_reads_bam}
+```
+
+### Read indexing and resquiggling
+
+Nanopolish is required to realign raw signal to the expected reference sequence. Reads have to be indexed first with nanopolish index, realigned with nanopolish eventalign and finally the data has to be collapsed per kmer and indexed by NanopolishComp Eventalign_collapse.
+
+Example with [Nanopolish v0.10.1](https://github.com/jts/nanopolish) and [NanopolishComp v0.4.3](https://github.com/a-slide/NanopolishComp)
+
+```bash
+nanopolish index -s {sequencing_summary.txt} -d {raw_fast5_dir} {basecalled_fastq}
+
+nanopolish eventalign --reads {basecalled_fastq} --bam {aligned_reads_bam} --genome {transcriptome_fasta} --samples --print-read-names --scale-events --samples > {eventalign_reads_tsv}
+
+NanopolishComp Eventalign_collapse -i {eventalign_reads_tsv} -o {eventalign_collapsed_reads_tsv}
+
+```