Add TOPMed Imputation Server specific changes

* Documentation updates
* SNPs per chunk limit
* Duplicate chromosome filter

.github/workflows/maven.yml

@@ -0,0 +1,27 @@
+# This workflow will build a Java project with Maven
+# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
+
+name: Java Build with Maven
+
+on: [push, pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up JDK 1.11
+        uses: actions/setup-java@v2
+        with:
+          java-version: 1.11
+          distribution: 'adopt'
+      - name: Run tests excluding ChrX
+        run: mvn -Dtest=AllTestsNoChrX test
+      - name: Package using Maven
+        run: mvn package -DskipTests
+      - name: Upload a Build Artifact
+        uses: actions/upload-artifact@v2
+        with:
+          path: target/imputationserver.zip

.gitignore

@@ -1,2 +1,3 @@
 /target/
 /dependency-reduced-pom.xml
+.idea

docs/api.md

@@ -1,20 +1,21 @@
 # API Reference
 
-The REST APIs provide programmatic ways to submit new jobs and to download data from Michigan Imputation Server. It identifies users using authentication tokens, responses are provided in JSON format.
-
+REST APIs provide programmatic ways to submit new jobs and to download data from both [Michigan Imputation Server](https://imputationserver.sph.umich.edu) and the [TOPMed Imputation Server](https://imputation.biodatacatalyst.nhlbi.nih.gov). It identifies users using authentication tokens, responses are provided in JSON format.
 
 ## Authentication
-Michigan Imputation Server uses a token-based authentication. The token is required for all future interaction with the server. The token can be created and downloaded from your user profile (username -> Profile):
+Both Michigan and TOPMed Imputation Server use a token-based authentication mechanism. The token is required for all future interaction with the server. The token can be created and downloaded from your user profile (username -> Profile):
 
 ![Activate API](https://raw.githubusercontent.com/genepi/imputationserver-docker/master/images/api.png)
 
-For security reasons, Api Tokens are valid for 30 days. You can check the status in the web interface.
+_**Note:** the tokens from Michigan Imputation Server and TOPMed Imputation Server are unique to each server_
 
+## Job Submission
+The API allows setting several imputation parameters.
 
-## Job Submission for Whole Genome Imputation
-The API allows to submit imputation jobs and to set several parameters. For HLA imputation, please see below. 
+### Michigan Imputation Server Job Submission
 
-### POST /jobs/submit/minimac4
+URL: https://imputationserver.sph.umich.edu/api/v2
+POST /jobs/submit/minimac4
 
 The following parameters can be set:
 
@@ -24,31 +25,32 @@ The following parameters can be set:
 | mode          | `qconly`<br> `phasing` <br> `imputation`     | `imputation`   | |
 | password      | user-defined password      |  auto generated and send by mail  | |
 | files-source  | `file-upload`<br> `sftp`<br> `http`     |  `file-upload`  | |
-| refpanel      | `hrc-r1.1` <br> `1000g-phase-3-v5` <br>  `gasp-v2` <br>  `genome-asia-panel`  <br> `1000g-phase-1` <br> `cappa` <br> `hapmap-2` | - | **x** |
+| refpanel      | `apps@hapmap-2`<br> `apps@hrc-r1.1`<br> `apps@1000g-phase-1`<br> `apps@1000g-phase-3-v5` <br> `apps@genome-asia-panel@1.0.0` <br> `apps@cappa` | - | **x** |
 | phasing     | `eagle`<br> `no_phasing`      |  `eagle`  | |
-| population  | `eur`<br> `afr`<br> `asn`<br> `amr`<br> `sas`<br> `eas`<br> `AA`<br> `mixed` <br> `all`   |  -  | **x** |
+| population  | `eur`<br> `afr`<br> `asn`<br> `amr`<br> `sas`<br> `eas`<br> `AA`<br> `mixed`      |  -  | **x** |
 | build       | `hg19`<br> `hg38` | `hg19`  | |
 | r2Filter    | `0` <br> `0.001` <br> `0.1` <br> `0.2` <br> `0.3` | `0`  | |
 
-## Job Submission for HLA Imputation
-The API also allows to submit imputation jobs using the HLA application. Please note, that the population parameter can be skipped here. 
+### TOPMed Imputation Server Job Submission
 
-### POST /jobs/submit/imputationserver-hla
+URL: https://imputation.biodatacatalyst.nhlbi.nih.gov/api/v2
+POST /jobs/submit/imputationserver@1.2.7
 
 The following parameters can be set:
 
-| Parameter        | Values           | Default Value  |  Required  |
-| ------------- |:-------------| :-----|---|
-| files         | /path/to/file |  | **x** |
-| mode          | `qconly`<br> `phasing` <br> `imputation`     | `imputation`   | |
-| password      | user-defined password      |  auto generated and send by mail  | |
-| files-source  | `file-upload`<br> `sftp`<br> `http`     |  `file-upload`  | |
-| refpanel      | `multiethnic-hla-panel-Ggroup` <br>  `multiethnic-hla-panel-4digit` | - | **x** |
-| phasing     | `eagle`<br> `no_phasing`      |  `eagle`  | |
-| build       | `hg19`<br> `hg38` | `hg19`  | |
-| r2Filter    | `0` <br> `0.001` <br> `0.1` <br> `0.2` <br> `0.3` | `0`  | |
+| Parameter        | Values           | Default Value  |
+| ------------- |:-------------| :-----|
+| input-files      | /path/to/file |  |
+| input-mode | qconly, imputation     | imputation   |
+| input-password | user-defined password      |  auto  |
+| input-files-source | file-upload, sftp, http     |  default: file-upload  |
+| input-refpanel     | apps@topmed-r2@1.0.0     | - |
+| input-phasing | eagle      |  eagle  |
+| input-population | all, mixed      |  all  |
 
 
+### Examples
+
 ### Examples: curl
 
 #### Submit a single file
@@ -64,7 +66,7 @@ TOKEN="YOUR-API-TOKEN";
 curl https://imputationserver.sph.umich.edu/api/v2/jobs/submit/minimac4 \
   -H "X-Auth-Token: $TOKEN" \
   -F "files=@/path-to/file.vcf.gz" \
-  -F "refpanel=1000g-phase-3-v5" \
+  -F "refpanel=apps@1000g-phase-3-v5" \
   -F "population=eur"
 ```
 
@@ -78,7 +80,15 @@ Response:
 }
 ```
 
-#### Submit multiple files
+#### Submit a single file using TOPMed 
+
+To submit a job please change `/path-to-file` to the actual path.
+
+```sh
+curl -H "X-Auth-Token: <your-API-token>" -F "input-files=@/path-to-file" -F "input-refpanel=apps@topmed-r2@1.0.0" -F "input-phasing=eagle" https://imputation.biodatacatalyst.nhlbi.nih.gov/api/v2/jobs/submit/imputationserver@1.2.7
+```
+
+#### Submit multiple files using 1000 Genomes Phase 3
 
 Submits multiple vcf files and impute against 1000 Genomes Phase 3 reference panel.
 
@@ -91,7 +101,7 @@ curl https://imputationserver.sph.umich.edu/api/v2/jobs/submit/minimac4 \
   -H "X-Auth-Token: $TOKEN" \
   -F "files=@/path-to/file1.vcf.gz" \
   -F "files=@/path-to/file2.vcf.gz" \
-  -F "refpanel=1000g-phase-3-v5" \
+  -F "refpanel=apps@1000g-phase-3-v5" \
   -F "population=eur"
 ```
 
@@ -119,7 +129,7 @@ curl  https://imputationserver.sph.umich.edu/api/v2/jobs/submit/minimac4 \
   -H "X-Auth-Token: $TOKEN" \
   -F "files=https://imputationserver.sph.umich.edu/static/downloads/hapmap300.chr1.recode.vcf.gz" \
   -F "files-source=http" \
-  -F "refpanel=hrc-r1.1" \
+  -F "refpanel=apps@hrc-r1.1" \
   -F "population=eur" \
   -F "mode=qconly"
 ```
@@ -150,7 +160,7 @@ token = 'YOUR-API-TOKEN';
 # add token to header (see Authentication)
 headers = {'X-Auth-Token' : token }
 data = {
-  'refpanel': '1000g-phase-3-v5',
+  'refpanel': 'apps@1000g-phase-3-v5',
   'population': 'eur'
 }
 
@@ -180,7 +190,7 @@ token = 'YOUR-API-TOKEN';
 # add token to header (see Authentication)
 headers = {'X-Auth-Token' : token }
 data = {
-  'refpanel': '1000g-phase-3-v5',
+  'refpanel': 'apps@1000g-phase-3-v5',
   'population': 'eur'
 }
 
@@ -286,7 +296,7 @@ Response:
 
 ```json
 {
-  "application":"Michigan Imputation Server (Minimac4) 1.5.8",
+  "application":"Michigan Imputation Server (Minimac4) 1.1.4",
   "applicationId":"minimac4",
   "deletedOn":-1,
   "endTime":1462369824173,

docs/contact.md

@@ -1,12 +1,19 @@
 # Contact
 
-The complete Michigan Imputation Server source code is available on [GitHub](https://github.com/genepi/imputationserver). Feel free to create issues and pull requests. Before contacting us, please have a look at the [FAQ page](/faq) first. 
+The complete Imputation Server source code is available on [GitHub](https://github.com/genepi/imputationserver). Feel free to create issues and pull requests. Before contacting us, please have a look at the [FAQ page](/faq) first. 
 
-Please contact [Christian Fuchsberger](mailto:cfuchsb@umich.edu) in case of other problems.
+Please contact [Michigan Imputation Server](mailto:imputationserver@umich.edu) in case of other problems.
+
+## TOPMed Imputation Server Team
+
+[TOPMed Imputation Server](https://imputation.biodatacatalyst.nhlbi.nih.gov) provides free genotype imputation for the TOPMed reference panel. You can upload phased or unphased GWAS agenotypes and receive phased and imputed genomes in return. For uploaded data sets an extensive QC is performed.
+
+* [Albert Smith](mailto:albertvs@umich.edu)
+* [Jacob Pleiness](mailto:pleiness@umich.edu)
 
 ## Michigan Imputation Server Team
 
-Michigan Imputation Server provides a free genotype imputation service using Minimac4. You can upload phased or unphased GWAS genotypes and receive phased and imputed genomes in return. For all uploaded data sets an extensive QC is performed.
+[Michigan Imputation Server](https://imputationserver.sph.umich.edu) provides a free genotype imputation service using Minimac4, supporting imputation 1000 Genome and Haplotype Reference Consortium Panels . You can upload phased or unphased GWAS genotypes and receive phased and imputed genomes in return. For all uploaded data sets an extensive QC is performed.
 
 * [Christian Fuchsberger](mailto:cfuchsb@umich.edu)
 * [Lukas Forer](mailto:lukas.forer@i-med.ac.at)

docs/data-sensitivity.md

@@ -1,6 +1,6 @@
 # Data Security
 
-Since data is transfered to our server located in Michigan, a wide array of security measures are in force:
+For TOPMed Imputation, data is transfered to a secure server hosted on Amazon Web Servies, a wide array of security measures are in force:
 
 - The complete interaction with the server is secured with HTTPS.
 - Input data is deleted from our servers as soon it is not needed anymore.
@@ -19,11 +19,11 @@ To upload and download data, users must register with a unique e-mail address an
 
 A wide array of security measures are in force on the imputation servers:
 
-- SSH login to the servers is restricted to only systems administrators.
-- Direct root login via SSH is not allowed from the public Internet.
-- The public-facing side of the servers sits behind the School of Public Health's Checkpoint virtual firewall instance where a default-deny policy is used on inbound traffic; only explicitly allowed TCP ports are passed.
-- The School of Public Health also makes use of NIDS technologies such as Snort and Peakflow on its network links for traffic analysis and threat detection.
-- On imputation server itself, updates are run regularly by systems administrators who follow several zero-day computer security announcement lists; the OSSEC HIDS is used for log analysis and anomaly detection; and Denyhosts is used to thwart brute-force SSH login attacks.
+-	All stored data is encrypted at rest using FIPS 140-2 validated cryptographic software as well as encrypted in transit.
+-	Access controls follow principles of least privilege. All administrative access is secured via two-factor authentication using roll-based access controls and temporary credentials.
+-	Network access is restricted and filtered via web application firewalls, network access control lists, and security groups. Public/private network segmentation also ensures only the services that need to be are exposed to the public internet. All internal traffic and requests are logged and scanned for malicious or unusual activity.
+-	Advanced DDOS protection is in place to assure consistent site availability.
+-	All administrative user activities, system activities, and network traffic is logged and scanned for anomalies and malicious activity. Findings are alerted to administrative users.
 
 
 ## What encryption of the data is used while the data are present?

docs/getting-started.md

@@ -1,26 +1,34 @@
 # Getting started
 
-To use Michigan Imputation Server, a [registration](https://imputationserver.sph.umich.edu/index.html#!pages/register) is required.
+To use TOPMed Imputation Server, a [registration](https://imputation.biodatacatalyst.nhlbi.nih.gov/index.html#!pages/register) is required.
 We send an activation mail to the provided address. Please follow the instructions in the email to activate your account. If it doesn't arrive, ensure you have entered the correct email address and check your spam folder.
 
 **After the email address has been verified, the service can be used without any costs.**
 
-Please cite this paper if you use Michigan Imputation Server in your GWAS study:
+Please cite this paper if you use the Imputation Server in your GWAS study:
 
 > Das S, Forer L, Schönherr S, Sidore C, Locke AE, Kwong A, Vrieze S, Chew EY, Levy S, McGue M, Schlessinger D, Stambolian D, Loh PR, Iacono WG, Swaroop A, Scott LJ, Cucca F, Kronenberg F, Boehnke M, Abecasis GR, Fuchsberger C. [Next-generation genotype imputation service and methods](https://www.ncbi.nlm.nih.gov/pubmed/27571263). Nature Genetics 48, 1284–1287 (2016).
 
 
 ## Setup your first imputation job
 
-Please [login](https://imputationserver.sph.umich.edu/index.html#!pages/login) with your credentials and click on the **Run** tab to start a new imputation job. The submission dialog allows you to specify the properties of your imputation job.
+Please [login](https://imputation.biodatacatalyst.nhlbi.nih.gov/index.html#!pages/login) with your credentials and click on the **Run** tab to start a new imputation job. The submission dialog allows you to specify the properties of your imputation job.
 
 ![](images/submit-job01.png)
 
 The following options are available:
 
-### Reference Panel
+### Reference Panels
 
-Our server offers genotype imputation from different reference panels. The most accurate and largest panel is **HRC (Version r1.1 2016)**. Please select one that fulfills your needs and supports the population of your input data:
+#### TOPMed Imputation Server
+
+The TOPMed Imputation Server offers genotype imputation for the TOPMed reference panel, which is the largest and most accurate panel available amongst the two imputation servers.
+
+- TOPMed (Version r2 2020)
+
+#### Michigan Imputation Server
+
+The Michigan Imputation Server has several additional reference panels available.  Please select one that fulfills your needs and supports the population of your input data:
 
 - HRC (Version r1.1 2016)
 - HLA Imputation Panel: two-field (four-digit) and G-group resolution
@@ -29,6 +37,7 @@ Our server offers genotype imputation from different reference panels. The most
 - 1000 Genomes Phase 1 (Version 3)
 - CAAPA - African American Panel
 - HapMap 2
+- TOPMed Freeze5 (in preparation)
 
 More details about all available reference panels can be found [here](/reference-panels/).
 
@@ -89,9 +98,9 @@ Please select the population of your uploaded samples. This information is used
 
 |  Population | Supported Reference Panels |
 | ----------- | ---------------------------|
-| **AFR** | all |
-| **AMR** | all |
-| **EUR** | all |
+| **AFR** | all except TOPMed-r2 |
+| **AMR** | all except TOPMed-r2 |
+| **EUR** | all except TOPMed-r2 |
 | **Mixed** | all |
 | **AA** | CAAPA |
 | **ASN** | 1000 Genomes Phase 1 (Version 3) |

docs/index.md

@@ -1,11 +1,20 @@
+# TOPMed Imputation Server
+
+[TOPMed Imputation Server](https://imputation.biodatacatalyst.nhlbi.nih.gov) provides a free genotype imputation service using [Minimac4](http://genome.sph.umich.edu/wiki/Minimac4). You can upload phased or unphased GWAS genotypes and receive phased and imputed genomes in return. This server offers imputation from the [TOPMed](http://nhlbiwgs.org/) reference panel. For all uploaded datasets an extensive QC is performed.
+
+![](images/index-tm.png)
+
 # Michigan Imputation Server
 
+!!! note "ASHG2020 Workshop"
+    Click [here](/workshops/ASHG2020) for additional resources and tutorials. For questions: [mis-ashg2020@umich.edu](mailto:mis-ashg2020@umich.edu). To sign-up for the dedicated Slack-channel: [Slack sign-up](https://join.slack.com/t/eurac-workspace/shared_invite/zt-iqlxpl01-PwAxoTvlcXpDZo04ZKCBZQ)
+
 
 [Michigan Imputation Server](https://imputationserver.sph.umich.edu) provides a free genotype imputation service using [Minimac4](http://genome.sph.umich.edu/wiki/Minimac4). You can upload phased or unphased GWAS genotypes and receive phased and imputed genomes in return. Our server offers imputation from 1000 Genomes (Phase 1 and 3), CAAPA, [HRC](http://www.haplotype-reference-consortium.org/) and the [TOPMed](http://nhlbiwgs.org/) reference panel. For all uploaded datasets an extensive QC is performed.
 
 ![](images/index.png)
 
-Please cite this paper if you use Michigan Imputation Server in your publication:
+Please cite this paper if you use Imputation Server in your publication:
 
 > Das S, Forer L, Schönherr S, Sidore C, Locke AE, Kwong A, Vrieze S, Chew EY, Levy S, McGue M, Schlessinger D, Stambolian D, Loh PR, Iacono WG, Swaroop A, Scott LJ, Cucca F, Kronenberg F, Boehnke M, Abecasis GR, Fuchsberger C. [Next-generation genotype imputation service and methods](https://www.ncbi.nlm.nih.gov/pubmed/27571263). Nature Genetics 48, 1284–1287 (2016).