Auxiliary Data Import

These instructions provide examples of importing certain auxiliary annotation data used the GMS, such as reference genomes, feature sets, and transcript annotations.

These instructions assume that you have followed the installation instructions up to and including the prime-system.pl command. If do not require the demonstration dataset, it is possible to prime the system without downloading the demonstration data by using the --data=none option to prime-system.pl.

Importing a New Human Reference Genome

A new human reference genome can be imported by defining a new imported-reference-sequence model using the genome model define imported-reference-sequence command. Several parameters must be specified when defining this model:

• the file system path of the reference fasta file
• the processing profile ID (typically 1990904)
• the name of the reference's species (eg, human)
• the reference version
• a prefix which classifies the source of the reference
• a name for the assembly of the reference (typically includes the prefix and version)
• a build name which acts as a local name for the reference
• a URL which identifies the original source of the reference

To import the soft-masked primary assembly from Ensembl, first download the compressed reference fasta from Ensembl's ftp server. After downloading the compressed reference fasta, calculate the check sum and compare the result to the checksum result in Ensembl's CHECKSUMS file (optional, but recommended). Decompress the downloaded file using the gunzip utility. Then run the genome model define imported-reference-sequence. Defining the imported reference sequence model automatically starts a build of this model to import the reference.

$ URI='ftp://ftp.ensembl.org/pub/release-76/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz'
$ wget $URI
$ sum Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz
$ gunzip Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz
$ genome model define imported-reference-sequence                    \
  --fasta-file=$PWD/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa   \
  --processing-profile-id=1990904                                    \
  --species-name=human                                               \
  --version=38                                                       \
  --prefix=GRC                                                       \
  --assembly-name=GRCh38-ensembl-primary                             \
  --build-name=GRCh38-ensembl-primary                                \
  --sequence-uri=$URI

Creating a Modified Reference based on an Existing Reference Genome

An existing reference genome may be used as the basis for a new reference genome which is the combination both the existing reference and new fasta file. In order to create the modified reference, the build id of the existing reference and the path to the new fasta file are required.

The existing reference genomes may be listed to show the existing reference genomes along with their build IDs:

$ genome model build list --filter="model.type_name='imported reference sequence'" 

Once the build ID of the existing reference and the path to the new fasta are both known, a new model can be defined. Defining this model automatically starts a build of this model, so there is no need to separately start a build.

The following command is an example of appending the sequences given in a file named ERCC92.fa to the GRCh37-lite reference.

$ genome model define imported-reference-sequence  \
  --append-to=106942997                            \
  --fasta-file=/home/ubuntu/ERCC92.fa              \
  --use-default-sequence-uri                       \
  --species-name=human                             \
  --version=37_ERCC

Importing a Variation List

import-dbsnp-build

A list of variants may be imported into the GMS from outside sources using genome model imported-variation-list. You may import variants directly from dbSNP with the import-dbsnp-build sub-command. To import a variation list, the build id for a reference sequence using the same coordinates as the variation list must be supplied.

Notice the use of http:// at the beginning of the vcf file url. The ftp protocol is not supported.

$ genome model imported-variation-list import-dbsnp-build \
  --version 141 --reference-sequence-build 106942997      \
  --vcf-file-url 'http://ftp.ncbi.nih.gov/snp/organisms/human_9606_b141_GRCh37p13/VCF/00-All.vcf.gz'

A copy of dbSNP vesion 141 for the GRCh37-lite reference is imported into GMS during prime-system.pl and assigned the build ID 127786607.

import-variants

Other variants in vcf or bed format may be imported with the import-variants sub-command.

$ genome model imported-variation-list import-variants  \
  --format vcf                                          \
  --input-path $PWD/clinvar_20140807.sorted.vcf         \
  --source dbsnp-clinvar                                \
  --version 20140807                                    \
  --description 'dbsnp clinvar 20140807'                \
  --reference-sequence-build 106942997                  \
  --variant-type snv

Importing a New Version of Ensembl

Ensembl transcript annotations can be imported for rna-seq analysis or for annotating variants. Depending on the type of analysis you need to have performed, the import procedure is different.

Rna-seq analysis

$ genome model define imported-annotation                         \
  --reference-sequence-build=0bfc1bd5fcfc474c9db737a520ae109d     \
  --species-name=human                                            \
  --version=74_37ercc                                             \
  --build-name=NCBI-human.ensembl/74_37ercc                       \
  --processing-profile=2725293                                    \
  --gtf-file=all_sequences.gtf                                    \
  --annotation-import-version=2

Annotating variants