htsget-axum

htsget-axum

Framework dependent code for a local instance of htsget-rs, using Axum.

Overview

This crate is used for running a server instance of htsget-rs. It is based on:

• Axum for endpoints, routes, and middleware.
• htsget-http for htsget-rs specific HTTP responses

Usage

For running htsget-rs as an application

This crate uses htsget-config for configuration. See htsget-config for details on how to configure this crate.

To run an instance of this crate, execute the following command:

cargo run -p htsget-axum

Using the default configuration, this will start a ticket server on 127.0.0.1:8080 and a data block server on 127.0.0.1:8081 with data accessible from the data directory. This application supports storage backends defined in htsget-storage.

To use S3Storage, compile with the s3-storage feature:

cargo run -p htsget-axum --features s3-storage

This will start a ticket server with S3Storage using a bucket called "data".

To use UrlStorage, compile with the url-storage feature.

See htsget-search for details on how to structure files.

Using TLS

There two server instances that are launched when running this crate. The ticket server, which returns a list of ticket URLs that a client must fetch. And the data block server, which responds to the URLs in the tickets. By default, the data block server runs without TLS. To run the data block server with TLS, pem formatted X.509 certificates are required.

For development and testing purposes, self-signed certificates can be used. For example, to generate self-signed certificates run:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes -subj '/CN=localhost'

It is not recommended to use self-signed certificates in a production environment as this is considered insecure.

Example requests

Using default configuration settings, this crate responds to queries referencing files in the data directory. Some example requests using curl are shown below:

• GET

curl '127.0.0.1:8080/variants/data/vcf/sample1-bcbio-cancer'

• POST

curl --header "Content-Type: application/json" -d '{}' '127.0.0.1:8080/variants/data/vcf/sample1-bcbio-cancer'

• Parametrised GET

curl '127.0.0.1:8080/variants/data/vcf/sample1-bcbio-cancer?format=VCF&class=header'

• Parametrised POST

curl --header "Content-Type: application/json" -d '{"format": "VCF", "regions": [{"referenceName": "chrM"}]}' '127.0.0.1:8080/variants/data/vcf/sample1-bcbio-cancer'

• Service info

curl '127.0.0.1:8080/variants/service-info'

Crypt4GH

The htsget-rs server experimentally supports serving Crypt4GH encrypted files to clients. See the Crypt4GH section in the configuration for more details on how to configure this.

Run the server with the following to enable Crypt4GH support using the example config:

cargo run -p htsget-axum --features experimental -- --config htsget-config/examples/config-files/c4gh.toml

Crypt4GH encrypted byte ranges can be queried:

curl 'http://localhost:8080/reads/data/c4gh/htsnexus_test_NA12878?referenceName=11&start=5000000&end=5050000'

The output consists of the Crypt4GH header, which includes the original header, the edit lists, and the re-encrypted header that the recipient can use to decrypt bytes:

{
  "htsget": {
    "format": "BAM",
    "urls": [
      {
        "url": "data:;base64,Y3J5cHQ0Z2gBAAAAAwAAAA=="
      },
      {
        "url": "http://127.0.0.1:8081/data/c4gh/htsnexus_test_NA12878.bam.c4gh",
        "headers": {
          "Range": "bytes=16-123"
        }
      },
      {
        "url": "data:;base64,bAAAAAAAAABPIoRdk+d+ifp2PWRFeXoe6Z9kPOj+HrREhzxZ3QiDa2SYh+0Gy8aKpFic4MtTa+ywMpkHziJgojVbcmbvBAr3G7o01lDubsBW98aQ/U1AcalIUCp0fGNkrtdTBN4NaVNIdtQmbAAAAAAAAABPIoRdk+d+ifp2PWRFeXoe6Z9kPOj+HrREhzxZ3QiDa+xJ+yh+52zHvw8qQXMyCtqT6jTFvaYhRPw/6ZzvOdt98YPQgCcTIut58VeTGmR3ien0TdcQFxmfE10MH4qapF2blgjX"
      },
      {
        "url": "http://127.0.0.1:8081/data/c4gh/htsnexus_test_NA12878.bam.c4gh",
        "headers": {
          "Range": "bytes=124-1114711"
        }
      },
      {
        "url": "http://127.0.0.1:8081/data/c4gh/htsnexus_test_NA12878.bam.c4gh",
        "headers": {
          "Range": "bytes=2557120-2598042"
        }
      }
    ]
  }
}                       

For example, using a htsget client, the data can be concatenated, and then decrypted using the Crypt4GH CLI:

htsget 'http://localhost:8080/reads/data/c4gh/htsnexus_test_NA12878?referenceName=11&start=5000000&end=5050000' > out.c4gh
crypt4gh decrypt --sk data/c4gh/keys/alice.sec < out.c4gh > out.bam
samtools view out.bam

As a library

This crates has some components which may be useful to other crates. Namely, in contains Axum routing functions for htsget-rs. It also contains the data block server which fetches data from a LocalStorage storage backend using htsget-storage.

Feature flags

This crate has the following features:

• s3-storage: used to enable S3Storage functionality.
• url-storage: used to enable UrlStorage functionality.
• experimental: used to enable experimental features that aren't necessarily part of the htsget spec, such as Crypt4GH support through C4GHStorage.

License

This project is licensed under the MIT license.