Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve dependency installation on cloud #56

Open
aryarm opened this issue Aug 28, 2020 · 0 comments
Open

improve dependency installation on cloud #56

aryarm opened this issue Aug 28, 2020 · 0 comments
Labels
enhancement New feature or request gcp

Comments

@aryarm
Copy link
Owner

aryarm commented Aug 28, 2020

the google life sciences api creates a new snakemake container to run each of our jobs
so our dependencies must be installed anew by snakemake every time it runs a job, since each job is run in a new, empty container
this process may add substantially to the runtime of some of our instances and the cost of using them

we have two options:

  1. separate our dependencies into different env files to install only the necessary dependencies for each job
  2. create a custom snakemake container with our dependencies already installed in it and force snakemake to use our container instead of its own
@aryarm aryarm added enhancement New feature or request gcp labels Aug 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request gcp
Projects
None yet
Development

No branches or pull requests

1 participant