Launch Gen3 using Helm/K8s

[lbeckman314]

Gen3-Helm Deployment Instructions: https://github.com/uc-cdis/gen3-helm

[lbeckman314]

• Deploy on https://aced-training.compbio.ohsu.edu
• Successfully log in as Mock user (test@example.com)
• Add Google and Microsoft logins
• Fix /data/schema 404 error
• Add data upload/ETL workflow
  • Mount volumes in ETL pod
    • S3:// data (e.g. aced.json)
    • Sheepdog credentials (e.g. sheepdog-credentials.yaml)
    • User config (e.g. user.yaml)
    • User credentials (e.g. fence.yaml)
  • Add service connections in ETL pod
    • Arborist
    • PostgreSQL
    • Elasticsearch
  • Build and push ETL image to quay.io
  • Add ETL upload script with Gen3 SDK
  • Connect OHSU Minio bicket
  • Connect AWS Minio buckets
• Add RDS and connect to Helm
• Add OpenSearch and connect to Helm

---

ETL:

• Map Fence/ETL Bot AWS Access Keys to ETL pod
• Map Fence.ALLOWED_DATA_UPLOAD_BUCKETS to ETL pod
• Create an ETL Bot IAM user modeled after Fence Bot in AWS (different policies?)
• How best to separate development/staging environments for ETL pod?
  • Separate etl.yaml files with different configmaps?
  • Switch or variable to tell Helm to use a particular environment
  • Current differences center on studies/output location and NodeSelector

[tbaxter73]

meeting with Jawad(UC) to talk about issues loading images

[lbeckman314]

Tracking PR in gen3-helm: calypr/gen3-helm#2