Skip to content

Commit f0c1cdd

Browse files
denis-yuendenis-yuen
authored
Feature/consonance cleanup (#128)
* Refactor container-admin docker image * Resolve work with develop * Consonance cleanup * cleanup creation of postgres user * use docker-compose v2 syntax * properly split out webservice and use non-superuser for client * Hook up Consonance user properly * Update readme * Support 'exit' and running again to re-enter * Update readme * Add bootstrap script in progress (needs json to properties file) * Continue work on bootstrap script and mustache templating * Updates * Fix variable references * Extend timeouts * Pin the version of mustache * the version that comes with ubuntu 14.04 will not work wget and install it manually if not on 16.04 * Start webservice before daemons * Add sudo and less for troubleshooting * Make sure container does not exit * Need to be able to restart daemons * Split out daemons to control order of start up * Need ansible 2.0 compatible playbook * Typo * Update README.md * Setup templates for ip address * Update README.md * Need versioned version of bamstats * Update README.md
1 parent 05ed182 commit f0c1cdd

18 files changed

+676
-141
lines changed

‎container-admin/Dockerfile

Lines changed: 0 additions & 57 deletions
This file was deleted.

‎container-admin/Dockerfile_client

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
FROM ubuntu:16.04
2+
3+
# Update the APT cache
4+
# prepare for Java download
5+
RUN apt-get update \
6+
&& apt-get upgrade -y \
7+
&& apt-get install -y \
8+
python-software-properties \
9+
software-properties-common \
10+
tree \
11+
vim \
12+
sudo \
13+
less \
14+
telnet \
15+
postgresql-client \
16+
&& apt-get clean
17+
18+
# grab oracle java (auto accept licence)
19+
RUN add-apt-repository -y ppa:webupd8team/java \
20+
&& apt-get update \
21+
&& echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections \
22+
&& apt-get install -y oracle-java8-installer
23+
24+
# install Consonance services
25+
ENV consonance_version=2.0-alpha.9
26+
27+
RUN wget --no-verbose https://seqwaremaven.oicr.on.ca/artifactory/seqware-release/io/consonance/consonance-arch/${consonance_version}/consonance-arch-${consonance_version}.jar
28+
29+
RUN useradd -ms /bin/bash ubuntu
30+
# the web and Consonance config
31+
WORKDIR /home/ubuntu
32+
33+
# install dockerize
34+
ENV DOCKERIZE_VERSION v0.2.0
35+
36+
RUN wget --no-verbose https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
37+
&& tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
38+
39+
40+
# TODO: make sure you create these from the .template files and customize them
41+
RUN mkdir -p /home/ubuntu/.youxia /home/ubuntu/.consonance /home/ubuntu/.consonance/self-installs /home/ubuntu/.ssh /home/ubuntu/bin
42+
COPY config /home/ubuntu/.consonance/config
43+
COPY key.pem /home/ubuntu/.ssh/key.pem
44+
COPY init_client.sh /home/ubuntu/init_client.sh
45+
46+
RUN chmod 600 /home/ubuntu/.ssh/key.pem
47+
RUN chmod a+wrx /home/ubuntu/init_client.sh
48+
RUN chown -R ubuntu /home/ubuntu
49+
USER ubuntu
50+
RUN echo "postgres:5432:*:*:postgres" > /home/ubuntu/.pgpass && chmod 600 /home/ubuntu/.pgpass
51+
52+
53+
# for youxia and the consonance command line on the main box
54+
RUN wget --no-verbose https://github.com/Consonance/consonance/releases/download/${consonance_version}/consonance && mv consonance /home/ubuntu/bin && chmod a+x /home/ubuntu/bin/consonance
55+
RUN wget --no-verbose --no-check-certificate http://seqwaremaven.oicr.on.ca/artifactory/seqware-release/io/consonance/consonance-client/${consonance_version}/consonance-client-${consonance_version}.jar && mv consonance-client-${consonance_version}.jar /home/ubuntu/.consonance/self-installs/
56+
57+
# now get a sample CWL and test JSON
58+
RUN wget --no-verbose https://raw.githubusercontent.com/CancerCollaboratory/dockstore-tool-bamstats/1.25-6_1.0/Dockstore.cwl
59+
RUN wget --no-verbose https://raw.githubusercontent.com/CancerCollaboratory/dockstore-tool-bamstats/1.25-6_1.0/sample_configs.json
60+
61+
ENV PATH="/home/ubuntu/bin:${PATH}"
62+
63+
CMD ["dockerize", "-wait", "tcp://webservice:8080", "-timeout", "60s", "/home/ubuntu/init_client.sh"]

‎container-admin/Dockerfile_coordinator

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
FROM ubuntu:16.04
2+
3+
# Update the APT cache
4+
# prepare for Java download
5+
RUN apt-get update \
6+
&& apt-get upgrade -y \
7+
&& apt-get install -y \
8+
python-software-properties \
9+
software-properties-common \
10+
telnet \
11+
&& apt-get clean
12+
13+
# grab oracle java (auto accept licence)
14+
RUN add-apt-repository -y ppa:webupd8team/java \
15+
&& apt-get update \
16+
&& echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections \
17+
&& apt-get install -y oracle-java8-installer
18+
19+
# install Consonance services
20+
ENV consonance_version=2.0-alpha.9
21+
22+
RUN wget --no-verbose https://seqwaremaven.oicr.on.ca/artifactory/seqware-release/io/consonance/consonance-arch/${consonance_version}/consonance-arch-${consonance_version}.jar
23+
24+
# install dockerize
25+
ENV DOCKERIZE_VERSION v0.2.0
26+
27+
RUN wget --no-verbose https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
28+
&& tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
29+
30+
COPY config .
31+
ADD init_coordinator.sh .
32+
33+
34+
RUN chmod u+x init_coordinator.sh
35+
COPY config /root/.consonance/config
36+
RUN mkdir /consonance_logs && chmod a+rx /consonance_logs
37+
38+
39+
# TODO: 1) update the above to have my AWS creds in it and 2) create the admin user in postgres db
40+
# Waiting for postgres and rabbitmq services
41+
CMD ["dockerize", "-wait", "tcp://webservice:8080", "-timeout", "60s", "./init_coordinator.sh"]

‎container-admin/Dockerfile_provisioner

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
FROM ubuntu:16.04
2+
3+
# Update the APT cache
4+
# prepare for Java download
5+
RUN apt-get update \
6+
&& apt-get upgrade -y \
7+
&& apt-get install -y \
8+
python-software-properties \
9+
software-properties-common \
10+
git \
11+
telnet \
12+
&& apt-get clean
13+
14+
RUN apt-get install software-properties-common \
15+
&& apt-add-repository ppa:ansible/ansible \
16+
&& apt-get update \
17+
&& apt-get install -y ansible
18+
19+
# grab oracle java (auto accept licence)
20+
RUN add-apt-repository -y ppa:webupd8team/java \
21+
&& apt-get update \
22+
&& echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections \
23+
&& apt-get install -y oracle-java8-installer
24+
25+
# install Consonance services
26+
ENV consonance_version=2.0-alpha.9
27+
28+
RUN wget --no-verbose https://seqwaremaven.oicr.on.ca/artifactory/seqware-release/io/consonance/consonance-arch/${consonance_version}/consonance-arch-${consonance_version}.jar
29+
30+
# install dockerize
31+
ENV DOCKERIZE_VERSION v0.2.0
32+
33+
RUN wget --no-verbose https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
34+
&& tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
35+
36+
# the web and Consonance config
37+
COPY config .
38+
ADD init_provisioner.sh .
39+
40+
RUN git clone https://github.com/ICGC-TCGA-PanCancer/container-host-bag.git /container-host-bag
41+
# we need the branch that is compatible with Ansible 2
42+
RUN (cd /container-host-bag && git checkout develop_2)
43+
44+
RUN chmod u+x init_provisioner.sh
45+
# TODO: make sure you create these from the .template files and customize them
46+
RUN mkdir -p /root/.youxia /root/.consonance /root/.consonance/self-installs /root/.ssh
47+
COPY youxia_config /root/.youxia/config
48+
COPY config /root/.consonance/config
49+
COPY key.pem /root/.ssh/key.pem
50+
RUN chmod 600 /root/.ssh/key.pem
51+
COPY aws.config /root/.aws/config
52+
RUN mkdir /consonance_logs && chmod a+rx /consonance_logs
53+
54+
# the Ansible-based setup
55+
COPY bag_params.json /container-host-bag/example_params.json
56+
COPY example_tags.json /container-host-bag/example_tags.json
57+
58+
# TODO: 1) update the above to have my AWS creds in it and 2) create the admin user in postgres db
59+
# Waiting for postgres and rabbitmq services
60+
CMD ["dockerize", "-wait", "tcp://webservice:8080", "-timeout", "60s", "./init_provisioner.sh"]

‎container-admin/Dockerfile_webservice

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
FROM ubuntu:16.04
2+
3+
# Update the APT cache
4+
# prepare for Java download
5+
RUN apt-get update \
6+
&& apt-get upgrade -y \
7+
&& apt-get install -y \
8+
python-software-properties \
9+
software-properties-common \
10+
telnet \
11+
&& apt-get clean
12+
13+
# grab oracle java (auto accept licence)
14+
RUN add-apt-repository -y ppa:webupd8team/java \
15+
&& apt-get update \
16+
&& echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections \
17+
&& apt-get install -y oracle-java8-installer
18+
19+
# install Consonance services
20+
ENV consonance_version=2.0-alpha.9
21+
22+
RUN wget --no-verbose https://seqwaremaven.oicr.on.ca/artifactory/seqware-release/io/consonance/consonance-webservice/${consonance_version}/consonance-webservice-${consonance_version}.jar
23+
24+
# install dockerize
25+
ENV DOCKERIZE_VERSION v0.2.0
26+
27+
RUN wget --no-verbose https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
28+
&& tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
29+
30+
# the web and Consonance config
31+
COPY config .
32+
COPY web.yml .
33+
ADD init_webservice.sh .
34+
35+
RUN chmod u+x init_webservice.sh
36+
# TODO: make sure you create these from the .template files and customize them
37+
COPY config /root/.consonance/config
38+
RUN mkdir /consonance_logs && chmod a+rx /consonance_logs
39+
40+
# TODO: 1) update the above to have my AWS creds in it and 2) create the admin user in postgres db
41+
# Waiting for postgres and rabbitmq services
42+
CMD ["dockerize", "-wait", "tcp://postgres:5432", "-wait", "tcp://rabbitmq:5672", "-timeout", "60s", "./init_webservice.sh"]

‎container-admin/README.md

Lines changed: 14 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
## Prereqs
22

3-
Install [docker-compose](https://docs.docker.com/compose/install/) on a Ubuntu 14.04+ VM and it's dependencies.
3+
Install [docker-compose](https://docs.docker.com/compose/install/) on a Ubuntu 16.04+ VM and it's dependencies.
44

55
**NOTE:** this isn't production ready, there is some manual config you need to do. Read all the directions below before running.
66

@@ -10,72 +10,35 @@ Install [docker-compose](https://docs.docker.com/compose/install/) on a Ubuntu 1
1010

1111
To run the webservice and command-line tools (still a work in progress, the ip address for swagger and other app level issues persist)
1212

13-
docker-compose build
14-
docker-compose up
13+
bash install_bootstrap
1514

16-
**NOTE:** this `docker-compose up` currently isn't working since you need to do some manual configuration before you can start the daemons. See the next section.
15+
You can exit and re-enter via:
16+
17+
exit
18+
docker-compose run client
19+
20+
**NOTE:** The Bash install_bootstrap script depends on Ubuntu 16.04 but sets up the templates required to run `docker-compose up` if needed.
1721

1822
## Developing
1923

20-
Create these files from templates:
24+
The following files are created from templates by the install script:
2125

2226
* youxia_config -> need to update aws key and various settings
2327
* config -> not much to do
2428
* key.pem -> need your AWS SSH key
2529
* aws.config -> need your AWS API keys
2630
* *the above files won’t be checked in due to the .gitignore policy*
2731

28-
Build with
29-
30-
docker-compose build
31-
32-
Start with
33-
34-
docker-compose run admin bash
35-
36-
Now, inside the admin container you just launched:
37-
38-
Start the webservice in the container
39-
40-
nohup java -jar consonance-webservice-*.jar server web.yml &> web.log &
41-
42-
You need to create the database, password is postgres (unless you changed it)
43-
44-
psql -h postgres -U postgres -W postgres
45-
> insert into consonance_user(user_id, admin, hashed_password, name) VALUES (1,true,'8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918','admin@admin.com');
46-
47-
Customize `/container-host-bag/example_params.json`, specifically, you need to add your aws keys. You might also want to customize `/container-host-bag/example_tags.json`.
48-
49-
Start the two daemons
50-
51-
nohup java -cp consonance-arch-*.jar io.consonance.arch.coordinator.Coordinator --config config --endless &> coordinator.log &
52-
nohup java -cp consonance-arch-*.jar io.consonance.arch.containerProvisioner.ContainerProvisionerThreads --config config --endless &> provisioner.log &
53-
5432
Now, you should have your webservice running on port 8080, you can monitor rabbitmq on port 15672.
5533

5634
You are now ready to submit some work (from within the admin docker container).
5735

58-
# get a sample CWL and JSON param file
59-
wget https://raw.githubusercontent.com/briandoconnor/dockstore-tool-bamstats/develop/Dockstore.cwl
60-
wget https://raw.githubusercontent.com/briandoconnor/dockstore-tool-bamstats/develop/sample_configs.json
61-
export CONSONANCE_ROOT=1
62-
consonance run --flavour m1.xlarge \
63-
--image-descriptor Dockstore.cwl \
64-
--run-descriptor sample_configs.json
65-
66-
# --extra-file /root/.aws/config=/home/ubuntu/.aws/config=false
67-
68-
69-
TODO: there's some sort of problem with the admin user getting wiped, need to disable in web.yml
70-
71-
TODO: I need a /root/.aws/config file after all -- DONE
72-
73-
TODO: chmod the key.pem -- DONE
74-
75-
TODO: note about security group ssh to itself
36+
consonance run --flavour m1.xlarge --image-descriptor Dockstore.cwl --run-descriptor sample_configs.json
37+
38+
Note that you will also need to configure your security group to allow for SSH access between nodes in the security group on public IP addresses.
7639

7740
TODO: how to get Youxia to launch m1.xlarge and have it attach all 4 ephemerial drives, only 1 is attached -- WORKED AROUND
7841

79-
TODO: the ecryptfs setup is using a hard-coded password for now, see lvm/tasks/mount.yml
42+
NOTE: We make the simplfying assumption that the ip address at eth0 of the launcher is reachable from the children. If it is different (i.e. a public ip address is preferred, modify sample_params.json in /container-host-bag in the provisioner container before launching jobs)
8043

81-
TODO: need to download the latest CLI and depenencies for Dockstore and then write an updated launcher that uses it instead of the CWL launcher from an older release
44+
Take a look at `/consonance_logs` for daemon and webservice logs in any container

‎container-admin/aws.config.template

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
[default]
2-
aws_secret_access_key = FILLMEIN
3-
aws_access_key_id = FILLMEIN
2+
aws_secret_access_key = {{ AWS_SECRET_ACCESS_KEY }}
3+
aws_access_key_id = {{ AWS_ACCESS_KEY_ID }}
44
region = us-east-1

‎container-admin/bag_params.json.template

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
11
{
22
"SENSU_SERVER_IP_ADDRESS": "localhost",
33
"FLEET_NAME": "fleet_name",
4-
"aws_key": "FOOBAR",
5-
"aws_secret_key": "FOOBAR",
4+
"aws_key": "{{ AWS_ACCESS_KEY_ID }}",
5+
"aws_secret_key": "{{ AWS_SECRET_ACCESS_KEY }}",
66
"seqware_version": "1.1.1",
77
"workflow_name": "HelloWorld",
88
"workflows": [
99
"Workflow_Bundle_HelloWorld_1.0-SNAPSHOT_SeqWare_1.1.0"
1010
],
1111
"install_workflow": "false",
1212
"test_workflows": "true",
13-
"queueHost": "localhost",
13+
"queueHost": "{{ LAUNCHER_IP_ADDRESS }}",
1414
"queueName": "consonance_arch",
1515
"queueUser": "guest",
1616
"queuePassword": "guest",
1717
"lvm_device_whitelist": "/dev/xvdb",
1818
"single_node_lvm": true,
19-
"consonance_arch_version": "2.0-alpha.6",
19+
"consonance_arch_version": "2.0-alpha.9",
2020
"seqware_engine": "whitestar",
2121
"seqware_use_custom_settings": false,
2222
"azure": false

0 commit comments

Comments
 (0)