Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ICGC Bucket Names to SCORe in Docker-Compose and Makefile #849

Merged
merged 2 commits into from
Aug 22, 2024
Merged

Refactor ICGC Bucket Names to SCORe in Docker-Compose and Makefile #849

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9a28e29
Refactor ICGC Bucket Names to SCORe in Docker-Compose and Makefile
Azher2Ali Aug 21, 2024
e29722c
Renaming bucket_name_state to score.data
Azher2Ali Aug 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,18 +90,18 @@ _ping_song_db:


_setup-object-storage:
@echo $(YELLOW)$(INFO_HEADER) "Setting up bucket oicr.icgc.test and heliograph" $(END)
@if $(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 ls s3://oicr.icgc.test ; then \
@echo $(YELLOW)$(INFO_HEADER) "Setting up bucket score.data and heliograph" $(END)
@if $(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 ls s3://score.data ; then \
echo $(YELLOW)$(INFO_HEADER) "Bucket already exists. Skipping creation..." $(END); \
else \
$(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 mb s3://oicr.icgc.test; \
$(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 mb s3://score.data; \
fi
@$(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 cp /score-data/heliograph s3://oicr.icgc.test/data/heliograph
@$(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 cp /score-data/heliograph s3://score.data/data/heliograph

_destroy-object-storage:
@echo $(YELLOW)$(INFO_HEADER) "Removing bucket oicr.icgc.test" $(END)
@if $(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 ls s3://oicr.icgc.test ; then \
$(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 rb s3://oicr.icgc.test --force; \
@echo $(YELLOW)$(INFO_HEADER) "Removing bucket score.data" $(END)
@if $(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 ls s3://score.data ; then \
$(DOCKER_COMPOSE_CMD) run aws-cli --endpoint-url http://object-storage:9000 s3 rb s3://score.data --force; \
else \
echo $(YELLOW)$(INFO_HEADER) "Bucket does not exist. Skipping..." $(END); \
fi
Expand Down
4 changes: 2 additions & 2 deletions dev/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ services:
AUTH_SERVER_UPLOADSCOPE: song.upload
SERVER_URL: http://localhost:8087
SERVER_PORT: 8087
BUCKET_NAME_OBJECT: oicr.icgc.test
BUCKET_NAME_STATE: oicr.icgc.test
BUCKET_NAME_OBJECT: score.data
BUCKET_NAME_STATE: score.test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rename this to score.state?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like in score we made this score.test, missed this detail earlier. I'll update that to make consistent.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cancel the above notes... There was a reason they used the same bucket name for both of these buckets... there is mounted directory for the test data and we renamed the bucket to score.data, we would either need to create a directory for the state bucket or reuse the data bucket name.

Therefore, I think its easiest to keep both bucket names the same for the local docker compose setup. Let's update this to BUCKET_NAME_STATE: score.data.

COLLABORATORY_DATA_DIRECTORY: data
OBJECT_SENTINEL: heliograph
METADATA_URL: "http://localhost:8080"
Expand Down
6 changes: 3 additions & 3 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ services:
SPRING_PROFILES_ACTIVE: amazon,collaboratory,prod,secure
SERVER_PORT: 8080
OBJECT_SENTINEL: heliograph
BUCKET_NAME_OBJECT: oicr.icgc.test
BUCKET_NAME_STATE: oicr.icgc.test
BUCKET_NAME_OBJECT: score.data
BUCKET_NAME_STATE: score.test
COLLABORATORY_DATA_DIRECTORY: data
METADATA_URL: http://song-server:8080
S3_ENDPOINT: http://object-storage:9000
Expand Down Expand Up @@ -129,7 +129,7 @@ services:
AWS_SECRET_ACCESS_KEY: minio123
AWS_DEFAULT_REGION: us-east-1
volumes:
- "./docker/object-storage-init/data/oicr.icgc.test/data:/score-data:ro"
- "./docker/object-storage-init/data/score.data/data:/score-data:ro"

song-client:
build:
Expand Down
0 ...-init/data/oicr.icgc.test/data/heliograph → ...rage-init/data/score.data/data/heliograph
View file
Open in desktop
File renamed without changes.