Automating image tag update for admin dashboard #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bumped up base server image tag
Bump up base server image tag
Added working code from join repo to fetch docker image tags using artifact download.
Combining both artifact and matrix methods since both workflow is triggered by both push event and workflow_dispatch event. Workflow dispatch event receives tag via input parameter sent via server workflow. Push event does not receive any tag and the DOCKER_IMAGE_TAG_2 would have empty value since workflow triggering event and hence input parameter would be empty. So, was facing the issue of having empty timestamp being suffixed to the docker image tag in the Dockerfiles. Hence, using the logic of fetching the latest run id and then downloading the artifact uploaded by server workflow to ensure that a non-empty value is also retrieved for the timestamp. This value is stored in DOCKER_IMAGE_TAG_1 and can be used for building docker image. Now, depending on the trigger event, the appropriate docker image tag can be used in the docker build command for the --build-arg flag. Additionally, Dockerfiles now use ARG to use the tags passed from the docker build command, hence using environment variables. Docker-compose files similarly have the args config parameter set. Developers would have to set the correct server image tags manually here for the docker-compose command to pass the value to the ARG in the Dockerfile and correctly set the image tag to point to the appropriate server image. Need to mention somewhere in the ReadME.md to refer to the server image list in Dockerhub to choose the latest image tag. While pushing the image in the GitHub actions, it'll be done manually. Todo: Change branch name to tags-combo-approach in fetch_runID.py
Was unable to see docker_image_tag in echo statement in logs. Added docker_image_tag print statement to see retrieved image tag. Ah! Was using run_id instead of docker_image_tag as the output value in outputs section in fetch_tag job.
Still not seeing the env vars echo-ed. Debugging by printing various log statements.
Still not seeing the env vars echo-ed. Debugging by printing various log statements.
Working finally!
Changed the deprecated set-output command to use {key}={value} format with echo command.
Also, adding commented out build and push commands to see if build, push is successful.
For this commit, not changing branch = tags-artifact in fetch_runID.py
Once, server code has latest working yml file, then will come back and push another commit to change branch to tags-combo-approach.
Removed an extra echo statement.
Updating Dockerfiles to use ARG environment variable with latest timestamp that will be passed through: - `docker build --build-arg` command in Github actions in the workflow for automated pushing to Docker hub. - `args: ` config field in docker-compose which will need to be set manually by developers locally. Also, changing branch in fetch_runID and Dockerfiles to tags-combo-approach.
For public-dash, admin-dash where ARGS are now being used, need to add the args under build command in the docker compose files. Gives error if arg is at the same hierarchical level as build. Also, public-dash docker-compose.yml (non-dev) version changed to have build: context, dockerfile ; unlike only build: frontend. This allows adding args under build. Similar to how currently being built in docker-compose.dev.yml. Also, args to be added under notebook-server and not dashboard since viz_scripts builds off of server image and not frontend, which is a node image.
Adding .env file which stores only docker image timestamp for the latest dockerhub e-mission-server image already pushed. .env file overwritten in both types of trigger events - push and workflow_dispatch. Added commit and push github actions as well for pushing latest changes to the .env file made via the workflow. Lastly, docker-compose now also mentions the ENV variable name to be read from the .env file for the ARG value in the Dockerfile. No changes required in the Dockerfiles.
Reminder for things to change as per master branch of e-mission-server once changes are finalized.
Added another TODO. evious Push event triggers run failed Error occurred in GitHub actions git add, commit, push step. If file with no changes operated upon, it leaves an error: “nothing to commit, working tree clean Error: Process completed with exit code 1.” Need to fix. —— Quick fix is to make changes to .env file only if workflow_dispatch event is the trigger. Don’t do anything for push event. So, in case anyone modifies .env file on their own by using their own timestamp during testing, and pushes it as a part of their PR, then Shankari will have to ask them to revert the changes. Else, their custom timestamp will make it to the repo code base. Found something: https://www.reddit.com/r/github/comments/ju3ipr/commit_from_github_action_only_when_changes_exist/ It should work but there’s a drawback of using “exit 0” - it will mask all errors generated during “git commit”. This is bad and we won’t be able to see the reason why something wrong happened as the workflow would be shown as successful with a green tick. Found a solution with git diff: https://github.com/simonw/til/blob/main/github-actions/commit-if-file-changed.md $ git diff --quiet || (git add README.md && git commit -m "Updated README") However, I won’t be able to log any message saying that no changes to commit, tag not modified. Hence, will possibly use just “git diff —quiet” with an if-else block. Expected results: - Push event triggers workflow. - It writes DOCKER_IMAGE_TAG_1 fetched from last successful completed run to .env file. - It sees that there is a difference in the latest committed .env file in the dashboard repo which includes older timestamp. - Hence it runs git commit part of the script to reset to latest server timestamp.
Reverting the Dockerfile tag for testing so that checks can run properly
Also editing this file so that checks pass for now
Changing this to build from actual e-mission server in hopes of merging!
Potentially finalizing the dockerfile in the docker repo.
|
Removed conflicts and started to make some cleanup changes so it's prepared to merge into actual emission admin dash. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR includes @MukuFlash03 work for finalizing the docker image build tag auto updating upon running the image-build-push.yml file in the server directory.