update: publish artifacts nightlies by kiran-thumma · Pull Request #148 · ROCm/FlyDSL

kiran-thumma · 2026-02-25T00:07:59Z

Motivation

Add AWS S3 publishing support to the FlyDSL CI workflow for distributing nightly wheel builds.

Technical Details

Added OIDC AWS credentials step, AWS CLI install, and aws s3 cp to publish .whl artifacts to s3://framework-whls-nightlies/flydsl/.
Added id-token: write permission for GitHub Actions OIDC federation with IAM role framework-flydsl-nightlies.
Added scripts/install_awscli.sh for portable AWS CLI installation on runners.

Test Plan

Verify workflow runs successfully on linux-flydsl-mi325-1 and linux-flydsl-mi355-1 runners and wheels are uploaded to S3.

Test Result

Pending first CI run after OIDC IAM role and S3 bucket are provisioned via Terraform.

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Copilot

Pull request overview

Adds AWS S3 publishing plumbing to the FlyDSL GitHub Actions CI workflow, aiming to distribute nightly wheel builds via OIDC-authenticated AWS credentials.

Changes:

Added job-level id-token: write permissions and steps to configure AWS credentials via OIDC.
Added an AWS CLI installation script intended for CI runners.
Added a (currently stubbed) step intended to publish wheel artifacts to an S3 bucket path.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
`scripts/install_awscli.sh`	New helper script to download/install AWS CLI v2 on Linux CI runners.
`.github/workflows/flydsl.yaml`	Adds OIDC permissions and AWS credential/config/publish steps to the FlyDSL CI workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T00:12:27Z

.github/workflows/flydsl.yaml

+      - name: Publish packages to S3
+        if: ${{ always() && !github.event.pull_request.head.repo.fork }}
+        run: |
+          # aws s3 cp ${{ github.workspace }}/dist/ s3://framework-whls-nightlies/flydsl/ --recursive --exclude "*" --include "*.whl"
+


The publish step is currently a no-op (the aws s3 cp command is commented out), so no wheels will be uploaded despite the PR goal. Also, the source path uses ${{ github.workspace }}/dist/, but this workflow checks out the repo into flydsl-test/, so wheel artifacts would typically be under ${{ github.workspace }}/flydsl-test/dist/ (and there is no step here that actually builds wheels into dist/). Please add/enable the wheel build step (e.g., run the existing wheel build script) and upload from the correct dist directory.

Copilot · 2026-02-25T00:12:28Z

.github/workflows/flydsl.yaml

+        if: ${{ always() && !github.event.pull_request.head.repo.fork }}
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          aws-region: us-east-1
+          role-to-assume: arn:aws:iam::661452401056:role/framework-flydsl-nightlies
+
+      - name: Install AWS CLI
+        if: always()
+        run: bash ./flydsl-test/scripts/install_awscli.sh
+
+      - name: Publish packages to S3
+        if: ${{ always() && !github.event.pull_request.head.repo.fork }}


always() here means AWS role assumption / publishing can run even when tests or builds fail, and it will also run on pull_request events for branches within the same repo (non-forks). For an artifact publishing role, it’s safer to gate this to successful runs on trusted events only (e.g., push to main and/or workflow_dispatch) to avoid uploading unreviewed PR artifacts or partial/failed outputs.

Suggested change

if: ${{ always() && !github.event.pull_request.head.repo.fork }}

uses: aws-actions/configure-aws-credentials@v4

with:

aws-region: us-east-1

role-to-assume: arn:aws:iam::661452401056:role/framework-flydsl-nightlies

- name: Install AWS CLI

if: always()

run: bash ./flydsl-test/scripts/install_awscli.sh

- name: Publish packages to S3

if: ${{ always() && !github.event.pull_request.head.repo.fork }}

if: ${{ success() && (github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref == 'refs/heads/main')) }}

uses: aws-actions/configure-aws-credentials@v4

with:

aws-region: us-east-1

role-to-assume: arn:aws:iam::661452401056:role/framework-flydsl-nightlies

- name: Install AWS CLI

if: ${{ success() && (github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref == 'refs/heads/main')) }}

run: bash ./flydsl-test/scripts/install_awscli.sh

- name: Publish packages to S3

if: ${{ success() && (github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref == 'refs/heads/main')) }}

Copilot · 2026-02-25T00:12:28Z

.github/workflows/flydsl.yaml

+        if: always()
+        run: bash ./flydsl-test/scripts/install_awscli.sh


This installs AWS CLI on the runner on every run (if: always()), even when publishing is skipped. On self-hosted runners this mutates the host environment across jobs and can introduce flaky behavior/version drift. Consider installing into a job-local directory (or using an action-provided AWS CLI) and gate the step to the same condition as publishing (and ideally only after wheel build succeeds).

Suggested change

if: always()

run: bash ./flydsl-test/scripts/install_awscli.sh

if: ${{ always() && !github.event.pull_request.head.repo.fork }}

uses: aws-actions/aws-cli-setup@v4

Copilot · 2026-02-25T00:12:28Z

scripts/install_awscli.sh

+curl --silent --fail --show-error --location \
+    "https://awscli.amazonaws.com/awscli-exe-linux-${ARCH}.zip" \
+    --output "awscliv2.zip"
+
+unzip -qq awscliv2.zip
+
+if [ "$EUID" -ne 0 ]; then
+  sudo ./aws/install --update
+else
+  ./aws/install --update
+fi


This script installs AWS CLI system-wide via ./aws/install --update and leaves the downloaded zip/extracted aws/ directory behind. On CI (especially self-hosted runners) that can cause cross-job side effects and workspace bloat. Consider installing into a temporary directory (e.g., under $RUNNER_TEMP) with a trap-based cleanup, and exposing it via PATH for the remainder of the job instead of updating the host installation.

coderfeli · 2026-02-25T11:24:40Z

@gyohuangxin help to take a look?

gyohuangxin · 2026-02-25T14:14:59Z

@kiran-thumma You mentioned this is for nightly wheel builds, but these changes will publish packages in every PR tests?

coderfeli · 2026-02-28T04:25:09Z

still failed @kiran-thumma

…ucket name

…elease_type

gyohuangxin

LGTM, just a minor question.

gyohuangxin · 2026-03-02T06:49:21Z

.github/workflows/flydsl.yaml

            echo "logs.tgz not found; skipping log extraction"
          fi

+      - name: Configure AWS credentials


Need we still need those steps in PR tests since we have a nightly one?

update: publish artifacts nightlies

69f3f63

Copilot AI review requested due to automatic review settings February 25, 2026 00:08

Copilot started reviewing on behalf of kiran-thumma February 25, 2026 00:09 View session

Copilot AI reviewed Feb 25, 2026

View reviewed changes

ramsagunner assigned sunway513 Feb 25, 2026

ramsagunner requested a review from sunway513 February 25, 2026 00:58

coderfeli requested a review from gyohuangxin February 25, 2026 11:24

kiran-thumma added 3 commits February 27, 2026 08:33

Add modular nightly CI: build, test (mi325+mi355), promote workflows

f614094

update build wheel

7e36ba8

fix: install python version

77bd3c7

kiran-thumma added 8 commits March 1, 2026 16:14

fix: extract and ship flir-opt binary to test containers

94a4423

update: aws command fix

e0dbcd0

update: temp nightlies as default

d9189c7

update fix promote

b7f6c94

fix: download/upload promote to avoid GetObjectTagging, use dynamic b…

6700bad

…ucket name

feat: wire generate_summary.py into build, test, and promote workflows

2152f51

update: schedule, workflow_disptach and fork protection

4cf4e41

add docker_image/llvm_commit dispatch inputs, dynamic OIDC role per r…

6bbf066

…elease_type

gyohuangxin approved these changes Mar 2, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update: publish artifacts nightlies#148

update: publish artifacts nightlies#148
kiran-thumma wants to merge 12 commits intomainfrom
kithumma/upload-packages-s3-nightlies

kiran-thumma commented Feb 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 25, 2026

Uh oh!

Copilot AI Feb 25, 2026

Uh oh!

Copilot AI Feb 25, 2026

Uh oh!

Copilot AI Feb 25, 2026

Uh oh!

coderfeli commented Feb 25, 2026

Uh oh!

gyohuangxin commented Feb 25, 2026

Uh oh!

coderfeli commented Feb 28, 2026

Uh oh!

gyohuangxin left a comment

Uh oh!

gyohuangxin Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		if: always()
		run: bash ./flydsl-test/scripts/install_awscli.sh

Conversation

kiran-thumma commented Feb 25, 2026

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

coderfeli commented Feb 25, 2026

Uh oh!

gyohuangxin commented Feb 25, 2026

Uh oh!

coderfeli commented Feb 28, 2026

Uh oh!

gyohuangxin left a comment

Choose a reason for hiding this comment

Uh oh!

gyohuangxin Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants