Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dummy to run workflow #1542

Closed
wants to merge 39 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f0ae89e
new workflow that accepts agent s3 path
dricross Feb 5, 2025
64287df
Add on: pull_request to trigger the workflow once
dricross Feb 5, 2025
bb08026
Fix output variable
dricross Feb 5, 2025
bc3132c
Create vars for windows and mac
dricross Feb 5, 2025
94b3e93
Move test repo branch to input argument
dricross Feb 5, 2025
c0ba607
Move checkout after determinig repo branch
dricross Feb 5, 2025
4dfad15
Fix default value for test repo branch
dricross Feb 5, 2025
4d2c711
Alphabetize the arguments
dricross Feb 5, 2025
5828cfe
Enable EC2 Linux integration tests
dricross Feb 5, 2025
b8c14bc
Fix terraform destroy command
dricross Feb 5, 2025
2967523
Enable more tests
dricross Feb 6, 2025
249d1e1
use build release ID inplace of github sha
dricross Feb 7, 2025
60b6000
temporarily remove onprem from stoplocalstack
dricross Feb 7, 2025
a42385c
Remove most of the workflow. Download release artifacts
dricross Feb 7, 2025
a478176
Add credentials
dricross Feb 7, 2025
ec66167
Add checkout
dricross Feb 7, 2025
0f36639
Add permissions
dricross Feb 7, 2025
f6a371c
download zipfile
dricross Feb 7, 2025
5a7b560
Download and re-upload artifacts
dricross Feb 7, 2025
57f9e89
show no progress on aws s3 cp
dricross Feb 7, 2025
0d5c55f
Re-add checkout/go setup for generating matrix
dricross Feb 7, 2025
d7f1465
Re-add matrix generation
dricross Feb 7, 2025
b31470c
Fix path for arm
dricross Feb 7, 2025
f676c1f
Fix agent path in cftest
dricross Feb 7, 2025
4fc6de7
workflow updates
dricross Feb 7, 2025
7534f27
Upload to cn and itar
dricross Feb 10, 2025
532e939
Fix regions for credentials
dricross Feb 10, 2025
07c73eb
reupload agent image
dricross Feb 10, 2025
8aa8c7b
trigger on pull request
dricross Feb 10, 2025
07dbf70
Push the image
dricross Feb 10, 2025
a0bad66
Rename the release candidate file so we can run the workflow
dricross Feb 10, 2025
6bb6a55
Remove on pull_request
dricross Feb 10, 2025
e519c1c
Fix variable references
dricross Feb 10, 2025
262ffe3
rework variables
dricross Feb 10, 2025
5a91e5c
try to fix registry reference
dricross Feb 10, 2025
6549df7
fix docker commands
dricross Feb 10, 2025
70798d1
Fix integ test registry
dricross Feb 10, 2025
0806241
call test-artifacts
dricross Feb 10, 2025
5b9e76d
dummy commit to create the workflow
dricross Feb 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 16 additions & 14 deletions .github/workflows/ec2-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ env:
on:
workflow_call:
inputs:
github_sha:
build_id:
required: true
type: string
test_repo_name:
Expand Down Expand Up @@ -92,26 +92,27 @@ jobs:

terraform init
if terraform apply --auto-approve \
-var="ssh_key_value=${{env.PRIVATE_KEY}}" \
-var="github_test_repo=${{ inputs.test_repo_url }}" \
-var="test_name=${{ matrix.arrays.os }}" \
-var="cwa_github_sha=${{inputs.github_sha}}" \
-var="install_agent=${{ matrix.arrays.installAgentCommand }}" \
-var="github_test_repo_branch=${{inputs.test_repo_branch}}" \
-var="ec2_instance_type=${{ matrix.arrays.instanceType }}" \
-var="user=${{ matrix.arrays.username }}" \
-var="agent_start=${{ matrix.arrays.agentStartCommand }}" \
-var="ami=${{ matrix.arrays.ami }}" \
-var="ca_cert_path=${{ matrix.arrays.caCertPath }}" \
-var="arc=${{ matrix.arrays.arc }}" \
-var="binary_name=${{ matrix.arrays.binaryName }}" \
-var="ca_cert_path=${{ matrix.arrays.caCertPath }}" \
-var="cwa_github_sha=${{inputs.build_id}}" \
-var="ec2_instance_type=${{ matrix.arrays.instanceType }}" \
-var="excluded_tests='${{ matrix.arrays.excludedTests }}'" \
-var="github_test_repo=${{ inputs.test_repo_url }}" \
-var="github_test_repo_branch=${{inputs.test_repo_branch}}" \
-var="install_agent=${{ matrix.arrays.installAgentCommand }}" \
-var="local_stack_host_name=${{ inputs.localstack_host }}" \
-var="plugin_tests='${{ github.event.inputs.plugins }}'" \
-var="region=${{ inputs.region }}" \
-var="s3_bucket=${{ inputs.s3_integration_bucket }}" \
-var="plugin_tests='${{ github.event.inputs.plugins }}'" \
-var="excluded_tests='${{ matrix.arrays.excludedTests }}'" \
-var="ssh_key_name=${{env.KEY_NAME}}" \
-var="ssh_key_value=${{env.PRIVATE_KEY}}" \
-var="test_dir=${{ matrix.arrays.test_dir }}" \
-var="agent_start=${{ matrix.arrays.agentStartCommand }}"; then terraform destroy -var="region=${{ inputs.region }}" -var="ami=${{ matrix.arrays.ami }}" -auto-approve
-var="test_name=${{ matrix.arrays.os }}" \
-var="user=${{ matrix.arrays.username }}"; then
terraform destroy -var="region=${{ inputs.region }}" -var="ami=${{ matrix.arrays.ami }}" -auto-approve
else
terraform destroy -var="region=${{ inputs.region }}" -var="ami=${{ matrix.arrays.ami }}" -auto-approve && exit 1
fi
Expand All @@ -123,4 +124,5 @@ jobs:
max_attempts: 2
timeout_minutes: 8
retry_wait_seconds: 5
command: cd ${{ inputs.test_dir }} && terraform destroy -var="region=${{ inputs.region }}" -var="ami=${{ matrix.arrays.ami }}" --auto-approve
command: |
cd ${{ inputs.test_dir }} && terraform destroy -var="region=${{ inputs.region }}" -var="ami=${{ matrix.arrays.ami }}" --auto-approve
6 changes: 3 additions & 3 deletions .github/workflows/integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ on:
description: 'The ID of the build-test-artifacts workflow run'
type: number
required: true
build_sha:
build_id:
description: 'The SHA of the build-test-artifacts workflow run'
type: string
required: true
Expand All @@ -40,7 +40,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- run: |
if [[ ${{ inputs.build_sha }} == ${{ github.sha }} ]]; then
if [[ ${{ inputs.build_id }} == ${{ github.sha }} ]]; then
echo "Build SHA matches test SHA"
else
echo "Build SHA does not match test SHA"
Expand Down Expand Up @@ -157,7 +157,7 @@ jobs:
- name: Test cf
run: |
cd test/test/cloudformation
go test -timeout 1h -package_path=s3://${S3_INTEGRATION_BUCKET}/integration-test/binary/${{ github.sha }}/linux/amd64/amazon-cloudwatch-agent.rpm -iam_role=${CF_IAM_ROLE} -key_name=${CF_KEY_NAME} -metric_name=mem_used_percent
go test -timeout 1h -package_path=s3://${S3_INTEGRATION_BUCKET}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.rpm -iam_role=${CF_IAM_ROLE} -key_name=${CF_KEY_NAME} -metric_name=mem_used_percent

StartLocalStack:
name: 'StartLocalStack'
Expand Down
61 changes: 61 additions & 0 deletions .github/workflows/release-candidate-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT

name: Test Release Candidate
env:
TERRAFORM_AWS_ASSUME_ROLE: ${{ vars.TERRAFORM_AWS_ASSUME_ROLE }}
TERRAFORM_AWS_ASSUME_ROLE_DURATION: 14400 # 4 hours
S3_INTEGRATION_BUCKET: ${{ vars.S3_INTEGRATION_BUCKET }}
S3_RELEASE_BUCKET: amazon-cloud-watch-agent
S3_RELEASE_REPO: cloudwatch-agent
CWA_GITHUB_TEST_REPO_BRANCH: "main"
TERRAFORM_AWS_ASSUME_ROLE_ITAR: ${{ vars.TERRAFORM_AWS_ASSUME_ROLE_ITAR }}
S3_INTEGRATION_BUCKET_ITAR: ${{ vars.S3_INTEGRATION_BUCKET_ITAR }}
TERRAFORM_AWS_ASSUME_ROLE_CN: ${{ vars.TERRAFORM_AWS_ASSUME_ROLE_CN }}
S3_INTEGRATION_BUCKET_CN: ${{ vars.S3_INTEGRATION_BUCKET_CN }}

on:
workflow_dispatch:
inputs:
build_id:
description: 'The build ID (release candidate build number or GitHub commit SHA)'
type: string
required: true
test_repo_branch:
description: 'Override for the test repo branch (default is main)'
type: string

jobs:
OutputEnvVariables:
name: 'OutputEnvVariables'
runs-on: ubuntu-latest
outputs:
CWA_GITHUB_TEST_REPO_BRANCH: ${{ steps.set-outputs.outputs.CWA_GITHUB_TEST_REPO_BRANCH }}
steps:
- name: SetOutputs
id: set-outputs
run: |
echo "CWA_GITHUB_TEST_REPO_BRANCH=${CWA_GITHUB_TEST_REPO_BRANCH:-${{ env.CWA_GITHUB_TEST_REPO_BRANCH }}}" >> "$GITHUB_OUTPUT"

- name: Echo test variables
run: |
echo "CWA_GITHUB_TEST_REPO_BRANCH: ${{ steps.set-outputs.outputs.CWA_GITHUB_TEST_REPO_BRANCH }}"

RepackageArtifacts:
name: 'RepackageArtifacts'
uses: ./.github/workflows/repackage-release-artifacts.yml
secrets: inherit
permissions:
id-token: write
contents: read
with:
build_id: ${{ inputs.build_id }}

StartIntegrationTests:
needs: [ RepackageArtifacts, OutputEnvVariables ]
runs-on: ubuntu-latest
steps:
# Avoid the limit of 5 nested workflows by executing the workflow in this manner
- run: gh workflow run test-artifacts.yml --ref ${{ github.ref_name }} --repo $GITHUB_REPOSITORY -f build_id=${{ inputs.build_id }} -f test_repo_branch=${{ needs.OutputEnvVariables.outputs.CWA_GITHUB_TEST_REPO_BRANCH }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
171 changes: 171 additions & 0 deletions .github/workflows/repackage-release-artifacts.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT

name: Repackage Release Artifacts
env:
TERRAFORM_AWS_ASSUME_ROLE: ${{ vars.TERRAFORM_AWS_ASSUME_ROLE }}
TERRAFORM_AWS_ASSUME_ROLE_DURATION: 14400 # 4 hours
S3_INTEGRATION_BUCKET: ${{ vars.S3_INTEGRATION_BUCKET }}
S3_RELEASE_BUCKET: amazon-cloud-watch-agent
S3_RELEASE_REPO: cloudwatch-agent
TERRAFORM_AWS_ASSUME_ROLE_ITAR: ${{ vars.TERRAFORM_AWS_ASSUME_ROLE_ITAR }}
S3_INTEGRATION_BUCKET_ITAR: ${{ vars.S3_INTEGRATION_BUCKET_ITAR }}
TERRAFORM_AWS_ASSUME_ROLE_CN: ${{ vars.TERRAFORM_AWS_ASSUME_ROLE_CN }}
S3_INTEGRATION_BUCKET_CN: ${{ vars.S3_INTEGRATION_BUCKET_CN }}

on:
workflow_call:
inputs:
build_id:
description: 'The build ID (release candidate build number or GitHub commit SHA)'
type: string
required: true

jobs:
RepackageS3Artifcats:
name: 'RepackageS3Artifcats'
runs-on: ubuntu-latest
strategy:
fail-fast: false
permissions:
id-token: write
contents: read
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE }}
aws-region: us-west-2
role-duration-seconds: ${{ env.TERRAFORM_AWS_ASSUME_ROLE_DURATION }}

- name: Download Artifacts
id: download-artifacts
run: |
mkdir -p windows/amd64/
aws s3 cp --no-progress s3://${{ env.S3_RELEASE_BUCKET }}/windows/amd64/${{ inputs.build_id }}/AmazonCloudWatchAgent.zip ./windows/amd64/

mkdir -p linux/amd64
aws s3 cp --no-progress s3://${{ env.S3_RELEASE_BUCKET }}/linux/amd64/${{ inputs.build_id }}/AmazonCloudWatchAgent.zip ./linux/amd64

mkdir -p linux/arm64
aws s3 cp --no-progress s3://${{ env.S3_RELEASE_BUCKET }}/linux/arm64/${{ inputs.build_id }}/AmazonCloudWatchAgent.zip ./linux/arm64

mkdir -p darwin/amd64
aws s3 cp --no-progress s3://${{ env.S3_RELEASE_BUCKET }}/darwin/amd64/${{ inputs.build_id }}/AmazonCloudWatchAgent.zip ./darwin/amd64

mkdir -p darwin/arm64
aws s3 cp --no-progress s3://${{ env.S3_RELEASE_BUCKET }}/darwin/arm64/${{ inputs.build_id }}/AmazonCloudWatchAgent.zip ./darwin/arm64

- name: Re-upload Artifacts
id: upload-artifacts
run: |
pushd windows/amd64/
unzip AmazonCloudWatchAgent.zip
aws s3 cp --no-progress ./amazon-cloudwatch-agent.msi s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/packaging/${{ inputs.build_id }}/amazon-cloudwatch-agent.msi
popd

pushd linux/amd64
unzip AmazonCloudWatchAgent.zip
aws s3 cp --no-progress ./amazon-cloudwatch-agent.rpm s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.rpm
aws s3 cp --no-progress ./amazon-cloudwatch-agent.deb s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.deb
popd

pushd linux/arm64
unzip AmazonCloudWatchAgent.zip
aws s3 cp --no-progress ./amazon-cloudwatch-agent.rpm s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/binary/${{ inputs.build_id }}/linux/arm64/amazon-cloudwatch-agent.rpm
aws s3 cp --no-progress ./amazon-cloudwatch-agent.deb s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/binary/${{ inputs.build_id }}/linux/arm64/amazon-cloudwatch-agent.deb
popd

pushd darwin/amd64
unzip AmazonCloudWatchAgent.zip
aws s3 cp --no-progress ./amazon-cloudwatch-agent.pkg s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/packaging/${{ inputs.build_id }}/arm64/amazon-cloudwatch-agent.pkg
popd

pushd darwin/arm64
unzip AmazonCloudWatchAgent.zip
aws s3 cp --no-progress ./amazon-cloudwatch-agent.pkg s3://${{ env.S3_INTEGRATION_BUCKET }}/integration-test/packaging/${{ inputs.build_id }}/arm64/amazon-cloudwatch-agent.pkg
popd

- name: Configure AWS Credentials (CN)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE_CN }}
aws-region: cn-north-1
role-duration-seconds: ${{ env.TERRAFORM_AWS_ASSUME_ROLE_DURATION }}

- name: Re-upload Artifacts (CN)
id: upload-artifacts-cn
run: |
pushd linux/amd64
aws --region cn-north-1 s3 cp --no-progress ./amazon-cloudwatch-agent.rpm s3://${{ env.S3_INTEGRATION_BUCKET_CN }}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.rpm
aws --region cn-north-1 s3 cp --no-progress ./amazon-cloudwatch-agent.deb s3://${{ env.S3_INTEGRATION_BUCKET_CN }}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.deb
popd

pushd linux/arm64
aws --region cn-north-1 s3 cp --no-progress ./amazon-cloudwatch-agent.rpm s3://${{ env.S3_INTEGRATION_BUCKET_CN }}/integration-test/binary/${{ inputs.build_id }}/linux/arm64/amazon-cloudwatch-agent.rpm
aws --region cn-north-1 s3 cp --no-progress ./amazon-cloudwatch-agent.deb s3://${{ env.S3_INTEGRATION_BUCKET_CN }}/integration-test/binary/${{ inputs.build_id }}/linux/arm64/amazon-cloudwatch-agent.deb
popd

- name: Configure AWS Credentials (ITAR)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE_ITAR }}
aws-region: us-gov-east-1
role-duration-seconds: ${{ env.TERRAFORM_AWS_ASSUME_ROLE_DURATION }}

- name: Re-upload Artifacts (ITAR)
id: upload-artifacts-itar
run: |
pushd linux/amd64
aws --region us-gov-east-1 s3 cp --no-progress ./amazon-cloudwatch-agent.rpm s3://${{ env.S3_INTEGRATION_BUCKET_ITAR }}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.rpm
aws --region us-gov-east-1 s3 cp --no-progress ./amazon-cloudwatch-agent.deb s3://${{ env.S3_INTEGRATION_BUCKET_ITAR }}/integration-test/binary/${{ inputs.build_id }}/linux/amd64/amazon-cloudwatch-agent.deb
popd

pushd linux/arm64
aws --region us-gov-east-1 s3 cp --no-progress ./amazon-cloudwatch-agent.rpm s3://${{ env.S3_INTEGRATION_BUCKET_ITAR }}/integration-test/binary/${{ inputs.build_id }}/linux/arm64/amazon-cloudwatch-agent.rpm
aws --region us-gov-east-1 s3 cp --no-progress ./amazon-cloudwatch-agent.deb s3://${{ env.S3_INTEGRATION_BUCKET_ITAR }}/integration-test/binary/${{ inputs.build_id }}/linux/arm64/amazon-cloudwatch-agent.deb
popd

RepackageECRImage:
name: 'RepackageECRImage'
runs-on: ubuntu-latest
strategy:
fail-fast: false
permissions:
id-token: write
contents: read
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE }}
aws-region: us-west-2
role-duration-seconds: ${{ env.TERRAFORM_AWS_ASSUME_ROLE_DURATION }}

- name: Login to Release Artifacts Amazon ECR
id: login-artifacts-ecr
uses: aws-actions/amazon-ecr-login@v2
with:
registries: ${{ secrets.RELEASE_ARTIFACTS_ACCOUNT_ID }}

- name: Pull Image
id: pull-image
env:
ARTIFACTS_REGISTRY: ${{ steps.login-artifacts-ecr.outputs.registry }}
run: |
docker pull ${{ env.ARTIFACTS_REGISTRY }}/cloudwatch-agent:${{ inputs.build_id }}

- name: Login to Integ Test Amazon ECR
id: login-integ-test-ecr
uses: aws-actions/amazon-ecr-login@v2

- name: Push Image
id: push-image
env:
ARTIFACTS_REGISTRY: ${{ steps.login-artifacts-ecr.outputs.registry }}
INTEG_TEST_REGISTRY: ${{ steps.login-integ-test-ecr.outputs.registry }}
run: |
docker tag ${{ env.ARTIFACTS_REGISTRY }}/cloudwatch-agent:${{ inputs.build_id }} ${{ env.INTEG_TEST_REGISTRY }}/cwagent-integration-test:${{ inputs.build_id }}
docker push ${{ env.INTEG_TEST_REGISTRY }}/cwagent-integration-test:${{ inputs.build_id }}


2 changes: 1 addition & 1 deletion .github/workflows/start-localstack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,5 +83,5 @@ jobs:
-var="ssh_key_name=${{env.KEY_NAME}}" &&
LOCAL_STACK_HOST_NAME=$(terraform output -raw public_dns) &&
echo $LOCAL_STACK_HOST_NAME &&
echo "::set-output name=local_stack_host_name::$LOCAL_STACK_HOST_NAME" &&
echo "local_stack_host_name=$LOCAL_STACK_HOST_NAME" >> "$GITHUB_OUTPUT" &&
aws s3 cp terraform.tfstate s3://${{inputs.s3_integration_bucket}}/integration-test/local-stack-terraform-state/${{inputs.github_sha}}/terraform.tfstate
3 changes: 3 additions & 0 deletions .github/workflows/stop-localstack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ jobs:
- name: Copy state
run: aws s3 cp s3://${{inputs.s3_integration_bucket}}/integration-test/local-stack-terraform-state/${{inputs.github_sha}}/terraform.tfstate .

- name: Install Terraform
uses: hashicorp/setup-terraform@v3

- name: Verify Terraform version
run: terraform --version

Expand Down
Loading