Add E2E Test[Ubuntu AMD64] as Gate for Pull Requests by nimanch · Pull Request #5717 · Azure/iotedge

nimanch · 2021-10-19T18:18:26Z

This PR Adds a Gate which triggers E2E Test Run for a Pull Request. Currently only Ubuntu 18.04 AMD64 E2E Test has been added to save time. As we optimize more, more platforms would be added

Added e2e-checkin.yaml.
Modularize images.yaml and packages.yaml to allow using build components for e2e checkin gate.
Add Conditions to selectively trigger Building Images and Packages. Edgelet Packages will only be built if there are changes in
edgelet directory. Images will only be built if there are changes outside of test or doc directory

TODO AFTER Merging PR

Add new pipeline and configure it to use e2e-checkin.yaml
Work with Repo Admin to add Status Check to run E2E Pipeline
Notify Team about Comment Triggers

Testing

Testing was done via setting up a private pipeline : https://dev.azure.com/msazure/One/_build/results?buildId=48064485&view=results
As a test , created a PR to in my private fork : Personal/nimanch/e2echeckin nimanch/iotedge#4 which trigger an E2E Test run : https://dev.azure.com/msazure/One/_build/results?buildId=48067190&view=results
Created a dummy Pull Request to my private fork with an external alias. Verified the E2E Gate does not trigger automatically. Update devguide.md nimanch/iotedge#5
Created a dummy PR to my private fork with an external alias. Verified that after Commenting "/azp run" with my MSFT Github account, A Pipeline run was triggered : Dummy PR for External Trigger nimanch/iotedge#6
Verified Existing Build Images and Build Packages Pipelines work as expected
Verified that existing CI E2E Test Run uses the correct artifacts ( Resource Artifacts) -> https://msazure.visualstudio.com/One/_build/results?buildId=48091890&view=logs&j=f1061aa8-bee6-5103-5b5c-2b2c91b98c58
Verified that an External Alias cannot trigger E2E Test Run by Commenting /azp run on the PR . Update devguide.md nimanch/iotedge#5

Testing done

Azure IoT Edge PR checklist:

This checklist is used to make sure that common guidelines for a pull request are followed.

General Guidelines and Best Practices

I have read the contribution guidelines.
Title of the pull request is clear and informative.
Description of the pull request includes a concise summary of the enhancement or bug fix.

Testing Guidelines

Pull request includes test coverage for the included changes.
Description of the pull request includes
- concise summary of tests added/modified
- local testing done.

Draft PRs

Open the PR in Draft mode if it is:
- Work in progress or not intended to be merged.
- Encountering multiple pipeline failures and working on fixes.

Note: We use the kodiakhq bot to merge PRs once the necessary checks and approvals are in place. When it merges a PR, kodiakhq converts the PR title to the commit title, PR description to the commit description, and squashes all the commits in the PR to a single commit. The net effect is that entire PR becomes a single commit. Please follow the best practices mentioned here for the PR title and description

This PR Re-enables Rust Coverage in Check in Pipelines 1. Remove the Dependency to on nightly build on Rust. Use the same version of rust as other pipelines 2. Use Cargo Tarpaulin for Create Code Coverage Reports. This was recommended by other teams within Microsoft developing rust code. (https://lib.rs/crates/cargo-tarpaulin) 3. Publish HTML Code Coverage Report Based on Coverage Report Note : The Current Code Coverage stands at 44%. I had to reduce the code coverage gate for Rust to 44%. The intent is to increase the code coverage and then increase the gate. Sample Build : https://iotedge.visualstudio.com/iotedge/_build/results?buildId=83180&view=codecoverage-tab

nimanch · 2021-10-21T01:26:56Z

builds/misc/templates/build-packages.yaml

+
+stages:
+
+  ################################################################################


I added this additional stage to prevent building edgelet packages in case of no edgelet changes

builds/misc/templates/build-images.yaml

builds/checkin/e2e-checkin.yaml

builds/misc/templates/build-images.yaml

builds/checkin/e2e-checkin.yaml

nimanch · 2021-10-21T20:29:45Z

builds/misc/templates/build-packages.yaml

+              echo "##vso[task.setvariable variable=PACKAGE_ARCH;]$(arch)"
+              echo "##vso[task.setvariable variable=PACKAGE_OS;]$(os)"
+            displayName: Set Version
+            condition: or(eq(${{ parameters['E2EBuild'] }}, false), eq(variables.arch,'amd64'))


I added a condition here to make sure that we only build amd64 packages for E2E Build.
Example can be seen here :https://dev.azure.com/msazure/One/_build/results?buildId=48098034&view=logs&s=3af0183b-493c-5fc6-b74c-e772c3065c36

damonbarry

You changed 18.04 to 20.04 in the checkin pipeline (thanks!), but we're still using 18.04 in the templates (build images, edgelet packages, etc.). That's probably necessary because other pipelines that depend on these templates have not upgraded to 20.04. If the mismatch causes problems, feel free to revert the e2e checkin pipeline back to 18.04.

nimanch · 2021-10-21T20:33:35Z

You changed 18.04 to 20.04 in the checkin pipeline (thanks!), but we're still using 18.04 in the templates (build images, edgelet packages, etc.). That's probably necessary because other pipelines that depend on these templates have not upgraded to 20.04. If the mismatch causes problems, feel free to revert the e2e checkin pipeline back to 18.04.

@damonbarry : I ran with pipeline with this change and didn't see any issues: https://dev.azure.com/msazure/One/_build?definitionId=243203&_a=summary

builds/misc/templates/build-images.yaml

builds/misc/templates/build-packages.yaml

nlcamp

Nice job on this!

This PR is a manual port of #5717 , #5743 to Enable E2E Check-in Gate for Release/1.1 Branch. Due to the way Images are built (Single Stage vs Multi-stage), Some minor modifications had to be made in creating the build-images.yaml and build-packages.yaml template. Deviation from #5717 1. Use Powershell Scrip in e2e-setup.yaml since windows platform is also used in 1.1 2. Add Debug Log to Print Edge Hub and Edge Agent Image 3. Fix Artifact Filter Pattern (Edgelet artifacts involve both .gz and .deb packages) Testing Done 1. E2E Checkin Gate: https://dev.azure.com/msazure/One/_build/results?buildId=48273899&view=results 2. Build Images Pipeline: https://dev.azure.com/msazure/One/_build/results?buildId=48274103&view=results 3. Build Packages Pipeline: https://dev.azure.com/msazure/One/_build/results?buildId=48274136&view=results 4. CI E2E Test: https://dev.azure.com/msazure/One/_build/results?buildId=48266631&view=logs&j=9ec97737-d75d-57d0-984a-aca7d53a27f6&t=dcdcf879-b5e7-5ee9-8160-60c54b14cdfe ## Azure IoT Edge PR checklist: This checklist is used to make sure that common guidelines for a pull request are followed. ### General Guidelines and Best Practices - [x] I have read the [contribution guidelines](https://github.com/azure/iotedge#contributing). - [x] Title of the pull request is clear and informative. - [x] Description of the pull request includes a concise summary of the enhancement or bug fix. ### Testing Guidelines - [x] Pull request includes test coverage for the included changes. - Description of the pull request includes - [x] concise summary of tests added/modified - [x] local testing done. ### Draft PRs - Open the PR in `Draft` mode if it is: - Work in progress or not intended to be merged. - Encountering multiple pipeline failures and working on fixes. _Note: We use the kodiakhq bot to merge PRs once the necessary checks and approvals are in place. When it merges a PR, kodiakhq converts the PR title to the commit title, PR description to the commit description, and squashes all the commits in the PR to a single commit. The net effect is that entire PR becomes a single commit. Please follow the best practices mentioned [here](https://chris.beams.io/posts/git-commit/#:~:text=The%20seven%20rules%20of%20a%20great%20Git%20commit,what%20and%20why%20vs.%20how%20For%20example%3A%20) for the PR title and description_

…Pull Requests (#5755) This PR is a manual port of the E2E Check-in Changes made in #5717 #5743 Changes had to be manually ported over due to additional artifacts that get build for Master Branch Deviation from #5717 1. Manually Port Build Images Instead of Cherry Picking since Master images.yaml builds builds more images than Release/1.2 2. Use Master Resources instead of release/1.2 Resources for downloading build artifacts Testing 1. E2E Check-in Test run : https://dev.azure.com/msazure/One/_build/results?buildId=48267053&view=results 2. Build Images Pipeline: https://msazure.visualstudio.com/One/_build/results?buildId=48269740&view=results 3. Build Packages Pipeline: https://msazure.visualstudio.com/One/_build/results?buildId=48269877&view=results 4. CI E2E Test Run: https://dev.azure.com/msazure/One/_build/results?buildId=48272784&view=results ## Azure IoT Edge PR checklist:

…Pull Requests (Azure#5755) This PR is a manual port of the E2E Check-in Changes made in Azure#5717 Azure#5743 Changes had to be manually ported over due to additional artifacts that get build for Master Branch Deviation from Azure#5717 1. Manually Port Build Images Instead of Cherry Picking since Master images.yaml builds builds more images than Release/1.2 2. Use Master Resources instead of release/1.2 Resources for downloading build artifacts Testing 1. E2E Check-in Test run : https://dev.azure.com/msazure/One/_build/results?buildId=48267053&view=results 2. Build Images Pipeline: https://msazure.visualstudio.com/One/_build/results?buildId=48269740&view=results 3. Build Packages Pipeline: https://msazure.visualstudio.com/One/_build/results?buildId=48269877&view=results 4. CI E2E Test Run: https://dev.azure.com/msazure/One/_build/results?buildId=48272784&view=results ## Azure IoT Edge PR checklist:

nimanch and others added 30 commits October 7, 2021 04:21

Increase Coverage to 47

32b7151

Merge branch 'Azure:release/1.2' into release/1.2

6ddbfdc

Merge branch 'Azure:release/1.2' into release/1.2

b82bd1e

Merge branch 'Azure:release/1.2' into release/1.2

5d7f815

Add Initial

46560d5

Fix Path

cd968f4

Fix Path again

8efe398

Fix Parameter syntax

416d0d9

Fix Parameter syntax

046b242

Fix Logic

5e52ba2

Add E2E Test Run

44a0f66

Add E2E Test Run

20666f2

Add E2E Test Condition

387577c

Add E2E Test Condition

eca99a3

Add E2E Test Condition

4b5e072

Add E2E Test Condition

f74d19b

Add E2E Test Condition

29bbf3c

Add E2E Test Condition final

8d5fe5e

Add E2E Test Condition final

302b804

Add E2E Test Condition final

1ade1ce

Add E2E Test Condition final

7459068

Fix E2E Test Condition

e30d405

Fix E2E Test Condition

4c2de58

Fix E2E Test Condition

d25b9cb

Added Pre Conditions

ed14e74

Added Pre Conditions

43d1339

Added Pre Conditions

8333789

Added Pre Conditions

b9222a4

Added Pre Conditions

5a4477c

nimanch changed the title ~~Add E2E Test as Gate for Pull Requests~~ Add E2E Test[Ubuntu AMD64] as Gate for Pull Requests Oct 21, 2021

Merge branch 'release/1.2' into personal/nimanch/e2echeckin

b234b2a

nimanch marked this pull request as ready for review October 21, 2021 01:23

nimanch requested review from and-rewsmith, damonbarry and scb01 October 21, 2021 01:24

nimanch commented Oct 21, 2021

View reviewed changes

builds/misc/templates/build-images.yaml Show resolved Hide resolved

damonbarry requested changes Oct 21, 2021

View reviewed changes

builds/checkin/e2e-checkin.yaml Outdated Show resolved Hide resolved

builds/checkin/e2e-checkin.yaml Outdated Show resolved Hide resolved

builds/checkin/e2e-checkin.yaml Outdated Show resolved Hide resolved

builds/misc/templates/build-images.yaml Show resolved Hide resolved

nimanch and others added 2 commits October 21, 2021 11:33

Merge branch 'release/1.2' into personal/nimanch/e2echeckin

ca3f637

Use Ubuntu 20.04 for E2E Test

84cc20b

nimanch requested a review from varunpuranik October 21, 2021 19:24

damonbarry reviewed Oct 21, 2021

View reviewed changes

builds/checkin/e2e-checkin.yaml Outdated Show resolved Hide resolved

Remove comment

8ffb995

nimanch commented Oct 21, 2021

View reviewed changes

nimanch requested a review from nlcamp October 21, 2021 20:32

damonbarry approved these changes Oct 21, 2021

View reviewed changes

Merge branch 'release/1.2' into personal/nimanch/e2echeckin

9005883

nlcamp reviewed Oct 22, 2021

View reviewed changes

builds/misc/templates/build-images.yaml Show resolved Hide resolved

nlcamp reviewed Oct 22, 2021

View reviewed changes

builds/misc/templates/build-packages.yaml Show resolved Hide resolved

nlcamp approved these changes Oct 22, 2021

View reviewed changes

nimanch added the ready-to-merge label Oct 22, 2021

kodiakhq bot merged commit 8e61ce0 into Azure:release/1.2 Oct 22, 2021

This was referenced Oct 26, 2021

Cherry Pick E2E Checkin for Master #5745

Closed

[Manual Cherry-Pick] [Master] Add E2E Test[Ubuntu AMD64] as Gate for Pull Requests #5755

Merged

[Release1.1][Manual Cherry-Pick] E2E Checkin Gate #5758

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add E2E Test[Ubuntu AMD64] as Gate for Pull Requests#5717

Add E2E Test[Ubuntu AMD64] as Gate for Pull Requests#5717
kodiakhq[bot] merged 83 commits intoAzure:release/1.2from
nimanch:personal/nimanch/e2echeckin

nimanch commented Oct 19, 2021 •

edited

Loading

Uh oh!

nimanch Oct 21, 2021 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nimanch Oct 21, 2021 •

edited

Loading

Uh oh!

damonbarry left a comment

Uh oh!

nimanch commented Oct 21, 2021

Uh oh!

Uh oh!

Uh oh!

nlcamp left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		stages:

		################################################################################

Conversation

nimanch commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Azure IoT Edge PR checklist:

General Guidelines and Best Practices

Testing Guidelines

Draft PRs

Uh oh!

nimanch Oct 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nimanch Oct 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

damonbarry left a comment

Choose a reason for hiding this comment

Uh oh!

nimanch commented Oct 21, 2021

Uh oh!

Uh oh!

Uh oh!

nlcamp left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nimanch commented Oct 19, 2021 •

edited

Loading

nimanch Oct 21, 2021 •

edited

Loading

nimanch Oct 21, 2021 •

edited

Loading