Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repeat option for e2e tests and incrementing timeout #6655

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 29 additions & 9 deletions .github/workflows/android-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,19 +32,20 @@ on:
description: Override container image
type: string
required: false
run_e2e_tests:
description: Run e2e tests
type: boolean
required: false
run_firebase_tests:
description: Run firebase tests
type: boolean
required: false
mockapi_test_repeat:
description: Mockapi test repeat
description: Mockapi test repeat(self hosted)
default: '1'
required: true
type: string
e2e_test_repeat:
description: e2e test repeat(self hosted)
default: '0'
required: true
type: string
# Build if main is updated to ensure up-to-date caches are available
push:
branches: [main]
Expand Down Expand Up @@ -407,7 +408,6 @@ jobs:
instrumented-tests:
name: Run instrumented tests
runs-on: [self-hosted, android-device]
timeout-minutes: 30
needs: [build-app, build-instrumented-tests]
strategy:
fail-fast: false
Expand Down Expand Up @@ -447,8 +447,14 @@ jobs:
name: ${{ matrix.test-type }}-instrumentation-apks
path: ${{ matrix.path }}

- name: Calculate timeout
id: calculate-timeout
run: echo "timeout=$(( ${{ matrix.test-repeat }} * 10 ))" >> $GITHUB_OUTPUT
shell: bash

- name: Run instrumented test script
if: ${{ matrix.test-repeat != 0 }}
timeout-minutes: ${{ fromJSON(steps.calculate-timeout.outputs.timeout) }}
shell: bash -ieo pipefail {0}
env:
AUTO_FETCH_TEST_HELPER_APKS: true
Expand All @@ -471,11 +477,15 @@ jobs:
name: Run instrumented e2e tests
# Temporary workaround for targeting the runner android-runner-v1
runs-on: [self-hosted, android-device, android-emulator]
if: github.event_name == 'schedule' || github.event.inputs.run_e2e_tests == 'true'
timeout-minutes: 30
if: github.event_name == 'schedule' || ${{ github.event.inputs.e2e_test_repeat }} != '0'
needs: [build-app, build-instrumented-tests]
strategy:
matrix:
include:
- test-repeat: ${{ github.event.inputs.e2e_test_repeat || 1 }}
steps:
- name: Prepare report dir
if: ${{ matrix.test-repeat != 0 }}
id: prepare-report-dir
env:
INNER_REPORT_DIR: /tmp/${{ github.run_id }}-${{ github.run_attempt }}
Expand All @@ -484,21 +494,31 @@ jobs:
echo "report_dir=$INNER_REPORT_DIR" >> $GITHUB_OUTPUT

- name: Checkout repository
if: ${{ matrix.test-repeat != 0 }}
uses: actions/checkout@v4

# Using v3 due to v4 being very slow for this artifact.
- uses: actions/download-artifact@v3
if: ${{ matrix.test-repeat != 0 }}
with:
name: apks
path: android/app/build/outputs/apk

# Using v3 due to v4 being very slow for this artifact.
- uses: actions/download-artifact@v3
if: ${{ matrix.test-repeat != 0 }}
with:
name: e2e-instrumentation-apks
path: android/test/e2e/build/outputs/apk

- name: Calculate timeout
id: calculate-timeout
run: echo "timeout=$(( ${{ matrix.test-repeat }} * 10 ))" >> $GITHUB_OUTPUT
shell: bash

- name: Run instrumented test script
if: ${{ matrix.test-repeat != 0 }}
timeout-minutes: ${{ fromJSON(steps.calculate-timeout.outputs.timeout) }}
shell: bash -ieo pipefail {0}
env:
AUTO_FETCH_TEST_HELPER_APKS: true
Expand All @@ -509,7 +529,7 @@ jobs:
INVALID_TEST_ACCOUNT_NUMBER: '0000000000000000'
ENABLE_HIGHLY_RATE_LIMITED_TESTS: ${{ github.event_name == 'schedule' && 'true' || 'false' }}
REPORT_DIR: ${{ steps.prepare-report-dir.outputs.report_dir }}
run: ./android/scripts/run-instrumented-tests.sh
run: ./android/scripts/run-instrumented-tests-repeat.sh ${{ matrix.test-repeat }}

firebase-tests:
name: Run firebase tests
Expand Down
Loading