TEST_PLAN.md

COVID Shield Test Plan

This test plan shows how to test Exposure Notfications (EN) in different scenarios and environments.

• Accessing EN framework on device
• Test exposed state on the same date without notification (as developer)
• Test exposed state with notification (as end user)
• Troubleshooting

Accessing EN framework on device

Android

Device Settings > Google > COVID-19 Exposure Notifications

iOS

Device Settings > Privacy > Bluetooth > COVID-19 Exposure Logging

Test exposed state on the same date without notification (as developer)

This test could be applied for all environments: Android + Android, iOS + iOS, Android + iOS.

Note: Because the server doesn't return Temporary Exposure Keys (TEKs) for current date (https://github.com/CovidShield/server/blob/master/pkg/server/retrieve.go#L83), for this test to work, the server needs to remove that restriction. This is possible if you run server locally or modify production server.

1. Run server locally, set SUBMIT_URL and RETRIEVE_URL in .env accordingly.
2. Build and run the app on two devices.
3. Go through onboarding flow on both devices. Make sure EN and bluetooth are turned on properly.
4. For Android only, see Debug mode section below.
5. Disable and re-enable COVID-19 Exposure Notifications (Android) or COVID-19 Exposure Notifications (iOS). This forces the EN framework to re-scan Bluetooth Random IDs.
6. On 1st device:
   1. Get OneTimeCode from server.
   2. Enter the code to set the device to positive.
   3. Verify that the positive state shows as expected.
7. On 2nd device:
   1. If you have Test Mode:
      1. Press on app logo or swipe right to left to show Test Mode.
      2. Press on Clear exposure history and run check. This will force the app to re-fetch all TEKs from the last 14 days.
      3. Expect to see the app changes to exposed state You have possibly been exposed to COVID-19.
   2. If you don't have Test Mode:
      1. Clear data or re-install the app.
      2. Go through onboarding flow.
      3. Expect to see the app changes to exposed state You have possibly been exposed to COVID-19.
   3. If you uninstall the app and re-install, expect to see exposed status show immediately after onboarding flow.

Test exposed state with notification (as end user)

This test runs everything on production. That means you only receive notification on the next one or two dates. Same date testing doesn't work for this scenario. Test mode cannot be used in this scenario.

Scenario 1: two devices haven't been using any Exposure Notifications app before.

1. Build and run the app on two devices.
2. Go through onboarding flow on both devices. Make sure EN and bluetooth are turned on properly.
3. For Android only, see Debug mode section below.
4. Keep two devices approximately close together for about 5 minutes.
5. On 1st device:
   1. Get OneTimeCode from server.
   2. Enter the code to set the device to positive.
   3. Verify that the positive state shows as expected.
6. Wait for next date (UTC timezone).
7. On 1st device:
   1. Expect to receive reminder notification about uploading new RANDOM_IDs.
   2. Submit new RANDOM_IDs.
8. On 2nd device:
   1. Open app.
   2. Expect the app in still in monitoring state.
9. Wait for next date (UTC timezone).
10. On 1st device:
    1. Expect to receive reminder notification about uploading new RANDOM_IDs.
    2. Submit new RANDOM_IDs.
11. On 2nd device:
    1. Expect to receive exposed notification saying You have possibly been exposed to COVID-19.
    2. Tap on notification will open the app.
    3. Expect to see exposed status showing in the app.
    4. If you uninstall the app and re-install, expect to see exposed status show immediately after onboarding flow.

Scenario 2: two devices have been using Exposure Notifications app for a few days.

1. Build and run the app on two devices.
2. Go through onboarding flow on both devices. Make sure EN and bluetooth are turned on properly.
3. For Android only, see Debug mode section below.
4. Keep two devices approximately close together for about 5 minutes.
5. Wait for next date (UTC timezone).
6. On 1st device:
   1. Get OneTimeCode from server.
   2. Enter the code to set the device to positive.
   3. Verify that the positive state shows as expected.
7. On 2nd device:
   1. Open app.
   2. Expect the app in still in monitoring state.
8. Wait for next date (UTC timezone).
9. On 1st device:
   1. Expect to receive reminder notification about uploading new RANDOM_IDs.
   2. Submit new RANDOM_IDs.
10. On 2nd device:
    1. Expect to receive exposed notification saying You have possibly been exposed to COVID-19.
    2. Tap on notification will open the app.
    3. Expect to see exposed status showing in the app.
    4. If you uninstall the app and re-install, expect to see exposed status show immediately after onboarding flow.

Troubleshooting

[Android] Debug mode

Debug mode is only available for developer email that has been whitelisted by Google to test EN. With this mode, you can use any APP_ID / Package Name that have been whitelisted by Google, ex: com.google.android.apps.exposurenotification to test the framework.

• If you have Debug mode, make sure Bypass app signature check toggle are ENABLED.
• If you don't, you have to use app with APP_ID / Package Name that has been whitelisted with correct signed key.

[General] Why same date testing doesn't work for me?

On devices that haven't been using any apps supporting Google / Apple Exposure Notifications before, the device hasn't been generating / collecting Temporary Exposure Keys (TEKs) yet. So, there is no key to submit to server after you enter OneTimeCode. In this case, the only option is to come back the next date and test again. Make sure you EN and bluetooth are enabled properly.

[General] What timezone is the app using for notification?

There are two types of notification:

• Reminder notification: it's about uploading new RANDOM_IDs and uses device timer.
• Exposed notification: it's about user has been exposed to COVID-19 and uses UTC timezone.