Previously Successful Build Suddenly Failing

[danielccdk]

If you are looking for help, please confirm the following...

• I confirm I have searched the Docs, Codemagic Sample Projects, and GitHub Discussions.
• I confirm the software versions on my local computer match the ones in CI environment (e.g. Xcode version, Gradle version). See here
• I confirm I am able to do a fresh clone of the repository and build the project on my local computer.

Which mobile framework are you using?

Flutter (Dart)

Steps to reproduce

This is our current codemagic.yaml:

workflows: integration_test_workflow: max_build_duration: 120 environment: flutter: "3.24.3" xcode: 16.2 groups: - bugsnag - database - okta - carfax triggering: events: - pull_request branch_patterns: - pattern: 'dev' include: true source: false scripts: - name: Integration Tests script: | # This command will will shutdown the existing simulators to save on resources. flutter doctor -v xcrun simctl shutdown all # Integration tests on iPhone (iOS 18.2) flutter emulators --launch apple_ios_simulator sleep 20 flutter devices xcrun simctl list devices --j BOOTED_DEVICE_UDID=$(xcrun simctl list devices --json | jq -r '.devices | .[] | .[] | select(.state == "Booted") | .udid') flutter -d $BOOTED_DEVICE_UDID test integration_test/test_master.dart \ --dart-define=isTesting=true \ --dart-define=BUGSNAG_API_KEY=$BUGSNAG_API_KEY \ --dart-define=CARFAX_OAUTH_CLIENT_ID=$CARFAX_OAUTH_CLIENT_ID \ --dart-define=OKTA_ISSUER=$OKTA_ISSUER \ --dart-define=OKTA_CLIENT_ID=$OKTA_CLIENT_ID \ --dart-define=OKTA_REDIRECT_URI=$OKTA_REDIRECT_URI

We've had to change to this because out of nowhere on this past Friday, 1/17, the build started failing out of nowhere.
This is the codemagic.yaml for when that started to occur (it had been working for months up until then):

workflows: integration_test_workflow: environment: flutter: "3.24.3" groups: - bugsnag - database - okta - carfax triggering: events: - pull_request branch_patterns: - pattern: 'dev' include: true source: false scripts: - name: Integration Tests script: | flutter emulators --launch apple_ios_simulator flutter -d iPhone test integration_test/test_master.dart \ --dart-define=isTesting=true \ --dart-define=BUGSNAG_API_KEY=$BUGSNAG_API_KEY \ --dart-define=CARFAX_OAUTH_CLIENT_ID=$CARFAX_OAUTH_CLIENT_ID \ --dart-define=OKTA_ISSUER=$OKTA_ISSUER \ --dart-define=OKTA_CLIENT_ID=$OKTA_CLIENT_ID \ --dart-define=OKTA_REDIRECT_URI=$OKTA_REDIRECT_URI

We were facing timeout errors out of nowhere and it was suddenly unable to find a device to test on.

Build is triggered by push to feature branch that points to dev.

Expected results

Build to run/pass successfully. This has been the norm until Friday 1/17 despite NO relevant codemagic changes.

Actual results

Below is the current error we are seeing. Nothing has changed from a code perspective outside of trying to change things in codemagic.yaml to achieve a successful build.

We have also noticed that it now takes a long time for the simulator/emulator to fire up. Any input on this?

11:58 +0: loading /Users/builder/clone/integration_test/test_master.dart 11:59 +0: loading /Users/builder/clone/integration_test/test_master.dart 12:00 +0: loading /Users/builder/clone/integration_test/test_master.dart 12:00 +0 -1: loading /Users/builder/clone/integration_test/test_master.dart [E] TimeoutException after 0:12:00.000000: Test timed out after 12 minutes. package:test_api/src/backend/invoker.dart 338:28 Invoker._handleError.<fn> dart:async/zone.dart 1391:47 _rootRun dart:async/zone.dart 1301:19 _CustomZone.run package:test_api/src/backend/invoker.dart 336:10 Invoker._handleError package:test_api/src/backend/invoker.dart 291:9 Invoker.heartbeat.<fn>.<fn> dart:async/zone.dart 1399:13 _rootRun dart:async/zone.dart 1301:19 _CustomZone.run package:test_api/src/backend/invoker.dart 290:38 Invoker.heartbeat.<fn> dart:async-patch/timer_patch.dart 18:15 Timer._createTimer.<fn> dart:isolate-patch/timer_impl.dart 398:19 _Timer._runTimers dart:isolate-patch/timer_impl.dart 429:5 _Timer._handleMessage dart:isolate-patch/isolate_patch.dart 184:12 _RawReceivePort._handleMessage

Build id (optional)

678f842617e6cec0c2b3e41e

[dam-ease]

Hello @danielccdk,
Comparing the successful and failed builds, the difference seems to be with the wait for Simulator Initialization i.e. time to allow the simulator to boot up fully. The failed build has sleep 20 giving maybe little time for simulator to build, hence the error. Can you please check this out?

[renekorss]

Hey @dam-ease,

I'm @danielccdk's coworker. We had builds running without any problems before without using any sleep command.

For example build ID: 678949bd50168caf95970fda

Command was as follows:

flutter emulators --launch apple_ios_simulator
flutter -d iPhone test integration_test/test_master.dart

This command now results in situation where there is no device to run integration tests on.

Since pricing is also per minute, it's not that wise to wait for 12 minutes just for simulator to start up. Since you upgraded default machines to M2 and XCode 16.2. We tried to switch back to XCode 16.0 and older simulator, but that also didn't help.

[dam-ease]

Hello @danielccdk,
Thanks for your detailed response and sharing these. I only compared the builds initially.
I looked through your failed build logs now and the actual error here is:

​"error": "TimeoutException after 0:12:00.000000: Test timed out after 12 minutes."

This is a known issue with Flutter which occurs intermittently. Please check this GH issue for more information.
The workaround is to use the flutter drive command:

flutter drive \
  --driver=test_driver/integration_test.dart \
  --target=integration_test/app_test.dart

Kindly check this and please do let me know how it goes on your end.
Also, you should have access to the chat widget support option as well for dedicated and faster responses.

[danielccdk]

Hi @dam-ease thank you so much

We are seeing this in regards to the app bundle. What are your thoughts?

06:50 +0: loading /Users/builder/clone/integration_test/test_master.dart                                                                                                                               Error: Unable to launch ai.predian.app on EE04D37A-FDE4-4222-B514-02A368ED2FB2:
ProcessException: Process exited abnormally with exit code 1:
An error was encountered processing the command (domain=FBSOpenApplicationServiceErrorDomain, code=1):
Simulator device failed to launch ai.predian.app.
The request was denied by service delegate (SBMainWorkspace) for reason: NotFound ("Unknown application display identifier ai.predian.app").
Underlying error (domain=FBSOpenApplicationServiceErrorDomain, code=1):
	The request to open "ai.predian.app" failed.
	The request was denied by service delegate (SBMainWorkspace) for reason: NotFound ("Unknown application display identifier ai.predian.app").
  Command: /usr/bin/arch -arm64e xcrun simctl launch EE04D37A-FDE4-4222-B514-02A368ED2FB2 ai.predian.app --enable-dart-profiling --disable-vm-service-publication --enable-checked-mode --verify-entry-points


06:50 +0 -1: loading /Users/builder/clone/integration_test/test_master.dart [E]                                                                                                                        
  TestDeviceException(Unable to start the app on the device.)
  package:flutter_tools/src/test/integration_test_device.dart 63:7  IntegrationTestTestDevice.start
  
  TimeoutException after 0:12:00.000000: Test timed out after 12 minutes.
  package:test_api/src/backend/invoker.dart 338:28  Invoker._handleError.<fn>
  dart:async/zone.dart 1391:47                      _rootRun
  dart:async/zone.dart 1301:19                      _CustomZone.run
  package:test_api/src/backend/invoker.dart 336:10  Invoker._handleError
  package:test_api/src/backend/invoker.dart 291:9   Invoker.heartbeat.<fn>.<fn>
  dart:async/zone.dart 1399:13                      _rootRun
  dart:async/zone.dart 1301:19                      _CustomZone.run
  package:test_api/src/backend/invoker.dart 290:38  Invoker.heartbeat.<fn>
  dart:async-patch/timer_patch.dart 18:15           Timer._createTimer.<fn>
  dart:isolate-patch/timer_impl.dart 398:19         _Timer._runTimers
  dart:isolate-patch/timer_impl.dart 429:5          _Timer._handleMessage
  dart:isolate-patch/isolate_patch.dart 184:12      _RawReceivePort._handleMessage

[danielccdk]

Hi @dam-ease

I have implented the driver approach into our codemagic call..

integration_test_workflow:
    environment:
      flutter: "3.24.3"
      groups:
        - bugsnag
        - database
        - okta
        - carfax
    triggering:
      events:
        - pull_request
      branch_patterns:
      - pattern: 'dev'
        include: true
        source: false 
    scripts:
      - name: Integration Tests 
        script: |
          # This command will will shutdown the existing simulators to save on resources.
          flutter doctor -v
          xcrun simctl shutdown all
          # Integration tests on iPhone (iOS 18.2)
          flutter emulators --launch apple_ios_simulator
          flutter devices
          xcrun simctl list devices --j
          BOOTED_DEVICE_UDID=$(xcrun simctl list devices --json | jq -r '.devices | .[] | .[] | select(.state == "Booted") | .udid')
          flutter drive \
            --device-id=$BOOTED_DEVICE_UDID \
            --driver=test_driver/integration_test.dart \
            --target=integration_test/test_master.dart \
            --dart-define=isTesting=true \
            --dart-define=BUGSNAG_API_KEY=$BUGSNAG_API_KEY \
            --dart-define=CARFAX_OAUTH_CLIENT_ID=$CARFAX_OAUTH_CLIENT_ID \
            --dart-define=OKTA_ISSUER=$OKTA_ISSUER \
            --dart-define=OKTA_CLIENT_ID=$OKTA_CLIENT_ID \
            --dart-define=OKTA_REDIRECT_URI=$OKTA_REDIRECT_URI`

And here is what we are now getting..

Error: Unable to launch ai.predian.app on EE04D37A-FDE4-4222-B514-02A368ED2FB2:
ProcessException: Process exited abnormally with exit code 1:
An error was encountered processing the command (domain=FBSOpenApplicationServiceErrorDomain, code=1):
Simulator device failed to launch ai.predian.app.
The request was denied by service delegate (SBMainWorkspace) for reason: NotFound ("Unknown application display identifier ai.predian.app").
Underlying error (domain=FBSOpenApplicationServiceErrorDomain, code=1):
	The request to open "ai.predian.app" failed.
	The request was denied by service delegate (SBMainWorkspace) for reason: NotFound ("Unknown application display identifier ai.predian.app").
  Command: /usr/bin/arch -arm64e xcrun simctl launch EE04D37A-FDE4-4222-B514-02A368ED2FB2 ai.predian.app --enable-dart-profiling --disable-vm-service-publication --start-paused --enable-checked-mode --verify-entry-points
Application failed to start on attempt: 1

With all due respect may I also ask why we are now getting all of these issues? We had code that was working and something has obviously changed on your end to break us.

This was our working code:

          flutter emulators --launch apple_ios_simulator
          flutter -d iPhone test integration_test/test_master.dart \
            --dart-define=isTesting=true \
            --dart-define=BUGSNAG_API_KEY=$BUGSNAG_API_KEY \
            --dart-define=CARFAX_OAUTH_CLIENT_ID=$CARFAX_OAUTH_CLIENT_ID \
            --dart-define=OKTA_ISSUER=$OKTA_ISSUER \
            --dart-define=OKTA_CLIENT_ID=$OKTA_CLIENT_ID \
            --dart-define=OKTA_REDIRECT_URI=$OKTA_REDIRECT_URI

Which gives us the following result:

Error: Unable to launch ai.predian.app on EE04D37A-FDE4-4222-B514-02A368ED2FB2:
ProcessException: Process exited abnormally with exit code 1:
An error was encountered processing the command (domain=FBSOpenApplicationServiceErrorDomain, code=1):
Simulator device failed to launch ai.predian.app.
The request was denied by service delegate (SBMainWorkspace) for reason: NotFound ("Unknown application display identifier ai.predian.app").
Underlying error (domain=FBSOpenApplicationServiceErrorDomain, code=1):
	The request to open "ai.predian.app" failed.
	The request was denied by service delegate (SBMainWorkspace) for reason: NotFound ("Unknown application display identifier ai.predian.app").
  Command: /usr/bin/arch -arm64e xcrun simctl launch EE04D37A-FDE4-4222-B514-02A368ED2FB2 ai.predian.app --enable-dart-profiling --disable-vm-service-publication --start-paused --enable-checked-mode --verify-entry-points
Application failed to start on attempt: 1`

---

BUT... it does pass:

flutter: 24:05 +33 ~2: (tearDownAll)
flutter: 24:05 +34 ~2: All tests passed!
All tests passed.

So I think there is an issue here at hand being that the command works but we are being given inaccurate errors? Can you please explain this specifically?

We have also tried the following:

scripts:
      - name: Integration Tests
        script: |
          # This command will will shutdown the existing simulators to save on resources.
          flutter doctor -v
          xcrun simctl shutdown all
          # Integration tests on iPhone (iOS 18.2)
          flutter emulators --launch apple_ios_simulator
          sleep 20
          flutter devices
          xcrun simctl list devices --j
          BOOTED_DEVICE_UDID=$(xcrun simctl list devices --json | jq -r '.devices | .[] | .[] | select(.state == "Booted") | .udid')
          flutter -d $BOOTED_DEVICE_UDID test integration_test/test_master.dart \
            --dart-define=isTesting=true \
            --dart-define=BUGSNAG_API_KEY=$BUGSNAG_API_KEY \
            --dart-define=CARFAX_OAUTH_CLIENT_ID=$CARFAX_OAUTH_CLIENT_ID \
            --dart-define=OKTA_ISSUER=$OKTA_ISSUER \
            --dart-define=OKTA_CLIENT_ID=$OKTA_CLIENT_ID \
            --dart-define=OKTA_REDIRECT_URI=$OKTA_REDIRECT_URI

Which then leads to the timeout error.

Can you please take this into account and provide us with the correct command to get this running in accordance with whatever changes occurred with codemagic?

Thank you so much!