The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2899500	dotnet-dotnet	Build	Log	#57704
2899478	dotnet-aspnetcore	Run build.cmd helix target	Log	#57696
2899443	dotnet-roslyn	Test	Log
2899373	dotnet-arcade-services	Run E2E tests	Log
2899417	dotnet-aspnetcore	Run build.cmd helix target	Log	#57695
2899449	dotnet-dotnet	Checkout dotnet-dotnet@dev/mthalman/license-scan to s	Log
2899410	dotnet-sdk	Run AoT Tests	Log	#57694
2899409	dotnet-sdk	Run TestBuild Tests	Log	#57693
1286355	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7578
1286337	dotnet/runtime	Checkout dotnet/runtime@refs/pull/123860/merge to s	Log	dotnet/runtime#123860
1286298	dotnet/runtime	Checkout dotnet/runtime@refs/pull/123323/merge to s	Log	dotnet/runtime#123323
1286281	dotnet/runtime	Checkout dotnet/runtime@refs/pull/124055/merge to s	Log	dotnet/runtime#124055
2899371	dotnet-roslyn	Test	Log
1286014	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124048
1286201	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65184
2899241	dotnet-arcade-services	Run E2E tests	Log
2899301	dotnet-sdk	Run Tests in Helix	Log	#57171
2899300	dotnet-sdk	Run Tests in Helix	Log	#57172
1286046	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124178
1285913	dotnet/runtime	Publish	Log	dotnet/runtime#124172
1286165	dotnet/aspire	Checkout dotnet/aspire@refs/pull/14004/merge to s	Log	dotnet/aspire#14004
1286077	dotnet/runtime	Checkout dotnet/runtime@release/8.0 to s	Log
1286023	dotnet/maui	Controls Layout	Log
2899267	dotnet-dotnet	🛡 Guardian: ESLint (JavaScript)	Log
2899201	dotnet-maui	Controls ScrollView	Log
1285433	dotnet/runtime	Send tests to Helix (Unix)	Log
1285725	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33961
1286060	dotnet/runtime	Send job to Helix (Unix)	Log
2899200	dotnet-maui	Run Integration Tests - Build	Log
1285886	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#64964
1285890	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65369
1285834	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
2898981	dotnet-runtime	Send ZipArchiveFuzzer to OneFuzz	Log
1284379	dotnet/maui	Run Integration Tests - Build windows	Log
2898993	dotnet-maui	Component Detection (auto-injected by policy)	Log
1285503	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33956
2898923	dotnet-dotnet	Build	Log
1285455	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33951
1285452	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33950
2898865	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2898862	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2898868	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2898867	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2898859	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
1285273	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#124166
1285325	dotnet/aspnetcore	Run Quarantined Tests	Log
1285253	dotnet/arcade	XHarness Apple Device Helix Testing	Log	dotnet/arcade#16423
1285244	dotnet/arcade	XHarness Apple Device Helix Testing	Log	dotnet/arcade#16444
1285249	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7570
1285234	dotnet/xharness	Run tests in Helix	Log	dotnet/xharness#1531
1285237	dotnet/xharness	Run tests in Helix	Log	dotnet/xharness#1532
1285080	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7578
1284990	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33944
1284924	dotnet/runtime	Send tests to Helix (Unix)	Log
1284926	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7578
2898509	dotnet-runtime	Generate CORE_ROOT	Log
1284715	dotnet/runtime	Publish	Log	dotnet/runtime#124139
2898363	dotnet-runtime	Send ZipArchiveFuzzer to OneFuzz	Log
1284585	dotnet/runtime	Send tests to Helix (Unix)	Log
1284564	dotnet/runtime	Send tests to Helix (Unix)	Log
2898149	dotnet-maui	Run Integration Tests - Build	Log
1284134	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33935
1284330	dotnet/runtime	Send tests to Helix (Unix)	Log
1284192	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33834
2898022	dotnet-sdk	🛡 Publish SDL Results to Advanced Security (1ES PT)	Log
1284197	dotnet/runtime	Send tests to Helix (Unix)	Log
2897971	dotnet-maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log
1284169	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124125
1284159	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124120
2897972	dotnet-maui	DeviceTestsAndroid (Windows)	Log
1284127	dotnet/maui	DeviceTestsAndroid (Windows)	Log
1284132	dotnet/maui	DeviceTestsAndroid (Windows)	Log
1284125	dotnet/maui	Run Integration Tests - Build windows	Log
2897970	dotnet-maui	Run Helix Tests	Log
2897893	dotnet-runtime	Send UTF8Fuzzer to OneFuzz	Log
1284068	dotnet/runtime	Send to Helix	Log
1284105	dotnet/aspnetcore	Run build.cmd helix target	Log
1284087	dotnet/runtime	Send to Helix	Log
2897908	dotnet-runtime	Send to Helix	Log
2897912	dotnet-dotnet	🔒 Publish BuildLogs	Log
1284082	dotnet/runtime	Send to Helix	Log
1284083	dotnet/runtime	Send to Helix	Log
1284049	dotnet/runtime	Send to Helix	Log
1284065	dotnet/runtime	Send to Helix	Log
2897849	dotnet-dotnet	Build (centOSStreamContainer)	Log
1284035	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124120
1284008	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124128
1283968	dotnet/runtime	Send to Helix	Log
2897760	dotnet-sdk	Run AoT Tests	Log	#57654
1283858	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124025
2897761	dotnet-sdk	Run AoT Tests	Log	#57655
1283715	dotnet/runtime	Send tests to Helix (Unix)	Log
2897720	dotnet-sdk	Run Tests in Helix	Log	#57171
2897719	dotnet-sdk	Run Tests in Helix	Log	#57172
1283474	dotnet/runtime	Send to Helix	Log
2897649	dotnet-aspnetcore	Run build.cmd helix target	Log
2897644	dotnet-aspnetcore	Run build.cmd helix target	Log
1283704	maestro-auth-test/helix-queue-insights-test	Run Tests (Windows)	Log
2897618	dotnet-roslyn	Test	Log
2897591	dotnet-efcore	Generate SBOM manifest	Log
Displaying 100 of 2606 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
52	499	2606

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez