Buildfarmer log 2024/09/23 - 2024/09/30

[github-actions]

Previous log #92

[github-actions]

Sep 23

Urgent

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
nightly_win_rep#3486	12	2024-09-22 04:02:49.039000	projectroot.gtest_multithreaded__rmw_connextdds	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#151	10	2024-09-21 09:19:45.450000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#151	8	2024-09-21 09:19:45.450000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!
gz_common-ci-gz-common5-homebrew-amd64#84	7	2024-09-21 17:06:11.815000	AssimpLoader.LoadBoxWithAnimationOutsideSkeleton AssimpLoader.LoadBoxInstControllerWithoutSkeleton AssimpLoader.LoadBoxMultipleInstControllers AssimpLoader.LoadBoxNestedAnimation AssimpLoader.LoadBoxWithDefaultStride	No reports found!
nightly_win_rep#3486	5	2024-09-22 04:02:49.039000	projectroot.test.rclcpp.test_clock rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock	No reports found!

Warnings

Reference build	Age	Failure DateTime	Errors	Reports
Jci__nightly-release_ubuntu_noble_amd64#54	-1	2024-09-20 14:10:39.267000	1 gcc warnings 1 clang-tidy warnings	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
Jci__nightly-connext_ubuntu_noble_amd64#56	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Rci__nightly-connext_ubuntu_noble_amd64 2024-09-21 2024-09-08 13 13 100.0% Jci__nightly-connext_ubuntu_noble_amd64 2024-09-20 2024-09-08 5 4 80.0% Ici__nightly-connext_ubuntu_jammy_amd64 2024-09-16 2024-09-10 5 2 40.0% nightly_win_deb 2024-09-18 2024-09-09 14 2 14.29% nightly_win_rep 2024-09-08 2024-09-08 14 1 7.14%	No reports found!
nightly_win_rep#3486	test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-08 14 10 71.43% nightly_win_deb 2024-09-19 2024-09-08 14 4 28.57%	No reports found!
nightly_linux-rhel_repeated#2035	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-22 2024-09-10 13 8 61.54%	No reports found!
nightly_win_rep#3486	projectroot.test_n_nodes__rmw_fastrtps_cpp test_rclcpp.TestNNodesAfterShutdown.test_10_nodes	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-08 14 7 50.0% nightly_win_deb 2024-09-15 2024-09-15 14 1 7.14% nightly_win_rel 2024-09-21 2024-09-21 15 1 6.67% nightly_linux-aarch64_repeated 2024-09-13 2024-09-13 15 1 6.67%	No reports found!
nightly_win_rep#3486	test_rclcpp.TestNNodes.test_10_nodes	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-08 14 7 50.0% nightly_linux-aarch64_repeated 2024-09-13 2024-09-13 15 1 6.67%	No reports found!
nightly_linux_iron_coverage#574	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux_humble_coverage 2024-09-19 2024-09-10 11 5 45.45% nightly_linux_iron_coverage 2024-09-22 2024-09-14 10 3 30.0%	https://github.com/ros2/launch_ros/issues/304 (Closed)
nightly_win_rep#3486 nightly_linux-rhel_repeated#2035	projectroot.test.rclcpp.test_executors_busy_waiting projectroot.test.rclcpp.test_executors_busy_waiting	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-08 14 6 42.86% nightly_win_deb 2024-09-18 2024-09-09 14 2 14.29% nightly_linux-rhel_repeated 2024-09-22 2024-09-22 13 1 7.69%	No reports found!
nightly_win_rep#3486 nightly_win_rel#3177	projectroot.test_tutorial_use_logger_service__rmw_connextdds projectroot.test_tutorial_use_logger_service__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-08 14 6 42.86% nightly_win_rel 2024-09-22 2024-09-20 15 2 13.33%	No reports found!
nightly_win_rep#3486	test_security.TestSecurePublisherSubscriberAfterShutdown.test_processes_finished_gracefully	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-14 14 6 42.86% nightly_linux_release 2024-09-12 2024-09-12 11 1 9.09% nightly_linux-aarch64_repeated 2024-09-08 2024-09-08 15 1 6.67%	No reports found!
nightly_win_rep#3486	projectroot.test_server	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-09 14 5 35.71% nightly_win_deb 2024-09-16 2024-09-16 14 1 7.14%	No reports found!
nightly_win_rep#3486	projectroot.test_cli_remapping	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-13 14 4 28.57% nightly_linux_repeated 2024-09-14 2024-09-12 11 3 27.27% nightly_linux-aarch64_repeated 2024-09-21 2024-09-18 15 2 13.33%	No reports found!
nightly_win_rep#3486	rclcpp_action.TestServer.publish_status_succeeded rclcpp_action.TestServer.publish_status_aborted rclcpp_action.TestServer.publish_feedback rclcpp_action.TestServer.get_result rclcpp_action.TestServer.get_result_deferred rclcpp_action.TestServer.deferred_execution	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-09 14 4 28.57% nightly_win_deb 2024-09-16 2024-09-16 14 1 7.14%	No reports found!
nightly_linux-aarch64_repeated#2892	projectroot.test_node_name__rmw_fastrtps_cpp test_rclcpp.test_node_name__rmw_fastrtps_cpp.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-aarch64_repeated 2024-09-22 2024-09-11 15 4 26.67%	No reports found!
nightly_win_rep#3486	rclcpp.TestBusyWaiting/EventsExecutor.test_spin	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-08 14 3 21.43%	No reports found!

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__nightly-release_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
gz_common-ci-gz-common5-homebrew-amd64	2024-08-28
gz_sim-7-win	2024-09-08
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
gz_gui-ci-ign-gui6-focal-amd64	2024-09-15
gz_gui-ci-gz-gui9-homebrew-amd64	2024-09-15

Pending

Test regressions all

Reference Build	Error Name
nightly_win_rep#3486	25 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 1)
nightly_win_rel#3177	30 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 1)
nightly_win_deb#3224	rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel (age: 2)
nightly_linux_repeated#3588	test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_consistency_cpp (age: 1)
nightly_linux_release#3209	launch_pytest.test.launch_pytest.tools.test_process.test_sync_process_tools (age: 1)
nightly_linux_iron_coverage#574	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (age: 1)
nightly_linux_coverage#2530	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (age: 5)
nightly_linux-rhel_repeated#2035	test_tracetools.test_pub_sub__rmw_fastrtps_cpp.xunit.missing_result (age: 1)
nightly_linux-aarch64_repeated#2892	test_rclcpp.test_node_name__rmw_fastrtps_cpp.xunit.missing_result (age: 1)
nightly_linux-aarch64_debug#2900	rosbag2_transport.xmllint.xunit.missing_result (age: 1)
gz_transport-ci-ign-transport11-homebrew-amd64#78	ignTest.TopicInfo (age: 21)
gz_transport-ci-gz-transport12-homebrew-amd64#80	UNIT_gz_TEST.xml.[failed-to-read] (age: 23)
gz_sim-7-win#101	INTEGRATION_multiple_servers.test_ran (age: 1)
gz_rendering-9-win#14	71 errors VisualTest.Clone (age: 14)
gz_rendering-8-win#87	70 errors VisualTest.Clone (age: 18)
gz_rendering-7-win#79	67 errors VisualTest.Clone (age: 18)
gz_common-ci-gz-common5-homebrew-amd64#84	AssimpLoader.LoadBoxWithDefaultStride (age: 7)
gz_common-6-win#10	SignalHandler.Single (age: 1)
Rci__nightly-connext_ubuntu_noble_amd64#151	14 errors rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content (age: 8)
Rci__benchmark_ubuntu_noble_amd64#104	projectroot.benchmark_string_conversions (age: 104)
Jci__nightly-release_ubuntu_noble_amd64#54	1 clang-tidy warnings (age: -1)
Jci__nightly-connext_ubuntu_noble_amd64#56	13 errors rclpy.rclpy.test.test_client.TestClient.test_service_timestamps (age: 1)
Ici__nightly-performance_ubuntu_jammy_amd64#129	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 1)
Ici__nightly-connext_ubuntu_jammy_amd64#169	11 errors launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 70)
Hci__nightly-connext_ubuntu_jammy_amd64#275	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 6)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST

[github-actions]

Sep 24

Urgent

Build regressions

Reference Build	Failure DateTime	Failure Reason
nightly_win_rep#3488	2024-09-24 04:00	Number of failed test exceeds the threshold

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
Rci__nightly-connext_ubuntu_noble_amd64#153	12	2024-09-23 14:26:58.530000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#153	10	2024-09-23 14:26:58.530000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!

Warnings

Reference build	Age	Failure DateTime	Errors	Reports
Jci__nightly-release_ubuntu_noble_amd64#55	-1	2024-09-23 14:11:55.039000	1 gcc warnings 1 clang-tidy warnings	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
Jci__nightly-connext_ubuntu_noble_amd64#57	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Rci__nightly-connext_ubuntu_noble_amd64 2024-09-23 2024-09-09 14 14 100.0% Jci__nightly-connext_ubuntu_noble_amd64 2024-09-23 2024-09-11 5 4 80.0% Ici__nightly-connext_ubuntu_jammy_amd64 2024-09-16 2024-09-10 5 2 40.0% nightly_win_deb 2024-09-18 2024-09-09 15 2 13.33%	No reports found!
nightly_linux_humble_coverage#942 nightly_linux-rhel_repeated#2037	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux_humble_coverage 2024-09-24 2024-09-10 13 7 53.85% nightly_linux_iron_coverage 2024-09-22 2024-09-14 11 3 27.27% nightly_linux-rhel_repeated 2024-09-24 2024-09-23 15 2 13.33%	https://github.com/ros2/launch_ros/issues/304 (Closed)
Hci__nightly-connext_ubuntu_jammy_amd64#276	projectroot.test_buffer_client_tester.launch.py	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Hci__nightly-connext_ubuntu_jammy_amd64 2024-09-24 2024-09-09 6 2 33.33% nightly_win_rep 2024-09-20 2024-09-09 14 3 21.43%	No reports found!
Hci__nightly-connext_ubuntu_jammy_amd64#276	test_tf2.BufferClientTestAfterShutdown.test_exit_code	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Hci__nightly-connext_ubuntu_jammy_amd64 2024-09-24 2024-09-09 6 2 33.33% nightly_win_rep 2024-09-20 2024-09-09 14 3 21.43% nightly_win_deb 2024-09-18 2024-09-18 15 1 6.67%	No reports found!
nightly_linux-rhel_release#2028	launch_testing_examples.launch_testing_examples.hello_world_launch_test.launch_testing_examples.hello_world_launch_test	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux_repeated 2024-09-14 2024-09-10 12 3 25.0% nightly_linux-rhel_release 2024-09-24 2024-09-24 15 1 6.67% nightly_linux-rhel_repeated 2024-09-14 2024-09-14 15 1 6.67% nightly_linux-aarch64_repeated 2024-09-14 2024-09-14 16 1 6.25%	No reports found!
nightly_win_deb#3226	projectroot.test_node rclpy.test_node.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_deb 2024-09-24 2024-09-12 15 3 20.0% nightly_win_rep 2024-09-15 2024-09-15 14 1 7.14% nightly_win_rel 2024-09-13 2024-09-13 16 1 6.25%	No reports found!
nightly_linux-rhel_repeated#2037	projectroot.test_node_name__rmw_cyclonedds_cpp__rmw_fastrtps_cpp test_rclcpp.test_node_name__rmw_cyclonedds_cpp__rmw_fastrtps_cpp.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-24 2024-09-13 15 3 20.0% nightly_linux_repeated 2024-09-12 2024-09-12 12 1 8.33%	No reports found!

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__nightly-release_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
gz_sim-7-win	2024-09-08
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
gz_gui-ci-ign-gui6-focal-amd64	2024-09-15

Pending

Test regressions all

Reference Build	Error Name
nightly_win_deb#3226	13 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 1)
nightly_linux_repeated#3590	test_tracetools.test_generic_subscription__rmw_connextdds.xunit.missing_result (age: 1)
nightly_linux_humble_coverage#942	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (age: 2)
nightly_linux_coverage#2532	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (age: 7)
nightly_linux-rhel_repeated#2037	test_rclcpp.test_node_name__rmw_cyclonedds_cpp__rmw_fastrtps_cpp.xunit.missing_result (age: 1)
nightly_linux-rhel_release#2028	launch_testing_examples.launch_testing_examples.hello_world_launch_test.launch_testing_examples.hello_world_launch_test (age: 1)
nightly_linux-rhel_debug#2050	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn (age: 1)
nightly_linux-aarch64_repeated#2894	test_communication.TestActionClientServerAfterShutdown.test_processes_finished_gracefully[NestedMessage] (age: 1)
gz_transport-ci-ign-transport11-homebrew-amd64#78	ignTest.TopicInfo (age: 21)
gz_transport-ci-gz-transport12-homebrew-amd64#80	UNIT_gz_TEST.xml.[failed-to-read] (age: 23)
gz_sim-ign-gazebo6-win#90	INTEGRATION_multiple_servers.test_ran (age: 1)
gz_sim-ci-gz-sim8-jammy-amd64#154	ResetFixture.HandleReset (age: 2)
gz_sim-7-win#101	INTEGRATION_multiple_servers.test_ran (age: 1)
gz_rendering-9-win#15	71 errors VisualTest.Clone (age: 15)
gz_rendering-8-win#88	70 errors VisualTest.Clone (age: 19)
gz_rendering-7-win#79	67 errors VisualTest.Clone (age: 18)
gz_gui-ci-ign-gui6-focal-amd64#94	UNIT_Teleop_TEST.test_ran (age: 2)
gz_common-ign-common3-win#79	PluginSpecialization.AccessTime (age: 1)
gz_common-6-win#10	SignalHandler.Single (age: 1)
Rci__nightly-connext_ubuntu_noble_amd64#153	13 errors rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset (age: 153)
Rci__benchmark_ubuntu_noble_amd64#105	projectroot.benchmark_string_conversions (age: 105)
Jci__nightly-release_ubuntu_noble_amd64#55	1 clang-tidy warnings (age: -1)
Jci__nightly-connext_ubuntu_noble_amd64#57	13 errors tf2_ros.tf2_ros_test_message_filter.gtest.missing_result (age: 1)
Ici__nightly-connext_ubuntu_jammy_amd64#169	11 errors launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 70)
Hci__nightly-connext_ubuntu_jammy_amd64#276	test_tf2.BufferClientTestAfterShutdown.test_exit_code (age: 1)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST

[github-actions]

Sep 25

Urgent

Build regressions

Reference Build	Failure DateTime	Failure Reason
nightly_linux_debug#3191	2024-09-25 07:33	Early fail

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
nightly_win_rep#3489	15	2024-09-25 04:00:01.305000	projectroot.gtest_multithreaded__rmw_connextdds	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#154	13	2024-09-24 09:20:46.737000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#154	11	2024-09-24 09:20:46.737000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!
nightly_win_rep#3489	4	2024-09-25 04:00:01.305000	test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result	No reports found!

Warnings

Reference build	Age	Failure DateTime	Errors	Reports
Jci__nightly-release_ubuntu_noble_amd64#55	-1	2024-09-23 14:11:55.039000	1 gcc warnings 1 clang-tidy warnings	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
Jci__nightly-connext_ubuntu_noble_amd64#57	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Rci__nightly-connext_ubuntu_noble_amd64 2024-09-24 2024-09-10 14 14 100.0% Jci__nightly-connext_ubuntu_noble_amd64 2024-09-23 2024-09-11 5 4 80.0% Ici__nightly-connext_ubuntu_jammy_amd64 2024-09-16 2024-09-10 6 2 33.33% nightly_win_deb 2024-09-18 2024-09-18 15 1 6.67%	No reports found!
nightly_win_rep#3489	test_rclcpp.test_timeout_subscriber.timeout_subscriber	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-25 2024-09-10 14 10 71.43%	No reports found!
nightly_win_rep#3489	projectroot.gtest_timeout_subscriber__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-25 2024-09-10 14 8 57.14%	No reports found!
nightly_win_rep#3489	projectroot.test.rclcpp.test_clock rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-25 2024-09-14 14 8 57.14% nightly_linux_repeated 2024-09-19 2024-09-12 13 4 30.77%	No reports found!
nightly_linux-rhel_repeated#2038	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-25 2024-09-10 16 9 56.25% nightly_linux-aarch64_repeated 2024-09-23 2024-09-23 16 1 6.25%	No reports found!
nightly_win_rep#3489	projectroot.gtest_multiple_service_calls__rmw_connextdds test_rclcpp.gtest_multiple_service_calls__rmw_connextdds.gtest.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-25 2024-09-15 14 5 35.71%	No reports found!
nightly_linux_repeated#3591	projectroot.test_subscription__rmw_connextdds test_tracetools.test_subscription__rmw_connextdds.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux_repeated 2024-09-24 2024-09-14 13 4 30.77%	No reports found!
nightly_linux-rhel_release#2029	projectroot.test.test_graph__rmw_fastrtps_cpp rcl.test_graph__rmw_fastrtps_cpp.gtest.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_release 2024-09-25 2024-09-10 15 4 26.67%	No reports found!
nightly_win_rep#3489	projectroot.test_tutorial_add_two_ints_server_add_two_ints_client_async__rmw_fastrtps_cpp demo_nodes_cpp.TestExecutablesTutorialAfterShutdown.test_last_process_exit_code demo_nodes_cpp.TestExecutablesTutorialAfterShutdown.test_last_process_exit_code	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-25 2024-09-10 14 3 21.43%	No reports found!

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__nightly-release_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
gz_gui-ci-ign-gui6-focal-amd64	2024-09-15
Ici__nightly-performance_ubuntu_jammy_amd64	2024-09-17
nightly_linux-rhel_repeated	2024-09-17
Jci__nightly-performance_ubuntu_noble_amd64	2024-09-17

Pending

Test regressions all

Reference Build	Error Name
nightly_win_rep#3489	36 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 2)
nightly_win_deb#3227	rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel (age: 1)
nightly_linux_repeated#3591	test_tracetools.test_subscription__rmw_connextdds.xunit.missing_result (age: 1)
nightly_linux-rhel_repeated#2038	robot_state_publisher.ChangeFixedJointTestAfterShutdown.test_exit_code (age: 1)
nightly_linux-rhel_release#2029	ros2cli.ros2cli.test.test_strategy.test_enforce_no_daemon (age: 1)
nightly_linux-rhel_debug#2050	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn (age: 1)
nightly_linux-aarch64_repeated#2895	rosbag2_tests.test_rosbag2_play_end_to_end.gtest.missing_result (age: 1)
gz_transport-ci-ign-transport11-homebrew-amd64#78	ignTest.TopicInfo (age: 21)
gz_transport-ci-gz-transport12-homebrew-amd64#80	UNIT_gz_TEST.xml.[failed-to-read] (age: 23)
gz_sim-ign-gazebo6-win#90	INTEGRATION_multiple_servers.test_ran (age: 1)
gz_sim-ci-gz-sim8-jammy-amd64#154	ResetFixture.HandleReset (age: 2)
gz_rendering-9-win#15	71 errors VisualTest.Clone (age: 15)
gz_rendering-8-win#88	70 errors VisualTest.Clone (age: 19)
gz_rendering-7-win#80	67 errors VisualTest.Clone (age: 19)
gz_gui-ci-ign-gui6-focal-amd64#94	UNIT_Teleop_TEST.test_ran (age: 2)
gz_common-ign-common3-win#79	PluginSpecialization.AccessTime (age: 1)
Rci__nightly-connext_ubuntu_noble_amd64#154	13 errors rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content (age: 11)
Rci__benchmark_ubuntu_noble_amd64#106	projectroot.test.benchmark.benchmark_parameter_client (age: 1)
Jci__nightly-release_ubuntu_noble_amd64#55	1 clang-tidy warnings (age: -1)
Jci__nightly-connext_ubuntu_noble_amd64#57	13 errors tf2_ros.tf2_ros_test_message_filter.gtest.missing_result (age: 1)
Ici__nightly-connext_ubuntu_jammy_amd64#170	13 errors test_tf2.BufferClientTestAfterShutdown.test_exit_code (age: 1)
Hci__nightly-connext_ubuntu_jammy_amd64#276	test_tf2.BufferClientTestAfterShutdown.test_exit_code (age: 1)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST

[Blast545]

Wednesday Sep 25th

Manual check of the buildfarms:
https://ci.ros2.org/job/nightly_linux_debug/3190/ and https://ci.ros2.org/job/nightly_linux-rhel_debug/2051/ have been running for more than 24 hours, I'm going to manually abort them and take a look.
https://ci.ros2.org/job/nightly_linux-rhel_debug/2052/ has been running for 13 hours, given that the average for these jobs is 2hours, I will cancel this one as well. Pending investigation on rhel_debug.

Deleting this offline agent: https://build.ros2.org/computer/agent-5c0a567e/, capacity seems normal

Manual check of the dashboards:
A lot of the ROS 2 jobs nightlies failed on github timeouts:
https://ci.ros2.org/view/nightly/job/nightly_linux_debug/3191/console, https://ci.ros2.org/view/nightly/job/nightly_linux_release/3212/console,
https://ci.ros2.org/job/nightly_linux_humble_coverage/943/console,
https://ci.ros2.org/job/nightly_linux_iron_coverage/578/console,
https://ci.ros2.org/job/nightly_linux_jazzy_coverage/161/console,
https://ci.ros2.org/job/nightly_linux_coverage/2533/console
Gazebo dashboard looks OK

nightly_win_rep#3489 and remaining important test regressions in the dashboard will take some time to dig into, given that those are related to connext, I decided to prioritize taking a look to the warning in Jazzy that's keeping the jobs from green there.

That one should be fixed after ros2/rosidl#811 gets backported.
Backport PR: ros2/rosidl#827

[github-actions]

Sep 26

Urgent

Build regressions

Reference Build	Failure DateTime	Failure Reason
gz_transport-ci-ign-transport11-homebrew-amd64#79	2024-09-25 20:06	Unknown Reason

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
Rci__nightly-connext_ubuntu_noble_amd64#155	14	2024-09-25 09:32:29.882000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#155	12	2024-09-25 09:32:29.882000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!
nightly_linux_humble_coverage#944	4	2024-09-26 06:22:57.038000	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test	https://github.com/ros2/launch_ros/issues/304 (Closed)
Jci__nightly-connext_ubuntu_noble_amd64#58	3	2024-09-26 06:15:03.185000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!

Warnings

Reference build	Age	Failure DateTime	Errors	Reports
Jci__nightly-release_ubuntu_noble_amd64#55	-1	2024-09-23 14:11:55.039000	1 gcc warnings 1 clang-tidy warnings	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
nightly_linux_repeated#3592	projectroot.test.rclcpp.test_clock rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-25 2024-09-14 13 8 61.54% nightly_linux_repeated 2024-09-25 2024-09-12 13 5 38.46%	No reports found!
nightly_linux-rhel_repeated#2039	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-26 2024-09-11 16 9 56.25% nightly_linux-aarch64_repeated 2024-09-23 2024-09-23 16 1 6.25%	No reports found!
nightly_win_deb#3228	projectroot.test_tutorial_use_logger_service__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-13 13 4 30.77% nightly_win_rel 2024-09-22 2024-09-20 16 2 12.5% nightly_win_deb 2024-09-26 2024-09-26 16 1 6.25%	No reports found!

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__nightly-release_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
gz_gui-ci-ign-gui6-focal-amd64	2024-09-15
Ici__nightly-performance_ubuntu_jammy_amd64	2024-09-17
nightly_linux-rhel_repeated	2024-09-17
Jci__nightly-performance_ubuntu_noble_amd64	2024-09-17

Pending

Test regressions all

Reference Build	Error Name
nightly_win_deb#3228	14 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 1)
nightly_linux_repeated#3592	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock (age: 1)
nightly_linux_humble_coverage#944	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (age: 4)
nightly_linux_coverage#2534	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (age: 9)
nightly_linux-rhel_repeated#2039	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3] (age: 2)
nightly_linux-aarch64_repeated#2896	rosbag2_py.test_transport_py.xunit.missing_result (age: 1)
gz_transport-ci-gz-transport14-homebrew-amd64#13	NodeTest.relay (age: 1)
gz_transport-ci-gz-transport12-homebrew-amd64#81	UNIT_gz_TEST.xml.[failed-to-read] (age: 24)
gz_sim-ign-gazebo6-win#90	INTEGRATION_multiple_servers.test_ran (age: 1)
gz_sim-ci-gz-sim8-jammy-amd64#154	ResetFixture.HandleReset (age: 2)
gz_rendering-9-win#15	71 errors VisualTest.Clone (age: 15)
gz_rendering-8-win#88	70 errors VisualTest.Clone (age: 19)
gz_rendering-7-win#80	67 errors VisualTest.Clone (age: 19)
gz_gui-ci-ign-gui6-focal-amd64#94	UNIT_Teleop_TEST.test_ran (age: 2)
gz_fuel_tools-ci-ign-fuel-tools7-homebrew-amd64#78	CollectionTest/DownloadCollectionTest.AllItems/2 (age: 1)
gz_common-ign-common3-win#79	PluginSpecialization.AccessTime (age: 1)
Rci__nightly-performance_ubuntu_noble_amd64#113	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 1)
Rci__nightly-connext_ubuntu_noble_amd64#155	15 errors rclpy.rclpy.test.test_client.TestClient.test_service_timestamps (age: 1)
Rci__benchmark_ubuntu_noble_amd64#107	projectroot.benchmark_string_conversions (age: 107)
Jci__nightly-release_ubuntu_noble_amd64#55	1 clang-tidy warnings (age: -1)
Jci__nightly-connext_ubuntu_noble_amd64#58	11 errors rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset (age: 4)
Ici__nightly-performance_ubuntu_jammy_amd64#130	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 2)
Ici__nightly-connext_ubuntu_jammy_amd64#170	13 errors test_tf2.BufferClientTestAfterShutdown.test_exit_code (age: 1)
Hci__nightly-connext_ubuntu_jammy_amd64#276	test_tf2.BufferClientTestAfterShutdown.test_exit_code (age: 1)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST

[Blast545]

Thursday Sep 26th

Manual check of the buildfarms:
Servers look OK, deleting: https://build.osrfoundation.org/computer/linux%2Dec2.nv.focal%2Da71622fa/ as it seems it was disconnected and replaced automatically
Aborting this job as it seems to be stuck again: https://ci.ros2.org/job/nightly_linux-rhel_debug/2053/

From the report, https://build.osrfoundation.org/job/gz_transport-ci-ign-transport11-homebrew-amd64/79 failed because there was an update going on in the agent running the job. After that, the agent has been working alright, re-running the job and keeping track. https://build.osrfoundation.org/job/gz_transport-ci-ign-transport11-homebrew-amd64/80

Keeping track: Cleaning workspace and Re-running https://build.osrfoundation.org/job/gz_physics-6-win/91/ and taking a look

[Blast545]

INVESTIGATING: https://ci.ros2.org/job/nightly_linux-rhel_debug aborted builds

There doesn't seem to be anything relevant new in https://github.com/ros2/launch_ros/, taking a look to repos.diff
The only changes that might have affected to this problem are:
ros2/rmw_fastrtps@565bbc2
and
ros2/rosbag2@cd7bd63
Taking a closer look tomorrow to see if this happens again and if we can reproduce it reliably in the buildfarm.

[Blast545]

INVESTIGATING: https://ci.ros2.org/job/nightly_linux_humble_coverage/944 (age 4)

(AGE 4 but I seem to have seen this error before in launch_testing_ros)
Reported in a closed issue: ros2/launch_ros#304
Requires a complete flaky ratio calculation to make sure when it started happening again.

[github-actions]

Sep 27

Urgent

Build regressions

Reference Build	Failure DateTime	Failure Reason
gz-gui9-debbuilder#596	2024-09-27 01:37	Unknown Reason
gz-gui9-debbuilder#594	2024-09-27 01:15	Failed to compile and install build
gz-launch8-debbuilder#579	2024-09-27 05:02	Failed to compile and install build
gz-physics8-debbuilder#597	2024-09-26 23:54	Failed to compile and install build
gz-physics8-debbuilder#595	2024-09-26 23:08	Failed to compile and install build
gz-rendering9-debbuilder#592	2024-09-26 20:42	Failed to compile and install build
gz-sensors9-debbuilder#592	2024-09-27 00:41	Failed to compile and install build
gz-sim9-debbuilder#591	2024-09-27 03:42	Failed to compile and install build
gz-sim9-debbuilder#588	2024-09-27 02:51	Failed to compile and install build

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
nightly_win_rep#3492	18	2024-09-27 04:00:01.764000	projectroot.gtest_multithreaded__rmw_connextdds	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#156	15	2024-09-26 14:25:21.218000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
nightly_win_rep#3492	7	2024-09-27 04:00:01.764000	test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result	No reports found!
nightly_win_rep#3492	4	2024-09-27 04:00:01.764000	projectroot.gtest_multiple_service_calls__rmw_connextdds test_rclcpp.gtest_multiple_service_calls__rmw_connextdds.gtest.missing_result	No reports found!
nightly_linux-rhel_repeated#2040	3	2024-09-27 04:00:01.060000	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	No reports found!
Jci__nightly-connext_ubuntu_noble_amd64#58	3	2024-09-26 06:15:03.185000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
nightly_win_deb#3229	projectroot.gtest_multithreaded__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-12 14 14 100.0% nightly_win_deb 2024-09-27 2024-09-12 16 6 37.5%	No reports found!
nightly_win_deb#3229	test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-14 14 10 71.43% nightly_win_deb 2024-09-27 2024-09-18 16 3 18.75%	No reports found!
nightly_win_rep#3492	projectroot.test_n_nodes__rmw_fastrtps_cpp test_rclcpp.TestNNodesAfterShutdown.test_10_nodes	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-16 14 6 42.86% nightly_linux-aarch64_repeated 2024-09-23 2024-09-13 16 2 12.5% nightly_linux_repeated 2024-09-22 2024-09-22 14 1 7.14% nightly_win_rel 2024-09-21 2024-09-21 16 1 6.25% nightly_win_deb 2024-09-15 2024-09-15 16 1 6.25%	No reports found!
nightly_win_rep#3492	test_rclcpp.TestNNodes.test_10_nodes	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-16 14 6 42.86% nightly_linux-aarch64_repeated 2024-09-23 2024-09-13 16 2 12.5% nightly_linux_repeated 2024-09-22 2024-09-22 14 1 7.14%	No reports found!
gz_sim-ci-gz-sim9-noble-amd64#17	UNIT_Server_TEST.test_ran	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage gz_sim-ci-gz-sim9-noble-amd64 2024-09-27 2024-09-14 7 3 42.86%	https://github.com/gazebosim/gz-sim/issues/2282 (Closed)
nightly_win_rep#3492	projectroot.test_play_publish_clock__rmw_cyclonedds_cpp	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-12 14 5 35.71%	No reports found!
nightly_win_rep#3492	projectroot.test_cli_remapping	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-13 14 5 35.71% nightly_linux_repeated 2024-09-14 2024-09-12 14 3 21.43% nightly_linux-aarch64_repeated 2024-09-21 2024-09-18 16 2 12.5%	No reports found!
nightly_linux-rhel_repeated#2040	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[2-3]	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-27 2024-09-13 16 5 31.25% nightly_win_rep 2024-09-16 2024-09-16 14 1 7.14%	No reports found!
nightly_win_rep#3492	projectroot.test_subscription rclpy.test_subscription.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-15 14 3 21.43% nightly_win_rel 2024-09-13 2024-09-13 16 1 6.25%	No reports found!
nightly_linux_repeated#3593	ros2bag.test.test_record.test_record	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux_repeated 2024-09-26 2024-09-13 14 3 21.43% nightly_linux-aarch64_repeated 2024-09-26 2024-09-22 16 2 12.5%	No reports found!

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
gz_gui-ci-ign-gui6-focal-amd64	2024-09-15
Ici__nightly-performance_ubuntu_jammy_amd64	2024-09-17
nightly_linux-rhel_repeated	2024-09-17
Jci__nightly-performance_ubuntu_noble_amd64	2024-09-17
gz_sim-ign-gazebo6-win	2024-09-19
gz_common-ign-common3-win	2024-09-19

Pending

Test regressions all

Reference Build	Error Name
nightly_win_rep#3492	19 errors rclpy.test_subscription.xunit.missing_result (age: 1)
nightly_win_rel#3182	rcl_action.TestActionClientServerInteraction.test_interaction (age: 1)
nightly_win_deb#3229	11 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 2)
nightly_linux_repeated#3593	ros2bag.test.test_record.test_record (age: 1)
nightly_linux_coverage#2534	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (age: 9)
nightly_linux-rhel_repeated#2040	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 1)
nightly_linux-aarch64_repeated#2897	test_tracetools.test_lifecycle_node.xunit.missing_result (age: 1)
nightly_linux-aarch64_release#2869	rosbag2_tests.test_rosbag2_play_end_to_end.gtest.missing_result (age: 1)
gz_transport-ci-ign-transport11-homebrew-amd64#80	ignTest.TopicInfo (age: 23)
gz_transport-ci-gz-transport14-homebrew-amd64#13	NodeTest.relay (age: 1)
gz_transport-ci-gz-transport12-homebrew-amd64#81	UNIT_gz_TEST.xml.[failed-to-read] (age: 24)
gz_sim-ign-gazebo6-win#90	INTEGRATION_multiple_servers.test_ran (age: 1)
gz_sim-ci-gz-sim9-noble-amd64#17	UNIT_Server_TEST.test_ran (age: 1)
gz_rendering-9-win#15	71 errors VisualTest.Clone (age: 15)
gz_rendering-8-win#88	70 errors VisualTest.Clone (age: 19)
gz_rendering-7-win#80	67 errors VisualTest.Clone (age: 19)
gz_fuel_tools-ci-ign-fuel-tools7-homebrew-amd64#78	CollectionTest/DownloadCollectionTest.AllItems/2 (age: 1)
gz_common-ign-common3-win#79	PluginSpecialization.AccessTime (age: 1)
Rci__nightly-connext_ubuntu_noble_amd64#156	10 errors rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos (age: 156)
Rci__benchmark_ubuntu_noble_amd64#108	projectroot.benchmark_string_conversions (age: 108)
Jci__nightly-connext_ubuntu_noble_amd64#58	11 errors rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset (age: 4)
Ici__nightly-performance_ubuntu_jammy_amd64#130	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 2)
Ici__nightly-connext_ubuntu_jammy_amd64#170	13 errors test_tf2.BufferClientTestAfterShutdown.test_exit_code (age: 1)
Hci__nightly-connext_ubuntu_jammy_amd64#277	ros2param.ros2param.test.test_verb_list.test_verb_list (age: 1)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST

[github-actions]

Sep 28

Urgent

Build regressions

Reference Build	Failure DateTime	Failure Reason
nightly_win_rep#3493	2024-09-28 04:00	Number of failed test exceeds the threshold
gz_rendering-9-win#16	2024-09-27 18:48	Early fail

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
Rci__nightly-connext_ubuntu_noble_amd64#157	16	2024-09-27 09:20:35.807000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
nightly_linux-rhel_repeated#2041	4	2024-09-28 04:00:01.345000	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	No reports found!
Jci__nightly-connext_ubuntu_noble_amd64#58	3	2024-09-26 06:15:03.185000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
nightly_win_rel#3183	projectroot.gtest_multithreaded__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-13 13 13 100.0% nightly_win_deb 2024-09-27 2024-09-18 16 5 31.25% nightly_win_rel 2024-09-28 2024-09-28 16 1 6.25%	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#157	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Rci__nightly-connext_ubuntu_noble_amd64 2024-09-27 2024-09-13 14 13 92.86% Jci__nightly-connext_ubuntu_noble_amd64 2024-09-26 2024-09-14 5 4 80.0% Ici__nightly-connext_ubuntu_jammy_amd64 2024-09-16 2024-09-16 6 1 16.67% nightly_win_deb 2024-09-18 2024-09-18 16 1 6.25%	No reports found!
nightly_win_rel#3183	test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-27 2024-09-14 13 10 76.92% nightly_win_deb 2024-09-27 2024-09-18 16 3 18.75% nightly_win_rel 2024-09-28 2024-09-28 16 1 6.25%	No reports found!
gz_sim-ci-gz-sim9-noble-amd64#17	UNIT_Server_TEST.test_ran	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage gz_sim-ci-gz-sim9-noble-amd64 2024-09-27 2024-09-14 7 3 42.86%	https://github.com/gazebosim/gz-sim/issues/2282 (Closed)
nightly_linux-rhel_repeated#2041	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[2-3]	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-28 2024-09-13 16 6 37.5% nightly_win_rep 2024-09-16 2024-09-16 13 1 7.69%	No reports found!
nightly_win_rel#3183	projectroot.test_tutorial_use_logger_service__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-22 2024-09-13 13 4 30.77% nightly_win_rel 2024-09-28 2024-09-20 16 3 18.75% nightly_win_deb 2024-09-26 2024-09-26 16 1 6.25%	No reports found!
nightly_linux-aarch64_repeated#2898	launch_testing_examples.launch_testing_examples.check_multiple_nodes_launch_test.launch_testing_examples.check_multiple_nodes_launch_test	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-aarch64_repeated 2024-09-28 2024-09-17 16 3 18.75% nightly_linux-rhel_repeated 2024-09-27 2024-09-20 16 2 12.5% nightly_linux_repeated 2024-09-23 2024-09-23 14 1 7.14%	https://github.com/ros2/examples/issues/340 (Closed)

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
Ici__nightly-performance_ubuntu_jammy_amd64	2024-09-17
nightly_linux-rhel_repeated	2024-09-17
Jci__nightly-performance_ubuntu_noble_amd64	2024-09-17
gz_sim-ign-gazebo6-win	2024-09-19

Pending

Test regressions all

Reference Build	Error Name
nightly_win_rel#3183	10 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 1)
nightly_win_deb#3230	launch_testing_ros.test.examples.talker_listener_launch_test.talker_listener_launch_test (age: 1)
nightly_linux_repeated#3594	launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test (age: 1)
nightly_linux_coverage#2536	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (age: 11)
nightly_linux-rhel_repeated#2041	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[2-3] (age: 2)
nightly_linux-aarch64_repeated#2898	rosbag2_py.test_transport_py.xunit.missing_result (age: 1)
gz_transport-ci-ign-transport11-homebrew-amd64#80	ignTest.TopicInfo (age: 23)
gz_transport-ci-gz-transport12-homebrew-amd64#81	UNIT_gz_TEST.xml.[failed-to-read] (age: 24)
gz_sim-ci-gz-sim9-noble-amd64#17	UNIT_Server_TEST.test_ran (age: 1)
gz_rendering-9-win#17	71 errors VisualTest.Clone (age: 17)
gz_rendering-7-win#80	67 errors VisualTest.Clone (age: 19)
gz_fuel_tools-ci-ign-fuel-tools7-homebrew-amd64#78	CollectionTest/DownloadCollectionTest.AllItems/2 (age: 1)
Rci__nightly-performance_ubuntu_noble_amd64#115	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 1)
Rci__nightly-connext_ubuntu_noble_amd64#157	13 errors rclcpp.TestGenericService.generic_service_qos (age: 16)
Rci__benchmark_ubuntu_noble_amd64#109	projectroot.benchmark_string_conversions (age: 109)
Jci__nightly-connext_ubuntu_noble_amd64#58	11 errors rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset (age: 4)
Ici__nightly-performance_ubuntu_jammy_amd64#130	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 2)
Ici__nightly-connext_ubuntu_jammy_amd64#171	11 errors launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 72)
Hci__nightly-connext_ubuntu_jammy_amd64#277	ros2param.ros2param.test.test_verb_list.test_verb_list (age: 1)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST

[github-actions]

Sep 29

Urgent

Test regressions consecutive

Test regressions

Reference build	Age	Failure DateTime	Errors	Reports
Rci__nightly-connext_ubuntu_noble_amd64#158	17	2024-09-28 09:30:33.497000	projectroot.test.rclcpp.test_generic_service rclcpp.TestGenericService.generic_service_qos	No reports found!
nightly_linux-rhel_repeated#2042	5	2024-09-29 04:00:01.168000	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	No reports found!
Jci__nightly-connext_ubuntu_noble_amd64#59	4	2024-09-29 06:15:01.637000	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	No reports found!
nightly_win_rep#3494	3	2024-09-29 04:00:01.364000	projectroot.test_subscription rclpy.test_subscription.xunit.missing_result	No reports found!
Ici__nightly-performance_ubuntu_jammy_amd64#131	3	2024-09-29 03:22:08.146000	projectroot.test.test_performance_CycloneDDS_sync_Array8m projectroot.test.test_performance_CycloneDDS_sync_PointCloud8m buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS	No reports found!

Test regressions flaky

Test regressions

Reference builds	Errors	Flaky report	Reports
nightly_win_rep#3494 nightly_win_deb#3231	projectroot.gtest_multithreaded__rmw_connextdds projectroot.gtest_multithreaded__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 13 100.0% nightly_win_deb 2024-09-29 2024-09-18 16 6 37.5% nightly_win_rel 2024-09-28 2024-09-28 16 1 6.25%	No reports found!
Rci__nightly-connext_ubuntu_noble_amd64#158	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Rci__nightly-connext_ubuntu_noble_amd64 2024-09-28 2024-09-15 14 13 92.86% Jci__nightly-connext_ubuntu_noble_amd64 2024-09-29 2024-09-14 6 5 83.33% Ici__nightly-connext_ubuntu_jammy_amd64 2024-09-16 2024-09-16 5 1 20.0% nightly_win_deb 2024-09-18 2024-09-18 16 1 6.25%	No reports found!
nightly_win_rep#3494 nightly_win_deb#3231	test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 11 84.62% nightly_win_deb 2024-09-29 2024-09-18 16 4 25.0% nightly_win_rel 2024-09-28 2024-09-28 16 1 6.25%	No reports found!
nightly_win_rep#3494	projectroot.test.rclcpp.test_clock rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 9 69.23% nightly_linux_repeated 2024-09-25 2024-09-14 13 4 30.77%	No reports found!
nightly_win_rep#3494	test_rclcpp.test_timeout_subscriber.timeout_subscriber test_rclcpp.test_timeout_subscriber.timeout_subscriber	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 9 69.23%	No reports found!
Rci__nightly-performance_ubuntu_noble_amd64#116	projectroot.test.test_performance_CycloneDDS_sync_PointCloud8m buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage Ici__nightly-performance_ubuntu_jammy_amd64 2024-09-29 2024-09-20 5 3 60.0% Rci__nightly-performance_ubuntu_noble_amd64 2024-09-28 2024-09-25 14 3 21.43%	No reports found!
nightly_win_rep#3494	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-rhel_repeated 2024-09-29 2024-09-14 16 9 56.25% nightly_win_rep 2024-09-29 2024-09-29 13 1 7.69% nightly_linux-aarch64_repeated 2024-09-23 2024-09-23 16 1 6.25%	No reports found!
nightly_win_rep#3494	projectroot.gtest_timeout_subscriber__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 7 53.85%	No reports found!
nightly_win_rep#3494	projectroot.test_n_nodes__rmw_fastrtps_cpp test_rclcpp.TestNNodesAfterShutdown.test_10_nodes	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-16 13 7 53.85% nightly_linux_repeated 2024-09-22 2024-09-22 13 1 7.69% nightly_linux-aarch64_repeated 2024-09-23 2024-09-23 16 1 6.25% nightly_win_rel 2024-09-21 2024-09-21 16 1 6.25% nightly_win_deb 2024-09-15 2024-09-15 16 1 6.25%	No reports found!
nightly_win_rep#3494	test_rclcpp.TestNNodes.test_10_nodes	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-16 13 7 53.85% nightly_linux_repeated 2024-09-22 2024-09-22 13 1 7.69% nightly_linux-aarch64_repeated 2024-09-23 2024-09-23 16 1 6.25%	No reports found!
nightly_linux_humble_coverage#948	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux_humble_coverage 2024-09-29 2024-09-14 14 7 50.0% nightly_linux_iron_coverage 2024-09-24 2024-09-14 12 4 33.33% nightly_linux-rhel_repeated 2024-09-24 2024-09-23 16 2 12.5%	https://github.com/ros2/launch_ros/issues/304 (Closed)
gz_sim-ci-gz-sim9-noble-amd64#17	UNIT_Server_TEST.test_ran	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage gz_sim-ci-gz-sim9-noble-amd64 2024-09-27 2024-09-14 6 3 50.0%	https://github.com/gazebosim/gz-sim/issues/2282 (Closed)
nightly_win_rep#3494	projectroot.test_play_publish_clock__rmw_cyclonedds_cpp	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-16 13 5 38.46%	No reports found!
nightly_win_rep#3494	projectroot.test_tutorial_use_logger_service__rmw_connextdds	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 4 30.77% nightly_win_rel 2024-09-28 2024-09-20 16 3 18.75% nightly_win_deb 2024-09-26 2024-09-26 16 1 6.25%	No reports found!
nightly_linux-aarch64_repeated#2899	launch_testing_examples.launch_testing_examples.check_multiple_nodes_launch_test.launch_testing_examples.check_multiple_nodes_launch_test	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-aarch64_repeated 2024-09-29 2024-09-17 16 4 25.0% nightly_linux-rhel_repeated 2024-09-27 2024-09-20 16 2 12.5% nightly_linux_repeated 2024-09-23 2024-09-23 13 1 7.69%	https://github.com/ros2/examples/issues/340 (Closed)
nightly_win_rep#3494	projectroot.test_action_communication__rmw_fastrtps_cpp	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-19 13 3 23.08%	No reports found!
nightly_win_rep#3494	projectroot.gtest_timeout_subscriber__rmw_cyclonedds_cpp	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-14 13 3 23.08%	No reports found!
nightly_win_rep#3494	projectroot.test_time_source projectroot.test_timer projectroot.test_waitable projectroot.test_wait_for_message rclpy.test_wait_for_message.xunit.missing_result rclpy.test_timer.xunit.missing_result rclpy.test_time_source.xunit.missing_result rclpy.test_waitable.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-15 13 3 23.08%	No reports found!
nightly_win_rep#3494	projectroot.test_record_all__rmw_cyclonedds_cpp	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_rep 2024-09-29 2024-09-17 13 3 23.08%	No reports found!
nightly_win_rep#3494 nightly_win_rel#3184 nightly_win_deb#3231	projectroot.test_node rclpy.test_node.xunit.missing_result projectroot.test_node rclpy.test_node.xunit.missing_result projectroot.test_node rclpy.test_node.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_win_deb 2024-09-29 2024-09-14 16 3 18.75% nightly_win_rep 2024-09-29 2024-09-15 13 2 15.38% nightly_win_rel 2024-09-29 2024-09-29 16 1 6.25%	No reports found!
nightly_linux-rhel_repeated#2042	projectroot.test_node_name__rmw_fastrtps_cpp test_rclcpp.test_node_name__rmw_fastrtps_cpp.xunit.missing_result	Job Name Last Fail First Fail Build Count Failure Count Failure Percentage nightly_linux-aarch64_repeated 2024-09-23 2024-09-20 16 3 18.75% nightly_linux-rhel_repeated 2024-09-29 2024-09-29 16 1 6.25%	No reports found!

Maintenance

Jobs last success date

Job Name	Last Success
Hci__nightly-connext_ubuntu_jammy_amd64	Never
Ici__nightly-connext_ubuntu_jammy_amd64	Never
Rci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-8-win	Never
gz_rendering-7-win	Never
Jci__nightly-connext_ubuntu_noble_amd64	Never
Jci__benchmark_ubuntu_noble_amd64	Never
gz_rendering-9-win	Never
nightly_win_rep	2024-01-12
Rci__nightly-connext_ubuntu_noble_amd64	2024-05-04
nightly_linux_coverage	2024-05-24
nightly_win_deb	2024-06-03
gz_rendering-ign-rendering6-win	2024-07-02
gz_transport-ci-gz-transport12-homebrew-amd64	2024-07-07
gz_rendering-ign-rendering3-win	2024-07-07
gz_transport-ci-ign-transport11-homebrew-amd64	2024-07-08
nightly_linux_repeated	2024-08-14
nightly_linux-aarch64_repeated	2024-09-09
gz_sim-ci-gz-sim8-jammy-amd64	2024-09-15
Ici__nightly-performance_ubuntu_jammy_amd64	2024-09-17
nightly_linux-rhel_repeated	2024-09-17
Jci__nightly-performance_ubuntu_noble_amd64	2024-09-17
gz_sim-ign-gazebo6-win	2024-09-19

Pending

Test regressions all

Reference Build	Error Name
nightly_win_rep#3494	70 errors test_rclcpp.TestNNodesAfterShutdown.test_10_nodes (age: 1)
nightly_win_rel#3184	rclpy.test_client.xunit.missing_result (age: 1)
nightly_win_deb#3231	53 errors demo_nodes_cpp.TestExecutablesTutorial.test_processes_output (age: 1)
nightly_linux_repeated#3594	launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test (age: 1)
nightly_linux_humble_coverage#948	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (age: 1)
nightly_linux_coverage#2537	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (age: 12)
nightly_linux-rhel_repeated#2042	test_rclcpp.test_node_name__rmw_fastrtps_cpp.xunit.missing_result (age: 1)
nightly_linux-aarch64_repeated#2899	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 1)
gz_transport-ci-ign-transport11-homebrew-amd64#80	ignTest.TopicInfo (age: 23)
gz_transport-ci-gz-transport12-homebrew-amd64#81	UNIT_gz_TEST.xml.[failed-to-read] (age: 24)
gz_sim-ign-gazebo6-win#91	ServerRepeat/MultipleServers.TwoServersNonBlocking/0 (age: 1)
gz_sim-ci-gz-sim9-noble-amd64#17	UNIT_Server_TEST.test_ran (age: 1)
gz_rendering-9-win#17	71 errors VisualTest.Clone (age: 17)
gz_rendering-8-win#89	70 errors VisualTest.Clone (age: 20)
gz_rendering-7-win#81	67 errors VisualTest.Clone (age: 20)
gz_fuel_tools-ci-ign-fuel-tools7-homebrew-amd64#78	CollectionTest/DownloadCollectionTest.AllItems/2 (age: 1)
Rci__nightly-performance_ubuntu_noble_amd64#116	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 2)
Rci__nightly-connext_ubuntu_noble_amd64#158	15 errors rclpy.rclpy.test.test_client.TestClient.test_service_timestamps (age: 1)
Rci__benchmark_ubuntu_noble_amd64#110	projectroot.benchmark_string_conversions (age: 110)
Jci__nightly-connext_ubuntu_noble_amd64#59	13 errors rclpy.rclpy.test.test_client.TestClient.test_service_timestamps (age: 1)
Ici__nightly-performance_ubuntu_jammy_amd64#131	buildfarm_perf_tests.PerformanceTestResults.test_results_CycloneDDS (age: 3)
Ici__nightly-connext_ubuntu_jammy_amd64#171	11 errors launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test (age: 72)
Hci__nightly-connext_ubuntu_jammy_amd64#277	ros2param.ros2param.test.test_verb_list.test_verb_list (age: 1)

Test regressions known

Ros

Open issues

Issue	Jobs Name	Errors Name
https://github.com/ros2/demos/issues/693	nightly_win_rep nightly_win_deb nightly_linux-rhel_repeated	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output projectroot.test_tutorial_parameter_events_async__rmw_connextdds projectroot.test_tutorial_parameter_events__rmw_connextdds
https://github.com/ros2/rosidl/issues/812	nightly_linux_coverage	projectroot.test_multi_nested_message_introspection rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
https://github.com/ros2/rcl/issues/1162	nightly_win_deb	projectroot.test_action_interaction__rmw_connextdds rcl_action.TestActionClientServerInteraction.test_interaction rcl_action.TestActionClientServerInteraction.test_interaction_with_cancel
https://github.com/ros2/rclcpp/issues/2582	Jci__nightly-connext_ubuntu_noble_amd64 Rci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_service rclcpp.TestService.server_qos projectroot.test.rclcpp.test_client_common rclcpp.TestAllClientTypesWithServer/Client.client_qos rclcpp.TestAllClientTypesWithServer/GenericClient.client_qos
https://github.com/ros2/rmw_connextdds/issues/155	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	11 errors projectroot.test.rclcpp.test_client rclcpp.TestClient.on_new_response_callback projectroot.test.rclcpp.test_qos_event rclcpp.TestQosEvent.test_on_new_event_callback rclcpp.TestQosEvent.test_invalid_on_new_event_callback projectroot.test.rclcpp.test_service rclcpp.TestService.on_new_request_callback projectroot.test.rclcpp.test_subscription rclcpp.TestSubscription.on_new_message_callback projectroot.test.rclcpp.test_executors rclcpp.TestIntraprocessExecutors/EventsExecutor.testIntraprocessRetrigger
https://github.com/ros2/rclcpp/issues/2588	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64	projectroot.test.rclcpp.test_subscription_topic_statistics rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_for_message_no_header rclcpp.TestSubscriptionTopicStatisticsFixture.test_receive_stats_include_window_reset
https://github.com/ros2/rosidl_typesupport_fastrtps/issues/119	Rci__benchmark_ubuntu_noble_amd64 Jci__benchmark_ubuntu_noble_amd64	projectroot.benchmark_string_conversions
https://github.com/ros2/launch_ros/issues/408	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_ros.test.examples.check_node_launch_test.check_node_launch_test
https://github.com/ros2/rclcpp/issues/2611	Ici__nightly-connext_ubuntu_jammy_amd64	rclcpp.TestClient.client_qos
https://github.com/ros2/examples/issues/391	Hci__nightly-connext_ubuntu_jammy_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	launch_testing_examples.launch_testing_examples.set_param_launch_test.launch_testing_examples.set_param_launch_test launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
https://github.com/ros2/rclpy/issues/1347	Hci__nightly-connext_ubuntu_jammy_amd64	projectroot.test_client rclpy.rclpy.test.test_client.TestClient.test_service_timestamps
https://github.com/ros2/rclcpp/issues/2613	Rci__nightly-connext_ubuntu_noble_amd64 Jci__nightly-connext_ubuntu_noble_amd64 Ici__nightly-connext_ubuntu_jammy_amd64	projectroot.test.rclcpp.test_service_introspection rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
https://github.com/ros2/launch/issues/796	nightly_linux_coverage	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time

Gazebo

Open issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-rendering/issues/745	gz_rendering-7-win gz_rendering-8-win gz_rendering-9-win	71 errors CameraTest.ViewProjectionMatrix CameraTest.RenderTexture CameraTest.TrackFollow CameraTest.AddRemoveRenderPass CameraTest.VisibilityMask CameraTest.IntrinsicMatrix GpuRaysTest.Configure LidarVisualTest.Configure LidarVisualTest.LidarVisual ProjectorTest.Projector SceneTest.Scene SceneTest.SceneGradient SceneTest.Nodes SceneTest.RemoveNodes SceneTest.DestroyNodes SceneTest.NodeCycle SceneTest.Materials SceneTest.Time SceneTest.BackgroundMaterial ThermalCameraTest.ThermalCamera ReloadEngineTest.Scene ArrowVisualTest.ArrowVisual AxisVisualTest.AxisVisual COMVisualTest.COMVisual CapsuleTest.Capsule GizmoVisualTest.GizmoVisual GizmoVisualTest.Material GridTest.Grid InertiaVisualTest.InertiaVisual LightVisualTest.LightVisual LightTest.Light MarkerTest.Marker MaterialTest.MaterialProperties MeshDescriptorTest.Descriptor MeshTest.MeshSubMesh MeshTest.MeshSkeleton MeshTest.MeshSkeletonAnimation MeshTest.MeshClone MoveToHelperTest.MoveTo NodeTest.Pose OrbitViewControllerTest.OrbitViewControl OrbitViewControllerTest.Constructor OrbitViewControllerTest.Control OrthoViewControllerTest.OrthoViewControl OrthoViewControllerTest.Control ParticleEmitterTest.ParticleEmitter RayQueryTest.RayQuery RenderTargetTest.RenderTexture RenderTargetTest.RenderWindow RenderTargetTest.AddRemoveRenderPass RenderingIfaceTest.GetEngine TextTest.Text TransformControllerTest.TransformControl TransformControllerTest.WorldSpace TransformControllerTest.LocalSpace TransformControllerTest.Control2d VisualTest.Material VisualTest.Children VisualTest.Scale VisualTest.UserData VisualTest.Geometry VisualTest.VisibilityFlags VisualTest.BoundingBox VisualTest.Wireframe VisualTest.Clone WavesTest.Waves MeshTest.NormalMapWithoutTexCoord GizmoVisualTest.LookAt FontTestInstantiation/FontTest.SupportedFont/0 FontTestInstantiation/FontTest.SupportedFont/1 FontTestInstantiation/FontTest.SupportedFont/2
https://github.com/gazebosim/gz-transport/issues/509	gz_transport-ci-gz-transport12-homebrew-amd64	UNIT_gz_TEST.xml.[failed-to-read]
https://github.com/gazebosim/gz-transport/issues/510	gz_transport-ci-ign-transport11-homebrew-amd64	ignTest.TopicInfo

Disabled issues

Issue	Jobs Name	Errors Name
https://github.com/gazebosim/gz-transport/issues/195	gz_transport-ign-transport8-win gz_transport-ign-transport11-win gz_transport-12-win gz_transport-13-win	playback.ReplayStep
https://github.com/gazebosim/gz-rendering/issues/847	gz_rendering-ci-ign-rendering3-homebrew-amd64 gz_rendering-ci-ign-rendering6-homebrew-amd64	34 errors UNIT_Visual_TEST.test_ran UNIT_TransformController_TEST.test_ran UNIT_ThermalCamera_TEST.test_ran UNIT_Scene_TEST.test_ran UNIT_RenderingIface_TEST.test_ran UNIT_RenderTarget_TEST.test_ran UNIT_RayQuery_TEST.test_ran UNIT_OrbitViewController_TEST.test_ran UNIT_Node_TEST.test_ran UNIT_MoveToHelper_TEST.test_ran UNIT_Mesh_TEST.test_ran UNIT_Material_TEST.test_ran UNIT_Marker_TEST.test_ran UNIT_Light_TEST.test_ran UNIT_Grid_TEST.test_ran UNIT_Camera_TEST.test_ran UNIT_AxisVisual_TEST.test_ran UNIT_ArrowVisual_TEST.test_ran INTEGRATION_shadows.test_ran INTEGRATION_scene.test_ran INTEGRATION_render_pass.test_ran INTEGRATION_depth_camera.test_ran INTEGRATION_camera.test_ran INTEGRATION_thermal_camera.test_ran INTEGRATION_gpu_rays.test_ran INTEGRATION_lidar_visual.test_ran UNIT_Capsule_TEST.test_ran UNIT_Heightmap_TEST.test_ran UNIT_JointVisual_TEST.test_ran UNIT_LidarVisual_TEST.test_ran UNIT_LightVisual_TEST.test_ran UNIT_OrthoViewController_TEST.test_ran UNIT_Text_TEST.test_ran UNIT_WireBox_TEST.test_ran
https://github.com/gazebosim/gz-sim/issues/1887	gz_sim-ci-ign-gazebo6-homebrew-amd64	.gz-sim.python.test.testFixture_TEST