Skip to content

Conversation

@DenzelPenzel
Copy link
Contributor

@DenzelPenzel DenzelPenzel commented Nov 20, 2025

@DenzelPenzel DenzelPenzel requested review from a team as code owners November 20, 2025 11:31
@DenzelPenzel DenzelPenzel added R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests. labels Nov 20, 2025
@DenzelPenzel DenzelPenzel changed the title Add substrate-zombienet-sdk-tests package and update workflows [DRAFT] Add substrate-zombienet-sdk-tests package and update workflows Nov 20, 2025
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch 2 times, most recently from 55b8f31 to 003b354 Compare November 20, 2025 15:18
@DenzelPenzel DenzelPenzel changed the title [DRAFT] Add substrate-zombienet-sdk-tests package and update workflows Add substrate-zombienet-sdk-tests package and update workflows Nov 20, 2025
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch 2 times, most recently from e8ef9a7 to 1da2ab1 Compare November 20, 2025 20:03
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch 2 times, most recently from 269e621 to cf1e1f0 Compare November 21, 2025 12:08
let db_snapshot_height = resolve_db_snapshot_height(&network).await?;

verify_bootstrap_height(&network, db_snapshot_height).await?;
// verify_new_blocks(&network, db_snapshot_height).await?;
Copy link
Contributor Author

@DenzelPenzel DenzelPenzel Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pepoviola this func failed for block_height{status="best"} and height > db_snapshot_height if delay <= 75 sec, maybe need to incr timeout wdyt?

Copy link
Contributor Author

@DenzelPenzel DenzelPenzel Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

increased timeout to 120 sec, tests passed

@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch 2 times, most recently from 850dfae to d22f46d Compare November 21, 2025 17:02
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch 4 times, most recently from aa0aea7 to ed7f157 Compare November 24, 2025 17:22
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch from ed7f157 to fa3d009 Compare November 25, 2025 17:37
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch from 8c3c197 to 5f1adb3 Compare November 26, 2025 13:38
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch from 5f1adb3 to 9e38b28 Compare November 26, 2025 18:37
@pepoviola
Copy link
Contributor

@DenzelPenzel can you include the metrics of a long run, like 30/50 times for each test.
Thx!

@pepoviola
Copy link
Contributor

ping @skunert / @michalkucharczyk / @lrubasze to check that we don't miss any behavior assertion in the migration.
Thx!

@DenzelPenzel
Copy link
Contributor Author

DenzelPenzel commented Nov 27, 2025

Zombienet Substrate Test Stability Report

Test Name Iterations Result
zombienet-substrate-0000-block-building 30 ✅ No errors found
zombienet-substrate-0001-basic-warp-sync 40 ✅ No errors found
zombienet-substrate-0003-block-building-warp-sync ~40 3 errors

Failure Analysis: zombienet-substrate-0003-block-building-warp-sync

Example of the issue: View Log (Job #56330027506)

Error Details:
The test failed with exit code 100 due to a timeout waiting for the block height metric.

Error: Timeout (95), waiting for metric block_height{status="best"} pass the predicate

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
test zombie_ci::block_building_warp_sync::block_building_warp_sync ... FAILED

failures:
    zombie_ci::block_building_warp_sync::block_building_warp_sync

@pepoviola
Copy link
Contributor

@DenzelPenzel is weird that the logs of this one Job #56330027506 show that the three node are up ok. Maybe we need to re-run with a more verbose debug level to check the return value from prometheus.

Thx!

@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch 3 times, most recently from 69c0945 to fb9503c Compare November 27, 2025 19:07
…k-tests' into zombienet-substrate-zombienet-sdk-tests
@DenzelPenzel DenzelPenzel force-pushed the zombienet-substrate-zombienet-sdk-tests branch from fb9503c to f17c3af Compare November 27, 2025 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants