Skip to content

Comments

[MrBot] Add timed test suite validation to build_test_folder#277

Open
parth21999 wants to merge 4 commits intomasterfrom
paaggarwal/timed-test-suite-validation
Open

[MrBot] Add timed test suite validation to build_test_folder#277
parth21999 wants to merge 4 commits intomasterfrom
paaggarwal/timed-test-suite-validation

Conversation

@parth21999
Copy link
Member

Summary

  • Adds validate_timed_test_suite() CMake function that checks integration test files use TIMED_TEST_SUITE_INITIALIZE/CLEANUP macros
  • Calls validation automatically from build_test_folder when building integration tests (*int*) on Windows
  • Supports skip regex via -Dtimed_test_suite_skip_regex="folder_a|folder_b" for folders that need exemption

Depends on c-build-tools PR that moves the validation script to build_functions/validate_timed_test_suite.ps1.

Test plan

  • In a consumer repo with integration tests, run cmake ... -Drun_int_tests=ON and verify validation fires automatically
  • With skip regex set, confirm matching test folders are not validated
  • Verify unit tests still pass: cmake --build cmake --target test_validate_timed_test_suite

🤖 Generated with Claude Code

@parth21999
Copy link
Member Author

/AzurePipelines run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@parth21999 parth21999 force-pushed the paaggarwal/timed-test-suite-validation branch from 9b86126 to 69e29f3 Compare February 12, 2026 00:15
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@parth21999 parth21999 force-pushed the paaggarwal/timed-test-suite-validation branch from 69e29f3 to 853d82e Compare February 12, 2026 00:24
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@parth21999
Copy link
Member Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).


# Note: We don't add test_validate_timed_test_suite_detection as a dependency because
# it should fail with exit code 1 (which would fail the build).
# Instead, it's available to run manually to verify detection works.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

add all tests as dependencies. even the ones that fail. add an NumExpectedViolation flag to the script that makes the script exit 0 even if number of expected violations matches the number of actual violations. flag is default 0 obviously.

WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "Testing timed test suite validation on clean files"
)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests should use the cmake function validate_timed_test_suite.

Add a test cases where the skip regex causes the check to be skipped.

… regex test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant