-
Notifications
You must be signed in to change notification settings - Fork 3
Fix flaky timed tests #344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #344 +/- ##
=======================================
Coverage 40.02% 40.02%
=======================================
Files 45 45
Lines 4240 4240
Branches 380 380
=======================================
Hits 1697 1697
Misses 2523 2523
Partials 20 20
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR addresses flaky timing-dependent tests by replacing real time delays with a deterministic clock mechanism. Instead of relying on actual wall-clock time (which can vary across machines and CI environments), the tests now use a patched time.monotonic that can be manually advanced.
Key Changes
- Added a
patched_clockpytest fixture that provides aFakeClockclass to replacetime.monotonicwith deterministic time control - Modified three test files to use the
patched_clockfixture instead oftime.sleep()for time advancement - Updated the README.rst Zenodo badge to use a DOI-specific URL
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/python/conftest.py | Adds the patched_clock fixture with a FakeClock class that provides deterministic time control for tests |
| tests/python/unit/services/test_services_usage.py | Removes time import and @flaky decorator; updates tests to use patched_clock.advance() instead of time.sleep() |
| tests/python/unit/services/test_services_base.py | Removes time import and @flaky decorator; updates test to use patched_clock.advance() instead of time.sleep() |
| tests/python/integration/test_integrated.py | Adds import for retry_module; creates fake_sleep to advance patched clock during async sleep; patches retry module's asyncio and random for deterministic behavior |
| README.rst | Updates Zenodo badge URL to include specific DOI reference |
A few of our tests kept failing on GHA (and presumably other machines) because they relied on precise timings. In this PR, we remove the dependency on machine timing by patching the time module and manually advancing the clock to make sure our code behaves as expected when the time changes.