add return_stale_on_timeout parameter for stale-while-revalidate caching
#297
Conversation
…e caching Adds new `return_stale_on_timeout` parameter that enables returning stale cached values when `wait_for_calc_timeout` expires, instead of triggering a new calculation. This implements a stale-while-revalidate pattern that keeps applications responsive while ensuring background cache refresh. Key changes: - Add return_stale_on_timeout parameter to all cache backends - Modify core logic to return stale values on RecalculationNeeded exception - Fix memory core timeout handling to properly check wait_for_calc_timeout - Add comprehensive test suite with 7 test cases - Update documentation and maintain backward compatibility When enabled, the behavior follows this pattern: 1. Fresh values (≤ stale_after) return immediately 2. Stale values trigger background refresh 3. Caller waits up to wait_for_calc_timeout for refresh to complete 4. If timeout expires, return stale value instead of blocking 5. Background refresh continues for next request Closes: Implements stale-while-revalidate caching pattern
|
bugbot run |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #297 +/- ##
===========================================
- Coverage 97.84% 63.39% -34.46%
===========================================
Files 8 12 +4
Lines 511 948 +437
Branches 88 108 +20
===========================================
+ Hits 500 601 +101
- Misses 10 329 +319
- Partials 1 18 +17
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
|
Please rebase over master before continuing to work - a lot of new code was added recently. @zscgeek |
Hi! I will do in the next day or so. |
Borda
changed the title
return_stale_on_timeout parameter for stale-while-revalidate caching
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new return_stale_on_timeout parameter that implements stale-while-revalidate caching behavior. When enabled, the feature returns cached stale values if wait_for_calc_timeout expires, instead of triggering a new calculation, helping keep applications responsive during cache refreshes.
Key Changes:
- Add
return_stale_on_timeoutparameter to all cache backends and core logic - Fix memory core timeout handling to properly respect
wait_for_calc_timeout - Implement comprehensive test coverage with 7 test scenarios
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/cachier/core.py |
Main decorator updated with new parameter and stale-return logic |
src/cachier/cores/base.py |
Base class updated to accept new parameter |
src/cachier/cores/memory.py |
Memory backend enhanced with proper timeout handling |
src/cachier/cores/pickle.py |
Pickle backend updated with new parameter |
src/cachier/cores/mongo.py |
MongoDB backend updated with new parameter |
src/cachier/cores/sql.py |
SQL backend updated with new parameter |
src/cachier/cores/redis.py |
Redis backend updated with new parameter |
src/cachier/config.py |
Global configuration updated with new parameter |
tests/test_return_stale_on_timeout.py |
Comprehensive test suite for the new feature |
demo_return_stale_on_timeout.py |
Demo script showcasing the feature |
| # If we weren't signaled and the entry is still processing, check timeout | ||
| if not signaled: | ||
| time_spent += 1 | ||
| self.check_calc_timeout(time_spent) |
There was a problem hiding this comment.
The variable hash_key is used but not defined in this method. It should be key (which is the parameter) or self._hash_func_key(key) to get the correct hash key.
| self.check_calc_timeout(time_spent) | |
| self.check_calc_timeout(time_spent, hash_key) |
Copilot uses AI. Check for mistakes.
|
@zscgeek I have resolved conflicts, but I am unable to push the update to your PR :( |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR is being reviewed by Cursor Bugbot
Details
You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
| except RecalculationNeeded: | ||
| if _return_stale_on_timeout and entry and entry.value is not None: | ||
| _print("Timeout reached, returning stale value.") | ||
| return entry.value |
There was a problem hiding this comment.
Bug: Cache Stale Value Handling Issues
The return_stale_on_timeout logic at lines 378 and 399 has two issues. It prevents returning stale None values when allow_none=True, inconsistent with other None handling. Additionally, returning a stale value on timeout leaves the cache entry marked as _processing=True, causing an inconsistent state for subsequent lookups.
Additional Locations (1)
| # If we weren't signaled and the entry is still processing, check timeout | ||
| if not signaled: | ||
| time_spent += 1 | ||
| self.check_calc_timeout(time_spent) |
There was a problem hiding this comment.
Bug: Race Condition and Unnecessary Timeout Check
The wait_on_entry_calc method has two issues. A race condition can cause an AttributeError if entry._condition becomes None between releasing and re-acquiring the lock in the waiting loop. Additionally, the timeout check may be called unnecessarily, potentially raising RecalculationNeeded even if the calculation completes successfully, if it finishes right after a wait timeout.
3 participants
feat: add return_stale_on_timeout parameter for stale-while-revalidate caching
Adds new
return_stale_on_timeoutparameter that enables returning stalecached values when
wait_for_calc_timeoutexpires, instead of triggeringa new calculation. This implements a stale-while-revalidate pattern that
keeps applications responsive while ensuring background cache refresh.
Key changes:
When enabled, the behavior follows this pattern: