Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add cache for all filters #153

Merged
merged 11 commits into from
Sep 28, 2024
Merged

feat: Add cache for all filters #153

merged 11 commits into from
Sep 28, 2024

Conversation

shouya
Copy link
Owner

@shouya shouya commented Sep 28, 2024

Background

In PR #112, @olehbozhok implemented caching for the full_text filter, significantly boosting performance on resource-constrained devices. This improvement is particularly effective for filters requiring network fetches or HTML processing.

Current Implementation

This PR extends the caching behavior to all filters, with two levels of cache granularity:

  1. Feed-level cache (default):

    • Skips the filter if the input feed is found in the cache
    • Expected to be effective due to infrequent feed changes between fetches
  2. Post-level cache (opt-in):

    • Caches individual posts
    • Run the filter on a modified feed with posts not found in the cache
    • Correctness depends on filter implementation, thus opt-in by certain filters only
    • Feed-level cache is still checked first

Both cache levels use expiring LRU (Least Recently Used) caches with hard expiration times:

  • Feed-level: size=5, TTL=12 hours
  • Post-level: size=40, TTL=1 hour

Additional Features

  • Cache parameters are currently hard-coded but can be made configurable upon request
  • Caches persist across hot-reloads due to config changes
    • Only caches for changed filters are invalidated during hot-reloads

@shouya shouya merged commit 952e7da into master Sep 28, 2024
2 checks passed
@shouya shouya deleted the filter-cache branch September 28, 2024 02:53
@shouya shouya mentioned this pull request Sep 30, 2024
shouya added a commit that referenced this pull request Sep 30, 2024
This PR combines two features:

- make `console.{log,error}` outputs more clear (fixes #152)
- show js logs on webui (fixes #71)

Screenshot:


![image](https://github.com/user-attachments/assets/f696d1e4-a146-4202-b0d9-b620f5d81e4b)

Note: due to [the way caching is
implemented](#153), refreshing
the webpage without changing the config may skip js evaluation, thus the
log may not show up as expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant