feat(views): Add IP-based rate limit tracking for 404 errors#19468
Draft
miketheman wants to merge 1 commit intomainfrom
Draft
feat(views): Add IP-based rate limit tracking for 404 errors#19468miketheman wants to merge 1 commit intomainfrom
miketheman wants to merge 1 commit intomainfrom
Conversation
This adds an observation-mode rate limit implementation for excessive 404 errors from a given client IP. Currently the implementation: - Tracks 404 requests per IP using the existing rate limiting infrastructure - Emits metrics (warehouse.notfound.ratelimiter.hit and warehouse.notfound.ratelimiter.exceeded) for monitoring - Logs warnings when rate limits are exceeded for investigation The rate limit is currently configured at 50 requests per 5 minutes (configurable via NOTFOUND_RATELIMIT_STRING env var) and operates in observation mode only - it does not block requests. Once the tunable values are determined from live data, blocking can be enabled. This helps identify misconfigured clients and crawlers that abuse the index by generating excessive 404s. https://claude.ai/code/session_01F8syZGanCFSwRMVwCD1493
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds an observation-mode rate limit implementation for excessive 404
errors from a given client IP. Currently the implementation:
warehouse.notfound.ratelimiter.exceeded) for monitoring
The rate limit is currently configured at 50 requests per 5 minutes
(configurable via NOTFOUND_RATELIMIT_STRING env var) and operates in
observation mode only - it does not block requests. Once the tunable
values are determined from live data, blocking can be enabled.
This helps identify misconfigured clients and crawlers that abuse the
index by generating excessive 404s.
https://claude.ai/code/session_01F8syZGanCFSwRMVwCD1493