Implement rate-limiting for OpenAI API calls #11
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces a rate-limiting mechanism to handle API rate limits more gracefully. The most important changes include adding a
RateLimitGate
component, modifying theChatRequestHandler
to use this new component, and updating theRateLimitException
handling to consider an additional header.This works towards #8
Rate Limiting Mechanism:
src/main/java/com/penguineering/hareairis/rmq/RateLimitGate.java
: Added theRateLimitGate
component to manage rate limits by waiting for the next available time before executing a protected call.Integration with Chat Request Handler:
src/main/java/com/penguineering/hareairis/rmq/ChatRequestHandler.java
: Updated theChatRequestHandler
to include theRateLimitGate
and use it in theonMessage
method to handle chat requests within rate limits. [1] [2] [3]Exception Handling:
src/main/java/com/penguineering/hareairis/ai/RateLimitException.java
: Enhanced theRateLimitException
handling to check for thex-ratelimit-timeremaining
header if theRetry-After
header is not present.Configuration Changes:
src/main/java/com/penguineering/hareairis/rmq/RabbitMQConfig.java
: Updated the RabbitMQ configuration to ensure theRateLimitGate
bean is initialized before thechatRequestsContainer
bean. [1] [2]