feat(microservices): add topic-consumers option to server-kafka#16555
Open
suuuuuuminnnnnn wants to merge 4 commits intonestjs:masterfrom
Open
feat(microservices): add topic-consumers option to server-kafka#16555suuuuuuminnnnnn wants to merge 4 commits intonestjs:masterfrom
suuuuuuminnnnnn wants to merge 4 commits intonestjs:masterfrom
Conversation
Pull Request Test Coverage Report for Build 157b7e61-c1ed-4ba3-99e3-02b51ffa7962Details
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Checklist
Please check if your PR fulfills the following requirements:
PR Type
What kind of change does this PR introduce?
What is the current behavior?
ServerKafkauses a single KafkaJS consumer subscribed to all registered topics, so a slow handler on one topic can effectively block message processing on other topics (topic-level backpressure). Partition-level concurrency (partitionsConsumedConcurrently) does not address cross-topic blocking.Issue Number: #14590
What is the new behavior?
Adds an opt-in
topicConsumers: booleanoption for Kafka microservices.When
topicConsumers: true:{groupId}-{topic}as the consumer group id (independent offset tracking per topic)close()options.runwhen building per-topicrun()optionsDefault remains
false, preserving the existing single-consumer behavior.Does this PR introduce a breaking change?
Note: enabling
topicConsumers: truecreates new consumer groups ({groupId}-{topic}), so migrating an existing deployment may require offset management.Other information
Tests
npm test -- packages/microservices/test/server/server-kafka.spec.tsAll tests pass, including new unit tests for
topicConsumersmode.