Skip to content

[Enhancement] Support LMQ in CombineConsumeQueueStore without migration to RocksDB CQ #10173

@imzs

Description

@imzs

Before Creating the Enhancement Request

  • I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

Support LMQ in CombineConsumeQueueStore without requiring full migration to RocksDB CQ, clusters that need LMQ support no longer need to complete a full migration first.

Motivation

CombineConsumeQueueStore was originally designed as a transient migration bridge to help brokers smoothly transition from file-based ConsumeQueueStore to RocksDBConsumeQueueStore via dual-write. The intended final state was a complete cutover to RocksDB, which was a prerequisite for enabling LMQ support.

This creates a hard dependency between LMQ capability and full RocksDB migration. Brokers in the dual-write transition state cannot support LMQ even though RocksDBConsumeQueueStore is already active, forcing operators to complete a potentially long migration before LMQ can be enabled.

Describe the Solution You'd Like

Enhance CombineConsumeQueueStore with topic-aware routing: LMQ topics are exclusively dispatched to and served from RocksDBConsumeQueueStore, while non-LMQ topics continue to follow the existing dual-write logic. This allows CombineConsumeQueueStore to serve as a stable long-term hybrid state.

Describe Alternatives You've Considered

.

Additional Context

No impact on existing behavior for non-LMQ topics.
The original migration path to a pure-RocksDB final state is fully preserved.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions