Skip to content

Bug: rabbitmq consumer recovery #2758

@paul-godhouse

Description

@paul-godhouse

Describe the bug
Provide a clear and concise description of the bug.

I'm using faststream with RabbitMQ on Kubernetes. RabbitMQ is deployed through RabbitMQCluster with 3 replicas (= pods).
From time to time, kubernetes puts a node in maintenance mode, which causes the the deletion of a RabbitMQ replica.

Therefore, the connection on faststream is lost and then recovered. Once it is recovered, the queues exist but not the consumers.

The only workaround I found for now is to setup an alert when a rabbitmq queue has no consumers and when the alert is triggered, I manually kill a faststream pod. The pod is then restarted from scratch which also causes the creation of the consumers.

For the record, I have also an app on my k8s cluster which is using rabbitmq with celery and this issue do not happen. So I guess there is either a problem in my config or a problem with faststream (or perhaps aio_pika)

How to reproduce
Here is an example of a subscriber I have

from faststream.rabbit import RabbitRouter
audience_router = RabbitRouter()

@audience_router.subscriber(queue=RabbitQueue(
        name="QUEUE_NAME",
        durable=True,
        auto_delete=False,
        exclusive=False,
    ))
async def audience_refresh_collection() -> None:
   ...

...

And/Or steps to reproduce the behavior:

  1. ...

Expected behavior
Explain what you expected to happen clearly and concisely.

Observed behavior
Describe what is actually happening clearly and concisely.

Screenshots
If applicable, attach screenshots to help illustrate the problem.

Environment
Include the output of the faststream -v command to display your current project and system environment.

Additional context
Provide any other relevant context or information about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions