How to produce the undelivered messages after timeout of producer.flush(1.0)

[shuvohishab]

Description

I'm producing some messages in our Kafka server, but getting acknowledgement of message delivery(produccer.flush()) is taking a huge time. That's why I added timeout of 1 second.
But I noticed that there are some messages left to deliver from our end.
How to produce such undelivered messages? Any ways?

How to reproduce

producer.poll(0.0)
producer.produce(
topic=topic,
key=str(uuid4()),
value=output_msg,
on_delivery=delivery_report_wrapper,
headers=logger_extra,
)
pending_messages = producer.flush(1.0)
logger.info(f"number of messages still in queue: {pending_messages}")

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): confluent_kafka==1.9.0
• Apache Kafka broker version:
• Client configuration: {...}
• Operating system: Linux
• Provide client logs (with 'debug': '..' as necessary): number of messages still in queue: 1
• Provide broker log excerpts
• Critical issue

[pranavrth]

I see that you are using an old version of the client. Can you use the latest one?

Can you also provide the debug logs for the same by setting debug: broker,topic,msg?

[shuvohishab]

Hi @pranavrth, thanks for your quick response.
I'll come back with your answer. But can I know, do I have to update my python protobuf for this? Or, just updating confluent-kafka will work?

[pranavrth]

I think just confluent_kafka should work.