Skip to content

Conversation

leo198706
Copy link

  1. fluent-plugin-kafka 0.18.1
  2. fluentd 1.14.6

error.log

According to the above error, I analyzed the reasons as follows:

When Kafka does not have a topic to write to and does not allow automatic creation, an error UnknownTopicOrPartition will be reported, but it will be caught by Kafka::Error
https://github.com/fluent/fluent-plugin-kafka/blob/v0.18.1/lib/fluent/plugin/kafka_producer_ext.rb#L254

and finally thrown in rescue Exception => e.
https://github.com/fluent/fluent-plugin-kafka/blob/master/lib/fluent/plugin/out_kafka2.rb#L411

This is not an ideal situation.

So I adjusted the code to handle the above situation and avoid repeated retries.

Signed-off-by: Bing Wang <bing.wang@derbysoft.net>
@JensErat
Copy link

JensErat commented Apr 8, 2025

Is this related to #311 ?

@Watson1978
Copy link
Contributor

We think the behavior you reported is not bug.

To avoid repeated retries, you might use discard_kafka_delivery_failed option.

Copy link

github-actions bot commented Sep 8, 2025

This PR has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this PR will be closed in 30 days

@github-actions github-actions bot added the stale label Sep 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants