InfluxDB 3 rejects Telegraf elasticsearch_http metrics with very long lines (>200k chars)

[n4cer]

Steps to reproduce:
Telegraf configuration

[[inputs.elasticsearch]]
  servers = ["http://elastic:token@host:9200"]

Run Telegraf

telegraf --once --debug --input-filter elasticsearch

Write target: InfluxDB 3 /api/v2/write

Expected behaviour:
All line protocol generated by the official Telegraf Elasticsearch plugin should be accepted by InfluxDB 3 if it is already accepted by InfluxDB 2.

Actual behaviour:
Metrics of measurement:

elasticsearch_http

are rejected by InfluxDB 3 with:

parsing for line protocol failed

The identical payload is accepted by InfluxDB 2.

Environment info:

• InfluxDB version: InfluxDB 3 Core 3.8.0
• Endpoint used: /api/v2/write
• Telegraf version: 1.37.1
• Operating system: SLES Linux
• Output of uname -srm: Linux 6.4.0-150700.53.28-default x86_64
• Running InfluxDB 3 in a containerized environment (Docker)

Config:

[[inputs.elasticsearch]]
  servers =  ["http://elastic:token@host:9200"]
  http_timeout = "5s"
  local = true
  cluster_health = true
  cluster_health_level = "indices"
  cluster_stats = true
  cluster_stats_only_from_master = true
  indices_include = ["_all"]
  indices_level = "shards"

Logs:

Error while handling request error=write buffer error:
parsing for line protocol failed method=POST path=/api/v2/write

Investigation Result:

By sending lines one-by-one to InfluxDB 3, we identified that a single line of measurement elasticsearch_http with a length of approximately 200,000+ characters causes the failure.

Dropping this measurement in Telegraf resolves the issue completely:

namedrop = ["elasticsearch_http"]

This strongly suggests a parser limitation in InfluxDB 3 regarding maximum line length or field count.

[hiltontj]

@n4cer thanks for opening the issue. Currently there is a limit imposed on string fields by the parser of 64KB (see here).

We are working on raising this to 1MB to be compatible with the v1 write API, which accepts string fields up to that size.