Fluentd is not able to send .json extension file logs to logstash (fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-opensearch-amd64-2.0) #4612
Replies: 3 comments 9 replies
-
I moved this issue here. |
Beta Was this translation helpful? Give feedback.
-
Looks like this filter setting is wrong. <filter **>
@type parser
format json
key_name log
reserve_data true
</filter> |
Beta Was this translation helpful? Give feedback.
-
Hi @daipom <source>
@type tail
path /opt/jenkins/deploymentlogs/output.json
pos_file /var/log/td-agent/httpd-access.log.pos
#format json
time_format %Y-%m-%dT%H:%M:%S
tag test
read_from_head true
#exclude_path ["/var/log/fluent*"]
#from_encoding UTF-8
#encoding UTF-8
#<parse>
# @type json
#</parse>
<parse>
@type multi_format
<pattern>
format json
time_key time
keep_time_key true
time_format %Y-%m-%dT%H:%M:%S.%NZ
</pattern>
</parse>
</source>
<filter **>
@type grep
<exclude>
key message
pattern /force flushing buffer events/
</exclude>
</filter>
<filter **>
@type grep
<exclude>
key message
pattern /flushing all buffer forcedly/
</exclude>
</filter> |
Beta Was this translation helpful? Give feedback.
-
Describe the bug
Hi,
We are trying to send json logs from the file extension .json to logstash but we are getting below error.
2024-08-21 07:30:44 +0000 [warn]: #0 pattern not matched: "{"applicationName":"abc"}}"
2024-08-21 07:30:44 +0000 [warn]: #0 dump an error event: error_class=ArgumentError error="log does not exist" location=nil tag="fluent.warn" time=2024-08-21 07:30:44.830543276 +0000 record={"message"=>"pattern not matched: "{\"applicationName\":\"abc\"}}""}
2024-08-21 07:30:44.830543276 +0000 fluent.warn: {"message":"pattern not matched: "{\"applicationName\":\"abc\"}}""}
Sample logs:
{"applicationName":"abc"}
fluentd config:-
Could you please guide me how to send .json extension file json logs to logstash?
Thanks
To Reproduce
result is expected json logs
Expected behavior
result is expected json logs
Your Environment
Your Configuration
Your Error Log
Additional context
No response
Beta Was this translation helpful? Give feedback.
All reactions