log/loki: limit line size #2475
Conversation
This commit limits the maximum size of each single log line to 4096 bytes in production workloads (i.e. for consumers of `loki.New()`). This prevents pushing log lines too long for our instance of Loki, avoiding potential DoS. As with batchMax, maxLogLineLen is not configurable except in tests.
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #2475 +/- ##
==========================================
- Coverage 53.59% 53.55% -0.05%
==========================================
Files 198 198
Lines 26711 26715 +4
==========================================
- Hits 14317 14307 -10
- Misses 10608 10625 +17
+ Partials 1786 1783 -3
☔ View full report in Codecov by Sentry. |
| @@ -101,10 +104,15 @@ func (c *Client) Run() { | |||
| for { | |||
| select { | |||
| case line := <-c.input: | |||
| if len(line) > c.maxLogLineLen && c.maxLogLineLen != 0 { | |||
| continue // Silently drop log line longer than maxLogLineLen if set | |||
There was a problem hiding this comment.
not sure about silent, what about using a log filter?
There was a problem hiding this comment.
I'm doing what batch does pretty much, we discussed this briefly on Slack and you suggested to just drop it.
What behavior would you like to see instead?
There was a problem hiding this comment.
well, log it using a log filter: log.Warn(ctx, "Dropping to large logging line", z.Str("prefix", line[:10]), dropFilter with dropFilter being dropFilter := log.Filter().
but sure, this is only loki client itself, which is not our primary logging framework, it is best effort, so yeah, maybe silent drop is fine.
gsora
added
the
merge when ready
This PR limits the maximum size of each single log line to 4096 bytes in production workloads (i.e. for consumers of
loki.New()).This prevents pushing log lines too long for our instance of Loki, avoiding potential DoS.
As with
batchMax,maxLogLineLenis not configurable except in tests.category: misc
ticket: none