Add write-ahead log settings for prometheus destinations (#948)

Signed-off-by: Pete Wall <pete.wall@grafana.com>

charts/k8s-monitoring/destinations/prometheus-values.yaml

@@ -251,6 +251,20 @@ queueConfig:
   # @section -- Queue Configuration
   sampleAgeLimit: 0s
 
+# Write-Ahead Log (WAL) settings. Only applies when protocol is "remote_write"
+writeAheadLog:
+  # -- How frequently to clean up the write-ahead log.
+  # @section -- Write-Ahead Log
+  truncateFrequency: 2h
+
+  # -- Minimum time to keep data in the write-ahead log before it can be removed.
+  # @section -- Write-Ahead Log
+  minKeepaliveTime: 5m
+
+  # -- Maximum time to keep data in the write-ahead log before removing it.
+  # @section -- Write-Ahead Log
+  maxKeepaliveTime: 8h
+
 # -- Settings for converting OpenTelemetry ecosystem metrics to Prometheus ecosystem metrics.
 # @section -- OpenTelemetry Conversion
 openTelemetryConversion:

charts/k8s-monitoring/docs/destinations/prometheus.md

@@ -130,6 +130,14 @@ This defines the options for defining a destination for metrics that use the Pro
 | tls.keyFile | string | `""` | The client key for the server (as a path to a file). |
 | tls.keyFrom | string | `""` | Raw config for accessing the client key. |
 
+### Write-Ahead Log
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| writeAheadLog.maxKeepaliveTime | string | `"8h"` | Maximum time to keep data in the write-ahead log before removing it. |
+| writeAheadLog.minKeepaliveTime | string | `"5m"` | Minimum time to keep data in the write-ahead log before it can be removed. |
+| writeAheadLog.truncateFrequency | string | `"2h"` | How frequently to clean up the write-ahead log. |
+
 ## Examples
 
 ### Local Prometheus