Skip to content

Add loki.enrich component #2882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Mar 20, 2025
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
1e9197c
Add loki.enricher
v-zhuravlev Mar 2, 2025
3587566
Add missing comma
v-zhuravlev Mar 2, 2025
2405517
Fix config.alloy
v-zhuravlev Mar 2, 2025
7165296
Fix integration test
v-zhuravlev Mar 2, 2025
d2521c3
Fix tests
v-zhuravlev Mar 2, 2025
3308f9d
Register in all.go
v-zhuravlev Mar 2, 2025
601e0ad
Update docs
v-zhuravlev Mar 2, 2025
dccff19
rename to loki.enrich
v-zhuravlev Mar 2, 2025
8238809
Update docs
v-zhuravlev Mar 2, 2025
68aa3a0
rename to network
v-zhuravlev Mar 2, 2025
3b8cf7d
Update docs
v-zhuravlev Mar 4, 2025
812da2d
Remove redundant label
v-zhuravlev Mar 4, 2025
04ad59b
Update test
v-zhuravlev Mar 4, 2025
ecf9e1a
rename int test
v-zhuravlev Mar 4, 2025
d721016
Rename
v-zhuravlev Mar 4, 2025
2b8f416
Add hidden label
v-zhuravlev Mar 4, 2025
69cb0cc
Fix tests
v-zhuravlev Mar 4, 2025
69ee31b
Update changelog
v-zhuravlev Mar 4, 2025
bb4e734
Make target_label optional
v-zhuravlev Mar 4, 2025
e8d79cd
Rename to logs_match_label, target_match_label. Make logs_match_label…
v-zhuravlev Mar 4, 2025
634190c
Update test
v-zhuravlev Mar 4, 2025
200d27e
rename to labels_to_copy
v-zhuravlev Mar 12, 2025
52a78ca
Update docs/sources/reference/components/loki/loki.enrich.md
v-zhuravlev Mar 14, 2025
364be62
Apply suggestions from code review
v-zhuravlev Mar 14, 2025
6696539
Update docs
v-zhuravlev Mar 14, 2025
c4e200c
Add limitation notice
v-zhuravlev Mar 14, 2025
c37d243
Update docs/sources/reference/components/loki/loki.enrich.md
v-zhuravlev Mar 14, 2025
3e2e3e4
Update docs/sources/reference/components/loki/loki.enrich.md
v-zhuravlev Mar 17, 2025
ba566fd
Add caution in docs
v-zhuravlev Mar 17, 2025
ec1fd2c
Merge branch 'main' into vzhuravlev/loki_enricher
v-zhuravlev Mar 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 31 additions & 30 deletions docs/sources/reference/components/loki/loki.enrich.md
Original file line number Diff line number Diff line change
@@ -1,59 +1,64 @@
---
aliases:
- /docs/alloy/latest/reference/components/loki/loki.enrich/
canonical: /docs/alloy/latest/reference/components/loki/loki.enrich/
canonical: https://grafana.com/docs/alloy/latest/reference/components/loki/loki.enrich/
title: loki.enrich
labels:
stage: experimental
description: The loki.enrich component enriches logs with labels from service discovery.
---

# loki.enrich
# `loki.enrich`

{{< docs/shared lookup="stability/experimental.md" source="alloy" version="<ALLOY_VERSION>" >}}

The `loki.enrich` component enriches logs with additional labels from service discovery targets. It matches a label from incoming logs against a label from discovered targets, and copies specified labels from the matched target to the log entry.
The `loki.enrich` component enriches logs with additional labels from service discovery targets.
It matches a label from incoming logs against a label from discovered targets, and copies specified labels from the matched target to the log entry.

## Usage

```alloy
loki.enrich "LABEL" {
loki.enrich "<LABEL>" {
// List of targets from a discovery component
targets = DISCOVERY_COMPONENT.targets
targets = <DISCOVERY_COMPONENT>.targets

// Which label from discovered targets to match against
match_label = "LABEL"
match_label = "<LABEL>"

// Which label from incoming logs to match against
source_label = "LABEL"
source_label = "<LABEL>"

// List of labels to copy from discovered targets to logs
target_labels = ["LABEL", ...]
labels_to_copy = ["<LABEL>", ...]

// Where to send enriched logs
forward_to = [RECEIVER_LIST]
forward_to = [<RECEIVER_LIST>]
}
```

## Arguments

The following arguments are supported:

Name | Type | Description | Default | Required
---- | ---- | ----------- | ------- | --------
`targets` | `[]discovery.Target` | List of targets from a discovery component. | | yes
`target_match_label` | `string` | Which label from discovered targets to match against (e.g., "__inventory_consul_service"). | | yes
`logs_match_label` | `string` | Which label from incoming logs to match against discovered targets (e.g., "service_name"). If not specified, `target_match_label` will be used. | `target_match_label` | no
`target_labels` | `[]string` | List of labels to copy from discovered targets to logs. If empty, all labels will be copied. | | no
`forward_to` | `[]loki.LogsReceiver` | List of receivers to send enriched logs to. | | yes
| Name | Type | Description | Default | Required |
| -------------------- | --------------------- | ------------------------------------------------------------------------------------------------ | -------------------- | -------- |
| `forward_to` | `[]loki.LogsReceiver` | List of receivers to send enriched logs to. | | yes |
| `target_match_label` | `string` | The label from discovered targets to match against, for example, `"__inventory_consul_service"`. | | yes |
| `targets` | `[]discovery.Target` | List of targets from a discovery. component. | | yes |
| `labels_to_copy` | `[]string` | List of labels to copy from discovered targets to logs. If empty, all labels will be copied. | | no |
| `logs_match_label` | `string` | The label from incoming logs to match against discovered targets, for example `"service_name"`. | `target_match_label` | no |

## Blocks

The `loki.enrich` component doesn't support any blocks. You can configure this component with arguments.

## Exports

The following values are exported:

Name | Type | Description
---- | ---- | -----------
`receiver` | `loki.LogsReceiver` | A receiver that can be used to send logs to this component.
| Name | Type | Description |
| ---------- | ------------------- | ----------------------------------------------------------- |
| `receiver` | `loki.LogsReceiver` | A receiver that can be used to send logs to this component. |

## Example

Expand Down Expand Up @@ -112,8 +117,6 @@ loki.enrich "default" {
// Match hostname from logs to DNS name
target_match_label = "primary_ip"



forward_to = [loki.write.default.receiver]
}
```
Expand All @@ -122,17 +125,15 @@ loki.enrich "default" {

The component matches logs to discovered targets and enriches them with additional labels:

1. For each log entry, it looks up the value of `logs_match_label` from the log's labels (or `target_match_label` if `logs_match_label` is not specified)
2. It matches this value against the `target_match_label` in discovered targets
3. If a match is found, it copies the requested `target_labels` from the discovered target to the log entry (if `target_labels` is empty, all labels are copied)
4. The log entry (enriched or unchanged) is forwarded to the configured receivers
1. For each log entry, it looks up the value of `logs_match_label` from the log's labels or `target_match_label` if `logs_match_label` is not specified.
1. It matches this value against the `target_match_label` in discovered targets.
1. If a match is found, it copies the requested `labels_to_copy` from the discovered target to the log entry. If `labels_to_copy` is empty, all labels are copied.
1. The log entry, enriched or unchanged, is forwarded to the configured receivers.

## See also
By default, `loki.enrich` is ready as soon as it starts, even if no targets have been discovered.
If telemetry is sent to this component before the metadata is synced, there will be no labels to enrich the telemetry with.

* [loki.source.syslog](../loki.source.syslog/)
* [loki.source.api](../loki.source.api/)
* [discovery.relabel](../discovery/discovery.relabel/)
* [discovery.http](../discovery/discovery.http/) <!-- START GENERATED COMPATIBLE COMPONENTS -->
<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ loki.source.api "network_device" {
loki.enrich "enricher" {

targets = discovery.relabel.network_devices.output
// Labels to copy from targets
target_labels = [
// List of labels to copy from discovered targets to logs
labels_to_copy = [
"environment",
"datacenter",
"role",
Expand Down
8 changes: 4 additions & 4 deletions internal/component/loki/enrich/enrich.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ type Arguments struct {
// If not specified, TargetMatchLabel will be used
LogsMatchLabel string `alloy:"logs_match_label,attr,optional"`

// List of target labels to copy to logs. If empty, all labels will be copied.
TargetLabels []string `alloy:"target_labels,attr,optional"`
// List of labels to copy from discovered targets to logs. If empty, all labels will be copied.
LabelsToCopy []string `alloy:"labels_to_copy,attr,optional"`

// Where to forward logs after enrichment
ForwardTo []loki.LogsReceiver `alloy:"forward_to,attr"`
Expand Down Expand Up @@ -139,14 +139,14 @@ func (c *Component) processLog(entry *logproto.Entry, labels model.LabelSet) err

// Copy labels from target to log labels
newLabels := labels.Clone()
if len(c.args.TargetLabels) == 0 {
if len(c.args.LabelsToCopy) == 0 {
// If no specific labels are requested, copy all labels
for k, v := range targetLabels {
newLabels[k] = v
}
} else {
// Copy only requested labels
for _, label := range c.args.TargetLabels {
for _, label := range c.args.LabelsToCopy {
if value := targetLabels[model.LabelName(label)]; value != "" {
newLabels[model.LabelName(label)] = value
}
Expand Down
8 changes: 4 additions & 4 deletions internal/component/loki/enrich/enrich_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ func TestEnricher(t *testing.T) {
},
TargetMatchLabel: "service",
LogsMatchLabel: "service_name",
TargetLabels: []string{"env", "owner"},
LabelsToCopy: []string{"env", "owner"},
},
inputLog: &logproto.Entry{
Timestamp: time.Now(),
Expand All @@ -68,7 +68,7 @@ func TestEnricher(t *testing.T) {
},
TargetMatchLabel: "service",
LogsMatchLabel: "service_name",
TargetLabels: []string{"env"},
LabelsToCopy: []string{"env"},
},
inputLog: &logproto.Entry{
Timestamp: time.Now(),
Expand Down Expand Up @@ -123,7 +123,7 @@ func TestEnricher(t *testing.T) {
},
TargetMatchLabel: "service",
// LogsMatchLabel intentionally omitted as 'service' label exists in both.
TargetLabels: []string{"env", "owner"},
LabelsToCopy: []string{"env", "owner"},
},
inputLog: &logproto.Entry{
Timestamp: time.Now(),
Expand Down Expand Up @@ -195,7 +195,7 @@ func TestUpdate(t *testing.T) {
Targets: newTargets,
TargetMatchLabel: "service",
LogsMatchLabel: "service_name",
TargetLabels: []string{"env"},
LabelsToCopy: []string{"env"},
})
require.NoError(t, err)
}
Expand Down