You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have configured the otel collector with Prometheus as the receiver and Data Prepper as the exporter.
So, while I can retrieve data from the OpenSearch dashboard, it appears that all data is grouped under the name field.
To create a dashboard, I believe the fields need to be distinguished.
What additional settings should I add to the otel collector configmap to differentiate fields?
I am attaching the yaml file of the otel collector configmap that I have written. Thank you.
[otel collector configmap yaml]
data:
relay: |
exporters:
otlp/metrics:
endpoint: data-prepper-headless.opensearch.svc:21891
tls:
insecure: true
extensions:
health_check:
endpoint: ${env:K8S_POD_IP}:13133
processors:
memory_limiter:
check_interval: 5s
limit_percentage: 80
spike_limit_percentage: 25
receivers:
prometheus/internal:
config:
scrape_configs:
- job_name: apps
kubernetes_sd_configs:
- role: pod
selectors:
- role: pod
# only scrape data from pods running on the same node as collector
field: "spec.nodeName=${NODE_NAME}"
relabel_configs:
# scrape pods annotated with "prometheus.io/scrape: true"
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
regex: "true"
action: keep
# read the port from "prometheus.io/port: " annotation and update scraping address accordingly
- source_labels: [address, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
target_label: address
regex: ([^:]+)(?::\d+)?;(\d+)
# escaped $1:$2
replacement: $$1:$$2
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
- job_name: 'otel-collector'
scrape_interval: 5s
static_configs:
- targets: ['prometheus-k8s.monitoring.svc:9090']
metric_relabel_configs:
- source_labels: [ name ]
regex: '.grpc_io.'
action: drop
otlp:
protocols:
grpc:
endpoint: ${env:K8S_POD_IP}:4317
http:
endpoint: ${env:K8S_POD_IP}:4318
service:
extensions:
- health_check
pipelines:
metrics/internal:
exporters:
- otlp/metrics
processors:
- memory_limiter
receivers:
- prometheus/internal
- otlp
The text was updated successfully, but these errors were encountered:
I have configured the otel collector with Prometheus as the receiver and Data Prepper as the exporter.
So, while I can retrieve data from the OpenSearch dashboard, it appears that all data is grouped under the name field.
To create a dashboard, I believe the fields need to be distinguished.
What additional settings should I add to the otel collector configmap to differentiate fields?
I am attaching the yaml file of the otel collector configmap that I have written. Thank you.
[otel collector configmap yaml]
data:
relay: |
exporters:
otlp/metrics:
endpoint: data-prepper-headless.opensearch.svc:21891
tls:
insecure: true
extensions:
health_check:
endpoint: ${env:K8S_POD_IP}:13133
processors:
memory_limiter:
check_interval: 5s
limit_percentage: 80
spike_limit_percentage: 25
receivers:
prometheus/internal:
config:
scrape_configs:
- job_name: apps
kubernetes_sd_configs:
- role: pod
selectors:
- role: pod
# only scrape data from pods running on the same node as collector
field: "spec.nodeName=${NODE_NAME}"
relabel_configs:
# scrape pods annotated with "prometheus.io/scrape: true"
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
regex: "true"
action: keep
# read the port from "prometheus.io/port: " annotation and update scraping address accordingly
- source_labels: [address, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
target_label: address
regex: ([^:]+)(?::\d+)?;(\d+)
# escaped $1:$2
replacement: $$1:$$2
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
- job_name: 'otel-collector'
scrape_interval: 5s
static_configs:
- targets: ['prometheus-k8s.monitoring.svc:9090']
metric_relabel_configs:
- source_labels: [ name ]
regex: '.grpc_io.'
action: drop
otlp:
protocols:
grpc:
endpoint: ${env:K8S_POD_IP}:4317
http:
endpoint: ${env:K8S_POD_IP}:4318
service:
extensions:
- health_check
pipelines:
metrics/internal:
exporters:
- otlp/metrics
processors:
- memory_limiter
receivers:
- prometheus/internal
- otlp
The text was updated successfully, but these errors were encountered: