Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I configured the Prometheus receiver, the fields were not differentiated. #4302

Open
sky9700 opened this issue Mar 20, 2024 · 1 comment

Comments

@sky9700
Copy link

sky9700 commented Mar 20, 2024

I have configured the otel collector with Prometheus as the receiver and Data Prepper as the exporter.
So, while I can retrieve data from the OpenSearch dashboard, it appears that all data is grouped under the name field.
To create a dashboard, I believe the fields need to be distinguished.
What additional settings should I add to the otel collector configmap to differentiate fields?
I am attaching the yaml file of the otel collector configmap that I have written. Thank you.

[otel collector configmap yaml]
data:
relay: |
exporters:
otlp/metrics:
endpoint: data-prepper-headless.opensearch.svc:21891
tls:
insecure: true
extensions:
health_check:
endpoint: ${env:K8S_POD_IP}:13133
processors:
memory_limiter:
check_interval: 5s
limit_percentage: 80
spike_limit_percentage: 25
receivers:
prometheus/internal:
config:
scrape_configs:
- job_name: apps
kubernetes_sd_configs:
- role: pod
selectors:
- role: pod
# only scrape data from pods running on the same node as collector
field: "spec.nodeName=${NODE_NAME}"
relabel_configs:
# scrape pods annotated with "prometheus.io/scrape: true"
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
regex: "true"
action: keep
# read the port from "prometheus.io/port: " annotation and update scraping address accordingly
- source_labels: [address, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
target_label: address
regex: ([^:]+)(?::\d+)?;(\d+)
# escaped $1:$2
replacement: $$1:$$2
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
- job_name: 'otel-collector'
scrape_interval: 5s
static_configs:
- targets: ['prometheus-k8s.monitoring.svc:9090']
metric_relabel_configs:
- source_labels: [ name ]
regex: '.grpc_io.'
action: drop
otlp:
protocols:
grpc:
endpoint: ${env:K8S_POD_IP}:4317
http:
endpoint: ${env:K8S_POD_IP}:4318
service:
extensions:
- health_check
pipelines:
metrics/internal:
exporters:
- otlp/metrics
processors:
- memory_limiter
receivers:
- prometheus/internal
- otlp

@kkondaka
Copy link
Collaborator

Please provide your data prepper config yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

2 participants