Support entity for OTLP custom metrics

[musa-asad]

Description of the issue

To support the Explore related feature in CloudWatch, the CloudWatch Agent sends an "Entity", which includes relevant metadata to correlate metrics or logs between resources (e.g., an EKS cluster) and services (e.g., a Java application). When the CloudWatch Agent runs in a Kubernetes cluster, we need to collect the namespace, workload name, and node name to populate the "Entity".

However, we currently only get Kubernetes metadata when Application Signals is enabled. For OTLP custom metrics, if Application Signals isn't configured, then we don't have a way to fetch Kubernetes metadata. To achieve this, we must migrate the logic the Application Signals Processor uses to get Kuberbetes metadata to a global extension, which can be used by other pipelines.

Description of changes

Important

Co-PRs: amazon-contributing/opentelemetry-collector-contrib#291; aws-observability/helm-charts#171

• Improve cluster detection for components during translation.
• Add fix in entity logic to fallback to workload name when unknown service is set by OTEL SDK.
• Set the service name source to Instrumentation in entity when customer sets the service name OTEL resource attribute.
• Add Kubernetes Metadata Extension to Translation for the OTLP Pipeline (translator/translate/otel)
  • Update sample yaml files to include new extension.
  • Create extension/k8smetadata/translator.go to reference the extension in translation.
  • Add translators.Extensions.Set(k8smetadata.NewTranslator()) to pipeline/host/translator.go for OTLP pipeline in Kubernetes contexts.
  • Enable entity in EMF logs for OTLP.
• Update service/defaultcomponents/components.go to include "k8smetadata".
• Get podIp from connection context in entity processor to extract the pod metadata in map.
• Add Instance ID entity attribute only if the application node is same as agent node.
• Update Application Signals Processor to use extension if available.
• Add unit tests for new functionality.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Integration Tests

View Workflow Run #14350034749

Application Signals E2E Tests

View Workflow Run #14350034748

Manual Testing

Agent Logs

│ 2025-04-09T06:04:39Z D! {"caller":"k8smetadata/extension.go:106","msg":"GetPodMetadataFromPodIP: found metadata","kind":"extension","name":"k8smetadata","ip":"XX │
│ X.XX.XX.XX","workload":"sample-app","namespace":"default","node":"ip-XXX-XX-XX-XXX.us-west-2.compute.internal"}                                                   │
│ 2025-04-09T06:04:42Z D! {"caller":"awsemfexporter@v0.115.0/emf_exporter.go:105","msg":"Start processing resource metrics","kind":"exporter","data_type":"metrics" │
│ ,"name":"awsemf","labels":{"com.amazonaws.cloudwatch.entity.internal.aws.account.id":"506463145083","com.amazonaws.cloudwatch.entity.internal.deployment.environm │
│ ent":"eks:cwagent-otlp-beta-testing/default","com.amazonaws.cloudwatch.entity.internal.instance.id":"i-03ad05b8dc69166f1","com.amazonaws.cloudwatch.entit │
│ y.internal.k8s.cluster.name":"cwagent-otlp-beta-testing","com.amazonaws.cloudwatch.entity.internal.k8s.namespace.name":"default","com.amazonaws.cloudwatc │
│ h.entity.internal.k8s.node.name":"ip-XXX-XX-XX-XXX.us-west-2.compute.internal","com.amazonaws.cloudwatch.entity.internal.k8s.workload.name":"sample-app","com.ama │
│ zonaws.cloudwatch.entity.internal.platform.type":"AWS::EKS","com.amazonaws.cloudwatch.entity.internal.service.name":"sample-app","com.amazonaws.cloudwatch.entity │
│ .internal.service.name.source":"K8sWorkload","com.amazonaws.cloudwatch.entity.internal.type":"Service","host.arch":"amd64","host.name":"sample-app-55dc5df99b-rwg │
│ bw","os.description":"Linux 6.1.131-143.221.amzn2023.x86_64","os.type":"linux","process.command_line":"/usr/lib/jvm/java-17-openjdk-amd64:bin:java -javaagent:/aw │
│ s-observability/classpath/aws-opentelemetry-agent-1.17.0.jar","process.executable.path":"/usr/lib/jvm/java-17-openjdk-amd64:bin:java","process.pid":"","process.r │
│ untime.description":"Debian OpenJDK 64-Bit Server VM 17.0.4+8-Debian-1deb11u1","process.runtime.name":"OpenJDK Runtime Environment","process.runtime.version":"17 │
│ .0.4+8-Debian-1deb11u1","service.name":"unknown_service:java","telemetry.auto.version":"1.17.0-aws","telemetry.sdk.language":"java","telemetry.sdk.name":"opentel │
│ emetry","telemetry.sdk.version":"1.17.0"}}
│ 2025-04-09T06:05:59Z D! {"caller":"k8sclient/endpointslicewatcher.go:218","msg":"Received EndpointSlice Update","kind":"extension","name":"k8smetadata","oldSlice │
│ UID":"1f78b8db-8aca-43d5-b2da-df751cd686c8","newSliceUID":"1f78b8db-8aca-43d5-b2da-df751cd686c8","name":"sample-app-g7l4h","namespace":"default"}                 │
│ 2025-04-09T06:05:59Z D! {"caller":"k8sclient/endpointslicewatcher.go:276","msg":"Finished handling EndpointSlice Update","kind":"extension","name":"k8smetadata", │
│ "sliceUID":"1f78b8db-8aca-43d5-b2da-df751cd686c8"}                                                                                                                │
│ 2025-04-09T06:06:00Z D! {"caller":"k8sclient/endpointslicewatcher.go:118","msg":"Processing endpoint","kind":"extension","name":"k8smetadata","podName":"sample-a │
│ pp-55dc5df99b-bmg4n","namespace":"default","nodeName":"ip-XXX-XX-XX-XXX.us-west-2.compute.internal"}

PMD

{"Entities":[{"__type":"com.amazonaws.observability#Entity","KeyAttributes":{"Environment":"eks:cwagent-otlp-beta-testing/default","Type":"Service","Name":"sample-app"}},{"__type":"com.amazonaws.observability#Entity","Attributes":{"AWS.ServiceNameSource":"K8sWorkload"},"KeyAttributes":{"Environment":"eks:cwagent-otlp-beta-testing/default","Type":"Service","Name":"sample-app"}}]}

EMF

{ "AWS.ServiceNameSource": "K8sWorkload", "CloudWatchMetrics": [ { "Namespace": "CWAgent", "Dimensions": [ ["endpoint", "telemetry.sdk.language", "telemetry.sdk.version", "telemetry.sdk.name", "service.name" ] ], "Metrics": [ {"Name": "request_count", "Unit": "", "StorageResolution": 60 } ] } ], "EC2.InstanceId": "i-05240b6269dfa8f69","EKS.Cluster": "cwagent-otlp-beta-testing", "Environment": "eks:cwagent-otlp-beta-testing/default","K8s.Namespace": "default", "K8s.Node": "ip-XXX-XX-XX-XXX.us-west-2.compute.internal", "K8s.Workload": "otlp-http-metrics-app", "PlatformType": "AWS::EKS", "Service": "otlp-http-metrics-app", "Timestamp": "1744169779952","Version": "0", "endpoint": "/simulate", "service.name": "unknown_service", "telemetry.sdk.language": "python","telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.18.0", "request_count": 14 }

Requirements

Before commit the code, please do the following steps.

1. Run make fmt and make fmt-sh
2. Run make lint

[github-actions]

This PR was marked stale due to lack of activity.

[github-actions]

This PR was marked stale due to lack of activity.

[github-actions]

This PR was marked stale due to lack of activity.