Skip to content

Conversation

liustve
Copy link
Contributor

@liustve liustve commented Sep 19, 2025

Description of changes:
Java Version of these PRs:

This PR introduces the complete CloudWatch EMF (Embedded Metric Format) exporter implementation for sending OpenTelemetry metrics directly to CloudWatch without requiring a Collector or Agent.

In order to enable this exporter, users MUST set the following environment variables:

  • OTEL_METRICS_EXPORTER=awsemf -
  • OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=<log-group-name>,x-aws-log-stream=<log-stream-name>, x-aws-metric-namespace=<namespace>
  • AWS_REGION=<region> OR AWS_DEFAULT_REGION=<region>

This PR includes:

  • EMF MetricRecord translation for for unified representation of all OTel metric types with log creation with unit mapping from OpenTelemetry to CloudWatch-compatible units
  • Automatic log group and stream creation with retry logic
  • Supported CloudWatch Logs integration with batching and constraint handling (256KB event limit, 1MB request limit, timestamp limits)
  • Support for metric grouping by attributes and timestamps for EMF log generation
  • Support for Gauge, Sum, Histogram, and ExponentialHistogram metric types

TODO:

Testing:

  • Added unit tests to validate EMF exporter configuration scenarios, including parameterized tests for both valid configurations (supporting AWS_REGION and AWS_DEFAULT_REGION) and invalid configurations (missing headers, wrong exporter type, missing region). The tests ensure the EMF exporter is correctly enabled only when all required environment variables are properly configured.

  • Manual end to end testing with the following environment variables to ensure the EMF logs show up:

    • AWS_REGION=us-east-1
    • OTEL_METRICS_EXPORTER=awsemf
    • OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=test,x-aws-log-stream=default,x-aws-metric-namespace=testNamespace
    • OTEL_RESOURCE_ATTRIBUTES=service.name=testService,aws.log.group.names=test,cloud.resource_id=agent-12345
    • OTEL_LOGS_EXPORTER=none
    • OTEL_TRACES_EXPORTER=none

Example EMF log emitted:

{
    "otel.resource.process.command_args": "[/Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home/bin/java, -javaagent:/Users/liustve/aws-otel-java-instrumentation/otelagent/build/libs/aws-opentelemetry-agent-2.18.0-SNAPSHOT.jar, -Dfile.encoding=UTF-8, -Dsun.stdout.encoding=UTF-8, -Dsun.stderr.encoding=UTF-8, -jar, /Users/liustve/aws-otel-java-instrumentation/sample-apps/springboot/build/libs/springboot-2.11.0-SNAPSHOT.jar]",
    "otel.resource.host.arch": "aarch64",
    "otel.resource.host.name": "7cf34dd812df",
    "otel.resource.service.instance.id": "a0399d3c-b856-43ae-b374-fe66dee41ce8",
    "otel.resource.aws.log.group.names": "test",
    "jvm.class.unloaded": 1,
    "otel.resource.service.name": "testSErvice",
    "_aws": {
        "CloudWatchMetrics": [
            {
                "Metrics": [
                    {
                        "Unit": "Count",
                        "Name": "jvm.cpu.count"
                    },
                    {
                        "Unit": "Count",
                        "Name": "jvm.class.loaded"
                    },
                    {
                        "Name": "jvm.cpu.recent_utilization"
                    },
                    {
                        "Unit": "Seconds",
                        "Name": "jvm.cpu.time"
                    },
                    {
                        "Unit": "Count",
                        "Name": "jvm.class.count"
                    },
                    {
                        "Unit": "Count",
                        "Name": "jvm.class.unloaded"
                    }
                ],
                "Namespace": "testNamespace"
            }
        ],
        "Timestamp": 1758761023497
    },
    "otel.resource.cloud.resource_id": "agent-12345",
    "jvm.class.count": 14264,
    "Version": "1",
    "otel.resource.process.pid": "46822",
    "otel.resource.os.description": "Mac OS X 15.6.1",
    "otel.resource.telemetry.distro.name": "opentelemetry-java-instrumentation",
    "otel.resource.os.type": "darwin",
    "otel.resource.telemetry.sdk.name": "opentelemetry",
    "otel.resource.telemetry.distro.version": "2.18.0-aws-SNAPSHOT",
    "otel.resource.process.runtime.description": "Amazon.com Inc. OpenJDK 64-Bit Server VM 21.0.8+9-LTS",
    "otel.resource.process.runtime.version": "21.0.8+9-LTS",
    "jvm.cpu.recent_utilization": 0,
    "otel.resource.process.executable.path": "/Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home/bin/java",
    "otel.resource.telemetry.sdk.version": "1.52.0",
    "jvm.cpu.count": 12,
    "jvm.class.loaded": 14265,
    "otel.resource.process.runtime.name": "OpenJDK Runtime Environment",
    "otel.resource.telemetry.sdk.language": "java",
    "jvm.cpu.time": 7.756965
}

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@liustve liustve requested a review from a team as a code owner September 19, 2025 22:32
@liustve liustve changed the title [WIP][Java] EMF Exporter Implementation feat: [Java] EMF Exporter Implementation Sep 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant