Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation for OTel -> AWS infra relationships #20025

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
+85 −58
Open

Update documentation for OTel -> AWS infra relationships #20025

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d9602ce
Update documentation for OTel -> AWS infra relationships
alanwest Feb 21, 2025
9dd183c
Fix table row
alanwest Feb 21, 2025
7ddf2ee
Improve wording and format
alanwest Feb 21, 2025
ff98496
Improve wording
alanwest Feb 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 85 additions & 58 deletions ...nt/docs/opentelemetry/best-practices/opentelemetry-best-practices-resources.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,74 +135,101 @@ with [Amazon CloudWatch Metric Streams](/install/aws-cloudwatch/). This is not
an OpenTelemetry solution for monitoring your infrastructure. However, New
Relic supports a number of relationships between OpenTelemetry instrumented
services and infrastructure monitored using Amazon CloudWatch Metric Streams.
The relationships that are supported are listed below.

<Callout variant="important">
The relationships described below leverage OpenTelemetry semantic conventions.
Many of these conventions are not yet stable and are still evolving.
Some relationships are supported by instrumentation out-of-the-box. These
relationships leverage OpenTelemetry semantic conventions for correlating
telemetry from OpenTelemetry instrumented services to AWS resources
instrumented with CloudWatch Metric Streams. However, many of these
conventions are not yet stable and are still evolving.
Instrumentation that produces telemetry for the frameworks below vary in
their level of maturity and conformance to the semantic conventions. As
such, the instrumentation you use may not emit the attributes required for
these relationships to work. Please reach out to the maintainers of the
instrumentation you're using with any questions about the instrumentation's
maturity with respect to OpenTelemetry's semantic conventions.
</Callout>

#### ElastiCache (Memcached)

The Amazon CloudWatch Metric Streams integration for
[Elasticache](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-elasticache-monitoring-integration/)
generates ElastiCache entities in New Relic. These entities will have the
following entity tags:

* `aws.elasticache.configurationEndpointAddress` or `aws.elasticache.endpointAddress`
* `aws.elasticache.configurationEndpointPort` or `aws.elasticache.endpointPort`

If your services use OpenTelemetry instrumentation for Memcached it must emit
the following attributes for a relationship to be generated:

* `db.system`: This attribute must have the value `memcached`.
* `net.peer.name`: This attribute must match the corresponding endpoint tag of
the ElastiCache entity.
* `net.peer.port`: This attribute must match the corresponding port tag of the
ElastiCache entity.

#### Amazon MQ (ActiveMQ)

The Amazon CloudWatch Metric Streams integration for
[Amazon MQ](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-mq-integration)
generates Amazon MQ entities in New Relic. These entities will have the
following entity tag:

* `aws.mq.endpoint`

If your services use OpenTelemetry instrumentation for ActiveMQ it must
the following attributes for a relationship to be generated:

* `messaging.system`: This attribute must have the value `activemq`.
* `server.address`: This attribute must match the corresponding endpoint tag
of the Amazon MQ entity.
* `span.kind`: This relationship is generated from span data where `span.kind`
is `producer` or `consumer`.

#### Amazon Managed Kafka (MSK)

The Amazon CloudWatch Metric Streams integration for
[MSK](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-managed-kafka-msk-integration)
generates MSK entities in New Relic. These entities will have the
following entity tag:

* `aws.clusterName` or `aws.kafka.ClusterName`
* `aws.awsRegion` or `aws.region`
* `aws.topic` or `aws.kafka.Topic`

If your services use OpenTelemetry instrumentation for MSK it must have
the following attributes for a relationship to be generated:
In cases where the relationship is not supported by instrumentation, or
where the instrumentation available does not follow the standard conventions,
you must manually emit telemetry to signal to New Relic that a relationship
should be created. To do so, you must emit a log or a span with
the attribute `newrelic.aws_metric_streams.arn` containing the value of the
[ARN](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html)
for the resource you are calling. Emitting a log is preferred over emitting a
span because spans are often subject to sampling. The relationships
created have a TTL of 75 minutes, so this telemetry must continue to be sent
periodically.
</Callout>

* `messaging.destination.name`: The MSK topic to which the service is either producing messages or consuming messages.
* `server.address`: This attribute must match the corresponding endpoint tag
of the MSK entity.
* `span.kind`: This relationship is generated from span data where `span.kind`
is `producer` or `consumer`.
<table>
<thead>
<tr>
<th style={{ width: "150px" }}>
Supported by instrumentation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this column, given that not all instrumentations in all languages will contain the data required for relationships, do you think its worth it to discuss what the specific requirements are of instrumentation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to avoid that at least to begin with. In general, I think the rules are decent and will cover most of the instrumentation that exists out there. That said, if we discover customers are struggling then we can figure out how best to elaborate.

</th>
<th>
AWS product
</th>
</tr>
</thead>
<tbody>
<tr>
<td></td>
<td>
[DynamoDB](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-dynamodb-monitoring-integration)
</td>
</tr>
<tr>
<td>✅</td>
<td>
[ElastiCache](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-elasticache-monitoring-integration)
</td>
</tr>
<tr>
<td>✅</td>
<td>
[ElasticSearch](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration)
</td>
</tr>
<tr>
<td></td>
<td>
[Kinesis Data Firehose](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-kinesis-data-firehose-monitoring-integration)
</td>
</tr>
<tr>
<td></td>
<td>
[Kinesis Data Streams](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-kinesis-data-streams-monitoring-integration)
</td>
</tr>
<tr>
<td></td>
<td>
[Lambda](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-lambda-monitoring-integration)
</td>
</tr>
<tr>
<td>✅</td>
<td>
[MQ](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-mq-integration)
</td>
</tr>
<tr>
<td>✅</td>
<td>
[MSK](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-managed-kafka-msk-integration)
</td>
</tr>
<tr>
<td></td>
<td>
[SQS](/docs/infrastructure/amazon-integrations/aws-integrations-list/amazon-sqs-monitoring-integration)
</td>
</tr>
</tbody>
</table>

## Adding custom tags to an entity [#tags]

Expand Down