-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
045455c
commit 97bee5d
Showing
25 changed files
with
2,304 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
include ../../Makefile.Common |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
# Azure Event Hub Receiver | ||
|
||
<!-- status autogenerated section --> | ||
| Status | | | ||
| ------------- |-----------| | ||
| Stability | [alpha]: metrics, logs | | ||
| Distributions | [contrib], [observiq], [splunk], [sumo] | | ||
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fazureeventhub%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fazureeventhub) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fazureeventhub%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fazureeventhub) | | ||
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@atoulme](https://www.github.com/atoulme), [@djaglowski](https://www.github.com/djaglowski) | | ||
|
||
[alpha]: https://github.com/open-telemetry/opentelemetry-collector#alpha | ||
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib | ||
[observiq]: https://github.com/observIQ/observiq-otel-collector | ||
[splunk]: https://github.com/signalfx/splunk-otel-collector | ||
[sumo]: https://github.com/SumoLogic/sumologic-otel-collector | ||
<!-- end autogenerated section --> | ||
|
||
## Overview | ||
Azure resources and services can be | ||
[configured](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings) | ||
to send their logs to an Azure Event Hub. The Azure Event Hub receiver pulls logs from an Azure | ||
Event Hub, transforms them, and pushes them through the collector pipeline. | ||
|
||
## Configuration | ||
|
||
### connection (Required) | ||
A string describing the connection to an Azure event hub. | ||
|
||
### group (Optional) | ||
The Consumer Group to read from. If empty will default to the default Consumer Group $Default | ||
|
||
### partition (Optional) | ||
The partition to watch. If empty, it will watch explicitly all partitions. | ||
|
||
Default: "" | ||
|
||
### offset (Optional) | ||
The offset at which to start watching the event hub. If empty, it starts with the latest offset. | ||
|
||
Default: "" | ||
|
||
### format (Optional) | ||
Determines how to transform the Event Hub messages into OpenTelemetry logs. See the "Format" | ||
section below for details. | ||
|
||
Default: "azure" | ||
|
||
### Example Configuration | ||
|
||
```yaml | ||
receivers: | ||
azureeventhub: | ||
connection: Endpoint=sb://namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=superSecret1234=;EntityPath=hubName | ||
partition: foo | ||
group: bar | ||
offset: "1234-5566" | ||
format: "azure" | ||
``` | ||
This component can persist its state using the [storage extension]. | ||
## Format | ||
### raw | ||
The "raw" format maps the AMQP properties and data into the | ||
attributes and body of an OpenTelemetry LogRecord, respectively. | ||
The body is represented as a raw byte array. | ||
This format is not supported for Metrics. | ||
### azure | ||
The "azure" format extracts the Azure log records from the AMQP | ||
message data, parses them, and maps the fields to OpenTelemetry | ||
attributes. The table below summarizes the mapping between the | ||
[Azure common log format](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/resource-logs-schema) | ||
and the OpenTelemetry attributes. | ||
| Azure | OpenTelemetry | | ||
|----------------------------------|----------------------------------------| | ||
| callerIpAddress (optional) | net.sock.peer.addr (attribute) | | ||
| correlationId (optional) | azure.correlation.id (attribute) | | ||
| category (optional) | azure.category (attribute) | | ||
| durationMs (optional) | azure.duration (attribute) | | ||
| Level (optional) | severity_number, severity_text (field) | | ||
| location (optional) | cloud.region (attribute) | | ||
| — | cloud.provider (attribute) | | ||
| operationName (required) | azure.operation.name (attribute) | | ||
| operationVersion (optional) | azure.operation.version (attribute) | | ||
| properties (optional) | azure.properties (attribute, nested) | | ||
| resourceId (required) | azure.resource.id (resource attribute) | | ||
| resultDescription (optional) | azure.result.description (attribute) | | ||
| resultSignature (optional) | azure.result.signature (attribute) | | ||
| resultType (optional) | azure.result.type (attribute) | | ||
| tenantId (required, tenant logs) | azure.tenant.id (attribute) | | ||
| time or timeStamp (required) | time_unix_nano (time takes precedence) | | ||
| identity (optional) | azure.identity (attribute, nested) | | ||
Notes: | ||
* JSON does not distinguish between fixed and floating point numbers. All | ||
JSON numbers are encoded as doubles. | ||
For Metrics the Azure Metric Records are an array | ||
of "records" with the following fields. | ||
| Azure | Open Telemetry | | ||
|------------|---------------------------------------------| | ||
| time | time_unix_nano (field) | | ||
| resourceId | azure.resource.id (resource attribute) | | ||
| metricName | | | ||
| timeGrain | start_time_unix_nano (field) | | ||
| total | mapped to datapoint metricName + "_TOTAL" | | ||
| count | mapped to datapoint metricName + "_COUNT" | | ||
| minimum | mapped to datapoint metricName + "_MINIMUM" | | ||
| maximum | mapped to datapoint metricName + "_MAXIMUM" | | ||
| average | mapped to datapoint metricName + "_AVERAGE" | | ||
From this data a Metric of type Gauge is created | ||
with a Data Points that represents the values | ||
for the Metric including: Total, Minimum, Maximum, | ||
Average and Count. | ||
[storage extension]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/storage |
36 changes: 36 additions & 0 deletions
36
collector/components/azureeventhubreceiver/azureresourcelogs_unmarshaler.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package azureeventhubreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver" | ||
|
||
import ( | ||
"github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs" | ||
"go.opentelemetry.io/collector/component" | ||
"go.opentelemetry.io/collector/pdata/plog" | ||
"go.uber.org/zap" | ||
|
||
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/translator/azure" | ||
) | ||
|
||
type AzureResourceLogsEventUnmarshaler struct { | ||
unmarshaler *azure.ResourceLogsUnmarshaler | ||
} | ||
|
||
func newAzureResourceLogsUnmarshaler(buildInfo component.BuildInfo, logger *zap.Logger) eventLogsUnmarshaler { | ||
return AzureResourceLogsEventUnmarshaler{ | ||
unmarshaler: &azure.ResourceLogsUnmarshaler{ | ||
Version: buildInfo.Version, | ||
Logger: logger, | ||
}, | ||
} | ||
} | ||
|
||
// UnmarshalLogs takes a byte array containing a JSON-encoded | ||
// payload with Azure log records and transforms it into | ||
// an OpenTelemetry plog.Logs object. The data in the Azure | ||
// log record appears as fields and attributes in the | ||
// OpenTelemetry representation; the bodies of the | ||
// OpenTelemetry log records are empty. | ||
func (r AzureResourceLogsEventUnmarshaler) UnmarshalLogs(event *azeventhubs.ReceivedEventData) (plog.Logs, error) { | ||
return r.unmarshaler.UnmarshalLogs(event.Body) | ||
} |
118 changes: 118 additions & 0 deletions
118
collector/components/azureeventhubreceiver/azureresourcelogs_unmarshaler_test.go
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.