Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate otel metrics to use mdatagen: routing processor #33539

Merged
merged 2 commits into from
Jun 20, 2024

Conversation

jpkrohling
Copy link
Member

Before:

# HELP otelcol_routing_processor_non_routed_log_records Number of log records that were not routed to some or all exporters
# TYPE otelcol_routing_processor_non_routed_log_records counter
otelcol_routing_processor_non_routed_log_records{routing_key="",service_instance_id="c0c03666-9ef6-456e-ae9d-b5d8c2041ce4",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
# HELP otelcol_routing_processor_non_routed_metric_points Number of metric points that were not routed to some or all exporters.
# TYPE otelcol_routing_processor_non_routed_metric_points counter
otelcol_routing_processor_non_routed_metric_points{routing_key="",service_instance_id="c0c03666-9ef6-456e-ae9d-b5d8c2041ce4",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
# HELP otelcol_routing_processor_non_routed_spans Number of spans that were not routed to some or all exporters.
# TYPE otelcol_routing_processor_non_routed_spans counter
otelcol_routing_processor_non_routed_spans{routing_key="",service_instance_id="c0c03666-9ef6-456e-ae9d-b5d8c2041ce4",service_name="otelcontribcol",service_version="0.102.0-dev"} 2

After:

# HELP otelcol_routing_processor_non_routed_log_records Number of log records that were not routed to some or all exporters.
# TYPE otelcol_routing_processor_non_routed_log_records counter
otelcol_routing_processor_non_routed_log_records{routing_key="",service_instance_id="f27bc682-a3a0-44ed-9b88-b870dd4d096a",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
# HELP otelcol_routing_processor_non_routed_metric_points Number of metric points that were not routed to some or all exporters.
# TYPE otelcol_routing_processor_non_routed_metric_points counter
otelcol_routing_processor_non_routed_metric_points{routing_key="",service_instance_id="f27bc682-a3a0-44ed-9b88-b870dd4d096a",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
# HELP otelcol_routing_processor_non_routed_spans Number of spans that were not routed to some or all exporters.
# TYPE otelcol_routing_processor_non_routed_spans counter
otelcol_routing_processor_non_routed_spans{routing_key="",service_instance_id="f27bc682-a3a0-44ed-9b88-b870dd4d096a",service_name="otelcontribcol",service_version="0.102.0-dev"} 2

Configuration used for testing:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4317

processors:
  routing:
    default_exporters: [ debug/routed ]
    attribute_source: resource
    from_attribute: vip
    error_mode: ignore
    table:
      - value: "true"
        exporters:
          - debug/vip

exporters:
  debug:
    verbosity: detailed
  debug/routed:
    verbosity: detailed
  debug/vip:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [routing]
      exporters: [debug, debug/routed, debug/vip]
    logs:
      receivers: [otlp]
      processors: [routing]
      exporters: [debug, debug/routed, debug/vip]
    metrics:
      receivers: [otlp]
      processors: [routing]
      exporters: [debug, debug/routed, debug/vip]

Test commands:

telemetrygen metrics --metrics 1 --otlp-insecure --otlp-attributes='vip="false"'
telemetrygen logs --logs 1 --otlp-insecure --otlp-attributes='vip="false"'
telemetrygen traces --traces 1 --otlp-insecure --otlp-attributes='vip="false"'

Fixes #33526

Signed-off-by: Juraci Paixão Kröhling juraci@kroehling.de

@jpkrohling jpkrohling requested review from a team, bryan-aguilar and codeboten and removed request for bryan-aguilar June 13, 2024 09:29
@github-actions github-actions bot added the processor/routing Routing processor label Jun 13, 2024
Copy link
Contributor

@codeboten codeboten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one question about optional to avoid instantiating unnecessary instruments

processor/routingprocessor/metadata.yaml Show resolved Hide resolved
processor/routingprocessor/documentation.md Show resolved Hide resolved
Copy link
Contributor

@codeboten codeboten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go.mod file needs resolving

Before:
```
otelcol_routing_processor_non_routed_log_records{routing_key="",service_instance_id="c0c03666-9ef6-456e-ae9d-b5d8c2041ce4",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
otelcol_routing_processor_non_routed_metric_points{routing_key="",service_instance_id="c0c03666-9ef6-456e-ae9d-b5d8c2041ce4",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
otelcol_routing_processor_non_routed_spans{routing_key="",service_instance_id="c0c03666-9ef6-456e-ae9d-b5d8c2041ce4",service_name="otelcontribcol",service_version="0.102.0-dev"} 2
```

After:
```
otelcol_routing_processor_non_routed_log_records{routing_key="",service_instance_id="f27bc682-a3a0-44ed-9b88-b870dd4d096a",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
otelcol_routing_processor_non_routed_metric_points{routing_key="",service_instance_id="f27bc682-a3a0-44ed-9b88-b870dd4d096a",service_name="otelcontribcol",service_version="0.102.0-dev"} 1
otelcol_routing_processor_non_routed_spans{routing_key="",service_instance_id="f27bc682-a3a0-44ed-9b88-b870dd4d096a",service_name="otelcontribcol",service_version="0.102.0-dev"} 2
```

Configuration used for testing:
```yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4317

processors:
  routing:
    default_exporters: [ debug/routed ]
    attribute_source: resource
    from_attribute: vip
    error_mode: ignore
    table:
      - value: "true"
        exporters:
          - debug/vip

exporters:
  debug:
    verbosity: detailed
  debug/routed:
    verbosity: detailed
  debug/vip:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [routing]
      exporters: [debug, debug/routed, debug/vip]
    logs:
      receivers: [otlp]
      processors: [routing]
      exporters: [debug, debug/routed, debug/vip]
    metrics:
      receivers: [otlp]
      processors: [routing]
      exporters: [debug, debug/routed, debug/vip]
```

Test commands:
```
telemetrygen metrics --metrics 1 --otlp-insecure --otlp-attributes='vip="false"'
telemetrygen logs --logs 1 --otlp-insecure --otlp-attributes='vip="false"'
telemetrygen traces --traces 1 --otlp-insecure --otlp-attributes='vip="false"'
```

Fixes open-telemetry#33526

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
@jpkrohling jpkrohling force-pushed the jpkrohling/issue33526 branch from 2d15f42 to c580dc7 Compare June 20, 2024 09:01
@jpkrohling jpkrohling merged commit 6d9f9cc into open-telemetry:main Jun 20, 2024
154 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
processor/routing Routing processor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate otel metrics to use mdatagen: routing processor
3 participants