Add otel components command

[ChrsMark]

What does this PR do?

This PR adds the components command for otel mode.

Implementation is replicated from https://github.com/open-telemetry/opentelemetry-collector/blob/main/otelcol/command_components.go.

Why is it important?

otel components command can be used to list the supported components the EDOT includes.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

Build Elastic Agent and execute the command:

./build/elastic-agent otel components          
buildinfo:
    command: ./build/elastic-agent
    description: Elastic opentelemetry-collector distribution
    version: 9.0.0
receivers:
    - name: filebeatreceiver
      stability:
        logs: Alpha
        metrics: Undefined
        traces: Undefined
    - name: filelog
      stability:
        logs: Beta
        metrics: Undefined
        traces: Undefined
    - name: hostmetrics
      stability:
        logs: Development
        metrics: Beta
        traces: Undefined
    - name: httpcheck
      stability:
        logs: Undefined
        metrics: Alpha
        traces: Undefined
    - name: jaeger
      stability:
        logs: Undefined
        metrics: Undefined
        traces: Beta
    - name: k8s_cluster
      stability:
        logs: Beta
        metrics: Beta
        traces: Undefined
    - name: k8sobjects
      stability:
        logs: Beta
        metrics: Undefined
        traces: Undefined
    - name: kubeletstats
      stability:
        logs: Undefined
        metrics: Beta
        traces: Undefined
    - name: metricbeatreceiver
      stability:
        logs: Alpha
        metrics: Undefined
        traces: Undefined
    - name: otlp
      stability:
        logs: Beta
        metrics: Stable
        traces: Stable
    - name: prometheus
      stability:
        logs: Undefined
        metrics: Beta
        traces: Undefined
    - name: zipkin
      stability:
        logs: Undefined
        metrics: Undefined
        traces: Beta
processors:
    - name: attributes
      stability:
        logs: Beta
        metrics: Beta
        traces: Beta
    - name: batch
      stability:
        logs: Beta
        metrics: Beta
        traces: Beta
    - name: elasticinframetrics
      stability:
        logs: Undefined
        metrics: Alpha
        traces: Undefined
    - name: elastictrace
      stability:
        logs: Undefined
        metrics: Undefined
        traces: Alpha
    - name: filter
      stability:
        logs: Alpha
        metrics: Alpha
        traces: Alpha
    - name: k8sattributes
      stability:
        logs: Beta
        metrics: Beta
        traces: Beta
    - name: lsminterval
      stability:
        logs: Undefined
        metrics: Development
        traces: Undefined
    - name: memory_limiter
      stability:
        logs: Beta
        metrics: Beta
        traces: Beta
    - name: resource
      stability:
        logs: Beta
        metrics: Beta
        traces: Beta
    - name: resourcedetection
      stability:
        logs: Beta
        metrics: Beta
        traces: Beta
    - name: transform
      stability:
        logs: Alpha
        metrics: Alpha
        traces: Alpha
exporters:
    - name: debug
      stability:
        logs: Development
        metrics: Development
        traces: Development
    - name: elasticsearch
      stability:
        logs: Beta
        metrics: Development
        traces: Beta
    - name: file
      stability:
        logs: Alpha
        metrics: Alpha
        traces: Alpha
    - name: loadbalancing
      stability:
        logs: Beta
        metrics: Development
        traces: Beta
    - name: otlp
      stability:
        logs: Beta
        metrics: Stable
        traces: Stable
    - name: otlphttp
      stability:
        logs: Beta
        metrics: Stable
        traces: Stable
connectors:
    - name: routing
      stability:
        logs-to-logs: Alpha
        logs-to-metrics: Undefined
        logs-to-traces: Undefined
        metrics-to-logs: Undefined
        metrics-to-metrics: Alpha
        metrics-to-traces: Undefined
        traces-to-logs: Undefined
        traces-to-metrics: Undefined
        traces-to-traces: Alpha
    - name: signaltometrics
      stability:
        logs-to-logs: Undefined
        logs-to-metrics: Alpha
        logs-to-traces: Undefined
        metrics-to-logs: Undefined
        metrics-to-metrics: Alpha
        metrics-to-traces: Undefined
        traces-to-logs: Undefined
        traces-to-metrics: Alpha
        traces-to-traces: Undefined
    - name: spanmetrics
      stability:
        logs-to-logs: Undefined
        logs-to-metrics: Undefined
        logs-to-traces: Undefined
        metrics-to-logs: Undefined
        metrics-to-metrics: Undefined
        metrics-to-traces: Undefined
        traces-to-logs: Undefined
        traces-to-metrics: Alpha
        traces-to-traces: Undefined
extensions:
    - name: file_storage
      stability:
        extension: Beta
    - name: health_check
      stability:
        extension: Beta
    - name: memory_limiter
      stability:
        extension: Development
    - name: pprof
      stability:
        extension: Beta

Related issues

• Closes https://github.com/elastic/opentelemetry-dev/issues/396

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

[mergify]

This pull request does not have a backport label. Could you fix it @ChrsMark? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

backport-v8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

[swiatekm]

Could you add a simple integration test to verify that this actually works?

Also, could we use upstream's implementantion here instead of copying it?

[ChrsMark]

Also, could we use upstream's implementantion here instead of copying it?

I didn't find an obvious way to use it directly from go.opentelemetry.io/collector/otelcol since it's not exported: https://github.com/open-telemetry/opentelemetry-collector/blob/c14d4f96d69e0053544d5a68863998b2b69a267b/otelcol/command_components.go#L37

I see our validate implementation is also copied/re-implemented. Let me know if I miss anything here.

Could you add a simple integration test to verify that this actually works?

Shall we test against the real EDOT's output? This would mean that every time we include a new component we will need to update that test too.

[swiatekm]

Also, could we use upstream's implementantion here instead of copying it?

I didn't find an obvious way to use it directly from go.opentelemetry.io/collector/otelcol since it's not exported: https://github.com/open-telemetry/opentelemetry-collector/blob/c14d4f96d69e0053544d5a68863998b2b69a267b/otelcol/command_components.go#L37

Allright, let's leave it as is then.

I see our validate implementation is also copied/re-implemented. Let me know if I miss anything here.

Could you add a simple integration test to verify that this actually works?

Shall we test against the real EDOT's output? This would mean that every time we include a new component we will need to update that test too.

I was just thinking about checking if the output looks reasonable, without enumerating the components, but maybe that's a good idea? I'm not sure if we have an integration test that verifies which components are present right now.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[ChrsMark]

Another thing I see that is missing here is the module (see the sample output in the PR description) information. The reason is that we don't populate the modules when we instantiate the collector factories at

elastic-agent/internal/pkg/otel/components.go

Line 66 in dbfd447

func components(extensionFactories ...extension.Factory) func() (otelcol.Factories, error) {

like it's done in the otel builder.

Is this something that we should fix or we can just skip it from the output for now? Any thoughts?

[swiatekm]

Another thing I see that is missing here is the module (see the sample output in the PR description) information. The reason is that we don't populate the modules when we instantiate the collector factories at

elastic-agent/internal/pkg/otel/components.go

Line 66 in dbfd447

func components(extensionFactories ...extension.Factory) func() (otelcol.Factories, error) {

like it's done in the otel builder.

Is this something that we should fix or we can just skip it from the output for now? Any thoughts?

I'd skip it for now and open an issue to include it. We should be as close to builder output as possible.

[ChrsMark]

All right, I have removed the module information for now (filed #6541 for the generic fix) and added a test that checks that all expected components are found. The test can be further enhanced to check components' stability, buildinfo or whatever else we would like to include. For now I started with the bear minimum.

Feel free to take a look when you get the time.

[ChrsMark]

I wonder if the coverage failure is legit: #6539 (comment).

It complains that the new Components function is not covered, however it should be since it's called by the added test 🤔 .

[jlind23]

@ChrsMark we had similar problem in the past hence the reason why it became a non required check. Feel free to ignore it.

[andrzej-stencel]

Can we mention the command in the OTel README? I think it would help users make sure the command is supported. A single sentence with an example, similar to the one with validate command, would be fine, something like:

Use the components command to get the list of components included in the binary:

./elastic-agent otel components

[ChrsMark]

Can we mention the command in the OTel README? I think it would help users make sure the command is supported. A single sentence with an example, similar to the one with validate command, would be fine, something like:

Use the components command to get the list of components included in the binary:
./elastic-agent otel components

Done!

[ChrsMark]

@andrzej-stencel @michalpristas @swiatekm I have enabled auto-merge but I guess that's not going to work without SonarQube test going green, right?

[swiatekm]

@ChrsMark it will merge without SonarQube, you have a different failure coming from make check-ci. Whitespaces in documentation from what I've seen.

[elastic-sonarqube]

Quality Gate failed

Failed conditions
13.1% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube