feat(fargate): use adot image for otel

[InesNi]

Description

Running AWS Distro for OpenTelemetry (ADOT) as a sidecar in Fargate in place of the Datadog agent in order to enable supporting multiple publish-metrics reporters rather than just Datadog.

Context

Some observability platforms can't ingest OTLP data directly, but through either OpenTelemetry Collector or their own agent. Currently Artillery AWS Fargate runs support only one such platform - Datadog, by running the Datadog agent as a sidecar.

In order to enable providing Fargate support for other such reporters (primarily Cloudwatch reporter), in this PR the Datadog agent is replaced by ADOT.

ADOT provides an array of built-in components including exporters for Datadog, AWS X-Ray, Cloudwatch, Splunk (SAML and SignalFX), Prometheus etc. Due to this implementing ADOT will enable providing tracing support for multiple platforms with one sidecar.

Implementation

Multiple components needed to be implemented in order to configure and run ADOT:

• Access to a resolved config in the run-cluster.js file ( 3c03ac3, 17d0268, e2b9e68, ef54217, 3c03ac3) in order to:
  • check if ADOT sidecar needs to be set,
  • get the required reporter configurations
• Vendor specific translator functions (7de3f85):
  • translate the reporter configuration into OTel Collector configuration (used by ADOT),
  • provide any additional environment variables that need to be provided to ADOT
• In order for ADOT to use our custom configuration(6aacc28):

  • SSM parameter whose value is the full custom configuration is created

  • The AOT_CONFIG_CONTENT variable that points to the created parameter is set in the ADOT task definition

  • Any environment variables required by the custom config are provided in task overrides

  • The SSM parameter is deleted on cleanup

Testing

• Manually tested with both Playwright and HTTP engine scenarios.
• Unit tests added on publish-metrics side for functions in vendor-adot.js
• Unit test on CLI side added for new applyScriptChanges function responsible for resolving templates in config
• e2e test added on CLI side

[InesNi]

As we agreed @hassy , I'm filtering the publish-metrics config in a separate function here before resolving the config and env vars for ADOT.

However maybe it can be moved into the resolveADOTConfigSettings to allow for better separation of concerns: everything publish-metrics related would get done on publish-metrics side , and this function only does run-cluster type stuff (ssm, task definition, etc) ? Let me know.

[socket-security]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/chai@4.3.7	None	+2	778 kB	chai
npm/cheerio@1.0.0-rc.10	None	+1	227 kB	feedic
npm/ci-info@3.9.0	environment	0	26.1 kB	sibiraj-s
npm/cli-highlight@2.1.11	Transitive: environment, filesystem	+3	690 kB	felixfbecker

🚮 Removed packages: npm/artillery-plugin-fake-data@1.0.2

View full report↗︎