Update cloudwatch exporter dependency (#839)

* Update cloudwatch exporter dependency * Fix imports * Fix converter tests * Add documentation * Change CHANGELOG.md * Revert some changes in the test * Update CHANGELOG.md * Fix tests * Update go.mod * Add -ld_classic to the CGO_LDFLAGS * Update CHANGELOG.md Co-authored-by: Kyle Eckhart <kgeckhart@users.noreply.github.com> * Implements comments from the PR * Rename variable * Rename variable * Update docs/sources/reference/components/prometheus.exporter.cloudwatch.md Co-authored-by: Kyle Eckhart <kgeckhart@users.noreply.github.com> * Update go.mod * Update prometheus.exporter.cloudwatch.md * Add compatibility layer for the service aliases * Solve merge conflicts * Rename service name * Rename service name * Remove ld_classic tag * Update of the CHANGELOG.md * Revert "Remove ld_classic tag" This reverts commit e176bd4. --------- Co-authored-by: Kyle Eckhart <kgeckhart@users.noreply.github.com>
grafana · Sep 3, 2024 · 08573df · 08573df
1 parent 6fcf6cf
commit 08573df
Show file tree

Hide file tree

Showing 19 changed files with 732 additions and 573 deletions.
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -23,4 +23,4 @@ jobs:
         go-version: "1.22"
         cache: true
     - name: Test
-      run: make GO_TAGS="nodocker" test
+      run: CGO_LDFLAGS="-ld_classic $CGO_LDFLAGS" make GO_TAGS="nodocker" test
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -41,6 +41,8 @@ Main (unreleased)
 
 - Add extra validation in `beyla.ebpf` to avoid panics when network feature is enabled. (@marctc)
 
+- A new parameter `aws_sdk_version_v2` is added for the cloudwatch exporters configuration. It enables the use of aws sdk v2 which has shown to have significant performance benefits. (@kgeckhart, @andriikushch)
+
 ### Bugfixes
 
 - Fix a bug where custom components don't always get updated when the config is modified in an imported directory. (@ante012)
@@ -70,6 +72,8 @@ Main (unreleased)
 
 - Renamed standard library functions. Old names are still valid but are marked deprecated. (@wildum)
 
+- Aliases for the namespaces are deprecated in the Cloudwatch exporter. For example: "s3" is not allowed, "AWS/S3" should be used. Usage of the aliases will generate warnings in the logs. Support for the aliases will be dropped in the upcoming releases. (@kgeckhart, @andriikushch)
+
 v1.3.1
 -----------------
 

diff --git a/docs/sources/reference/components/prometheus/prometheus.exporter.cloudwatch.md b/docs/sources/reference/components/prometheus/prometheus.exporter.cloudwatch.md
@@ -92,10 +92,10 @@ To use all of the integration features, use the following AWS IAM Policy:
 
 ```alloy
 prometheus.exporter.cloudwatch "queues" {
-    sts_region = "us-east-2"
-
+    sts_region      = "us-east-2"
+    aws_sdk_version_v2 = "false"
     discovery {
-        type        = "sqs"
+        type        = "AWS/SQS"
         regions     = ["us-east-2"]
         search_tags = {
             "scrape" = "true",
@@ -124,6 +124,7 @@ Omitted fields take their default values.
 | Name                      | Type                | Description                                                                    | Default | Required |
 |---------------------------|---------------------|--------------------------------------------------------------------------------|---------|----------|
 | `sts_region`              | `string`            | AWS region to use when calling [STS][] for retrieving account information.     |         | yes      |
+| `aws_sdk_version_v2`      | `bool`              | Use AWS SDK version 2.                                                                                                                                                                                                                  | `false` | no       |
 | `fips_disabled`           | `bool`              | Disable use of FIPS endpoints. Set 'true' when running outside of USA regions. | `true`  | no       |
 | `debug`                   | `bool`              | Enable debug logging on CloudWatch exporter internals.                         | `false` | no       |
 | `discovery_exported_tags` | `map(list(string))` | List of tags (value) per service (key) to export in all metrics. For example, defining the `["name", "type"]` under `"AWS/EC2"` will export the name and type tags and its values as labels in all metrics. Affects all discovery jobs. | `{}` | no |
@@ -366,91 +367,100 @@ See the examples described under each [discovery][] and [static] sections.
 ## Supported services in discovery jobs
 
 The following is a list of AWS services that are supported in `cloudwatch_exporter` discovery jobs. When configuring a
-discovery job, the `type` field of each `discovery_job` must match either the desired job namespace or alias.
-
-- Namespace: `CWAgent` or Alias: `cwagent`
-- Namespace: `AWS/Usage` or Alias: `usage`
-- Namespace: `AWS/CertificateManager` or Alias: `acm`
-- Namespace: `AWS/ACMPrivateCA` or Alias: `acm-pca`
-- Namespace: `AmazonMWAA` or Alias: `airflow`
-- Namespace: `AWS/MWAA` or Alias: `mwaa`
-- Namespace: `AWS/ApplicationELB` or Alias: `alb`
-- Namespace: `AWS/AppStream` or Alias: `appstream`
-- Namespace: `AWS/Backup` or Alias: `backup`
-- Namespace: `AWS/ApiGateway` or Alias: `apigateway`
-- Namespace: `AWS/AmazonMQ` or Alias: `mq`
-- Namespace: `AWS/AppSync` or Alias: `appsync`
-- Namespace: `AWS/Athena` or Alias: `athena`
-- Namespace: `AWS/AutoScaling` or Alias: `asg`
-- Namespace: `AWS/ElasticBeanstalk` or Alias: `beanstalk`
-- Namespace: `AWS/Billing` or Alias: `billing`
-- Namespace: `AWS/Cassandra` or Alias: `cassandra`
-- Namespace: `AWS/CloudFront` or Alias: `cloudfront`
-- Namespace: `AWS/Cognito` or Alias: `cognito-idp`
-- Namespace: `AWS/DMS` or Alias: `dms`
-- Namespace: `AWS/DDoSProtection` or Alias: `shield`
-- Namespace: `AWS/DocDB` or Alias: `docdb`
-- Namespace: `AWS/DX` or Alias: `dx`
-- Namespace: `AWS/DynamoDB` or Alias: `dynamodb`
-- Namespace: `AWS/EBS` or Alias: `ebs`
-- Namespace: `AWS/ElastiCache` or Alias: `ec`
-- Namespace: `AWS/MemoryDB` or Alias: `memorydb`
-- Namespace: `AWS/EC2` or Alias: `ec2`
-- Namespace: `AWS/EC2Spot` or Alias: `ec2Spot`
-- Namespace: `AWS/ECS` or Alias: `ecs-svc`
-- Namespace: `ECS/ContainerInsights` or Alias: `ecs-containerinsights`
-- Namespace: `AWS/EFS` or Alias: `efs`
-- Namespace: `AWS/ELB` or Alias: `elb`
-- Namespace: `AWS/ElasticMapReduce` or Alias: `emr`
-- Namespace: `AWS/EMRServerless` or Alias: `emr-serverless`
-- Namespace: `AWS/ES` or Alias: `es`
-- Namespace: `AWS/Firehose` or Alias: `firehose`
-- Namespace: `AWS/FSx` or Alias: `fsx`
-- Namespace: `AWS/GameLift` or Alias: `gamelift`
-- Namespace: `AWS/GlobalAccelerator` or Alias: `ga`
-- Namespace: `Glue` or Alias: `glue`
-- Namespace: `AWS/IoT` or Alias: `iot`
-- Namespace: `AWS/Kafka` or Alias: `kafka`
-- Namespace: `AWS/KafkaConnect` or Alias: `kafkaconnect`
-- Namespace: `AWS/Kinesis` or Alias: `kinesis`
-- Namespace: `AWS/KinesisAnalytics` or Alias: `kinesis-analytics`
-- Namespace: `AWS/Lambda` or Alias: `lambda`
-- Namespace: `AWS/MediaConnect` or Alias: `mediaconnect`
-- Namespace: `AWS/MediaConvert` or Alias: `mediaconvert`
-- Namespace: `AWS/MediaLive` or Alias: `medialive`
-- Namespace: `AWS/MediaTailor` or Alias: `mediatailor`
-- Namespace: `AWS/Neptune` or Alias: `neptune`
-- Namespace: `AWS/NetworkFirewall` or Alias: `nfw`
-- Namespace: `AWS/NATGateway` or Alias: `ngw`
-- Namespace: `AWS/NetworkELB` or Alias: `nlb`
-- Namespace: `AWS/PrivateLinkEndpoints` or Alias: `vpc-endpoint`
-- Namespace: `AWS/PrivateLinkServices` or Alias: `vpc-endpoint-service`
-- Namespace: `AWS/Prometheus` or Alias: `amp`
-- Namespace: `AWS/QLDB` or Alias: `qldb`
-- Namespace: `AWS/RDS` or Alias: `rds`
-- Namespace: `AWS/Redshift` or Alias: `redshift`
-- Namespace: `AWS/Route53Resolver` or Alias: `route53-resolver`
-- Namespace: `AWS/Route53` or Alias: `route53`
-- Namespace: `AWS/S3` or Alias: `s3`
-- Namespace: `AWS/SES` or Alias: `ses`
-- Namespace: `AWS/States` or Alias: `sfn`
-- Namespace: `AWS/SNS` or Alias: `sns`
-- Namespace: `AWS/SQS` or Alias: `sqs`
-- Namespace: `AWS/StorageGateway` or Alias: `storagegateway`
-- Namespace: `AWS/TransitGateway` or Alias: `tgw`
-- Namespace: `AWS/TrustedAdvisor` or Alias: `trustedadvisor`
-- Namespace: `AWS/VPN` or Alias: `vpn`
-- Namespace: `AWS/ClientVPN` or Alias: `clientvpn`
-- Namespace: `AWS/WAFV2` or Alias: `wafv2`
-- Namespace: `AWS/WorkSpaces` or Alias: `workspaces`
-- Namespace: `AWS/AOSS` or Alias: `aoss`
-- Namespace: `AWS/SageMaker` or Alias: `sagemaker`
-- Namespace: `/aws/sagemaker/Endpoints` or Alias: `sagemaker-endpoints`
-- Namespace: `/aws/sagemaker/TrainingJobs` or Alias: `sagemaker-training`
-- Namespace: `/aws/sagemaker/ProcessingJobs` or Alias: `sagemaker-processing`
-- Namespace: `/aws/sagemaker/TransformJobs` or Alias: `sagemaker-transform`
-- Namespace: `/aws/sagemaker/InferenceRecommendationsJobs` or Alias: `sagemaker-inf-rec`
-- Namespace: `AWS/Sagemaker/ModelBuildingPipeline` or Alias: `sagemaker-model-building-pipeline`
+discovery job, the `type` field of each `discovery_job` must match the desired job namespace.
+
+- Namespace: `CWAgent`
+- Namespace: `AWS/Usage`
+- Namespace: `AWS/CertificateManager`
+- Namespace: `AWS/ACMPrivateCA`
+- Namespace: `AmazonMWAA`
+- Namespace: `AWS/MWAA`
+- Namespace: `AWS/ApplicationELB`
+- Namespace: `AWS/AppStream`
+- Namespace: `AWS/Backup`
+- Namespace: `AWS/ApiGateway`
+- Namespace: `AWS/AmazonMQ`
+- Namespace: `AWS/AppSync`
+- Namespace: `AWS/Athena`
+- Namespace: `AWS/AutoScaling`
+- Namespace: `AWS/ElasticBeanstalk`
+- Namespace: `AWS/Billing`
+- Namespace: `AWS/Cassandra`
+- Namespace: `AWS/CloudFront`
+- Namespace: `AWS/Cognito`
+- Namespace: `AWS/DataSync`
+- Namespace: `AWS/DMS`
+- Namespace: `AWS/DDoSProtection`
+- Namespace: `AWS/DocDB`
+- Namespace: `AWS/DX`
+- Namespace: `AWS/DynamoDB`
+- Namespace: `AWS/EBS`
+- Namespace: `AWS/ElastiCache`
+- Namespace: `AWS/MemoryDB`
+- Namespace: `AWS/EC2`
+- Namespace: `AWS/EC2Spot`
+- Namespace: `AWS/ECS`
+- Namespace: `ECS/ContainerInsights`
+- Namespace: `AWS/EFS`
+- Namespace: `AWS/ELB`
+- Namespace: `AWS/ElasticMapReduce`
+- Namespace: `AWS/EMRServerless`
+- Namespace: `AWS/ES`
+- Namespace: `AWS/Firehose`
+- Namespace: `AWS/FSx`
+- Namespace: `AWS/GameLift`
+- Namespace: `AWS/GatewayELB`
+- Namespace: `AWS/GlobalAccelerator`
+- Namespace: `Glue`
+- Namespace: `AWS/IoT`
+- Namespace: `AWS/Kafka`
+- Namespace: `AWS/KafkaConnect`
+- Namespace: `AWS/Kinesis`
+- Namespace: `AWS/KinesisAnalytics`
+- Namespace: `AWS/KMS`
+- Namespace: `AWS/Lambda`
+- Namespace: `AWS/Logs`
+- Namespace: `AWS/MediaConnect`
+- Namespace: `AWS/MediaConvert`
+- Namespace: `AWS/MediaLive`
+- Namespace: `AWS/MediaTailor`
+- Namespace: `AWS/Neptune`
+- Namespace: `AWS/NetworkFirewall`
+- Namespace: `AWS/NATGateway`
+- Namespace: `AWS/NetworkELB`
+- Namespace: `AWS/PrivateLinkEndpoints`
+- Namespace: `AWS/PrivateLinkServices`
+- Namespace: `AWS/Prometheus`
+- Namespace: `AWS/QLDB`
+- Namespace: `AWS/RDS`
+- Namespace: `AWS/Redshift`
+- Namespace: `AWS/Route53Resolver`
+- Namespace: `AWS/Route53`
+- Namespace: `AWS/RUM`
+- Namespace: `AWS/S3`
+- Namespace: `AWS/SecretsManager`
+- Namespace: `AWS/SES`
+- Namespace: `AWS/States`
+- Namespace: `AWS/SNS`
+- Namespace: `AWS/SQS`
+- Namespace: `AWS/StorageGateway`
+- Namespace: `AWS/TransitGateway`
+- Namespace: `AWS/TrustedAdvisor`
+- Namespace: `AWS/VPN`
+- Namespace: `AWS/ClientVPN`
+- Namespace: `AWS/WAFV2`
+- Namespace: `AWS/WorkSpaces`
+- Namespace: `AWS/AOSS`
+- Namespace: `AWS/SageMaker`
+- Namespace: `/aws/sagemaker/Endpoints`
+- Namespace: `/aws/sagemaker/TrainingJobs`
+- Namespace: `/aws/sagemaker/ProcessingJobs`
+- Namespace: `/aws/sagemaker/TransformJobs`
+- Namespace: `/aws/sagemaker/InferenceRecommendationsJobs`
+- Namespace: `AWS/Sagemaker/ModelBuildingPipeline`
+- Namespace: `AWS/IPAM`
+- Namespace: `AWS/Bedrock`
+- Namespace: `AWS/Events`
 
 <!-- START GENERATED COMPATIBLE COMPONENTS -->
 

diff --git a/go.mod b/go.mod
@@ -97,7 +97,7 @@ require (
 	github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
 	github.com/natefinch/atomic v1.0.1
 	github.com/ncabatoff/process-exporter v0.7.10
-	github.com/nerdswords/yet-another-cloudwatch-exporter v0.55.0
+	github.com/nerdswords/yet-another-cloudwatch-exporter v0.60.0
 	github.com/oklog/run v1.1.0
 	github.com/olekukonko/tablewriter v0.0.5
 	github.com/oliver006/redis_exporter v1.54.0
@@ -386,22 +386,23 @@ require (
 	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.7 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/amp v1.23.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/apigateway v1.22.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/apigatewayv2 v1.19.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/autoscaling v1.38.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/databasemigrationservice v1.36.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ec2 v1.149.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/amp v1.25.5 // indirect
+	github.com/aws/aws-sdk-go-v2/service/apigateway v1.23.7 // indirect
+	github.com/aws/aws-sdk-go-v2/service/apigatewayv2 v1.20.5 // indirect
+	github.com/aws/aws-sdk-go-v2/service/autoscaling v1.40.6 // indirect
+	github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.38.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/databasemigrationservice v1.38.5 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ec2 v1.161.1 // indirect
 	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 // indirect
 	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.3.0 // indirect
 	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.9 // indirect
 	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.17.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.20.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.21.5 // indirect
 	github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.27.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/shield v1.24.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/shield v1.25.5 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sso v1.20.9 // indirect
 	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.24.3 // indirect
-	github.com/aws/aws-sdk-go-v2/service/storagegateway v1.26.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/storagegateway v1.27.5 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sts v1.28.10 // indirect
 	github.com/aws/smithy-go v1.20.2 // indirect
 	github.com/axiomhq/hyperloglog v0.0.0-20240124082744-24bca3a5b39b // indirect