Opentelemetry tracing #2577

sbiscigl · 2023-07-13T14:15:07Z

Description of changes:

Creates a Smithy Reference Architecture interface for emitting traces and metrics. This will allow for client users to more easily monitor the telemetry of the SDK inside a running application to find metrics. We also provide a Open telemetry implementation of metrics and tracing so that a user could see immediate value. For example a user could configure the SDK as such

int main() {
    SDKOptions options;
    InitAPI(options);
    {
        S3ClientConfiguration configuration;
        configuration.telemetryProvider = OtelTelemetryProvider::CreateOtelProvider(
            opentelemetry::exporter::trace::OStreamSpanExporterFactory::Create(),
            std::make_unique<opentelemetry::exporter::metrics::OStreamMetricExporter>());
        auto s3 = Aws::MakeUnique<S3Client>("test-alloc", configuration);
        auto result = s3->ListBuckets();
        if (!result.IsSuccess()) {
            std::cout << "error:" << result.GetError().GetMessage() << "\n";
        }
        auto getObjResp = s3->GetObject(GetObjectRequest()
            .WithBucket("bucket-name")
            .WithKey("key-name"));
        if (!getObjResp.IsSuccess()) {
            std::cout << "error:" << getObjResp.GetError().GetMessage() << "\n";
        }
        // sleep for period metrics emitter
        std::this_thread::sleep_for(std::chrono::milliseconds(1000));
    }
    ShutdownAPI(options);
}

and receive the following output

{
  name          : S3.ListBuckets
  trace_id      : 0cb806c50dced1d97384b3891f3876c6
  span_id       : 9f1da789275c1a3c
  tracestate    : 
  parent_span_id: 0000000000000000
  start         : 1689257478862356000
  duration      : 376816125
  description   : 
  span kind     : Client
  status        : Unset
  attributes    : 
        rpc.system: aws-api
        rpc.service: S3
        rpc.method: ListBuckets
  events        : 
  links         : 
  resources     : 
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  instr-lib     : S3
}
{
  name          : S3.GetObject
  trace_id      : 054a45ec564bd3de9051d3720a5c76b2
  span_id       : 7c65af6ee7dc05b7
  tracestate    : 
  parent_span_id: 0000000000000000
  start         : 1689257479239449000
  duration      : 359973208
  description   : 
  span kind     : Client
  status        : Unset
  attributes    : 
        rpc.system: aws-api
        rpc.service: S3
        rpc.method: GetObject
  events        : 
  links         : 
  resources     : 
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  instr-lib     : S3
}
{
  scope name    : S3
  schema url    : 
  version       : 
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.deserialization_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 2
  min     : 2
  max     : 2
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: ListBuckets
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.http.ssl_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 507
  min     : 507
  max     : 507
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.http.dns_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 27
  min     : 27
  max     : 27
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 359
  min     : 359
  max     : 359
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.http.connect_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 356
  min     : 356
  max     : 356
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.serialization_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 0
  min     : 0
  max     : 2.22507e-308
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.attempts
  description   : 
  unit          : n
  type          : SumPointData
  value         : 1
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.service_call_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 357
  min     : 357
  max     : 357
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.auth.signing_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 0
  min     : 0
  max     : 2.22507e-308
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.http.throughput
  description   : 
  unit          : bytes/s
  type     : HistogramPointData
  count     : 1
  sum     : 16
  min     : 16
  max     : 16
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
  start time    : Thu Jul 13 14:11:18 2023
  end time      : Thu Jul 13 14:11:19 2023
  instrument name       : smithy.client.resolve_endpoint_duration
  description   : 
  unit          : ms
  type     : HistogramPointData
  count     : 1
  sum     : 0
  min     : 0
  max     : 2.22507e-308
  buckets     : [0, 5, 10, 25, 50, 75, 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000, ]
  counts     : [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ]
  attributes            : 
        rpc.method: GetObject
        rpc.service: S3
  resources     :
        service.name: unknown_service
        telemetry.sdk.language: cpp
        telemetry.sdk.name: opentelemetry
        telemetry.sdk.version: 1.10.0
}

While the above uses the OStream exporter, the SDK can be configured with any of the open telemetry exporters. In specific this can be used with the AWS Distro for OpenTelemetry to report metrics and traces straight to AWS.

Check all that applies:

Did a review by yourself.
Added proper tests to cover this PR. (If tests are not applicable, explain.)
Checked if this PR is a breaking (APIs have been changed) change.
Checked if this PR will not introduce cross-platform inconsistent behavior.
Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

generated/src/aws-cpp-sdk-AWSMigrationHub/source/MigrationHubClient.cpp

SergeyRyabinin · 2023-07-13T18:13:07Z

generated/src/aws-cpp-sdk-AWSMigrationHub/source/MigrationHubClient.cpp

-  return AssociateCreatedArtifactOutcome(MakeRequest(request, endpointResolutionOutcome.GetResult(), Aws::Http::HttpMethod::HTTP_POST, Aws::Auth::SIGV4_SIGNER));
+  auto tracer = m_telemetryProvider->getTracer(this->GetServiceClientName(), {});
+  auto span = tracer->CreateSpan(Aws::String(this->GetServiceClientName()) + ".AssociateCreatedArtifact",
+    {{ "rpc.method", request.GetServiceRequestName() }, { "rpc.service", this->GetServiceClientName() }, { "rpc.system", "aws-api" }},


if this->GetServiceClientName() is used to get over-writable service name (that can be updated by customer to update user agent) - then good

otherwise, SERVICE_NAME static variable will serve this purpose.

I dont think we will be able to use SERVICE_NAME because we add the service name dimensions in XmlClient/JsonClient/AwsClient where it is not accessible. Also SERVICE_NAME and GetServiceClientName could and do return different values, and they are needed to be the same for dimensions to work correctly.

that function is defined as

inline virtual const char* GetServiceClientName() const { return m_serviceName.c_str(); }

so i think it is ok to leave as is, let me know what you think though

src/aws-cpp-sdk-core/include/smithy/tracing/TracingUtils.h

src/aws-cpp-sdk-core/source/client/AWSClient.cpp

src/aws-cpp-sdk-core/source/monitoring/HttpClientMetrics.cpp

src/aws-cpp-sdk-core/source/smithy/tracing/impl/opentelemetry/MeterAdapters.cpp

SergeyRyabinin

Looks good.
Please also add doc comments for classes and methods definitions in headers.

…jects tracing and metrics into the SDK

SergeyRyabinin reviewed Jul 13, 2023

View reviewed changes

SergeyRyabinin approved these changes Jul 14, 2023

View reviewed changes

sbiscigl force-pushed the opentelemetry-tracing branch 4 times, most recently from e5224e5 to 0521c7b Compare July 14, 2023 21:41

Updates SRA tracing interfaces. Adds opentelemetry implementation, in…

c5983b8

…jects tracing and metrics into the SDK

sbiscigl force-pushed the opentelemetry-tracing branch from 0521c7b to c5983b8 Compare July 14, 2023 23:35

sbiscigl marked this pull request as ready for review July 17, 2023 17:29

sbiscigl merged commit 32d186c into main Jul 17, 2023

sbiscigl deleted the opentelemetry-tracing branch July 17, 2023 17:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Opentelemetry tracing #2577

Opentelemetry tracing #2577

sbiscigl commented Jul 13, 2023

SergeyRyabinin Jul 13, 2023

sbiscigl Jul 13, 2023

SergeyRyabinin left a comment

Opentelemetry tracing #2577

Opentelemetry tracing #2577

Conversation

sbiscigl commented Jul 13, 2023

SergeyRyabinin Jul 13, 2023

Choose a reason for hiding this comment

sbiscigl Jul 13, 2023

Choose a reason for hiding this comment

SergeyRyabinin left a comment

Choose a reason for hiding this comment