Skip to content

1.9.0

Compare
Choose a tag to compare
@github-actions github-actions released this 30 Jan 17:27
· 5196 commits to master since this release
4a1050c
As of ddtrace 1.9.0, CPU Profiling 2.0 is now in opt-in (that is, disabled by default) beta 2. (Click to expand for details) ​You can enable it:
  • Using an environment variable by setting DD_PROFILING_FORCE_ENABLE_NEW=true
  • Or via code by adding to your Datadog.configure block:
Datadog.configure do |c|
  # ... existing configuration ...
  c.profiling.advanced.force_enable_new_profiler = true
end

​What to expect from Ruby CPU Profiling 2.0 beta 2?

  • Finer-grained profiling data due to our sampling engine rewritten in C+Rust. The profiler will be able to run more often and get more information while keeping the same 2% overhead target you're used to, and with a lower impact on latency. Especially when looking at the "Code Hotspots" panel for a distributed trace, expect more and finer grained profiles.
  • Thread id information now includes the operating system thread id for Ruby 3.1+, so you'll be able to correlate your thread information when looking at other system monitoring tools
  • Thread names are now collected and you're able to filter your profiles by these names
  • Experimental support for capturing CPU and Wall-time spent doing Garbage Collection. This is disabled by default as we're still improving the performance of this feature and fixing a few incompatibilities with Ruby Ractors. You can enable it by adding DD_PROFILING_FORCE_ENABLE_GC=true or c.profiling.advanced.force_enable_gc_profiling = true to the instructions seen above.

...with more and faster improvements to come in early 2023!

Give it a try, and we'd love to hear your feedback. Below, you'll find a list of known issues that we're still looking into.

Known issues:

  • Profiling CPU-time overhead is not shown in flamegraphs (unlike with the existing profiler). We will be fixing this soon!

  • Rare incompatibilities with native extensions/libraries.

    Ruby CPU Profiling 2.0 gathers profiling data by sending SIGPROF unix signals to Ruby applications. This is a common approach used by many other profilers, and it may cause system calls performed by native extensions/libraries to be interrupted with an EINTR error code (reference).

    Most native extensions/libraries are unaffected by this issue, but we know of at least one case: when using the mysql2 gem together with versions of libmysqlclient older than 8.0.0 this can lead to failed database requests (reference). The affected libmysqlclient version is known to be present on Ubuntu 18.04, but not 20.04 and later releases.

    We expect these occurrences to be rare, and will be working to both improve the ecosystem as well as to deploy countermeasures in the profiler itself to avoid triggering these issues.

  • Ruby 2.5 and below are missing an API that allows the profiler to detect the currently-active Ruby thread. We have deployed a workaround, but suspect that it may lead to crashes in extremely rare situations. We are still researching a solution for this issue and do not plan on rolling out CPU Profiling 2.0 automatically to Ruby 2.5 and below applications until it is fixed.

  • The disabled-by-default experimental support for capturing CPU and Wall-time spent doing Garbage Collection is incompatible with Ractors due to Ruby upstream bugs (https://bugs.ruby-lang.org/issues/18464 and https://bugs.ruby-lang.org/issues/19112). We plan to work with the Ruby developers to incorporate fixes for these issues.

  • The disabled-by-default experimental support for capturing CPU and Wall-time spent doing Garbage Collection can cause a lot of overhead in Ruby applications with high object allocation rates. We will be fixing this soon!

Added

  • Tracing: Add Stripe instrumentation (#2557)
  • Tracing: Add configurable response codes considered as errors for Net/HTTP, httprb and httpclient (#2501, #2576)(@caramcc)
  • Tracing: Flexible header matching for HTTP propagator (#2504)
  • Tracing: OpenTelemetry Traces support (#2496)
  • Tracing: W3C: Propagate unknown values as-is (#2485)
  • AppSec: Add event kit API (#2512)
  • Profiling: Allow profiler development on arm64 macOS (#2573)
  • Core: Add profiling_enabled state to environment logger output (#2541)
  • Core: Add 'type' to OptionDefinition (#2493)
  • Allow debase-ruby_core_source 3.2.0 to be used (#2526)

Changed

  • Profiling: Upgrade to libdatadog to 1.0.1.1.0 (#2530)
  • Appsec: Update appsec rules 1.4.3 (#2580)
  • Ci: Update CI Visibility metadata extraction (#2586)

Fixed

  • Profiling: Fix wrong libdatadog version being picked during profiler build (#2531)
  • Tracing: Support PG calls with a block (#2522)
  • Ci: Fix error in teamcity env vars (#2562)

Read the full changeset and the release milestone