1.9.0
As of ddtrace 1.9.0, CPU Profiling 2.0 is now in opt-in (that is, disabled by default) beta 2. (Click to expand for details)
You can enable it:- Using an environment variable by setting
DD_PROFILING_FORCE_ENABLE_NEW=true
- Or via code by adding to your
Datadog.configure
block:
Datadog.configure do |c|
# ... existing configuration ...
c.profiling.advanced.force_enable_new_profiler = true
end
What to expect from Ruby CPU Profiling 2.0 beta 2?
- Finer-grained profiling data due to our sampling engine rewritten in C+Rust. The profiler will be able to run more often and get more information while keeping the same 2% overhead target you're used to, and with a lower impact on latency. Especially when looking at the "Code Hotspots" panel for a distributed trace, expect more and finer grained profiles.
- Thread id information now includes the operating system thread id for Ruby 3.1+, so you'll be able to correlate your thread information when looking at other system monitoring tools
- Thread names are now collected and you're able to filter your profiles by these names
- Experimental support for capturing CPU and Wall-time spent doing Garbage Collection. This is disabled by default as we're still improving the performance of this feature and fixing a few incompatibilities with Ruby Ractors. You can enable it by adding
DD_PROFILING_FORCE_ENABLE_GC=true
orc.profiling.advanced.force_enable_gc_profiling = true
to the instructions seen above.
...with more and faster improvements to come in early 2023!
Give it a try, and we'd love to hear your feedback. Below, you'll find a list of known issues that we're still looking into.
Known issues:
-
Profiling CPU-time overhead is not shown in flamegraphs (unlike with the existing profiler). We will be fixing this soon!
-
Rare incompatibilities with native extensions/libraries.
Ruby CPU Profiling 2.0 gathers profiling data by sending SIGPROF unix signals to Ruby applications. This is a common approach used by many other profilers, and it may cause system calls performed by native extensions/libraries to be interrupted with an EINTR error code (reference).
Most native extensions/libraries are unaffected by this issue, but we know of at least one case: when using the
mysql2
gem together with versions of libmysqlclient older than 8.0.0 this can lead to failed database requests (reference). The affected libmysqlclient version is known to be present on Ubuntu 18.04, but not 20.04 and later releases.
We expect these occurrences to be rare, and will be working to both improve the ecosystem as well as to deploy countermeasures in the profiler itself to avoid triggering these issues. -
Ruby 2.5 and below are missing an API that allows the profiler to detect the currently-active Ruby thread. We have deployed a workaround, but suspect that it may lead to crashes in extremely rare situations. We are still researching a solution for this issue and do not plan on rolling out CPU Profiling 2.0 automatically to Ruby 2.5 and below applications until it is fixed.
-
The disabled-by-default experimental support for capturing CPU and Wall-time spent doing Garbage Collection is incompatible with Ractors due to Ruby upstream bugs (https://bugs.ruby-lang.org/issues/18464 and https://bugs.ruby-lang.org/issues/19112). We plan to work with the Ruby developers to incorporate fixes for these issues.
-
The disabled-by-default experimental support for capturing CPU and Wall-time spent doing Garbage Collection can cause a lot of overhead in Ruby applications with high object allocation rates. We will be fixing this soon!
Added
- Tracing: Add
Stripe
instrumentation (#2557) - Tracing: Add configurable response codes considered as errors for
Net/HTTP
,httprb
andhttpclient
(#2501, #2576)(@caramcc) - Tracing: Flexible header matching for HTTP propagator (#2504)
- Tracing:
OpenTelemetry
Traces support (#2496) - Tracing: W3C: Propagate unknown values as-is (#2485)
- AppSec: Add event kit API (#2512)
- Profiling: Allow profiler development on arm64 macOS (#2573)
- Core: Add
profiling_enabled
state to environment logger output (#2541) - Core: Add 'type' to
OptionDefinition
(#2493) - Allow
debase-ruby_core_source
3.2.0 to be used (#2526)
Changed
- Profiling: Upgrade to
libdatadog
to1.0.1.1.0
(#2530) - Appsec: Update appsec rules
1.4.3
(#2580) - Ci: Update CI Visibility metadata extraction (#2586)
Fixed
- Profiling: Fix wrong
libdatadog
version being picked during profiler build (#2531) - Tracing: Support
PG
calls with a block (#2522) - Ci: Fix error in
teamcity
env vars (#2562)
Read the full changeset and the release milestone