[PROF-13732] Enable testing libdatadog-based features on macOS by ivoanjo · Pull Request #5351 · DataDog/dd-trace-rb

ivoanjo · 2026-02-12T12:51:46Z

What does this PR do?

With the upgrade to libdatadog v27 in #5274, we finally have libdatadog builds on rubygems.org. This PR adjusts our testing logic to:

Assume that if you're on macOS libdatadog is now available (and fail if it isn't)
Enable testing with libdatadog in CI

I ended up also de-duplicating some of our helper code, hopefully simplifying how we do this kind of integration.

Motivation:

Make sure libdatadog features are working and keep working on macOS.

Change log entry

Yes. Enable libdatadog-based features on macOS

Additional Notes:

This PR is on top of #5274, so we can only merge it once that one's in good shape.

I don't have a macOS machine available myself, so please let me know if I missed a spot!

How to test the change?

The existing tests are now executed and not skipped on macOS so that should be the validation :)

pr-commenter · 2026-02-12T13:21:36Z

Benchmarks

Benchmark execution time: 2026-03-16 15:36:22

Comparing candidate commit 9baf8ac in PR branch ivoanjo/prof-13732-improve-macos-testing with baseline commit 1fc7720 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

ivoanjo · 2026-02-12T14:08:32Z

TODO for myself : Since this branch is not (yet) targeting master, most of CI checks are not running. Once #5274 is merged and this branch I should check that the extra tests are successfully running on our macOS CI jobs.

marcotc

Super exciting!

Use of libdatadog will make this helper more and more common so let's put it in a central location.

I chose to keep the `skip_if_libdatadog_not_supported` as I like seeing the skip on the rspec output but this is very debatable so if you're unconvinced I don't mind too much going the other way.

Now that we do have separate libdatadog testing, it's time to decouple this from profiling!

ivoanjo · 2026-03-03T10:10:07Z

Update: Rebased this on top of current master (to pull in latest libdatadog v28)

This avoids the previous approach of manually keeping them in sync (and we were missing the new `libdatadog_extconf_helpers_spec.rb`). I've also removed any duplicates.

I think it's safe to assume folks on macOS on arm64 are not running 10 year old compilers.

ivoanjo · 2026-03-03T14:35:49Z

Now that we have this up-and-running in CI I've found a new more things that need work (which is why "Test macOS" is failing) so I'm moving this to draft while I work on them

Fix ``` ../../../../ext/datadog_profiling_native_extension/collectors_cpu_and_wall_time_worker.c:245:14: error: unused function 'rescued_after_gvl_running_from_postponed_job' [-Werror,-Wunused-function] 245 | static VALUE rescued_after_gvl_running_from_postponed_job(VALUE self_instance); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../../../ext/datadog_profiling_native_extension/collectors_cpu_and_wall_time_worker.c:1499:14: error: unused function 'handle_sampling_failure_rescued_after_gvl_running_from_postponed_job' [-Werror,-Wunused-function] 1499 | static VALUE handle_sampling_failure_rescued_after_gvl_running_from_postponed_job(VALUE self_instance, VALUE exception) { | ```

@y9v

BIG thanks to @y9v for helping me debug this!

Fix ``` ../../../../ext/datadog_profiling_native_extension/collectors_thread_context.c:293:14: error: unused function '_native_on_gvl_waiting' [-Werror,-Wunused-function] 293 | static VALUE _native_on_gvl_waiting(DDTRACE_UNUSED VALUE self, VALUE thread); | ^~~~~~~~~~~~~~~~~~~~~~ ../../../../ext/datadog_profiling_native_extension/collectors_thread_context.c:294:14: error: unused function '_native_gvl_waiting_at_for' [-Werror,-Wunused-function] 294 | static VALUE _native_gvl_waiting_at_for(DDTRACE_UNUSED VALUE self, VALUE thread); | ^~~~~~~~~~~~~~~~~~~~~~~~~~ ../../../../ext/datadog_profiling_native_extension/collectors_thread_context.c:295:14: error: unused function '_native_on_gvl_running' [-Werror,-Wunused-function] 295 | static VALUE _native_on_gvl_running(DDTRACE_UNUSED VALUE self, VALUE thread); | ^~~~~~~~~~~~~~~~~~~~~~ ../../../../ext/datadog_profiling_native_extension/collectors_thread_context.c:296:14: error: unused function '_native_sample_after_gvl_running' [-Werror,-Wunused-function] 296 | static VALUE _native_sample_after_gvl_running(DDTRACE_UNUSED VALUE self, VALUE collector_instance, VALUE thread, VALUE allow_exception); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../../../ext/datadog_profiling_native_extension/collectors_thread_context.c:297:14: error: unused function '_native_apply_delta_to_cpu_time_at_previous_sample_ns' [-Werror,-Wunused-function] 297 | static VALUE _native_apply_delta_to_cpu_time_at_previous_sample_ns(DDTRACE_UNUSED VALUE self, VALUE collector_instance, VALUE thread, VALUE delta_ns); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ```

This isn't working because of the wrong DYLIB issue + I suspect the rpath on the receiver is not correct, so for now we're disabling it and maybe we'll timebox a fix later.

ivoanjo · 2026-03-04T10:12:25Z

So right now the macOS tests are passing by using the hacks in extconf.rb but we might wait until the actual upstream libdatadog fix lands in DataDog/libdatadog#1646 so we can remove them.

…ibility Fix compatibility issues with MacOS

ivoanjo · 2026-03-13T10:32:26Z

Update: Libdatadog v29 is now on rubygems.org; once master gets moved to it I expect we'll finally be ready to land this PR

**What does this PR do?** This PR bumps the libdatadog dependency from version 28.0.2.1.0 to 29.0.0.1.0. This new version brings: * macOS build fixes needed to unblock #5351 * libdatadog support for the [OTel process context](open-telemetry/opentelemetry-specification#4719) (I plan to submit a PR with some integration for testing this separately -- commit is already waiting) **Motivation:** Adopt latest libdatadog. **Change log entry** Yes. Upgrade libdatadog dependency to version 29.0.0 **Additional Notes:** N/A **How to test the change?** Green CI is good, as usual.

ivoanjo · 2026-03-16T15:40:04Z

We're finally good to go! macOS testing all green!

ivoanjo · 2026-03-16T16:36:51Z

I'm going to go ahead and merge this one: this PR hasn't substantially changed from when it was initially reviewed, the only difference was merging in master which brought in libdatadog 29 which fixed all the issues we saw before.

This was changed in #5351 which was concurrently worked on with this PR.

ivoanjo requested review from a team as code owners February 12, 2026 12:51

ivoanjo requested review from vpellan and removed request for a team February 12, 2026 12:51

marcotc approved these changes Feb 13, 2026

View reviewed changes

vpellan approved these changes Feb 16, 2026

View reviewed changes

gyuheon0h force-pushed the ivoanjo/libdatadog-26-ruby branch 2 times, most recently from 29113a3 to 30c93b0 Compare February 24, 2026 12:39

gyuheon0h force-pushed the ivoanjo/libdatadog-26-ruby branch 4 times, most recently from 913e09d to 9330399 Compare March 2, 2026 18:47

Base automatically changed from ivoanjo/libdatadog-26-ruby to master March 2, 2026 21:34

ivoanjo added 8 commits March 3, 2026 10:08

Simplify docs now that there are libdatadog macOS builds

7386a56

Extract helper for libdatadog support to core helpers

bc5002d

Use of libdatadog will make this helper more and more common so let's put it in a central location.

Update profiling helper to also reuse libdatadog helper

18d3341

Avoid duplicating helper

9fd368c

I chose to keep the `skip_if_libdatadog_not_supported` as I like seeing the skip on the rspec output but this is very debatable so if you're unconvinced I don't mind too much going the other way.

Simplify skip helper usage

7bb53c0

Run libdatadog specs on macOS in CI

f97f382

Update LibdatadogExtconfHelpers spec

659d701

Now that we do have separate libdatadog testing, it's time to decouple this from profiling!

Fix leftover references to profiling in spec

897c78c

ivoanjo force-pushed the ivoanjo/prof-13732-improve-macos-testing branch from 3a574e5 to 897c78c Compare March 3, 2026 10:09

ivoanjo added 2 commits March 3, 2026 10:29

Directly set spec:main filter from CORE_WITH_LIBDATADOG_API

0f07741

This avoids the previous approach of manually keeping them in sync (and we were missing the new `libdatadog_extconf_helpers_spec.rb`). I've also removed any duplicates.

Use C11 on macOS when building extension

ff40ec5

I think it's safe to assume folks on macOS on arm64 are not running 10 year old compilers.

ivoanjo marked this pull request as draft March 3, 2026 14:35

ivoanjo added 4 commits March 3, 2026 16:10

Skip process discovery on macOS, it's not supported

8b72d3e

Fix crashtracking spec for macOS

47770f4

WIP workaround broken dylib path in libdatadog build

0474675

BIG thanks to @y9v for helping me debug this!

ivoanjo force-pushed the ivoanjo/prof-13732-improve-macos-testing branch from 2830ed2 to 0474675 Compare March 3, 2026 16:18

github-actions Bot added the core Involves Datadog core libraries label Mar 3, 2026

ivoanjo added 4 commits March 3, 2026 16:24

Disable crashtracking on macOS

825f4b4

This isn't working because of the wrong DYLIB issue + I suspect the rpath on the receiver is not correct, so for now we're disabling it and maybe we'll timebox a fix later.

Don't run the other crashtracking tests on macOS

0673fcb

Minor: Fixes to make profiler test suite pass on macOS CI

52b903a

gyuheon0h and others added 3 commits March 5, 2026 07:17

Fix compatibility issues with MacOS

e27ec8b

Update gated test for clarity on skip MacOS

48597c6

Merge pull request #5419 from DataDog/gyuheon0h/libdatadog-mac-compat…

50a55bf

…ibility Fix compatibility issues with MacOS

ivoanjo mentioned this pull request Mar 16, 2026

[PROF-13732] Upgrade libdatadog dependency to version 29.0.0 #5461

Merged

ivoanjo added 3 commits March 16, 2026 14:45

Merge branch 'master' into ivoanjo/prof-13732-improve-macos-testing

e18c674

Workaround Fiddle.free(42) not seeming to always cause crash on macOS

b4ae7eb

Avoid trying to crash Ruby on macOS with fiddle

9baf8ac

ivoanjo marked this pull request as ready for review March 16, 2026 15:40

ivoanjo merged commit 0c664fd into master Mar 16, 2026
631 checks passed

ivoanjo deleted the ivoanjo/prof-13732-improve-macos-testing branch March 16, 2026 16:36

github-actions Bot added this to the 2.30.0 milestone Mar 16, 2026

ivoanjo added a commit that referenced this pull request Mar 16, 2026

Fix usage of removed LibdatadogHelpers.supported?

c1dd3c7

This was changed in #5351 which was concurrently worked on with this PR.

Strech mentioned this pull request Mar 19, 2026

Bump to version 2.30.0 #5474

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PROF-13732] Enable testing libdatadog-based features on macOS #5351

[PROF-13732] Enable testing libdatadog-based features on macOS #5351
ivoanjo merged 30 commits intomasterfrom
ivoanjo/prof-13732-improve-macos-testing

ivoanjo commented Feb 12, 2026

Uh oh!

pr-commenter Bot commented Feb 12, 2026 •

edited

Loading

Explanation

More details about the CI and significant changes

Uh oh!

ivoanjo commented Feb 12, 2026

Uh oh!

marcotc left a comment

Uh oh!

ivoanjo commented Mar 3, 2026

Uh oh!

ivoanjo commented Mar 3, 2026

Uh oh!

ivoanjo commented Mar 4, 2026

Uh oh!

ivoanjo commented Mar 13, 2026

Uh oh!

ivoanjo commented Mar 16, 2026

Uh oh!

ivoanjo commented Mar 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

ivoanjo commented Feb 12, 2026

Uh oh!

pr-commenter Bot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Uh oh!

ivoanjo commented Feb 12, 2026

Uh oh!

marcotc left a comment

Choose a reason for hiding this comment

Uh oh!

ivoanjo commented Mar 3, 2026

Uh oh!

ivoanjo commented Mar 3, 2026

Uh oh!

ivoanjo commented Mar 4, 2026

Uh oh!

ivoanjo commented Mar 13, 2026

Uh oh!

ivoanjo commented Mar 16, 2026

Uh oh!

ivoanjo commented Mar 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pr-commenter Bot commented Feb 12, 2026 •

edited

Loading