Skip to content

ci(openai, langchain, llmobs): fix broken tests#5507

Merged
watson merged 3 commits intomasterfrom
sabrenner/fix-llmobs-embeddings-tests
Apr 1, 2025
Merged

ci(openai, langchain, llmobs): fix broken tests#5507
watson merged 3 commits intomasterfrom
sabrenner/fix-llmobs-embeddings-tests

Conversation

@sabrenner
Copy link
Copy Markdown
Collaborator

What does this PR do?

A new release of openai broke our tests that either test openai embeddings or use the library (in langchain's case). This is because they use base64 as a default encoding. This does not change our tracing functionality at all, but because we stub out responses, we need to make sure they're compatible with the decoding format in the openai library.

Motivation

Fix CI

Plugin Checklist

Additional Notes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 31, 2025

Overall package size

Self size: 9.13 MB
Deduped: 101.66 MB
No deduping: 102.18 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.5.0 | 29.83 MB | 29.83 MB | | @datadog/native-appsec | 8.5.1 | 19.26 MB | 19.27 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.6.0 | 9.79 MB | 10.16 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.4.0 | 2.77 MB | 5.42 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.13.1 | 117.64 kB | 839.26 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | dc-polyfill | 0.1.6 | 24.56 kB | 24.56 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 31, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.20%. Comparing base (b984e63) to head (43c3272).
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5507      +/-   ##
==========================================
- Coverage   79.24%   79.20%   -0.05%     
==========================================
  Files         512      513       +1     
  Lines       23152    23160       +8     
==========================================
- Hits        18347    18344       -3     
- Misses       4805     4816      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-datadog-prod-us1
Copy link
Copy Markdown

datadog-datadog-prod-us1 Bot commented Mar 31, 2025

Datadog Report

Branch report: sabrenner/fix-llmobs-embeddings-tests
Commit report: d87e1be
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 920 Passed, 0 Skipped, 12m 2.24s Total Time

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Mar 31, 2025

Benchmarks

Benchmark execution time: 2025-03-31 21:28:43

Comparing candidate commit 43c3272 in PR branch sabrenner/fix-llmobs-embeddings-tests with baseline commit b984e63 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 946 metrics, 17 unstable metrics.

@sabrenner sabrenner marked this pull request as ready for review April 1, 2025 00:15
@sabrenner sabrenner requested a review from a team as a code owner April 1, 2025 00:15
@watson watson merged commit 1bd4073 into master Apr 1, 2025
422 of 428 checks passed
@watson watson deleted the sabrenner/fix-llmobs-embeddings-tests branch April 1, 2025 09:14
wconti27 pushed a commit that referenced this pull request Apr 8, 2025
* fix openai tests

* skip langchain tests
@wconti27 wconti27 mentioned this pull request Apr 8, 2025
wconti27 pushed a commit that referenced this pull request Apr 9, 2025
* fix openai tests

* skip langchain tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants