Skip to content

core: add msgpack encoder#1491

Merged
Kyle-Verhoog merged 27 commits intomasterfrom
kylev/custom-msgpack
Jul 28, 2020
Merged

core: add msgpack encoder#1491
Kyle-Verhoog merged 27 commits intomasterfrom
kylev/custom-msgpack

Conversation

@Kyle-Verhoog
Copy link
Copy Markdown
Member

@Kyle-Verhoog Kyle-Verhoog commented Jun 8, 2020 •
edited
Loading

What?

This patch introduces the following:

Packer

A Cython msgpack encoder mostly derived from the official msgpack library's 0.6.2 implementation. The changes made were:

  • Removed functionality we don't require
  • Small optimizations since we have knowledge of what a trace and span roughly look like
    • Add custom span encoding which will intern the common strings (service, name, resource), etc
    • Process spans objects directly in Cython rather than serializing to a dict and then encoding

TraceMsgPackEncoder

A trace encoder using the above Packer to encode traces.

Why?

We support Python 2 but the msgpack package has dropped support for the C-extension in Python 2. Instead a pure-python fallback is used which is over an order of magnitude slower. This results in an unacceptable performance overhead to Python 2 projects using the tracer.

This also opens the door to further optimizations we can make to encoding.

Benchmarks

Python 2

We see a 23x improvement (244.49ms -> 10.35ms).

Note here that custom is the implementation added, fallback is the default that a user would have if they installed the tracer today without specifying their own version of msgpack.

-------------------------------------------------------------------------------------- benchmark 'encoding': 3 tests ---------------------------------------------------------------------------------------
Name (time in ms)                             Min                 Max                Mean            StdDev              Median               IQR            Outliers      OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_1000_span_trace_custom         9.8290 (1.0)       12.4619 (1.0)       10.4968 (1.0)      0.4931 (1.0)       10.3540 (1.0)      0.5691 (1.18)         19;4  95.2674 (1.0)          89           1
test_encode_1000_span_trace               12.2190 (1.24)      22.5139 (1.81)      12.9208 (1.23)     1.2610 (2.56)      12.6935 (1.23)     0.4821 (1.0)           2;5  77.3948 (0.81)         70           1
test_encode_1000_span_trace_fallback     239.6259 (24.38)    257.7190 (20.68)    244.4946 (23.29)    7.5664 (15.34)    241.5009 (23.32)    7.3441 (15.23)         1;0   4.0901 (0.04)          5           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------ benchmark 'encoding.join_encoded': 2 tests -----------------------------------------------------------------------------
Name (time in us)                 Min                 Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_join_encoded_custom     175.7908 (1.0)      212.5597 (1.0)      184.4673 (1.0)       7.9693 (1.0)      181.3912 (1.0)       9.2912 (1.0)           8;2        5.4210 (1.0)          47         100
test_join_encoded            179.4696 (1.02)     228.1904 (1.07)     193.5662 (1.05)     10.2773 (1.29)     191.0090 (1.05)     11.3487 (1.22)         12;3        5.1662 (0.95)         55         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'encoding.small': 2 tests ------------------------------------------------------------------------------------
Name (time in us)                       Min                 Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_custom      83.5681 (1.0)      150.4898 (1.0)       95.6845 (1.0)      10.5787 (1.0)       93.4911 (1.0)       7.1502 (1.0)          15;8       10.4510 (1.0)         108         100
test_encode_trace_small            131.4497 (1.57)     260.9396 (1.73)     144.0758 (1.51)     16.5305 (1.56)     140.1711 (1.50)     10.0195 (1.40)          3;3        6.9408 (0.66)         69         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'encoding.small.multi': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_multi_custom     4.1935 (1.0)       5.5765 (1.0)      4.5640 (1.0)      0.2989 (1.0)      4.4820 (1.0)      0.3368 (1.0)          27;5  219.1047 (1.0)         111           2
test_encode_trace_small_multi            6.9890 (1.67)     16.7830 (3.01)     7.9098 (1.73)     1.3714 (4.59)     7.6439 (1.71)     0.4773 (1.42)          3;5  126.4259 (0.58)        117           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Python 3

We get about a 1.4-2.0x improvement with the optimizations introduced.

---------------------------------------------------------------------------------------- benchmark 'encoding': 3 tests ----------------------------------------------------------------------------------------
Name (time in ms)                             Min                 Max                Mean             StdDev              Median                IQR            Outliers       OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_1000_span_trace_custom         5.9540 (1.0)       17.5237 (1.0)        6.8113 (1.0)       1.8651 (1.0)        6.2934 (1.0)       0.4221 (1.0)          9;17  146.8155 (1.0)         138           1
test_encode_1000_span_trace                8.6126 (1.45)      25.9230 (1.48)      10.8887 (1.60)      3.7886 (2.03)       9.1269 (1.45)      2.1952 (5.20)          6;6   91.8384 (0.63)         58           1
test_encode_1000_span_trace_fallback     185.2778 (31.12)    228.0931 (13.02)    199.9352 (29.35)    15.5621 (8.34)     196.9757 (31.30)    17.7143 (41.96)         1;0    5.0016 (0.03)          6           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'encoding.join_encoded': 2 tests -------------------------------------------------------------------------------
Name (time in us)                 Min                   Max                Mean              StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_join_encoded            176.3751 (1.0)        291.8260 (1.0)      194.2263 (1.0)       24.1176 (1.0)      186.0701 (1.0)      11.4473 (1.0)           6;7        5.1486 (1.0)          55         100
test_join_encoded_custom     183.8962 (1.04)     1,650.5986 (5.66)     383.0385 (1.97)     264.9502 (10.99)    303.6385 (1.63)     78.7307 (6.88)        20;25        2.6107 (0.51)        191          17
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'encoding.small': 2 tests ------------------------------------------------------------------------------------
Name (time in us)                       Min                 Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_custom      60.7158 (1.0)      102.8957 (1.0)       68.4087 (1.0)       7.0093 (1.0)       66.6154 (1.0)       4.6086 (1.0)         12;10       14.6180 (1.0)         156         100
test_encode_trace_small            121.2871 (2.00)     265.2528 (2.58)     141.9907 (2.08)     26.6266 (3.80)     131.9370 (1.98)     18.8154 (4.08)          8;8        7.0427 (0.48)         65         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'encoding.small.multi': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_multi_custom     3.0415 (1.0)      10.9975 (1.0)      4.5682 (1.0)      1.9561 (1.0)      3.6383 (1.0)      1.3941 (2.16)        24;23  218.9026 (1.0)         151           2
test_encode_trace_small_multi            6.0442 (1.99)     26.2392 (2.39)     7.0495 (1.54)     2.3930 (1.22)     6.6648 (1.83)     0.6442 (1.0)           2;7  141.8542 (0.65)        131           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Further optimizations

Outside the scope of this PR

  • LRU cache common strings
  • Reduce memory allocations with a larger initial buffer
  • Reduce memory allocations to 1 by using just one buffer over the lifespan of the tracer
  • Fix payloads encoding and dropping data
  • inlining more calls
  • join_encoded is actually pretty slow and can be quite easily done manually
    • could be done in-place if we take advantage of the encoder/payload being stateful

TODO

  • copy setup.py config from msgpack
  • further tests
  • license issues?

brettlangdon
brettlangdon reviewed Jun 8, 2020
View reviewed changes
Comment thread ddtrace/internal/_encoding.pyx Outdated
Comment thread ddtrace/internal/_encoding.pyx Outdated
Comment thread ddtrace/internal/_encoding.pyx Outdated
Comment thread ddtrace/internal/_encoding.pyx Outdated
Comment thread ddtrace/internal/_encoding.pyx Outdated
Comment thread ddtrace/internal/_encoding.pyx Outdated
Comment thread ddtrace/internal/_encoding.pyx Outdated
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/custom-msgpack branch from 59f9ff9 to d02932b Compare June 8, 2020 18:55
brettlangdon
brettlangdon reviewed Jun 8, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@brettlangdon brettlangdon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 on most of your responses, good to keep things 100% compatible now, then we can make riskier optimizations later

@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/custom-msgpack branch from d02932b to f91972c Compare June 8, 2020 20:02
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/custom-msgpack branch from f91972c to 43c5875 Compare June 8, 2020 20:43
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/custom-msgpack branch from 5e58b1d to 147bf90 Compare June 9, 2020 04:17
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/custom-msgpack branch from 5e58444 to c5c4658 Compare June 9, 2020 04:51
@brettlangdon brettlangdon mentioned this pull request Jun 30, 2020
Merged
@brettlangdon brettlangdon marked this pull request as ready for review July 1, 2020 18:12
@brettlangdon brettlangdon requested a review from a team as a code owner July 1, 2020 18:12
@Kyle-Verhoog Kyle-Verhoog added this to the 0.41.0 milestone Jul 15, 2020
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/custom-msgpack branch from f5c7c31 to d8f2068 Compare July 24, 2020 23:55
@Kyle-Verhoog
more cleanup
712099b 
- remove float implementation
@Kyle-Verhoog
Copy link
Copy Markdown
Member Author

Kyle-Verhoog commented Jul 28, 2020

Was able to run the tests under Windows:
image

@brettlangdon I think we're good for a final review 😄

jd
jd reviewed Jul 28, 2020
View reviewed changes
Comment thread ddtrace/internal/_encoding.pyx Outdated
@Kyle-Verhoog Kyle-Verhoog merged commit bc4cdcb into master Jul 28, 2020
@Kyle-Verhoog Kyle-Verhoog deleted the kylev/custom-msgpack branch July 28, 2020 21:03
@majorgreys majorgreys mentioned this pull request Oct 27, 2020
Merged
5 tasks
@Yun-Kim Yun-Kim mentioned this pull request Jun 7, 2021
Merged
@Kyle-Verhoog Kyle-Verhoog mentioned this pull request Nov 24, 2021
Merged
4 tasks
mergify Bot pushed a commit that referenced this pull request Nov 24, 2021
@Kyle-Verhoog
Add msgpack Packer based off msgpack-python 0.6.2
86e1f3e 
It's handy to have a general msgpack encoder that can be used for
encoding arbitrary payloads of primitive Python types (see #2915).

The encoder added here is based off the one added in #1491.

It might be useful to use as a fallback for encoding traces as well if
an issue with the custom msgpack encoder is suspected.

The relevant tests from the msgpack-python implementation are included
as well to ensure that the implementation is correct.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brettlangdon brettlangdon brettlangdon approved these changes

+1 more reviewer

@jd jd jd left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

0.41.0

Development

Successfully merging this pull request may close these issues.

3 participants

@Kyle-Verhoog @jd @brettlangdon