Skip to content

Conversation

@siladu
Copy link
Contributor

@siladu siladu commented Nov 12, 2025

Replaces tuweni impl, removing the megamorphic calls.
engine_getBlobsV2 uses this since returned blob data is bottlenecked on this conversion.
Other getPayload calls will be updated to use it in a followup PR.

1 blob = 128 KB = 131072 bytes + (1 * 128 proofs * 48 bytes) = 137,216 bytes
9 blobs = 9 * 131072 = 1,179,648 bytes + (9 * 128 proofs * 48 bytes) = 1,234,944 bytes (~1.2 MB)
21 blobs = 21 * 131072 = 2,752,512 bytes + (21 * 128 proofs * 48 bytes) = 2,881,536 bytes (~2.7 MB)
72 blobs = 72 * 131072 = 9,437,184 bytes + (72 * 128 proofs * 48 bytes) = 9,879,552 bytes (~9.4 MB)
128 blobs = 128 * 131072 = 16,777,216 bytes + (128 * 128 proofs * 48 bytes) = 17,563,648 bytes (~16.7 MB)
Benchmark                        (caseName)  Mode  Cnt          Score        Error  Units
# before
ToFastHexBenchmark.tuweni       SIZE_1_BLOB  avgt   15     796406.136 ±   4813.701  ns/op
ToFastHexBenchmark.tuweni      SIZE_9_BLOBS  avgt   15    7944644.764 ±  62536.148  ns/op
ToFastHexBenchmark.tuweni     SIZE_21_BLOBS  avgt   15   18469751.559 ± 130136.114  ns/op
ToFastHexBenchmark.tuweni     SIZE_72_BLOBS  avgt   15   63279210.037 ± 279280.513  ns/op
ToFastHexBenchmark.tuweni    SIZE_128_BLOBS  avgt   15  112467237.963 ± 311872.696  ns/op
# this PR
ToFastHexBenchmark.hexUtils     SIZE_1_BLOB  avgt   15     340773.984 ±   9617.594  ns/op
ToFastHexBenchmark.hexUtils    SIZE_9_BLOBS  avgt   15    3040690.635 ±  15067.695  ns/op
ToFastHexBenchmark.hexUtils   SIZE_21_BLOBS  avgt   15    6998597.390 ±  20288.077  ns/op
ToFastHexBenchmark.hexUtils   SIZE_72_BLOBS  avgt   15   24413063.387 ± 787985.528  ns/op
ToFastHexBenchmark.hexUtils  SIZE_128_BLOBS  avgt   15   43078088.889 ± 650158.811  ns/op

5-6x faster throughput

Before
Screenshot 2025-11-17 at 9 25 00 am

This PR
Screenshot 2025-11-17 at 9 25 19 am


Hoodi performance showing control, initial StringBuilder optimisation, final char array impl

Screenshot 2025-11-17 at 5 25 21 pm

Replaces tuweni impl, adapted from StructLog.toCompactHex but retains leading zeros

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Copy link
Contributor

@ahamlat ahamlat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you share JMH benchmarks to check before and after to see if the new implementation is better in terms of performance ?

Undo parallel streams

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
@siladu
Copy link
Contributor Author

siladu commented Nov 16, 2025

Input size is byte length representing blob (+proof) sizes used in responses for engine_getBlobsV2

1 blob = 128 KB = 131072 bytes + (1 * 128 proofs * 48 bytes) = 137,216 bytes
9 blobs = 9 * 131072 = 1,179,648 bytes + (9 * 128 proofs * 48 bytes) = 1,234,944 bytes (~1.2 MB)
21 blobs = 21 * 131072 = 2,752,512 bytes + (21 * 128 proofs * 48 bytes) = 2,881,536 bytes (~2.7 MB)
72 blobs = 72 * 131072 = 9,437,184 bytes + (72 * 128 proofs * 48 bytes) = 9,879,552 bytes (~9.4 MB)
128 blobs = 128 * 131072 = 16,777,216 bytes + (128 * 128 proofs * 48 bytes) = 17,563,648 bytes (~16.7 MB)
Benchmark                          (inputSize)  Mode  Cnt          Score         Error  Units
ToFastHexBenchmark.hexUtils             137216  avgt   15     347772.399 ±   11241.508  ns/op
ToFastHexBenchmark.hexUtils            1234944  avgt   15    3102369.431 ±   57631.705  ns/op
ToFastHexBenchmark.hexUtils            2881536  avgt   15    7237844.530 ±   81062.748  ns/op
ToFastHexBenchmark.hexUtils            9879552  avgt   15   24894359.969 ±  906055.271  ns/op
ToFastHexBenchmark.hexUtils           17563648  avgt   15   43469924.536 ±  337368.591  ns/op
ToFastHexBenchmark.tuweni               137216  avgt   15     808014.190 ±    2489.838  ns/op
ToFastHexBenchmark.tuweni              1234944  avgt   15    8084398.389 ±  199896.569  ns/op
ToFastHexBenchmark.tuweni              2881536  avgt   15   18816737.733 ±  402653.828  ns/op
ToFastHexBenchmark.tuweni              9879552  avgt   15   64368279.170 ±  385450.292  ns/op
ToFastHexBenchmark.tuweni             17563648  avgt   15  114858759.548 ± 2828625.738  ns/op

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
@siladu
Copy link
Contributor Author

siladu commented Nov 17, 2025

Using char array instead of StringBuilder

Benchmark                          (inputSize)  Mode  Cnt          Score        Error  Units
ToFastHexBenchmark.hexUtils             137216  avgt   15     128046.093 ±    547.659  ns/op
ToFastHexBenchmark.hexUtils            1234944  avgt   15    1383249.368 ± 366222.851  ns/op
ToFastHexBenchmark.hexUtils            2881536  avgt   15    3871263.657 ±  72802.766  ns/op
ToFastHexBenchmark.hexUtils            9879552  avgt   15   13176864.399 ± 109785.248  ns/op
ToFastHexBenchmark.hexUtils           17563648  avgt   15   23378568.582 ± 160927.048  ns/op
ToFastHexBenchmark.tuweni               137216  avgt   15     823368.888 ±   5551.835  ns/op
ToFastHexBenchmark.tuweni              1234944  avgt   15    8236590.901 ± 204055.673  ns/op
ToFastHexBenchmark.tuweni              2881536  avgt   15   19031861.216 ±  98560.324  ns/op
ToFastHexBenchmark.tuweni              9879552  avgt   15   65268289.048 ± 560765.539  ns/op
ToFastHexBenchmark.tuweni             17563648  avgt   15  115915573.133 ± 617974.569  ns/op

@siladu siladu marked this pull request as ready for review November 17, 2025 09:18
Copilot AI review requested due to automatic review settings November 17, 2025 09:18
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@siladu siladu enabled auto-merge (squash) November 18, 2025 07:04
@siladu siladu merged commit bdcb5b4 into hyperledger:main Nov 18, 2025
46 checks passed
@siladu siladu deleted the engineGetBlobsV2-optimise-toFastHex branch November 18, 2025 07:31
pinges pushed a commit to pinges/besu that referenced this pull request Dec 15, 2025
- Avoids Tuweni's megamorphic get() and size() calls
- Add jmh ToFastHexBenchmark

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants