improve text encoder encode performance

[anonrig]

Improves Next.js benchmarks by 4-7% and WebAssembly applications by 5%

Important disclaimer: Codspeed will not show any improvement (although you can look into previous edits of the codspeed post) because this change is behind an autogate.

---

Sharing @mhart's benchmark results:

Before:

  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    17.63ms    7.86ms 102.29ms   87.71%
    Req/Sec    57.83      9.24    80.00     75.59%
  Latency Distribution
     50%   14.25ms
     75%   17.30ms
     90%   33.63ms
     99%   41.37ms
  1736 requests in 30.06s, 517.73MB read
Requests/sec:     57.75

After:

  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.96ms    9.48ms 145.37ms   88.17%
    Req/Sec    61.43      9.89    80.00     74.92%
  Latency Distribution
     50%   13.27ms
     75%   16.94ms
     90%   32.61ms
     99%   41.71ms
  1842 requests in 30.05s, 549.21MB read
Requests/sec:     61.30

This is pretty consistent over multiple runs. Seeing 1.04x - 1.07x

---

Claude is used to write documentation in encoding-test.c++.

[codspeed-hq]

CodSpeed Performance Report

Merging #5448 will not alter performance

_{Comparing yagiz/experiment-value-view (1a2eab7) with main (49feab0)}

Summary

✅ 140 untouched
⏩ 38 skipped¹

Footnotes

1. 38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[jasnell]

Looks like there are still some bugs to work out here with the failing CI... but even then, my preference would be to settle on #5449 first before landing any changes here. Also, since we do utf8 conversions everywhere, not just in TextEncoder::encode, my preference would be to address this more generally. That is, the optimized encoding path -- if we decide it's worthwhile -- should go into jsg::JsString so that it can be used everywhere rather than just in TextEncoder::encode.

[erikcorry]

I don't think we need to speed optimize for the broken UTF-16 case unless and until someone shows it matters. The only reason to space-optimize would be to avoid throwing OOM, so if we can guarantee that doesn't happen I'm OK with some temporary blowup in space too.

[erikcorry]

https://paste.cfdata.org/GKKdyFGFqSks

[anonrig]

https://paste.cfdata.org/GKKdyFGFqSks

This helps a lot. I'll push with your changes. Thanks @erikcorry