-
Notifications
You must be signed in to change notification settings - Fork 4.1k
ImageParts sent via experimental_telemetry are bloated #6210
Description
Description
For doGenerate spans, image data is sent as Uint8Array, on which JSON.stringify acts very weirdly, resulting in a 5-10x bloat on average:
Welcome to Node.js v23.3.0.
Type ".help" for more information.
> JSON.stringify(new Uint8Array([42, 33, 22]))
'{"0":42,"1":33,"2":22}'
// I would expect '[42, 33, 22]' probably
As a telemetry provider (Laminar), we can implement some conversions on our backend, but I believe this is rather a client side bug than a feature. My understanding is that Uint8Arrays are compact (about 3/4 the size of a base64 string) and easy to carry around in JS, that's why we convert image data to that format. But I'd say such stringification goes against the whole purpose of using a compact data structure.
I tried tracing where and why this happens. Roughly:
JSON.stringify(promptMessages) in core/generate-text/generate-text.ts
->
convertToLanguageModelPrompt in core/prompt/convert-to-language-model-prompt.ts
-> (via several calls in the same file)
convertDataContentToUint8Array in core/prompt/convert-to-language-model-prompt.ts
->
convertBase64ToUint8Array in core/prompt/data-content.ts
See also MDN docs examples of JSON.stringify