Skip to content

ImageParts sent via experimental_telemetry are bloated #6210

@dinmukhamedm

Description

@dinmukhamedm

Description

For doGenerate spans, image data is sent as Uint8Array, on which JSON.stringify acts very weirdly, resulting in a 5-10x bloat on average:

Welcome to Node.js v23.3.0.
Type ".help" for more information.
> JSON.stringify(new Uint8Array([42, 33, 22]))
'{"0":42,"1":33,"2":22}'
// I would expect '[42, 33, 22]' probably

As a telemetry provider (Laminar), we can implement some conversions on our backend, but I believe this is rather a client side bug than a feature. My understanding is that Uint8Arrays are compact (about 3/4 the size of a base64 string) and easy to carry around in JS, that's why we convert image data to that format. But I'd say such stringification goes against the whole purpose of using a compact data structure.

I tried tracing where and why this happens. Roughly:

JSON.stringify(promptMessages) in core/generate-text/generate-text.ts
->
convertToLanguageModelPrompt in core/prompt/convert-to-language-model-prompt.ts
-> (via several calls in the same file)
convertDataContentToUint8Array in core/prompt/convert-to-language-model-prompt.ts
->
convertBase64ToUint8Array in core/prompt/data-content.ts

See also MDN docs examples of JSON.stringify

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai/corecore functions like generateText, streamText, etc. Provider utils, and provider spec.support

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions