Add code_mode output helpers for text and images#14244
Conversation
cconger
left a comment
cconger
left a comment
There was a problem hiding this comment.
nonblocking questions inline.
| if (typeof serialized === 'string') { | ||
| return serialized; | ||
| } | ||
| } catch {} |
There was a problem hiding this comment.
Do we have a mechanism by which we plan on surfacing errors. I feel like showing the model that there was a serialization error would be helpful here.
I see we have the typeerror below for image strings.
There was a problem hiding this comment.
Yeah, good idea we should just let this bubble up.
| function createCodeModeModule(context) { | ||
| const outputText = (value) => { | ||
| const item = { | ||
| type: 'input_text', |
There was a problem hiding this comment.
why are these keyed "input_*"
There was a problem hiding this comment.
https://developers.openai.com/api/reference/resources/responses/
| serde_json::json!({ | ||
| "type": "custom_tool_call_output", | ||
| "call_id": "call-1", | ||
| "output": [ |
There was a problem hiding this comment.
Is it important for output to be so structured here instead of freeform?
I feel like we might also want to provide some kind of typedArray uint8 to allow byte passing in and out instead of requiring data:base64 encoding if we want to allow the model to output image data as "I want to see this" payload.
There was a problem hiding this comment.
Is it important for output to be so structured here instead of freeform?
That's the content_item format from Responses API. That's how it distinguishes images that the model wants to see from raw text.
2 participants
Summary
output_text/output_imageand ensureadd_contentstays compatible@openai/code_modemodule that appends content items and validates inputsTesting