FEAT normalize messages before sending#1613
FEAT normalize messages before sending#1613hannahwestra25 wants to merge 8 commits intomicrosoft:mainfrom
Conversation
…ra/normalize_send_prompt
|
|
||
| @limit_requests_per_minute | ||
| async def send_prompt_async(self, *, message: Message) -> list[Message]: | ||
| async def _send_prompt_target_async(self, *, normalized_conversation: list[Message]) -> list[Message]: |
There was a problem hiding this comment.
What do you think about making normalized_conversation a data object (maybe in message.py) in order to call clearly-named properties like normalized_conversation.current_message and normalized_conversation.current_request instead of normalized_conversation[-1] and normalized_conversation[-1].message_pieces[0]? I'm thinking it might not be super clear what those indices are referring to when creating/editing target subclasses.
There was a problem hiding this comment.
this should probably use a normalized_conversation param instead of conversation_id to ensure consistency
There was a problem hiding this comment.
/ anywhere we use conversation_id to grab conversation history, we should probably use the normalized_conversation list instead since it includes history now.
| message = normalized_conversation[-1] | ||
| message_piece: MessagePiece = message.message_pieces[0] | ||
| json_config = _JsonResponseConfig(enabled=False) | ||
| if message.message_pieces: |
There was a problem hiding this comment.
NIT: we can get rid of this check?
3 participants
Description
Utilize the Normalization Pipeline in the Target Send Path
PR 4 of the TargetConfiguration roadmap
Problem
The
TargetConfiguration.normalize_asyncpipeline (system-squash, history-squash, etc.) was fully built in this PR but never called. Every target independently fetched conversation history, appended the current message, and sent it to the API — some with ad-hoc normalization (AzureMLChatTarget), most with none at all. This meant the centralized normalization pipeline was dead code, and normalization behavior was inconsistent across targets.Solution
Wire the normalization pipeline into the send path so that every prompt passes through
configuration.normalize_async()before reaching the target's API call. This is done by makingsend_prompt_asynca concrete template method onPromptTargetthat validates, fetches conversation from memory, runs the normalization pipeline, and delegates to a new_send_prompt_target_asyncabstract method for wire-format-specific logic.Changes
PromptTarget.send_prompt_async: Now a concrete method that callsself.configuration.normalize_async(messages=...)and passes the result to_send_prompt_target_asyncsend_prompt_async→_send_prompt_target_async, removed duplicated validation/memory-fetch boilerplate, now receive the pre-normalized conversation directlyAzureMLChatTarget:message_normalizerparameter deprecated with auto-translation toTargetConfiguration(policy={SYSTEM_PROMPT: ADAPT}); will be removed in v0.14.0Breaking Changes
_send_prompt_target_asyncinstead ofsend_prompt_asyncTests and Documentation
test_normalize_async_integration.py(395 lines) covering normalize-is-called, normalized-conversation-is-used, memory-not-mutated, and legacy deprecation pathswip: running integration tests