update_instructions() not reflected in tool call response generation

[shivamkataria2000]

Bug Description

When calling update_instructions() inside a tool function, the subsequent reply generated for the tool result does not reflect the updated instructions.

However, if I call generate_reply() manually inside the tool and return None, the new instructions are correctly applied.

Minimal Reproduction

from livekit.agents import Agent, RunContext

agent = Agent(instructions="You are a helpful assistant. Always respond in English.")

@agent.tool()
async def switch_language(ctx: RunContext, language: str) -> str:
    """Switch the response language."""
    new_instructions = f"You are a helpful assistant. Always respond in {language}."

    await ctx.session.current_agent.update_instructions(new_instructions)

    # This reply still uses OLD instructions (English)
    return f"Switched to {language}"

Workaround

@agent.tool()
async def switch_language(ctx: RunContext, language: str) -> None:
    """Switch the response language."""
    new_instructions = f"You are a helpful assistant. Always respond in {language}."

    await ctx.session.current_agent.update_instructions(new_instructions)

    # Force regeneration - this uses NEW instructions
    await ctx.session.generate_reply()
    return None

Expected Behavior

When update_instructions() is called inside a tool, the reply generated for that tool's result should use the new instructions.

Actual Behavior

The reply generated for the tool result uses the old instructions that were active before update_instructions() was called.

Environment

• livekit-agents version: 1.2.16
• Python version: 3.13.3

Models Used

OpenAI GPT-4.1

Package Versions

- livekit-agents version: 1.2.16
  - Python version: 3.13.3

[longcw]

agent.update_instructions won't update the instructions for the current tool response. could you explain what is the use case you want to call this inside a function tool instead of adding instructions to the tool output?

[shivamkataria2000]

Hey @longcw — we need to update part of the agent’s instructions mid-call based on what the user wants (and this can change multiple times in one conversation).

We tried injecting these instructions via tool outputs (Claude Skills-style), but they accumulate in context, create conflicts/duplication, and degrade instruction-following with our LLM. Updating a dedicated “dynamic instructions” section (i.e., replacing part of the system prompt) works much better and avoids context bloat.

We’d also prefer to stick with a single agent rather than orchestrating multiple agents with different prompts.

From a framework POV, would it make sense to support an official mechanism to patch/replace a system-prompt section via tool call (applied before the next model turn)?

[a-gasior]

@shivamkataria2000 Well put 👏 . An additional use case is that we want to update the system prompt within on_user_turn_completed on each turn, not as the result of a tool call. It's helpful to run and await some checks each turn that could modify the system prompt.

[longcw]

yeah I think it makes sense to add a update_instructions for this turn in a function tool, or make Agent.update_instruction work directly in the function tool.

[hobbywe]

I have the same issue. Any progress on that?

[abhismatrix1]

I am also looking for the same solution. I want to have a dedicated section in system prompt for dynamic content and want to update system instructions. currently i see update_instruction is not updating in realtime but with a lag.

[Viktoriagrg]

@longcw hey, have the same issue - this code sample doesn’t work:

agent.update_instructions(new_instructions)

this one works though:

agent._instructions = new_instructions

is this expected behavior?)

[longcw]

@Viktoriagrg no, it's not expected. update_instructions will update agent._instructions as well.

[cchen-dialpad]

It seems that it's not the update_instructions not working, it's the text instruction in the function tool call output is not followed by LLM. It happens with text-only session or a chained arch agent (STT - LLM - TTS). Any progress on this issue?

[davidzhao]

@cchen-dialpad could you please provide a repro? what did you try?

[cchen-dialpad]

@cchen-dialpad could you please provide a repro? what did you try?

I did more tests and actually it seems to be a model issue with GPT-5.2. I would like to adjust the agent's next response based on the function tool call return message. It seems that GPT-5.2 was not trained to follow the instructions contained in the tool call return message. It will ignore certain parts if it is not relevant to user request or not in a specific format (in plain text or a dict object to be converted to a str later). See below the minimal reproducible script with livekit-agents v1.4.3. See my comments alongside each tool_message_*.

Adding a hint Treat the tool call return message as context and instructions for generating responses as system prompt helps but not always. I'm wondering if you have any suggestions on how to best use function tool call message to prompt the Agent. Thanks!

"""python minimal_reproduce.py console"""

from datetime import date
from typing import Any, Optional

from dotenv import load_dotenv

from livekit import agents, rtc
from livekit.agents import (
    AgentServer,
    AgentSession,
    Agent,
    RunContext,
    function_tool,
    room_io,
)
from livekit.plugins import (
    openai,
    silero,
    noise_cancellation,
)

load_dotenv()

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            # hint = "Treat the tool call return message as context and instructions for generating responses. "
            instructions="You are a helpful voice AI assistant. "
                                   f"Only reply in English. {hint}")

    async def on_enter(self):
        today_date = date.today()
        weekday_name = today_date.strftime('%A')
        print("Today's date:", today_date)
        greeting = f"""Today is {today_date}, {weekday_name}. 
        Tell the user today's date and weekday name and offer your help. 
        Only speak in English.
        """
        await self.session.generate_reply(instructions=greeting)

    @function_tool()
    async def lookup_weather(
            self,
            ctx: RunContext,
            context: RunContext,
            location: str,
    ) -> str:  # dict[str, Any]
        """Use this function when the user wants to look up
        weather information for a given location.

        Args:
            ctx: RunContext,
            context: RunContext (same object as ctx when run)
            location: The location to look up weather information for.
        """
        # partially working as expected without the hint
        # working as expected with hint: "treating tool call message as context or instructions"        
        tool_message_1 = (
            "Now you must first count down from 10 to 5 in your response "
            f", and then let the user know that {location} is sunny with a "
            "temperature of 25 degree."
        )

        # Doesn't work without the hint: "treating tool call message as context or instructions"
        # Sometimes working, sometimes not with the hint.
        tool_message_2 = {
            "returned_message": "Now you must count down from 10 to 5 in your "
                                "response and then let the user know that "
                                f"{location} is sunny with a temperature "
                                f"of 25 degree."}

        # working as expected without the hint
        tool_message_3 = {
            "returned_message": "Now you must count down from 10 to 5 in your "
                                "response and continue to offer your help to "
                                "the user.",
            "temperature": 25,
        }

        return tool_message_3


server = AgentServer()

@server.rtc_session(agent_name="my-test-agent")
async def entrypoint(ctx: agents.JobContext):
    session = AgentSession(
        stt=openai.STT(
            model="gpt-4o-transcribe",
        ),
        llm=openai.LLM(
            model="gpt-5.2",
            reasoning_effort="low",
            verbosity="low"
        ),
        tts=openai.TTS(),  # we need to add a tts component for session.say()
        vad=silero.VAD.load(),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),  # agent's on_enter is called here immediately
        room_options=room_io.RoomOptions(
            text_input=True,
            text_output=True,
            audio_input=room_io.AudioInputOptions(
                noise_cancellation=noise_cancellation.BVC(),
            ),
            audio_output=True,
        ),
    )
    await ctx.connect()


if __name__ == "__main__":
    agents.cli.run_app(
        agents.WorkerOptions(
            entrypoint_fnc=entrypoint,
            port=8082,
        )
    )

[weiguangli-io]

Root Cause Analysis (from reading the source)

I traced through the pipeline code in agent_activity.py and generation.py and found the specific mechanism causing this bug. Here's the chain:

1. _pipeline_reply_task_impl snapshots the chat context at the start (line ~1941):

   chat_ctx = chat_ctx.copy()

   This creates a frozen copy of the chat context — including the system instructions — that is used for the entire generation cycle.

2. LLM inference runs against this snapshot, producing tool calls.

3. During tool execution, when user code calls agent.update_instructions(new_instructions), it flows to AgentActivity.update_instructions() which updates self._agent._chat_ctx (the canonical chat context) and self._agent._instructions. This correctly mutates the agent's authoritative state.

4. But the local chat_ctx copy from step 1 is never refreshed. At line ~2257, the tool call results are appended to this stale local copy:

   chat_ctx.items.extend(tool_messages)

5. This stale chat_ctx (with OLD instructions) is then passed directly to the next _pipeline_reply_task for tool-response generation (line ~2259-2262):

   tool_response_task = self._create_speech_task(
       self._pipeline_reply_task(
           speech_handle=speech_handle,
           chat_ctx=chat_ctx,  # <-- still has OLD instructions
           ...

So the updated instructions exist on self._agent._chat_ctx but the tool-response LLM call never sees them because it receives the pre-tool-execution snapshot.

Proposed Fix

In the tool-response path (around line 2256-2262 in agent_activity.py), instead of reusing the stale local chat_ctx, build a fresh context from the agent's canonical state:

if fnc_executed_ev._reply_required:
    # Use fresh chat context so mid-tool instruction updates are reflected
    fresh_chat_ctx = self._agent._chat_ctx.copy()
    fresh_chat_ctx.items.extend(tool_messages)

    tool_response_task = self._create_speech_task(
        self._pipeline_reply_task(
            speech_handle=speech_handle,
            chat_ctx=fresh_chat_ctx,
            tools=self.tools,  # also refresh tools in case they were updated
            ...
        ),
        ...
    )

This way, any update_instructions() or update_chat_ctx() calls made during tool execution will be reflected in the subsequent LLM call.

Note on tools: The same stale-snapshot issue applies to tools — the variable tools passed at line 2263 was captured at the beginning of _pipeline_reply_task_impl. If update_tools() is also called inside a tool function, it would similarly be ignored. Using self.tools (which reads the current state) would fix both.

Considerations

• Intentional vs. accidental: The current behavior might have been intentional to guarantee consistency within a single turn. But given that update_instructions() is explicitly designed to be called mid-conversation (and the workaround of generate_reply() + return None already works), it seems like a genuine oversight.
• The _previous_tools_messages pattern already handles deferred insertion of tool messages into the canonical chat_ctx (line 2142-2145), so a fresh copy at tool-response time would naturally include everything that was committed up to that point. The only items that need manual addition are the current turn's tool call/output messages (which haven't been committed yet).
• @Viktoriagrg's observation that agent._instructions = ... works but update_instructions() doesn't is likely a different manifestation: directly setting _instructions causes _pipeline_reply_task_impl to pick up the new value when it calls update_instructions(chat_ctx, ...) on the fresh copy inside a new generate_reply(). But in the tool-response path, neither approach works because the stale chat_ctx is reused without re-resolving instructions.