Skip to content

Add optional model parameter to runSubagent tool#298161

Closed
digitarald wants to merge 5 commits intomainfrom
digitarald/runsubagent-model-param
Closed

Add optional model parameter to runSubagent tool#298161
digitarald wants to merge 5 commits intomainfrom
digitarald/runsubagent-model-param

Conversation

@digitarald
Copy link
Copy Markdown
Contributor

@digitarald digitarald commented Feb 27, 2026

Adds an optional model parameter to the runSubagent tool, allowing the LLM to specify which language model a subagent should use.

Fixes #298380

Changes

Model parameter

  • Added model?: string to IRunSubagentToolInputParams interface and tool schema
  • Updated description to instruct the LLM to use qualified name format "Model Name (Vendor)"

Dynamic model enum

  • Schema model property includes a dynamic enum of available models filtered by suitableForAgentMode() (requires toolCalling capability and agentMode !== false)
  • Uses Event.any() to merge config changes and onDidChangeLanguageModels events, triggering tool re-registration when models change
  • Enum is conditionally omitted when no models are available (avoids invalid empty enum: [])

Model resolution priority

  • Explicit model (from model param) > Agent configured model > Main model
  • If explicit model is not found, falls through to agent/main model with a warning log
  • Multiplier-based cost guard applies to all sources — prevents subagent from using a more expensive model than the main agent

Tests

  • 23 tests covering: enum population, agent-mode filtering, agentMode: false exclusion, empty-model omission, explicit model resolution, fallback chains, multiplier guard

Copilot AI review requested due to automatic review settings February 27, 2026 04:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for explicitly selecting a language model when invoking the runSubagent built-in tool, enabling callers to pass a qualified model name (e.g. "Model (Vendor)") and have it participate in the existing model-resolution + cost-guard logic.

Changes:

  • Extend IRunSubagentToolInputParams and the tool input schema with optional model?: string.
  • Thread args.model through prepareToolInvocation() and invoke(), and prioritize it in resolveSubagentModel().
  • Add a new test suite covering schema presence and explicit-model resolution/fallback scenarios.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Adds model to schema/params; updates model-resolution to accept explicit qualified model name and apply multiplier fallback.
src/vs/workbench/contrib/chat/test/common/tools/builtinTools/runSubagentTool.test.ts Adds tests for explicit-model parameter behavior and schema presence.
Comments suppressed due to low confidence (1)

src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts:100

  • The model schema description says “If not provided, uses the current model”, but when agentName is provided the tool may instead use the agent’s configured model. Please clarify the description to match actual resolution behavior (explicit model → agent-configured model → current/main model).
				model: {
					type: 'string',
					description: 'Optional qualified model name to use for the subagent, in the format "Model Name (Vendor)". If not provided, uses the current model.'
				}

Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
@digitarald digitarald marked this pull request as ready for review February 27, 2026 04:54
@aeschli aeschli requested a review from roblourens February 27, 2026 08:26
@aeschli aeschli added this to the March 2026 milestone Feb 27, 2026
@aeschli
Copy link
Copy Markdown
Contributor

aeschli commented Feb 27, 2026

@digitarald Can you link to an issue that describes the need for this?
Also, this is for the next milestone, right?

roblourens
roblourens previously approved these changes Feb 27, 2026
@digitarald digitarald requested a review from Copilot February 27, 2026 23:33
@digitarald
Copy link
Copy Markdown
Contributor Author

Created #298380 and linked it. Yes, this is for the March 2026 milestone (already set).

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
@digitarald digitarald force-pushed the digitarald/runsubagent-model-param branch from 5bdda14 to 6d01047 Compare March 1, 2026 21:49
roblourens
roblourens previously approved these changes Mar 1, 2026
@digitarald digitarald force-pushed the digitarald/runsubagent-model-param branch from 6d01047 to 789488f Compare March 2, 2026 05:58
dmitrivMS
dmitrivMS previously approved these changes Mar 2, 2026
@digitarald digitarald modified the milestones: 1.111.0, 1.112.0 Mar 5, 2026
@digitarald digitarald marked this pull request as draft March 5, 2026 23:58
…che eviction

- Remove duplicate @ILanguageModelToolsService injection (toolsService), use
  languageModelToolsService consistently
- Improve model schema description with concrete examples for fast vs reasoning
  models
- Add _MaxCachedModels cap (100) with FIFO eviction in prepareToolInvocation
  to prevent unbounded _resolvedModels growth on cancellation
- Remove unnecessary 'outer:' label on single for-loop break
- Remove duplicate mockToolsService args in tests
@digitarald digitarald marked this pull request as ready for review March 11, 2026 23:06
@digitarald digitarald requested a review from aeschli March 11, 2026 23:06
) {
super();
this.onDidUpdateToolData = Event.filter(this.configurationService.onDidChangeConfiguration, e => e.affectsConfiguration(ChatConfiguration.SubagentToolCustomAgents));
this.onDidUpdateToolData = Event.debounce(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the rational for the delay?

@digitarald
Copy link
Copy Markdown
Contributor Author

@aeschli onDidChangeLanguageModels fires per vendor/model during startup, so without debouncing we'd churn through repeated getToolData() → re-registration → flushToolUpdates() cycles. The 100ms window coalesces those bursts into a single update. Will add a comment in code.

@digitarald digitarald modified the milestones: 1.112.0, 1.113.0 Mar 17, 2026
Copy link
Copy Markdown
Contributor

@aeschli aeschli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tool description now lists all model names. That's for me around 30 models and I could have many more with BYOK. This increasing the context size and COGS but I'm not sure I'll profit much from that model selection. Again, the question is still, how would the LM chose a model over the other. The model names are not descriptive
I suggest the main agent should explicitly name the models
Run the search subagent with either the 'Opus 4.6 (copilot)' model when extra reasoning is required.
Given this is an experimental feature this is the a cost efficient way.
That way you can also remove the onDidChangeLanguageModels event

@IProductService private readonly productService: IProductService,
) {
super();
// Debounce because onDidChangeLanguageModels fires per vendor/model during
Copy link
Copy Markdown
Contributor

@aeschli aeschli Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be improved in the onDidChangeLanguageModels then. Please file a issue against onDidChangeLanguageModels to collapse the events on start up and add the issue number in the comment.

@aeschli
Copy link
Copy Markdown
Contributor

aeschli commented Mar 18, 2026

@digitarald Happy to make the change, let me know if its ok for you:

  • remove the list of all modes from the tool description to save tokens
  • remove the listener as there's no longer a dependency of the tool data to the models
  • rely on the parent agent to provide a good list names of the models

@digitarald
Copy link
Copy Markdown
Contributor Author

@aeschli sorry for the delay, maybe we should do this after the #306871 .

The problem with just providing the model names is lacking context for when the agents should use each. For example Copilot CLI adds another label to clarify that:

List: claude-sonnet-4.6, claude-sonnet-4.5, claude-haiku-4.5, claude-opus-4.6, claude-opus-4.5, claude-sonnet-4, gpt-5.4, gpt-5.3-codex, gpt-5.2-codex, gpt-5.2, gpt-5.1-codex-max, gpt-5.1-codex, gpt-5.1, gpt-5.4-mini, gpt-5.1-codex-mini, gpt-5-mini, gpt-4.1.

  • Each model is tagged standard, fast/cheap, or premium.
  • Built-in agent defaults vary: explore/task use Haiku-class defaults; general-purpose uses
    Sonnet-class defaults.
  • Custom agents also support model override unless constrained by their implementation.

Thoughts on bringing in the labels? Not sure if we have them or could apply but we could also apply them on multiplier.

@RockyWearsAHat
Copy link
Copy Markdown

+1 didn't realize my PR was a duplicate of this, I second this behavior I like it and think it should be inbuilt. Agents shouldn't by default perform like this but having ability to tune agents for tasks in requests like already happening iun explore or agent instructions files is great and should be a feature, if I want Haiku to write my PR I shouldn't have to create a whole new agent (though in this example it probably would be good practice, it's just an example) I should just be able to say "after you've completed your changes run a haiku agent to upload to github".

+1000

@hediet
Copy link
Copy Markdown
Member

hediet commented Apr 9, 2026

Implemented in #308784

@hediet hediet closed this Apr 9, 2026
@RockyWearsAHat
Copy link
Copy Markdown

RockyWearsAHat commented Apr 9, 2026

@aeschli sorry for the delay, maybe we should do this after the #306871 .

The problem with just providing the model names is lacking context for when the agents should use each. For example Copilot CLI adds another label to clarify that:

List: claude-sonnet-4.6, claude-sonnet-4.5, claude-haiku-4.5, claude-opus-4.6, claude-opus-4.5, claude-sonnet-4, gpt-5.4, gpt-5.3-codex, gpt-5.2-codex, gpt-5.2, gpt-5.1-codex-max, gpt-5.1-codex, gpt-5.1, gpt-5.4-mini, gpt-5.1-codex-mini, gpt-5-mini, gpt-4.1.

  • Each model is tagged standard, fast/cheap, or premium.
  • Built-in agent defaults vary: explore/task use Haiku-class defaults; general-purpose uses
    Sonnet-class defaults.
  • Custom agents also support model override unless constrained by their implementation.

Thoughts on bringing in the labels? Not sure if we have them or could apply but we could also apply them on multiplier.

Possibly could be worth listing free models as a tier if available (especially for copilot pro+ users), would make users much happier if their requests weren't eaten by saying "use the cheapest/free subagents" and then it calls 35 instances of Opus 4.6 [happened to me, many times, fun times].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow runSubagent tool to specify a language model

8 participants