feat(chat): add tiers frontmatter and tier parameter to runSubagent tool by RockyWearsAHat · Pull Request #306846 · microsoft/vscode

RockyWearsAHat · 2026-03-31T16:39:44Z

Summary

Adds support for capability tiers in agent definitions (tiers frontmatter) and a tier parameter on the runSubagent tool, enabling semantic model selection at call time.

Resolves #306717

Related: #306836 / PR #306841 (call-time model parameter)

What Changed

Prompt file parser (`promptFileParser.ts`)

New tiers constant in PromptHeaderAttributes
New get tiers() getter on PromptHeader — parses YAML map of { tierName: { model: qualifiedName } }

Agent interface & service (`promptsService.ts`, `promptsServiceImpl.ts`)

Added tiers?: Readonly<Record<string, { readonly model: string }>> to ICustomAgent
Passthrough in service implementation

Prompt validation (`promptValidator.ts`, `promptFileAttributes.ts`)

tiers registered as allowed agent attribute (type: map)

RunSubagent tool (`runSubagentTool.ts`)

Added model?: string and tier?: string to IRunSubagentToolInputParams interface
Both parameters gated behind SubagentToolCustomAgents config flag
resolveSubagentModel() rewritten with 4-level resolution priority:
1. Call-time model parameter (highest)
2. Call-time tier parameter (looks up agent.tiers[tier].model)
3. Agent's top-level model: frontmatter
4. Parent model (default)
New resolveModelHint() helper: tries qualified name → direct ID → undefined
Multiplier-based cost cap moved outside conditional — applies uniformly to all resolution paths

Tests (`runSubagentTool.test.ts`)

createAgent() helper extended to accept tiers parameter
call-time model parameter suite (6 tests): schema, override via qualified name, override via direct ID, multiplier cap, unknown model fallback, precedence over tier
tier resolution suite (7 tests): correct model from tiers map, overrides agent default, multiplier cap, unknown tier fallback, no-agent ignored, no-tiers agent fallback, tier-with-tiers-defined

Example Agent Frontmatter

---
agent: DeepResearch
description: Research agent with tiered capabilities
model: claude-sonnet-4-6
tiers:
  fast:
    model: claude-haiku-4-5
  standard:
    model: claude-sonnet-4-6
  deep:
    model: claude-opus-4-6
---

// Select semantic capability level
runSubagent({ prompt: "quick lookup", tier: "fast", agentName: "DeepResearch" })
// Or explicit model override (takes precedence over tier)
runSubagent({ prompt: "complex analysis", model: "claude-opus-4-6", agentName: "DeepResearch" })

Add support for capability tiers in agent definitions, allowing agents to declare named model tiers (e.g., fast, standard, deep) in their frontmatter. The runSubagent tool gains a `tier` parameter that selects a model from these tiers at call time. Implementation: - Add `tiers` attribute to prompt file parser (YAML map of tier names to `{ model: string }` configurations) - Add `tiers` to ICustomAgent interface and service implementation - Register `tiers` in prompt validator and attribute definitions - Add `tier` parameter to runSubagent tool schema (gated behind SubagentToolCustomAgents config, alongside `model`) - Extend resolveSubagentModel() with 4-level priority: model param > tier param > agent frontmatter > parent model - Add resolveModelHint() helper for model string resolution - Multiplier-based cost cap applies uniformly to all resolution paths Tests: - Schema: model and tier properties present when custom agents enabled - Model param: override via qualified name, direct ID, multiplier cap, unknown model fallback, precedence over tier - Tier resolution: correct model from tiers map, overrides agent default, multiplier cap, unknown tier fallback, no-agent ignored, no-tiers agent fallback Resolves microsoft#306717

The tool's modelDescription now instructs the orchestrating agent to assess subtask complexity before choosing a model, use the cheapest model that can reliably complete each task, and prefer the latest version within each model family. Updated parameter descriptions for `model` and `tier` to include concrete cost-matching guidance instead of just listing examples. This addresses the behavior where agents default to using their own (often expensive) model for all subagent invocations regardless of subtask complexity.

Copilot

Pull request overview

Adds capability-tier model routing for custom agents by introducing tiers in agent frontmatter and supporting call-time model/tier overrides on the runSubagent tool, with updated validation and tests.

Changes:

Parse and surface tiers from agent YAML frontmatter through the prompts service layer.
Extend runSubagent tool input schema + resolution logic to support call-time model and tier selection with multiplier-based cost capping.
Add comprehensive unit tests for call-time model override and tier-based resolution behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts	Adds `model`/`tier` params, implements new model resolution priority + cost cap, and updates tool schema.
src/vs/workbench/contrib/chat/test/common/tools/builtinTools/runSubagentTool.test.ts	Adds new suites covering call-time model override and tier resolution, plus helper updates.
src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsService.ts	Extends `ICustomAgent` with optional `tiers` metadata.
src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts	Plumbs parsed `tiers` from AST header into the returned custom agent objects.
src/vs/workbench/contrib/chat/common/promptSyntax/promptFileParser.ts	Adds `tiers` attribute constant + frontmatter parsing getter on `PromptHeader`.
src/vs/workbench/contrib/chat/common/promptSyntax/languageProviders/promptValidator.ts	Allows `tiers` for agent frontmatter and adds localized description text.
src/vs/workbench/contrib/chat/common/promptSyntax/languageProviders/promptFileAttributes.ts	Registers `tiers` as an allowed custom agent attribute of type `map`.

Copilot · 2026-03-31T16:46:26Z

+			const mainMultiplier = mainModelMetadata?.multiplierNumeric;
+			const resolvedMultiplier = resolvedModelMetadata?.multiplierNumeric;
+			if (mainMultiplier !== undefined && resolvedMultiplier !== undefined && resolvedMultiplier > mainMultiplier) {
+				const source = callTimeModelHint ? 'Call-time' : callTimeTier ? `Tier '${callTimeTier}'` : subagent ? `Subagent '${subagent.name}'` : 'Unknown';
+				this.logService.warn(`[RunSubagentTool] ${source} requested model '${resolvedModelMetadata?.name}' (multiplier: ${resolvedMultiplier}) which exceeds the main agent model '${mainModelMetadata?.name}' (multiplier: ${mainMultiplier}). Falling back to the main agent model.`);


The multiplier-cap warning can misattribute the source of the resolved model. If callTimeModelHint/callTimeTier is provided but fails to resolve (and the agent/default model ends up being the one capped), source will still report "Call-time"/"Tier ..." even though that override was ignored. Track the actual resolution source (e.g., set a variable only when an override successfully changes modeModelId) and use that for the cap warning to avoid misleading logs.

Copilot · 2026-03-31T16:46:27Z

+	 * Maps tier names (e.g., 'fast', 'standard', 'deep') to model qualified names.
+	 * Allows a single agent to serve at different capability/cost levels.
+	 */


The doc comment says tier models are "qualified names", but the rest of this PR (schema examples + resolveModelHint for tiers) accepts either qualified names or direct model IDs. Please update this comment/type docs to reflect the accepted format so agent authors aren’t misled.

Suggested change

* Maps tier names (e.g., 'fast', 'standard', 'deep') to model qualified names.

* Allows a single agent to serve at different capability/cost levels.

*/

* Maps tier names (e.g., 'fast', 'standard', 'deep') to models, which may be

* either qualified model names or direct model IDs. Allows a single agent to

* serve at different capability/cost levels.

Copilot · 2026-03-31T16:46:27Z

+		function createAgent(name: string, modelQualifiedNames?: string[], tiers?: Record<string, { model: string }>): ICustomAgent {
+			return {
+				uri: URI.parse(`file:///test/${name}.md`),
+				name,
+				description: `Agent ${name}`,
+				tools: ['tool1'],
+				model: modelQualifiedNames,
+				tiers,
+				agentInstructions: { content: 'test', toolReferences: [] },
+				source: { storage: PromptsStorage.local },
+				target: Target.Undefined,
+				visibility: { userInvocable: true, agentInvocable: true }
+			};
+		}
+


This test file now has multiple near-identical helpers (createTool, createAgent, createMetadata) redefined in different suites (and there’s already a top-level createAgent). Consider reusing the existing helpers (or factoring shared helpers to the outer scope) to reduce duplication and future maintenance overhead.

Suggested change

function createAgent(name: string, modelQualifiedNames?: string[], tiers?: Record<string, { model: string }>): ICustomAgent {

return {

uri: URI.parse(`file:///test/${name}.md`),

name,

description: `Agent ${name}`,

tools: ['tool1'],

model: modelQualifiedNames,

tiers,

agentInstructions: { content: 'test', toolReferences: [] },

source: { storage: PromptsStorage.local },

target: Target.Undefined,

visibility: { userInvocable: true, agentInvocable: true }

};

}

Add structural validation for the new `tiers` attribute so agent prompts must provide a map of named tiers with string `model` entries, warn when the model is unknown, and flag stray properties when VS Code targets custom agents. Refactor the runSubagent tool tests to share the metadata, agent, and tool factory helpers (with an opt-in to custom agents), which cuts redundant copies across model/tier suites. This checkpoint keeps the tier feature consistent between prompt parsing and runtime coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Tier validation now checks both qualified names and direct model IDs before warning about unknown models, matching resolveModelHint’s runtime logic. This keeps prompt parsing aligned with how tiers are actually resolved at execution time, preventing false warnings when authors specify direct IDs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Copilot · 2026-04-02T21:04:33Z

 					const cached = this._resolvedModels.get(invocation.callId);
 					if (cached) {
 						this._resolvedModels.delete(invocation.callId);
 						modeModelId = cached.modeModelId;
 						resolvedModelName = cached.resolvedModelName;
 					} else {
 						// Fallback: resolve the model here if prepare didn't cache it
-						const resolved = this.resolveSubagentModel(subagent, invocation.modelId);
+						const resolved = this.resolveSubagentModel(subagent, invocation.modelId, args.model, args.tier);
 						modeModelId = resolved.modeModelId;


When a cached model resolution is present, modeModelId is assigned directly from cached.modeModelId. Because IToolInvocationPreparationContext.modelId is optional, the cached entry can legitimately contain undefined even when invocation.modelId is defined at invoke-time. This can unintentionally clear modeModelId and cause the subagent to run without an explicit model selection. Consider falling back to the current modeModelId (or invocation.modelId) when the cached value is undefined, similar to the no-subagent branch.

Copilot · 2026-04-02T21:04:34Z

+		if (modeModelId && modeModelId !== mainModelId) {
+			const mainModelMetadata = mainModelId ? this.languageModelsService.lookupLanguageModel(mainModelId) : undefined;
+			const resolvedModelMetadata = this.languageModelsService.lookupLanguageModel(modeModelId);
+			const mainMultiplier = mainModelMetadata?.multiplierNumeric;
+			const resolvedMultiplier = resolvedModelMetadata?.multiplierNumeric;
+			if (mainMultiplier !== undefined && resolvedMultiplier !== undefined && resolvedMultiplier > mainMultiplier) {
+				const source = callTimeModelHint ? 'Call-time' : callTimeTier ? `Tier '${callTimeTier}'` : subagent ? `Subagent '${subagent.name}'` : 'Unknown';
+				this.logService.warn(`[RunSubagentTool] ${source} requested model '${resolvedModelMetadata?.name}' (multiplier: ${resolvedMultiplier}) which exceeds the main agent model '${mainModelMetadata?.name}' (multiplier: ${mainMultiplier}). Falling back to the main agent model.`);
+				modeModelId = mainModelId;


The multiplier-cap warning message can misattribute the source of the resolved model. source is derived from whether callTimeModelHint/callTimeTier are set, even if the call-time model hint failed to resolve (and the final modeModelId came from the tier or agent default). This can produce misleading logs. Track the last successful resolution source (e.g., set a variable only when an override actually changes modeModelId) and use that for the warning.

Copilot · 2026-04-02T21:04:34Z

 					this._resolvedModels.delete(invocation.callId);
+					modeModelId = cached.modeModelId ?? modeModelId;
 					resolvedModelName = cached.resolvedModelName;
+				} else if (args.model) {
+					// Call-time model override without a named subagent
+					const resolved = this.resolveSubagentModel(undefined, invocation.modelId, args.model);
+					modeModelId = resolved.modeModelId;
+					resolvedModelName = resolved.resolvedModelName;
 				} else {


args.model is applied as a call-time override even when ChatConfiguration.SubagentToolCustomAgents is disabled. Since model/tier are introduced as parameters gated behind that config in getToolData(), it would be safer and more consistent to also ignore these parameters at runtime (both here and in prepareToolInvocation) unless the config flag is enabled. This prevents bypass via manually constructed tool invocations and keeps behavior aligned with the tool schema.

Copilot AI review requested due to automatic review settings March 31, 2026 16:39

RockyWearsAHat mentioned this pull request Mar 31, 2026

Chat: add 'tiers' frontmatter and 'tier' parameter to runSubagent for semantic model tier routing #306717

Open

Copilot started reviewing on behalf of RockyWearsAHat March 31, 2026 16:41 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

RockyWearsAHat and others added 2 commits March 31, 2026 11:12

vs-code-engineering Bot assigned roblourens Mar 31, 2026

roblourens assigned digitarald and aeschli and unassigned roblourens Mar 31, 2026

RockyWearsAHat requested a review from Copilot April 2, 2026 20:59

Copilot started reviewing on behalf of RockyWearsAHat April 2, 2026 21:00 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

austindyoung mentioned this pull request Apr 2, 2026

[mirror] microsoft/vscode#306846 feat(chat): add tiers frontmatter and tier parameter to runSubagent tool austindyoung/vscode-driftfence-mirror-fork#989

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(chat): add tiers frontmatter and tier parameter to runSubagent tool#306846

feat(chat): add tiers frontmatter and tier parameter to runSubagent tool#306846
RockyWearsAHat wants to merge 4 commits intomicrosoft:mainfrom
RockyWearsAHat:feature/runsubagent-tiers

RockyWearsAHat commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 31, 2026

Uh oh!

Copilot AI Mar 31, 2026

Uh oh!

Copilot AI Mar 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

-	 * Maps tier names (e.g., 'fast', 'standard', 'deep') to model qualified names.
-	 * Allows a single agent to serve at different capability/cost levels.
-	 */
+	 * Maps tier names (e.g., 'fast', 'standard', 'deep') to models, which may be
+	 * either qualified model names or direct model IDs. Allows a single agent to
+	 * serve at different capability/cost levels.

Conversation

RockyWearsAHat commented Mar 31, 2026

Summary

What Changed

Prompt file parser (promptFileParser.ts)

Agent interface & service (promptsService.ts, promptsServiceImpl.ts)

Prompt validation (promptValidator.ts, promptFileAttributes.ts)

RunSubagent tool (runSubagentTool.ts)

Tests (runSubagentTool.test.ts)

Example Agent Frontmatter

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Prompt file parser (`promptFileParser.ts`)

Agent interface & service (`promptsService.ts`, `promptsServiceImpl.ts`)

Prompt validation (`promptValidator.ts`, `promptFileAttributes.ts`)

RunSubagent tool (`runSubagentTool.ts`)

Tests (`runSubagentTool.test.ts`)