Skip to content

Add General Purpose agent support behind experimental setting#306871

Merged
digitarald merged 9 commits intomainfrom
users/digitarald/general-purpose-agent
Apr 2, 2026
Merged

Add General Purpose agent support behind experimental setting#306871
digitarald merged 9 commits intomainfrom
users/digitarald/general-purpose-agent

Conversation

@digitarald
Copy link
Copy Markdown
Contributor

@digitarald digitarald commented Mar 31, 2026

Add a built-in "General Purpose" agent to the runSubagent tool, gated behind the chat.generalPurposeAgent.enabled experimental setting.

Based on #295494, rebased and adapted to the current codebase.

Changes

constants.ts

  • Added GeneralPurposeAgentName = 'General Purpose' constant
  • Added GeneralPurposeAgentEnabled = 'chat.generalPurposeAgent.enabled' to ChatConfiguration enum

chat.contribution.ts

  • Registered the chat.generalPurposeAgent.enabled setting with type: 'boolean', default: false, tags: ['experimental', 'advanced'], experiment: { mode: 'auto' }
  • The advanced tag hides the setting from the Settings UI by default on Stable

runSubagentTool.ts

  • Replaced IWorkbenchAssignmentService experiment checks with synchronous configurationService.getValue(ChatConfiguration.GeneralPurposeAgentEnabled) reads
  • Removed cached _generalPurposeAgentEnabled field, _resolveExperiment() method, and onDidRefetchAssignments listener
  • Schema gating: agentName parameter only appears in the tool schema when SubagentToolCustomAgents or GeneralPurposeAgentEnabled is enabled; required when GP is enabled
  • Name normalization: When GP is enabled, undefined/empty/"General Purpose" agent names are normalized to effectiveSubAgentName = GeneralPurposeAgentName, used consistently in agentRequest.subAgentName and toolMetadata.agentName
  • Custom agent gating: getSubAgentByName() lookups are gated behind SubagentToolCustomAgents setting in both invoke() and prepareToolInvocation()
  • Error hints: Unknown agent errors now always include a base recovery hint ("Try again with the correct agent name, or omit agentName"), with an additional GP-specific hint when enabled
  • onDidUpdateToolData fires on config changes for both SubagentToolCustomAgents and GeneralPurposeAgentEnabled

computeAutomaticInstructions.ts

  • Removed IWorkbenchAssignmentService injection (replaced async getTreatment with synchronous configurationService.getValue)
  • When setting is enabled, renders GP agent entry first in the agents instruction block with a full description
  • Custom agents are appended after the GP agent entry

Tests

  • Removed all IWorkbenchAssignmentService stubs and NullWorkbenchAssignmentService usage from GP-related tests
  • GP-enabled tests now use TestConfigurationService({ [ChatConfiguration.GeneralPurposeAgentEnabled]: true })
  • Model resolution tests now enable SubagentToolCustomAgents via TestConfigurationService
  • Schema test validates agentName is absent when neither GP nor custom agents is enabled
  • Removed await Event.toPromise(tool.onDidUpdateToolData) waits (no longer async)

Decisions

  • Old experiment key chat.generalPurposeAgent intentionally dropped — this PR has not been merged yet, so no backward compatibility needed
  • Setting uses experiment: { mode: 'auto' } so experiments can still override the default value
  • Setting tagged ['experimental', 'advanced'] — hidden by default on Stable, visible on Insiders
  • agentName schema gated behind customAgentsEnabled || gpEnabled to avoid exposing the parameter when neither feature is active
  • Custom agent lookup gated behind SubagentToolCustomAgents to prevent resolving agents when that setting is off

Copilot AI review requested due to automatic review settings March 31, 2026 17:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a built-in “General Purpose” subagent pathway to the runSubagent tool and surfaces it in automatic agent instructions, gated behind the chat.generalPurposeAgent experiment (resolved via IWorkbenchAssignmentService).

Changes:

  • Add GeneralPurposeAgentName constant and integrate GP routing in RunSubagentTool (schema + invocation behavior).
  • Update ComputeAutomaticInstructions to include GP agent first (when experiment is enabled) and append custom agents after.
  • Refactor PromptsService caching/logging and update/extend tests for GP agent behavior and prompt parsing expectations.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
src/vs/workbench/contrib/chat/common/constants.ts Adds shared GeneralPurposeAgentName constant for GP agent references.
src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Introduces experiment-gated GP behavior and updates tool schema/update events accordingly.
src/vs/workbench/contrib/chat/common/promptSyntax/computeAutomaticInstructions.ts Renders GP agent entry first (when enabled) in the <agents> instruction block.
src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts Refactors cached results vs discovery-info logging and custom agent/skill/slash command computations.
src/vs/workbench/contrib/chat/test/common/tools/builtinTools/runSubagentTool.test.ts Adds GP-path tests and updates tool construction for new DI shape.
src/vs/workbench/contrib/chat/test/common/promptSyntax/computeAutomaticInstructions.test.ts Adds coverage asserting GP agent appears first when experiment is enabled.
src/vs/workbench/contrib/chat/test/common/promptSyntax/service/promptsService.test.ts Updates expectations for parsed variable references and tool-reference ranges.
Comments suppressed due to low confidence (2)

src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts:715

  • getAgentDiscoveryInfo() is recomputed just for discovery logging, which can significantly increase work for getCustomAgents() (re-parsing agent files) whenever sessionResource is provided. Please guard this behind this._onDidLogDiscovery.hasListeners() or restructure caching so logging can reuse the already-computed results.
	public async getCustomAgents(token: CancellationToken, sessionResource?: URI): Promise<readonly ICustomAgent[]> {
		const sw = StopWatch.create();
		const result = await this.cachedCustomAgents.get(token);
		if (sessionResource) {
			const elapsed = sw.elapsed();
			void this.getAgentDiscoveryInfo(token).catch(() => undefined).then(discoveryInfo => {
				const details = result.length === 1
					? localize("promptsService.resolvedAgent", "Resolved {0} agent in {1}ms", result.length, elapsed.toFixed(1))
					: localize("promptsService.resolvedAgents", "Resolved {0} agents in {1}ms", result.length, elapsed.toFixed(1));
				this._onDidLogDiscovery.fire({
					sessionResource,
					name: localize("promptsService.loadAgents", "Load Agents"),
					details,
					discoveryInfo,
					category: 'discovery',
				});
			});

src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts:1125

  • getSkillDiscoveryInfo() is recomputed purely for discovery logging inside findAgentSkills(), which can double skill parsing and folder diagnostics work whenever sessionResource is provided. Consider guarding the extra computation behind this._onDidLogDiscovery.hasListeners() or caching the discovery info with the computed skills.
		const result = await this.cachedSkills.get(token);
		if (sessionResource) {
			const elapsed = sw.elapsed();
			void this.getSkillDiscoveryInfo(token).catch(() => undefined).then(discoveryInfo => {
				const details = result.length === 1
					? localize("promptsService.resolvedSkill", "Resolved {0} skill in {1}ms", result.length, elapsed.toFixed(1))
					: localize("promptsService.resolvedSkills", "Resolved {0} skills in {1}ms", result.length, elapsed.toFixed(1));
				this._onDidLogDiscovery.fire({
					sessionResource,
					name: localize("promptsService.loadSkills", "Load Skills"),
					details,
					discoveryInfo,
					category: 'discovery',
				});
			});

Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/promptSyntax/computeAutomaticInstructions.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/promptSyntax/service/promptsServiceImpl.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/promptSyntax/computeAutomaticInstructions.ts Outdated
Copy link
Copy Markdown
Contributor

@aeschli aeschli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of the IWorkbenchAssignmentService don't we just use settings that then are controlled by experiments?

@digitarald
Copy link
Copy Markdown
Contributor Author

Instead of the IWorkbenchAssignmentService don't we just use settings that then are controlled by experiments?

Felt like an odd setting to add, as there is uncertain user value in toggling it on or off. I could make it a hidden internal setting, which also makes testing it easier.

Copy link
Copy Markdown
Contributor

@aeschli aeschli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between the default agent (no custom agent named in the subagent tool) and the General Purpose agent?

@aeschli
Copy link
Copy Markdown
Contributor

aeschli commented Apr 1, 2026

Yes, lets add an internal setting. From what I understand there's the convention that all experiments are now backed by a setting.

@digitarald digitarald changed the title Add General Purpose agent support behind experiment Add General Purpose agent support behind experimental setting Apr 2, 2026
@digitarald
Copy link
Copy Markdown
Contributor Author

What's the difference between the default agent (no custom agent named in the subagent tool) and the General Purpose agent?

Technically, there's no difference between current empty argument and the new general purpose. I keep getting reports that the Explorer subagent is overused, and also that the subagents are not automatically used for parallel implementation. Both issues I hope to see improve by giving the default agent a named entry.

Add a built-in 'General Purpose' agent to the runSubagent tool, gated
behind the 'chat.generalPurposeAgent' experiment treatment:

- Add GeneralPurposeAgentName constant
- Make agentName required and route undefined/GP names to built-in agent
- Render GP agent in automatic instructions agents block
- Clean up duplicate DI injection in RunSubagentTool
- Add unit tests for GP agent paths
…h, deterministic tests

- Replace unsafe 'configEvent as Event<void>' cast with dedicated Emitter
  (fixes tsgo typecheck CI failure)
- Fire onDidUpdateToolData when experiment resolution changes the value
- Add try/catch around getTreatment in computeAutomaticInstructions
- Replace flaky setTimeout(0) in tests with Event.toPromise(onDidUpdateToolData)
…ulation

The merge conflict resolution incorrectly replaced fullLength with
name.length + 1 for toolReference OffsetRange calculation, and removed
fullLength from variableReferences test expectations. Restore the
original behavior from main.
- Add error handler to _resolveExperiment() preventing unhandled
  promise rejections when getTreatment fails
- Decouple GP agent from SubagentToolCustomAgents config gate so
  experiment works independently of custom agents setting
- Fix redundant parens on Event listener arrow function
- Add test for GP agent rendering without custom agents config
Reset promptsServiceImpl.ts and promptsService.test.ts back to main.
These files contained an unrelated refactor (method renames, type
simplifications) that was accidentally carried over during the port
from PR #295494.
Replace direct IWorkbenchAssignmentService.getTreatment checks with
an experimental configuration setting chat.generalPurposeAgent.enabled.
This simplifies runtime code from async experiment resolution to
synchronous configurationService.getValue() calls.

- Add GeneralPurposeAgentEnabled to ChatConfiguration enum
- Register setting with experiment: { mode: 'auto' }, default: false
- Remove IWorkbenchAssignmentService DI from RunSubagentTool and
  ComputeAutomaticInstructions (zero other usages in either file)
- Make GP agent error hint conditional on setting being enabled
- Update tests to use config-based setup instead of assignment stubs
@digitarald digitarald force-pushed the users/digitarald/general-purpose-agent branch from 75a0c2d to 1eb1701 Compare April 2, 2026 04:46
@digitarald digitarald requested a review from Copilot April 2, 2026 04:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Comment thread src/vs/workbench/contrib/chat/common/tools/builtinTools/runSubagentTool.ts Outdated
@aeschli
Copy link
Copy Markdown
Contributor

aeschli commented Apr 2, 2026

Can we remove the SubagentToolCustomAgents ? That would simplify this PR a lot.

There's also the suggestion that we make the name of the subagentTool mandatory. We could then mention in the tool description that 'General Agent' is a good default.

What do you think?

Is the 'General Agent' a custom agent defined in the Copilot extension? How would core know about it? Maybe the setting could be to give the default agent's name?

@digitarald digitarald merged commit b955f7c into main Apr 2, 2026
19 checks passed
@digitarald digitarald deleted the users/digitarald/general-purpose-agent branch April 2, 2026 08:02
@vs-code-engineering vs-code-engineering Bot added this to the 1.115.0 milestone Apr 2, 2026
@aeschli
Copy link
Copy Markdown
Contributor

aeschli commented Apr 2, 2026

I approved the PR. Open issues:

  • Can we remove the SubagentToolCustomAgents ? That would simplify things
  • Where is the 'General Agent' defined, if not in Core, how does core know its name and whether it real exists
  • Why is the 'General Agent' not a regular custom agent contributed from the GitHub extension.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants