feat(sources): support well-known HTTP skill sources by gricha · Pull Request #77 · getsentry/dotagents

gricha · 2026-03-20T17:55:32Z

Summary

Adds support for bare HTTPS URLs as skill sources, resolving via the .well-known/skills/ convention (Issue #74).

dotagents add https://cli.sentry.dev now works — fetches {url}/.well-known/skills/index.json, downloads skill files into a TTL-based local cache
Bare HTTPS URLs not matching GitHub/GitLab are treated as well-known candidates; failure suggests the git: prefix
Trust validation uses git_domains with prefix-matching (e.g. trust add cli.sentry.dev)
Same offline behavior as git: stale cache used on fetch failure
Full add/install/resolve/wildcard/lockfile support

Changes

File	What
`src/sources/wellknown.ts`	New HTTP fetcher and cache (`fetchWellKnownIndex`, `ensureWellKnownCached`)
`src/skills/resolver.ts`	`parseSource` returns `"well-known"` for bare HTTPS; `resolveSkill`/`resolveWildcardSkills` handle it; `normalizeSource` lowercases hostname
`src/config/schema.ts`	Schema accepts bare `http://`/`https://` URLs
`src/trust/validator.ts`	`"well-known"` shares domain-matching logic with `"git"`
`src/lockfile/schema.ts`	`lockedWellKnownSkillSchema` (`source` + `resolved_url`)
`src/cli/commands/add.ts`	Well-known branch: fetch/cache → discover → picker (same UX as git)
`src/cli/commands/install.ts`	Lock entry creation for well-known sources
`specs/SPEC.md`, `docs/public/llms.txt`	Document `https://<domain>` source type

Closes #74

Test plan

pnpm check passes (lint + typecheck + 585 tests)
New tests in wellknown.test.ts: index fetch (valid/invalid/404/network error), caching (fresh/stale/offline)
New tests in resolver.test.ts: parseSource with well-known URLs, normalizeSource, sourcesMatch
New tests in validator.test.ts: well-known sources against git_domains
Updated test in schema.test.ts: bare HTTPS URLs now accepted
Manual smoke test: npx @sentry/dotagents add https://cli.sentry.dev (requires network + live endpoint)

Agent transcript: https://claudescope.sentry.dev/share/IWLZm9eChduh84Sfd0hyukgm_3Y8F0fp82DqNZzNZ6Q

Add support for bare HTTPS URLs (e.g. `https://cli.sentry.dev`) as skill sources. These are resolved via the `.well-known/skills/` convention: dotagents fetches `{url}/.well-known/skills/index.json` to discover available skills, then downloads each skill's files into a TTL-based cache. Bare HTTPS URLs that don't match GitHub/GitLab patterns are treated as well-known source candidates. If the endpoint doesn't exist, the error message suggests using the `git:` prefix. Closes #74 Agent transcript: https://claudescope.sentry.dev/share/zG7Tq2nDrDatJjftMjoLN2b_zvtpADH05RTyp_Zka3c

vercel · 2026-03-20T17:55:37Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
dotagents	Ready	Preview, Comment	Mar 20, 2026 6:54pm

gricha · 2026-03-20T18:04:21Z

src/skills/resolver.ts


+  // Bare HTTP(S) URL not matching GitHub/GitLab — candidate for well-known
+  if (/^https?:\/\//i.test(source)) {
+    return { type: "well-known", url: source.replace(/\/+$/, "") };


Fixed — replaced /\/+$/ regex with an iterative stripTrailingSlashes() helper that uses a simple while loop. No regex involved.

— Claude Code

gricha · 2026-03-20T18:04:22Z

src/sources/wellknown.ts

+export async function fetchWellKnownIndex(
+  baseUrl: string,
+): Promise<WellKnownIndex | null> {
+  const indexUrl = `${baseUrl.replace(/\/+$/, "")}/.well-known/skills/index.json`;


Fixed — same stripTrailingSlashes() helper approach, no regex.

— Claude Code

gricha · 2026-03-20T18:04:22Z

src/sources/wellknown.ts

+    return { cacheDir };
+  }
+
+  const baseUrl = opts.url.replace(/\/+$/, "");


Fixed — same stripTrailingSlashes() helper approach, no regex.

— Claude Code

src/sources/wellknown.ts

Replace /\/+$/ regex (flagged by CodeQL as polynomial on uncontrolled data) with a simple while-loop helper in wellknown.ts and resolver.ts, and an inline slice in add.ts.

src/config/schema.ts

src/skills/resolver.ts

src/sources/wellknown.ts

- Validate skill.name and file names from remote index.json against safe name pattern to prevent path traversal attacks - Restrict well-known sources to HTTPS only (reject http://) to prevent MITM attacks on skill content

src/sources/wellknown.ts

src/skills/resolver.ts

src/cli/commands/add.ts

- Use URL.host instead of URL.hostname to preserve port numbers in normalization and cache keys - Remove unused WellKnownError class - Move stripTrailingSlashes to shared utils/fs.ts (deduplicate)

src/skills/resolver.ts

parseSource now only classifies https:// URLs as well-known (not http://). The add command explicitly rejects bare http:// URLs with a clear error message, preventing MITM attacks via plain HTTP skill fetching.

src/trust/validator.ts

src/sources/wellknown.ts

extractDomain and extractDomainPath now use URL.host instead of URL.hostname, consistent with the resolver changes. This ensures git_domains entries with ports (e.g. "localhost:3000") match correctly.

src/sources/wellknown.ts

src/config/schema.ts

- File paths in well-known index can now contain subdirectories (e.g. "prompts/base.md") while still preventing path traversal - Schema rejects bare "https://" without a hostname - Create parent directories for nested file paths during download

src/cli/commands/add.ts

src/skills/resolver.ts

Move the http:// rejection check before parseSource and check against the raw URL with GitHub/GitLab regex exclusions, so http://cli.sentry.dev gets the "Insecure source" error instead of a confusing git clone failure.

src/sources/wellknown.ts

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

cursor · 2026-03-20T19:03:28Z

src/cli/commands/add.ts

+            `Specify skill names as arguments, use --skill to specify which ones, or --all for all skills.`,
+        );
+      }
+    }


Well-known and git add branches heavily duplicated

Low Severity

The well-known branch (lines 200–343) is nearly a verbatim copy of the git branch (lines 344–487). The interactive selection flow — clack.select for all-vs-pick, clack.multiselect, multi-name override handling, duplicate-skip logic, and addWildcard — is identical across both branches except for the cache source (cached.cacheDir vs cached.repoDir). A shared helper accepting the cache directory would eliminate ~140 lines of duplication and reduce the risk of fixing a bug in one branch but not the other.

vercel bot deployed to Preview March 20, 2026 17:55 View deployment

github-advanced-security bot found potential problems Mar 20, 2026

View reviewed changes

sentry bot reviewed Mar 20, 2026

View reviewed changes

src/sources/wellknown.ts Show resolved Hide resolved

fix: replace polynomial regex with iterative trailing-slash strip

2f59e78

Replace /\/+$/ regex (flagged by CodeQL as polynomial on uncontrolled data) with a simple while-loop helper in wellknown.ts and resolver.ts, and an inline slice in add.ts.

vercel bot deployed to Preview March 20, 2026 18:00 View deployment

sentry-warden bot reviewed Mar 20, 2026

View reviewed changes

src/config/schema.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/skills/resolver.ts Outdated Show resolved Hide resolved

src/sources/wellknown.ts Show resolved Hide resolved

src/sources/wellknown.ts Outdated Show resolved Hide resolved

src/sources/wellknown.ts Outdated Show resolved Hide resolved

sentry bot reviewed Mar 20, 2026

View reviewed changes

src/sources/wellknown.ts Outdated Show resolved Hide resolved

vercel bot deployed to Preview March 20, 2026 18:04 View deployment

sentry-warden bot reviewed Mar 20, 2026

View reviewed changes

src/sources/wellknown.ts Outdated Show resolved Hide resolved

sentry bot reviewed Mar 20, 2026

View reviewed changes

src/skills/resolver.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/cli/commands/add.ts Outdated Show resolved Hide resolved

fix: address review bot feedback

242e2d2

- Use URL.host instead of URL.hostname to preserve port numbers in normalization and cache keys - Remove unused WellKnownError class - Move stripTrailingSlashes to shared utils/fs.ts (deduplicate)

vercel bot deployed to Preview March 20, 2026 18:12 View deployment

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/skills/resolver.ts Show resolved Hide resolved

fix: reject http:// URLs in parseSource and add command

42df3ff

parseSource now only classifies https:// URLs as well-known (not http://). The add command explicitly rejects bare http:// URLs with a clear error message, preventing MITM attacks via plain HTTP skill fetching.

vercel bot deployed to Preview March 20, 2026 18:21 View deployment

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/trust/validator.ts Show resolved Hide resolved

sentry-warden bot reviewed Mar 20, 2026

View reviewed changes

src/sources/wellknown.ts Outdated Show resolved Hide resolved

fix: use URL.host in trust validator to preserve port numbers

7e49230

extractDomain and extractDomainPath now use URL.host instead of URL.hostname, consistent with the resolver changes. This ensures git_domains entries with ports (e.g. "localhost:3000") match correctly.

vercel bot deployed to Preview March 20, 2026 18:28 View deployment

sentry bot reviewed Mar 20, 2026

View reviewed changes

src/sources/wellknown.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/config/schema.ts Outdated Show resolved Hide resolved

vercel bot deployed to Preview March 20, 2026 18:38 View deployment

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/cli/commands/add.ts Show resolved Hide resolved

sentry-warden bot reviewed Mar 20, 2026

View reviewed changes

src/skills/resolver.ts Show resolved Hide resolved

fix: reject http:// URLs before parseSource to show clear error

0952632

Move the http:// rejection check before parseSource and check against the raw URL with GitHub/GitLab regex exclusions, so http://cli.sentry.dev gets the "Insecure source" error instead of a confusing git clone failure.

vercel bot deployed to Preview March 20, 2026 18:47 View deployment

cursor bot reviewed Mar 20, 2026

View reviewed changes

src/sources/wellknown.ts Show resolved Hide resolved

fix: validate files entries are strings in well-known index

7a7577b

vercel bot deployed to Preview March 20, 2026 18:54 View deployment

cursor bot reviewed Mar 20, 2026

View reviewed changes

gricha merged commit 0fa5b70 into main Mar 20, 2026
18 checks passed

gricha deleted the feat/wellknown-skill-sources branch March 20, 2026 19:05

Uh oh!

Conversation

gricha commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Uh oh!

vercel bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Check failure

gricha Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Check failure

gricha Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Check failure

gricha Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Mar 20, 2026

Choose a reason for hiding this comment

Well-known and git add branches heavily duplicated

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gricha commented Mar 20, 2026 •

edited

Loading

vercel bot commented Mar 20, 2026 •

edited

Loading