Skip to content

chore: upgrade afdocs to ^0.17.1#14

Closed
SahilAujla wants to merge 1 commit intofern-api:mainfrom
SahilAujla:chore/upgrade-afdocs-v0.17
Closed

chore: upgrade afdocs to ^0.17.1#14
SahilAujla wants to merge 1 commit intofern-api:mainfrom
SahilAujla:chore/upgrade-afdocs-v0.17

Conversation

@SahilAujla
Copy link
Copy Markdown

@SahilAujla SahilAujla commented Apr 27, 2026

Summary

This PR upgrades the afdocs dependency from 0.14.0 → 0.17.1, incorporating three minor releases of fixes and adapting the codebase to the breaking API changes introduced in v0.17.0.

It includes:

  • Minimal refactors in scoring logic to support the new nullable CategoryScore.score
  • Updates to renamed and split check IDs across both scoring code and marketing pages

Key user-facing improvement:
Sites that have both an apex llms.txt and a docs-specific llms.txt (e.g., Cloudflare, Supabase, Vercel, Resend, Pinecone, Clerk, Sentry, Datadog, MongoDB, Auth0, LaunchDarkly, etc.) will now correctly use the docs version as canonical. This comes from afdocs#54 and prevents scores from being skewed by marketing-site indexes.


What Changed in afdocs Since 0.14.0

Release Key Changes
0.15.x Canonical llms.txt selection (prefers most specific file). Adds --llms-txt-url CLI flag and llmsTxtUrl config.
0.16.x CategoryScore becomes an object instead of a number. Programmatic API improvements.
0.17.0 Spec v0.5.0. Renames llms-txt-freshnessllms-txt-coverage. Splits llms-txt-directive into HTML + Markdown variants. Adds nested index handling, strips <script>/<style> before sizing, introduces nullable CategoryScore.score, and validates options in createContext() / runChecks(). See the migration guide.
0.17.1 Patch fixes on top of 0.17.0.

Code Changes in This PR

Scoring Core

  • lib/scoring.ts

    • Introduces normalizeCategoryScores() helper
    • Handles new type:
      Record<string, number | { score: number | null }>
    • Drops null scores (new in v0.17 for low sample sizes), allowing fallback to existing local calculations
    • Result: preserves previous behavior with no UI changes
  • app/api/score/route.ts and scripts/score-single.ts

    • Replace inline conversions with the new helper
    • Add required specUrl: '' field to reports (now enforced by computeScore())

Renamed / Split Check IDs

  • scripts/debug-score.ts

    • Updated WEIGHTS map:

      • llms-txt-directive

        • llms-txt-directive-html (High / 7)
        • llms-txt-directive-md (Medium / 4)
      • llms-txt-freshnessllms-txt-coverage

  • app/master-plan/page.tsx

    • Updated methodology table
    • Rebalanced Discoverability weights (2 + 1) to maintain total = 100
  • app/page.tsx

    • Updated Discoverability check count: 1 → 2

Dependency Updates

  • package.json

    • afdocs: ^0.14.0^0.17.1
  • package-lock.json

    • Regenerated to resolve afdocs@0.17.1

Stored Data: Rescore Required

Existing Supabase rows still reference old check IDs:

  • llms-txt-directive
  • llms-txt-freshness

After this upgrade:

  • computeScore() will ignore these outdated checks
  • Result: slightly inaccurate scores for any non-rescored entries (effectively treated as 0-weight)

Recommended Post-Merge Steps

# 1. Preview score changes (no writes)
export $(grep -v '^#' .env.local | xargs)
npx tsx scripts/compare-scores.ts --changed

# 2. Rescore all sites
npx tsx scripts/rerun-all.ts

From the v0.17 migration guide:

  • Typical score movement: ±5 points

  • Directive split may reduce discoverability by 3–14 points

  • Gains often offset by:

    • HTML size improvements (script/style stripping)
    • Better coverage scoring (especially for nested llms.txt setups)

Net effect: generally small changes, with some significant upward movers.


Verification

Check Result
npm install Resolves afdocs@0.17.1
npm run build Succeeds; all 15 routes generated
tsc --noEmit Passes
Touched files (lib/scoring.ts, updated scripts) Clean
npx tsx scripts/debug-score.ts https://www.alchemy.com/docs Runs end-to-end; new check IDs appear; score = 97 (A)

Note:
Pre-existing type errors in the following files are unrelated and out of scope:

  • scripts/dedup-supabase.ts
  • scripts/reclassify-categories.ts
  • scripts/migrate-to-supabase.ts

Brings in three afdocs minor versions of fixes since this repo last bumped
to 0.14.0 in PR fern-api#2. The most user-visible change is that sites with both an
apex llms.txt and a docs-section llms.txt now get the docs file picked as
canonical (afdocs PR #54), so docs scores stop being polluted by marketing
indexes.

## Code changes for v0.17.0 breaking API

- `package.json`: `afdocs` `^0.14.0` → `^0.17.1`
- `lib/scoring.ts`: extract a `normalizeCategoryScores()` helper that
  handles the new `CategoryScore.score: number | null` shape (null means
  the category had fewer than 5 sampleable pages so afdocs marks it
  `notApplicable`). Drops null categories from the flat numeric map so
  consumers fall back to local calculation, matching previous behavior.
- `app/api/score/route.ts`, `scripts/score-single.ts`: switch the inline
  `Record<string, CategoryScore>` → `Record<string, number>` conversion
  to use the new helper. `score-single.ts` also picks up a small
  pre-existing fix: `computeScore()` now requires `specUrl` on its input,
  which the script wasn't passing (was uncaught because `scripts/` is
  excluded from the project tsconfig).

## Check ID renames (v0.17.0)

`afdocs` v0.17.0 split `llms-txt-directive` into `llms-txt-directive-html`
(High/7 weight, same as the old combined check) and `llms-txt-directive-md`
(Medium/4), and renamed `llms-txt-freshness` → `llms-txt-coverage`. Updated
in:

- `scripts/debug-score.ts`: `WEIGHTS` reference map
- `app/master-plan/page.tsx`: scoring methodology reference table.
  Rebalanced the simplified directive split (2 + 1) so the table still
  totals 100.
- `app/page.tsx`: `Discoverability` method-cell card check count `1` → `2`

## Score effects on stored data

Existing rows in Supabase still have `results` arrays with the old check
IDs (`llms-txt-directive`, `llms-txt-freshness`). Until those companies
are rescored, displayed `score`/`grade` will be slightly off because
`computeScore` no longer recognizes the old IDs.

Recommended follow-up after merge:

  npx tsx scripts/compare-scores.ts --changed   # see deltas
  export $(grep -v '^#' .env.local | xargs) && \\
    npx tsx scripts/rerun-all.ts                # rescore everything

Per the v0.17 migration guide, smoke testing showed sites moving by ±5
points on average with the new spec, mostly driven by the directive
split (content-discoverability typically drops 3-14 points) offset by
HTML page-size improvements (script/style elements now stripped before
sizing). Sites that nest their llms.txt may see big jumps in coverage
because the renamed `llms-txt-coverage` check correctly handles nested
indexes via the new "omitted subtrees" feature.

## Verified

- `npm run build` — succeeds; all 15 routes generate; tsc --noEmit passes
- `npx tsx scripts/debug-score.ts https://www.alchemy.com/docs` — runs
  end-to-end, all 22 checks execute, new IDs appear in output, score 97/100 (A)
- `tsc --noEmit` on lib/scoring.ts + the updated scripts — clean

Pre-existing scripts/ type errors unrelated to afdocs (`dedup-supabase.ts`,
`reclassify-categories.ts`, `migrate-to-supabase.ts`) are out of scope.

Made-with: Cursor
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 27, 2026

@SahilAujla is attempting to deploy a commit to the fern Team on Vercel.

A member of the Team first needs to authorize it.

@SahilAujla
Copy link
Copy Markdown
Author

SahilAujla commented Apr 27, 2026

@kgowru could I get your eyes on this pls 🙏🏻

@SahilAujla SahilAujla closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant