Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ structure, trust signals, and feedback.
```bash
pnpm exec docs doctor --agent
pnpm exec docs doctor --site
pnpm exec docs doctor --agent --json
```

Expected output looks like:
Expand Down Expand Up @@ -216,6 +217,24 @@ agent-ready or agent-optimized, and it works well as a CI check for the machine-
- reader feedback
- reading-time cues

Use `--json` when the result needs to feed another system instead of a person reading the terminal:

```bash
pnpm exec docs doctor --agent --json
pnpm exec docs doctor --site --json
```

That JSON form is useful for:

- CI quality gates
- GitHub Actions summaries or PR comments
- dashboards that track docs quality over time
- automation that reruns `docs agent compact --stale`
- other agents that need structured readiness signals instead of terminal text
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 "Warnings go to stderr" claim is not backed by the implementation

doctor.ts has no console.warn, console.error, or process.stderr writes of its own. All check results — including those with warn or fail status — are embedded in the JSON payload sent to stdout via console.log. The only stderr path is an incidental console.warn in config.ts that fires when config module evaluation falls back to static parsing, which is an unrelated infrastructure edge case.

The same claim appears in website/app/docs/cli/page.mdx ("Warnings still go to stderr, so stdout stays safe to parse") and skills/farming-labs/cli/SKILL.md ("JSON stays on stdout while warnings stay on stderr").

Consider either updating the copy to reflect actual behavior ("The full report, including any check warnings, is written as JSON to stdout") or adding a deliberate stderr path (e.g., process.stderr.write(...)) for operational messages such as config load failures before the JSON claim is made.


The JSON report itself is written to stdout. Separate loader notices, such as config fallback
warnings, are outside the JSON payload.

## Common Tasks

Use the full docs for feature-specific setup:
Expand Down
111 changes: 109 additions & 2 deletions packages/docs/src/cli/doctor.test.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { createServer } from "node:http";
import { mkdtempSync, mkdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import type { AddressInfo } from "node:net";
import { compactAgentDocs } from "./agent.js";
import { inspectAgentReadiness, inspectHumanReadiness, parseDoctorArgs } from "./doctor.js";
import {
inspectAgentReadiness,
inspectHumanReadiness,
parseDoctorArgs,
runDoctor,
} from "./doctor.js";

function writePackageJson(
rootDir: string,
Expand Down Expand Up @@ -56,6 +61,17 @@ describe("parseDoctorArgs", () => {
});
});

it("parses json output mode", () => {
expect(parseDoctorArgs(["--json"])).toEqual({
mode: "agent",
json: true,
});
expect(parseDoctorArgs(["--site", "--json"])).toEqual({
mode: "human",
json: true,
});
});

it("parses human mode aliases", () => {
expect(parseDoctorArgs(["--human"])).toEqual({ mode: "human" });
expect(parseDoctorArgs(["human", "--config=docs.config.ts"])).toEqual({
Expand Down Expand Up @@ -996,4 +1012,95 @@ Welcome to the docs.
expect(trustCheck?.status).toBe("pass");
expect(trustCheck?.detail).toBe("Edit links and last-updated metadata are configured.");
});

it("prints agent reports as JSON for automation consumers", async () => {
writePackageJson(tmpDir, "doctor-agent-json", { next: "16.0.0" });

writeFileSync(
path.join(tmpDir, "docs.config.ts"),
`export default {
entry: "docs",
llmsTxt: { enabled: true },
};`,
"utf-8",
);

mkdirSync(path.join(tmpDir, "docs"), { recursive: true });
writeFileSync(
path.join(tmpDir, "docs", "page.mdx"),
`---
title: "Overview"
description: "Docs home"
---

# Overview
`,
"utf-8",
);

process.chdir(tmpDir);

const logSpy = vi.spyOn(console, "log").mockImplementation(() => undefined);

try {
const report = await runDoctor({ mode: "agent", json: true });
expect(report.mode).toBe("agent");
expect(logSpy).toHaveBeenCalledTimes(1);

const serialized = logSpy.mock.calls[0]?.[0];
expect(typeof serialized).toBe("string");

const payload = JSON.parse(String(serialized)) as { mode: string; checks: unknown[] };
expect(payload.mode).toBe("agent");
expect(Array.isArray(payload.checks)).toBe(true);
} finally {
logSpy.mockRestore();
}
});

it("prints site reports as JSON with the public mode name", async () => {
writePackageJson(tmpDir, "doctor-site-json", { next: "16.0.0" });

writeFileSync(
path.join(tmpDir, "docs.config.ts"),
`export default {
entry: "docs",
contentDir: "docs",
search: true,
};`,
"utf-8",
);

mkdirSync(path.join(tmpDir, "docs"), { recursive: true });
writeFileSync(
path.join(tmpDir, "docs", "page.mdx"),
`---
title: "Overview"
description: "Docs home"
---

# Overview
`,
"utf-8",
);

process.chdir(tmpDir);

const logSpy = vi.spyOn(console, "log").mockImplementation(() => undefined);

try {
const report = await runDoctor({ mode: "human", json: true });
expect(report.mode).toBe("human");
expect(logSpy).toHaveBeenCalledTimes(1);

const serialized = logSpy.mock.calls[0]?.[0];
expect(typeof serialized).toBe("string");

const payload = JSON.parse(String(serialized)) as { mode: string; checks: unknown[] };
expect(payload.mode).toBe("site");
expect(Array.isArray(payload.checks)).toBe(true);
} finally {
logSpy.mockRestore();
}
});
});
31 changes: 31 additions & 0 deletions packages/docs/src/cli/doctor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ type DoctorMode = "agent" | "human";
export interface DoctorOptions {
configPath?: string;
mode?: DoctorMode;
json?: boolean;
}

export interface ParsedDoctorArgs extends DoctorOptions {
Expand Down Expand Up @@ -157,6 +158,11 @@ export function parseDoctorArgs(argv: string[]): ParsedDoctorArgs {
continue;
}

if (arg === "--json") {
parsed.json = true;
continue;
}

if (arg === "--human" || arg === "human" || arg === "--site" || arg === "site") {
parsed.mode = "human";
continue;
Expand Down Expand Up @@ -199,13 +205,15 @@ ${pc.dim("Usage:")}
pnpm exec docs doctor
pnpm exec docs doctor --agent
pnpm exec docs doctor --site
pnpm exec docs doctor --agent --json
pnpm exec docs doctor agent
pnpm exec docs doctor site

${pc.dim("Options:")}
${pc.cyan("--agent")} Score agent-readiness for the current docs app (default)
${pc.cyan("--site")} Score reader-facing docs quality for the current docs app
${pc.cyan("--human")} Alias for ${pc.cyan("--site")}
${pc.cyan("--json")} Print the report as JSON for CI, scripts, and other agents
${pc.cyan("--config <path>")} Use a custom docs config path instead of ${pc.dim("docs.config.ts[x]")}
${pc.cyan("-h, --help")} Show this help message
`);
Expand Down Expand Up @@ -1697,14 +1705,37 @@ export function printHumanDoctorReport(report: HumanDoctorReport) {
}
}

function serializeDoctorJsonReport(report: AgentDoctorReport | HumanDoctorReport) {
if (report.mode === "human") {
return {
...report,
mode: "site" as const,
};
}

return report;
}

export function printDoctorJsonReport(report: AgentDoctorReport | HumanDoctorReport) {
console.log(JSON.stringify(serializeDoctorJsonReport(report), null, 2));
}

export async function runDoctor(options: DoctorOptions = {}) {
if (options.mode === "human") {
const report = await inspectHumanReadiness(options);
if (options.json) {
printDoctorJsonReport(report);
return report;
}
printHumanDoctorReport(report);
return report;
}

const report = await inspectAgentReadiness(options);
if (options.json) {
printDoctorJsonReport(report);
return report;
}
printAgentDoctorReport(report);
return report;
}
1 change: 1 addition & 0 deletions packages/docs/src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@ ${pc.dim("Options for doctor:")}
${pc.cyan("doctor --agent")} Same as ${pc.cyan("doctor")}; explicit agent scoring mode
${pc.cyan("doctor --site")} Score the current docs app for reader-facing docs quality
${pc.cyan("doctor --human")} Alias for ${pc.cyan("doctor --site")}
${pc.cyan("doctor --json")} Print the report as JSON for CI, scripts, and automation
${pc.cyan("doctor agent")} Subcommand alias for agent scoring
${pc.cyan("doctor site")} Subcommand alias for reader-facing scoring
${pc.cyan("doctor human")} Legacy alias for reader-facing scoring
Expand Down
5 changes: 5 additions & 0 deletions skills/farming-labs/cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,7 @@ experience instead.
pnpm exec docs doctor
pnpm exec docs doctor --agent
pnpm exec docs doctor --site
pnpm exec docs doctor --agent --json
pnpm exec docs doctor agent
pnpm exec docs doctor site
pnpm exec docs doctor --agent --config docs.config.tsx
Expand Down Expand Up @@ -362,6 +363,10 @@ How to explain it:
GEO-friendly
- low `Explicit agent-friendly pages` does **not** mean pages are invisible to agents; it means
fewer pages have extra machine-only context through `agent.md` or `Agent` blocks
- `--json` is for CI, scripts, dashboards, and other agents that need structured output instead of
terminal formatting
- the JSON report itself is written to stdout; separate loader notices, such as config fallback
warnings, are outside the JSON payload

Useful checks:

Expand Down
43 changes: 43 additions & 0 deletions website/app/docs/cli/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,7 @@ Common forms:
pnpm exec docs doctor
pnpm exec docs doctor --agent
pnpm exec docs doctor --site
pnpm exec docs doctor --agent --json
pnpm exec docs doctor agent
pnpm exec docs doctor site
pnpm exec docs doctor --agent --config docs.config.tsx
Expand Down Expand Up @@ -395,6 +396,47 @@ Reader-facing runs use the same overall shape, but print `doctor — site` and s
- reader feedback
- reading-time cues

When you need structured output instead of terminal text, add `--json`:

```bash title="terminal"
pnpm exec docs doctor --agent --json
pnpm exec docs doctor --site --json
```

Sample shape:

```json title="~"
{
"mode": "agent",
"score": 87,
"maxScore": 100,
"grade": "Agent-ready",
"framework": "nextjs",
"entry": "docs",
"contentDir": "app/docs",
"coverage": {
"explicitPages": 10,
"totalPages": 41,
"explicitCoverage": 24
},
"checks": [
{
"id": "api-route",
"title": "Docs API route",
"status": "pass",
"score": 10,
"maxScore": 10
}
],
"recommendations": [
"Enable feedback.agent if you want agents to discover and post feedback through the shared docs API."
]
}
```

The JSON report itself is written to stdout. Separate loader notices, such as config fallback
warnings, are outside the JSON payload.

How to read it:

- the **score** is a quick summary, not the only thing that matters
Expand All @@ -410,6 +452,7 @@ When to use it:
- after changing `docs.config`, route wiring, `.well-known` endpoints, or MCP
- before opening a PR for docs infrastructure work
- in CI if you want a quick quality gate for the agent or human-facing surface
- when another agent, script, or dashboard needs structured results instead of terminal formatting

How essential it is:

Expand Down
Loading