[tests] Structure agent-driven PHPUnit JSON results

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Keep structured DevTools command output parseable and agent-friendly by exposing the PHPUnit agent-reporter payload under nested `output`, suppressing intermediary progress logs in JSON modes, and normalizing orchestrated subprocess payloads before the final JSON envelope is emitted (#248)
 - Register ``ergebnis/phpunit-agent-reporter`` in the packaged ``phpunit.xml`` so AI agents receive compact PHPUnit JSON summaries without changing consumer overrides manually (#327)
 - Let `wiki`, `docs`, `tests`, `metrics`, and `reports` skip gracefully for guide-only repositories while keeping wiki/report workflows and published preview links aligned with the artifacts that were actually generated (#325)
 

README.md

@@ -35,6 +35,8 @@ across Fast Forward libraries.
 - Synchronizes packaged skills and project-agent prompts into consumer
   `.agents/skills` and `.agents/agents` directories using safe link-based
   updates
+- Supports guide-only and automation-only repositories by skipping PHPUnit or
+  wiki generation gracefully when no runnable PHP surface exists
 - Works both as a Composer plugin and as a local binary
 - Preserves local overrides through consumer-first configuration resolution
 
@@ -71,6 +73,8 @@ You can also run individual commands for specific development tasks:
 ```bash
 # Run PHPUnit tests
 composer dev-tools tests
+composer dev-tools tests --json
+composer dev-tools tests --pretty-json
 
 # Analyze missing, unused, misplaced, and outdated Composer dependencies
 composer dependencies
@@ -120,6 +124,7 @@ composer phpdoc
 
 # Generate HTML API documentation using phpDocumentor
 composer docs
+composer docs --source=docs/user-guide
 
 # Generate Markdown documentation for the wiki
 composer wiki
@@ -180,6 +185,14 @@ The `metrics` command ships with `phpmetrics/phpmetrics` as a direct
 dependency of `fast-forward/dev-tools`, so consumer repositories can generate
 metrics reports without extra setup.
 
+Guide-only repositories and workflow-only repositories can still use the
+packaged command surface. When no runnable PHPUnit surface exists, `tests`
+returns a controlled warning instead of failing. When a repository ships
+guides without PSR-4 source paths, `docs` builds the guide site without trying
+to synthesize API pages. When `.github/wiki` is absent and no wiki has been
+initialized yet, `wiki` now skips generation with a warning instead of failing
+the whole automation run.
+
 The changelog commands manage Keep a Changelog 1.1.0 files without requiring
 extra tooling in the consumer repository. `changelog:entry` bootstraps a
 missing changelog file on first use, `changelog:check` enforces meaningful
@@ -209,16 +222,35 @@ automatically when the runtime is detected as agent-driven. For
 prints raw values so release workflows can keep capturing semantic versions
 and piping rendered release notes directly into GitHub releases.
 
+`--pretty-json` intentionally remains valid JSON. DevTools does not inject ANSI
+escape sequences into that mode today because preserving a parseable payload
+takes precedence over terminal-only color. Where orchestrated tools can expose
+structured subprocess results safely, DevTools prefers adding stable fields to
+the JSON context rather than coloring otherwise strict JSON output.
+
+For the `tests` command, structured runs now capture the bundled PHPUnit
+agent-reporter payload inside `context.output` while preserving the normal
+DevTools JSON envelope. That means the top-level command still emits one final
+document, and consumers can inspect stable nested keys such as
+`context.output.result`, `context.output.summary`, optional
+`context.output.details`, and `context.output.coverage` when minimum-coverage
+validation is active.
+
 Progress output is disabled by default on the commands that support transient
 rendering, and `--progress` re-enables it for human-readable terminal runs.
 When `--json` or `--pretty-json` is active on commands that orchestrate other
 tools, DevTools keeps progress suppressed, forwards JSON flags where the
 underlying tool supports structured output, and otherwise falls back to
-quieter subprocess modes so the captured payload stays machine-readable. In
-GitHub Actions, queued subprocess output is grouped into collapsible sections,
-and logged failures emit native workflow error annotations, including file and
-line metadata when commands provide it. The packaged tests, reports, wiki, and
-changelog workflows also append concise Markdown outcomes to
+quieter subprocess modes so the captured payload stays machine-readable. The
+`tests` command now captures the bundled PHPUnit agent-reporter payload in
+structured runs and stores it in `context.output`, preserving `result`,
+`summary`, optional `details`, and a `raw_output` fallback when coverage or
+other PHPUnit text is emitted before the final reporter JSON. In GitHub
+Actions, queued subprocess output is
+grouped into collapsible sections, and logged failures emit native workflow
+error annotations, including file and line metadata when commands provide it.
+The packaged tests, reports, wiki, and changelog workflows also append concise
+Markdown outcomes to
 `GITHUB_STEP_SUMMARY` so maintainers can scan versions, URLs, preview refs,
 verification status, and release results without expanding full logs. This
 repository also keeps a bounded retry workflow that reruns failed jobs once
@@ -324,8 +356,8 @@ authoring.
 - `Composer Plugin` - `FastForward\DevTools\Composer\Plugin` exposes the
   packaged command set to Composer and runs `dev-tools:sync` after install and
   update.
-- `DevTools Container` - `FastForward\DevTools\Console\DevTools::create()`
-  builds a shared container from `DevToolsServiceProvider`, which wires
+- `DevTools Container` - `FastForward\DevTools\Container\ContainerFactory::get(FastForward\DevTools\Console\DevTools::class)`
+  resolves the shared application from `DevToolsServiceProvider`, which wires
   process execution, filesystem access, changelog services, Git helpers,
   diffing, reporting, and template loading.
 - `Generic Link Synchronization` -

bin/dev-tools.php

@@ -20,10 +20,18 @@
 namespace FastForward\DevTools;
 
 use FastForward\DevTools\Console\DevTools;
+use FastForward\DevTools\Container\ContainerFactory;
 
-$projectVendorAutoload = \dirname(__DIR__, 4) . '/vendor/autoload.php';
-$pluginVendorAutoload = \dirname(__DIR__) . '/vendor/autoload.php';
+$autoloadCandidates = [\dirname(__DIR__, 4) . '/vendor/autoload.php', \dirname(__DIR__) . '/vendor/autoload.php'];
 
-require_once file_exists($projectVendorAutoload) ? $projectVendorAutoload : $pluginVendorAutoload;
+foreach ($autoloadCandidates as $autoloadCandidate) {
+    if (is_file($autoloadCandidate)) {
+        require_once $autoloadCandidate;
 
-DevTools::create()->run();
+        exit(ContainerFactory::get(DevTools::class)->run());
+    }
+}
+
+fprintf(\STDERR, "Could not locate Composer autoload.php for fast-forward/dev-tools.\n");
+
+exit(1);

composer.json

@@ -44,8 +44,8 @@
         "container-interop/service-provider": "^0.4.1",
         "dg/bypass-finals": "^1.9",
         "ergebnis/agent-detector": "^1.1",
-        "ergebnis/phpunit-agent-reporter": "^0.3",
         "ergebnis/composer-normalize": "^2.51",
+        "ergebnis/phpunit-agent-reporter": "^0.3",
         "ergebnis/rector-rules": "^1.18",
         "fakerphp/faker": "^1.24",
         "fast-forward/phpdoc-bootstrap-template": "^2.0",

docs/advanced/phpunit-extension.rst

@@ -19,6 +19,9 @@ Runtime Chain
    builds a summary notification and sends it when the run finishes.
 5. ``Ergebnis\PHPUnit\AgentReporter\Extension`` replaces PHPUnit's normal
    output with a compact JSON report when an agent runtime is detected.
+6. In structured DevTools runs, ``tests`` forces the same reporter path for
+   the PHPUnit subprocess so the final nested payload remains deterministic
+   even when the surrounding process would not naturally look agent-driven.
 
 Why This Helps Consumer Projects
 --------------------------------
@@ -29,6 +32,8 @@ Why This Helps Consumer Projects
   scrollback;
 - agent-driven runs consume far less terminal context while still keeping
   failure details and PHPUnit exit semantics intact;
+- DevTools can preserve a single top-level JSON document while nesting the
+  compact PHPUnit summary under ``context.output`` for bot-friendly parsing;
 - event counts are available to the notification layer without adding ad-hoc
   test code.
 

docs/api/composer-integration.rst

@@ -12,8 +12,9 @@ Startup Chain
    ``vendor/autoload.php`` and falls back to the package autoloader.
 3. ``bin/dev-tools.php`` starts ``FastForward\DevTools\Console\DevTools`` and appends
    ``--no-plugins``.
-4. ``FastForward\DevTools\Console\DevTools::create()`` builds the shared
-   container from ``FastForward\DevTools\ServiceProvider\DevToolsServiceProvider``.
+4. ``FastForward\DevTools\Container\ContainerFactory::get(FastForward\DevTools\Console\DevTools::class)``
+   resolves the shared application container from
+   ``FastForward\DevTools\ServiceProvider\DevToolsServiceProvider``.
 5. ``FastForward\DevTools\Console\CommandLoader\DevToolsCommandLoader``
    lazily discovers ``#[AsCommand]`` classes from the command namespace.
 6. ``FastForward\DevTools\Composer\Capability\DevToolsCommandProvider`` later

docs/commands/docs.rst

@@ -103,9 +103,14 @@ Exit Codes
 Behavior
 ---------
 
-- ``docs/`` must exist unless you pass another ``--source`` directory.
+- The default ``docs`` source directory is optional. When it is absent, the
+  command still generates API pages if PSR-4 source paths are available.
+- A custom explicit ``--source`` path must exist; otherwise the command fails
+  fast.
 - API pages are built from the PSR-4 paths declared in ``composer.json``.
 - Guide pages are built from the selected source directory.
+- Repositories without PSR-4 source paths can still generate a guides-only
+  site from the selected source directory.
 - Cache stays enabled by default; omit both flags to keep the command default,
   pass ``--cache`` to force it on, and pass ``--no-cache`` to force it off.
 - When ``--cache-dir`` is omitted, phpDocumentor keeps its default cache

docs/commands/reports.rst

@@ -123,6 +123,9 @@ Behavior
 - When ``--json`` or ``--pretty-json`` is active, it forwards JSON mode to the
   ``docs``, ``tests``, and ``metrics`` subprocesses and suppresses transient
   progress output where those tools support it.
+- Nested ``docs`` and ``tests`` stages can now skip gracefully with warnings in
+  guide-only or automation-only repositories, while ``reports`` still returns a
+  successful aggregate result for the stages that were actually generated.
 - Passes ``--junit <coverage>/junit.xml`` to the metrics step.
 - Used by the ``standards`` command as the final phase.
 - This is the reporting stage used by GitHub Pages.

docs/commands/tests.rst

@@ -71,7 +71,8 @@ Options
 ``--pretty-json``
    Emit the same structured payload with indentation for terminal inspection.
    This also suppresses PHPUnit progress output automatically so the JSON
-   payload is not polluted by transient progress rendering.
+   payload is not polluted by transient progress rendering. The output remains
+   valid JSON and intentionally does not include ANSI color escapes.
 
 Examples
 --------
@@ -112,6 +113,13 @@ Run with minimum coverage enforcement:
 
    composer tests --min-coverage=80
 
+Run with structured JSON output:
+
+.. code-block:: bash
+
+   composer tests --json
+   composer tests --pretty-json
+
 Run without cache:
 
 .. code-block:: bash
@@ -149,6 +157,10 @@ Behavior
 ---------
 
 - Local ``phpunit.xml`` is preferred over the packaged default.
+- When the default ``tests`` path is absent and the project exposes no
+  testable PHP source files, the command exits successfully with a warning
+  instead of failing the whole automation flow.
+- A custom explicit tests path that does not exist still fails fast.
 - Coverage filters are automatically applied to all PSR-4 paths from composer.json.
 - Multiple coverage formats are generated: HTML, Testdox HTML, Clover XML, and PHP.
 - Cache stays enabled by default; omit both flags to keep the command default,
@@ -160,5 +172,25 @@ Behavior
 - progress output is disabled by default.
 - ``--json`` and ``--pretty-json`` keep progress output disabled so the
   structured payload stays clean, even when ``--progress`` is provided.
+- in agent-driven runs outside the Composer test suite, the command also
+  switches to the same structured capture mode automatically.
+- when structured capture is active and PHPUnit emits agent-reporter JSON, the
+  command stores that payload inside ``output`` while keeping the standard
+  DevTools JSON envelope. ``--json`` and ``--pretty-json`` therefore expose
+  the same structured result, with formatting as the only difference.
+- the command forces the bundled PHPUnit agent-reporter extension for
+  structured runs so the nested payload stays compact and stable even when the
+  surrounding runtime would not normally look like an agent.
+- in structured mode, the command suppresses intermediary ``Running...`` log
+  records so the output stream contains a single final JSON document.
+- when structured capture is active but PHPUnit does not emit parseable JSON,
+  the command preserves the raw subprocess text inside ``output.raw_output``
+  instead of dropping it.
+- when coverage generation or other PHPUnit text appears before the final
+  reporter payload, the command preserves that prelude in
+  ``output.raw_output`` while keeping the main JSON result parseable.
+- when ``--min-coverage`` is used in structured mode, the command appends a
+  ``coverage`` object under ``output`` and flips ``output.result`` to
+  ``failure`` if the threshold is not met.
 - The command fails if minimum coverage is not met (when ``--min-coverage`` is set).
 - The packaged configuration registers the DevTools PHPUnit extension.

docs/commands/wiki.rst

@@ -90,6 +90,10 @@ Behavior
 ---------
 
 - Default output directory is ``.github/wiki``.
+- When the default target does not exist yet and ``--init`` is not requested,
+  the command skips with a controlled warning instead of failing.
+- Repositories without PHP API surface also skip wiki generation with a
+  warning, because the current wiki renderer only emits API pages.
 - Cache stays enabled by default; omit both flags to keep the command default,
   pass ``--cache`` to force it on, and pass ``--no-cache`` to force it off.
 - When ``--cache-dir`` is omitted, phpDocumentor keeps its default cache

docs/getting-started/quickstart.rst

@@ -4,16 +4,16 @@ Quickstart
 This walkthrough is the fastest way to get a new library into a healthy state.
 
 1. Install the package.
-2. Create a minimal guide directory.
+2. Create a guide directory if the repository will publish guides.
 3. Synchronize shared automation, packaged skills, and packaged agents.
 4. Run the focused commands once.
 5. Run the full suite before opening a pull request.
 
-Create the Minimum Guide
-------------------------
+Optional Guide Setup
+--------------------
 
-The ``docs`` command fails early when ``docs/`` does not exist. A tiny
-starting page is enough for the first successful run.
+If the repository will publish guides, a tiny starting page is enough for the
+first successful ``docs`` run.
 
 Create the directory:
 
@@ -30,10 +30,14 @@ Create ``docs/index.rst`` with content such as:
 
    Welcome to the project documentation.
 
+Repositories that only ship PHP code can skip this step and still generate API
+documentation. Repositories that only ship guides can also use the same
+``docs`` command even without PSR-4 source paths.
+
 Run the First Commands
 ----------------------
 
-Once the package is installed and the guide directory exists, run:
+Once the package is installed, run:
 
 .. code-block:: bash
 
@@ -57,9 +61,11 @@ What Each Command Proves
   safely into ``.agents/agents`` without copying files into the consumer
   repository.
 - ``composer tests`` proves the packaged or local PHPUnit
-  configuration can execute the current test suite.
+  configuration can execute the current test suite, or skip gracefully when
+  the repository intentionally has no runnable PHPUnit surface yet.
 - ``composer docs`` proves the PSR-4 source paths and the guide
-  directory are usable by phpDocumentor.
+  directory are usable by phpDocumentor, whichever of those surfaces the
+  repository actually provides.
 - ``composer dev-tools`` proves the complete pipeline can run in the expected
   order.
 

docs/internals/architecture.rst

@@ -10,8 +10,9 @@ Local Command Lifecycle
 1. ``bin/dev-tools`` loads ``bin/dev-tools.php``.
 2. ``bin/dev-tools.php`` prefers the consumer ``vendor/autoload.php`` and
    falls back to the package autoloader.
-3. ``FastForward\DevTools\Console\DevTools::create()`` builds a shared
-   container from ``FastForward\DevTools\ServiceProvider\DevToolsServiceProvider``.
+3. ``FastForward\DevTools\Container\ContainerFactory::get(FastForward\DevTools\Console\DevTools::class)``
+   resolves the shared application container from
+   ``FastForward\DevTools\ServiceProvider\DevToolsServiceProvider``.
 4. ``FastForward\DevTools\Console\CommandLoader\DevToolsCommandLoader``
    lazily discovers ``#[AsCommand]`` classes and resolves them from that
    container.
@@ -61,7 +62,7 @@ Dependency Injection
 --------------------
 
 ``DevToolsServiceProvider`` builds the shared application container used by
-``DevTools::create()``. Most commands receive collaborators through
+``ContainerFactory::get(DevTools::class)``. Most commands receive collaborators through
 constructor injection once resolved by that container, while command discovery
 itself stays lazy through ``DevToolsCommandLoader``.
 

docs/running/specialized-commands.rst

@@ -63,6 +63,12 @@ Important details:
 - ``--progress`` re-enables PHPUnit progress output in text mode;
 - ``--json`` and ``--pretty-json`` still suppress PHPUnit progress output
   automatically;
+- ``--pretty-json`` stays valid JSON and does not add ANSI color escapes;
+- in agent-driven runs, the command also captures PHPUnit output in structured
+  mode automatically and stores the bundled PHPUnit agent-reporter payload in
+  ``output``. ``--json`` and ``--pretty-json`` therefore keep the same
+  structured shape, with ``raw_output`` preserved under ``output`` when
+  PHPUnit writes extra text before the final JSON;
 - the packaged configuration registers the DevTools PHPUnit extension.
 
 ``dependencies``

docs/usage/testing-and-coverage.rst

@@ -17,6 +17,8 @@ When you run ``tests``, DevTools:
   explicit force-off flag for the current run;
 - uses the selected workspace ``cache/phpunit`` directory only when caching
   stays enabled;
+- skips the command with a warning when the repository has no default
+  ``tests`` directory and no testable PHP source surface;
 - can generate HTML coverage, Testdox, Clover, and raw PHP coverage output
   when ``--coverage`` is provided.
 
@@ -29,6 +31,8 @@ Useful Examples
    composer tests -- --filter=PluginTest
    composer tests --coverage=.dev-tools/coverage
    composer tests --no-cache --bootstrap=tests/bootstrap.php
+   composer tests --json
+   composer tests --pretty-json
 
 Coverage Outputs
 ----------------
@@ -59,6 +63,10 @@ The packaged ``phpunit.xml`` registers:
   default verbose terminal output with a compact JSON summary when an agent
   runtime is detected.
 
+When ``tests`` itself runs in structured mode, DevTools also forces the
+reporter path for the PHPUnit subprocess so the nested payload remains compact
+and parseable for bots and agent workflows.
+
 Programmatic Coverage Access
 -----------------------------
 
@@ -83,7 +91,10 @@ When to Override Locally
 
 Create your own ``phpunit.xml`` in the consumer project when you need a
 different bootstrap file, extra extensions, or alternative strictness flags.
-DevTools will prefer the local file automatically.
+DevTools will prefer the local file automatically, but consumer projects that
+replace the packaged configuration must re-register both bundled extensions if
+they still want desktop notifications, BypassFinals support, and compact
+agent-oriented JSON output.
 
 .. note::