docs(tests) document Stern Playwright asymmetry + add pinning test#70
Merged
Conversation
Phase 2 § Scope item 8 (Wave 2) — route (ii). Resolves the asymmetry between the family-wide HttpMessageHandler-based scraper-pipeline test infrastructure (covers 8 of 10 ISourceScrapers) and the two Stern Playwright scrapers (GamePageScraper, ServiceBulletinScraper) that drive a real browser and don't go through HttpClient. What ships: - tests/PinballWizard.Scraper.Tests/README.md — describes the test project layout, the family-wide scraper-pipeline integration test infrastructure (FakePolitenessGate + QueueingHttpMessageHandler), the proven 5-test template from CgcGamePageScraperTests, the 8 scrapers covered, the Stern Playwright asymmetry section (deliberately not covered + why HttpMessageHandler can't intercept Playwright + what coverage exists instead), revisit criteria (three concrete triggers including "≥ 2 manufacturers adopt Playwright" and "build cost of fake-Playwright fixture drops below ~4 hours"), and engineering conventions. - tests/PinballWizard.Scraper.Tests/Scraping/Stern/SternPlaywrightAsymmetryDocumentationTests.cs — single Fact pinning test (Stern_Playwright_Pipeline_Test_Asymmetry_IsAcknowledged) that asserts the README exists at the expected path and contains the load-bearing markers of the asymmetry section. If the README is deleted or the section is gutted, the test fails — forcing a re-read or a real fix (build a Playwright-route fixture and replace the asymmetry with actual coverage). - docs/guardrails.md R4 — updates the risk row to "Resolved (route ii)" and points at the new README + pinning test, so a search from the guardrails risk register lands on the documentation. Local review summary: 0 🔴 / 4⚠️ / 6 categories ✅.⚠️ — Fixed: - "Anchor on a unique phrase" — added an Assert.Contains for "deliberately not covered" (a phrase that appears only in the asymmetry section, catches the gutted-section false-pass case). - "Cross-reference from guardrails" — R4 row now points at the README + pinning test by name.⚠️ — Deferred: - "Helper drift" (FindRepoRoot duplicated across two test files): defer per the in-file comment's "extract on third consumer" rule. N=2 is below the extraction threshold; ~12 lines of stable filesystem-walk code; both copies are textually identical. - "Sharpen the 'behavior change' revisit trigger to a count": the other two triggers in the Revisit criteria section are already objective (manufacturer count, build-cost dollar threshold). Defer pending operational signal that the subjective trigger has bitten.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| public void Stern_Playwright_Pipeline_Test_Asymmetry_IsAcknowledged() | ||
| { | ||
| var repoRoot = FindRepoRoot(); | ||
| var readmePath = Path.Combine(repoRoot, "tests", "PinballWizard.Scraper.Tests", "README.md"); |
| // Mirrors the helper in IngestionSourceSeederTests; if a third | ||
| // consumer appears, extract to Scraping/_TestInfra/RepoPaths.cs. | ||
| var dir = new DirectoryInfo(AppContext.BaseDirectory); | ||
| while (dir is not null && !File.Exists(Path.Combine(dir.FullName, "PinballWizard.slnx"))) |
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes
docs/build-spec.mdPhase 2 § Scope item 8 (Wave 2) via route (ii).The family-wide scraper-pipeline integration test infrastructure (
FakePolitenessGate+QueueingHttpMessageHandler) wires at the typed-HttpClientlayer and covers 8 of 10ISourceScraperimplementations. The two remaining ones — Stern'sGamePageScraperandServiceBulletinScraper— drive a real Chromium browser via Playwright'sIBrowserContext/IPageAPIs and never callHttpClient.GetAsync, so the shared infra cannot intercept their page-load traffic.This PR resolves the asymmetry by documenting it permanently plus adding a pinning test that fails if the documentation is removed.
What ships
tests/PinballWizard.Scraper.Tests/README.md— first README for the test project. Describes the test-project layout, the family-wide infra and the proven 5-test template, the 8 covered scrapers, the Stern Playwright asymmetry section (why the template doesn't fit, what coverage exists instead, and three concrete revisit triggers), and engineering conventions for new tests.tests/PinballWizard.Scraper.Tests/Scraping/Stern/SternPlaywrightAsymmetryDocumentationTests.cs— singleFactpinning test (Stern_Playwright_Pipeline_Test_Asymmetry_IsAcknowledged, the identifier mandated bybuild-spec.md§ Scope item 8). Asserts the README exists at the expected path and contains six load-bearing markers including the unique phrase"deliberately not covered"(catches the gutted-section false-pass case).docs/guardrails.mdR4 — risk row updated from "Known" to "Resolved (route ii)" and now points at the README + pinning test. A search from the guardrails risk register lands on the documentation.Test Plan
dotnet test PinballWizard.slnx --nologo→ 516 / 516 passing (was 515; +1 pinning test)dotnet build PinballWizard.slnx --nologo→ clean, zero warningsLocal review
/local-reviewoutcome: 0 🔴 / 4Assert.Contains("deliberately not covered", ...), anchoring the pinning to a phrase that appears only inside the asymmetry justification.FindRepoRoothelper duplicated across two test files (this one andIngestionSourceSeederTests). The in-file comment sets a "extract on third consumer" rule (N=2 is below the extraction threshold; ~12 lines of stable filesystem-walk code; copies are textually identical).7-item self-audit
IngestionSourceSeederTests.cs(also has aFindRepoRoothelper); helpers are textually identical, drift acknowledged in the new file's comment.catch { }—FindRepoRootthrowsInvalidOperationExceptionwith a clear message; no swallowed exceptions.git log -1 --format='%an <%ae>'→Jim Keeley <94459922+jkeeley2073@users.noreply.github.com>✅Out of Scope
FakePlaywrightContext). Route (i) of Phase 2 § Scope item 8; deferred per the README's revisit criteria. Build it when ≥ 2 manufacturers adopt Playwright OR an official Microsoft.Playwright testing helper lands.--dry-run, item 7 round 2 clean Dependabot bumps) — separate PRs.🤖 Generated with Claude Code