Skip to content

test(infra) Barrels of Fun scraper-pipeline integration tests#45

Merged
jkeeley2073 merged 1 commit into
mainfrom
Dev-BofScraperTestInfra
May 3, 2026
Merged

test(infra) Barrels of Fun scraper-pipeline integration tests#45
jkeeley2073 merged 1 commit into
mainfrom
Dev-BofScraperTestInfra

Conversation

@jkeeley2073
Copy link
Copy Markdown
Contributor

Summary

Backfill of the PR #41 family-wide test-infra template for BofProductScraper. 5 new scraper-pipeline integration tests pinning yield order, provenance propagation, polite-gate routing invariants, and failure-isolation behavior.

BofProductScraper is a single-yield scraper (yields .Game only — no .Link items, since BoF's WooCommerce product pages host no firmware/document downloads), so the yield-order assertion is over .Game only. Otherwise the test class mirrors the CGC proof-of-concept exactly:

  • ScrapeAsync_HappyPath_YieldsGamesInIndexOrderWithProvenance — 3-machine category page + 3 product pages; asserts category-page order, full provenance fields (SourceType.BarrelsOfFunProductPage, DiscoveryContext = "Barrels of Fun Machines Category", DiscoveryUrl), GameRecord.Source.ScrapedFrom/GamePageUrl, DiscoveredOn carries the barrelsoffun_machines_category sentinel, and gate-vs-wire URL equality (so a future re-canonicalisation refactor can't silently throttle a different origin).
  • ScrapeAsync_PerPageFetchFailure_DoesNotAbortRun — middle product returns 500; siblings still yield; politeness invariants hold under failure (500 is reported back to the gate so the 429-streak detector can see it).
  • ScrapeAsync_DiscoveryFailure_AbortsThisSourceOnly/product-category/machines/ returns 500; scraper yield-breaks cleanly without throwing.
  • ScrapeAsync_PolitenessExceptionFromGate_PropagatesUp — gate throws on Acquire BEFORE any HTTP fires; assertions pin zero wire requests / zero reports so a regression that swallowed the exception would fail loudly.
  • ScrapeAsync_GateThrowsOnReport_BubblesUp — symmetric report-side path.

Touched files: 1 new test file + 1 CHANGELOG bullet (no production code changes).

Pre-push self-audit

Step 0 — /local-review: deferred to the human reviewer at merge time per the PR description (this is a backfill of an already-vetted template; the mechanical checks below are the binding gate).

Step 1 — 7-item mechanical checklist (all pass):

  1. Every option field is read — N/A (no new *Options).
  2. Sibling-diff for drift — diffed against CgcGamePageScraperTests. Differences are intentional and called out in remarks: single-yield (no .Link assertions), category-page path (/product-category/machines/) instead of menu, BoF source-type / discovery-context / sentinel strings, and the JSON-LD product schema in fixtures (instead of CGC's title / anchor extraction). Test structure, helper signatures, gate/wire assertions, and BuildScraper shape are byte-identical to the template.
  3. No bare catch { } — confirmed; no catches in test code.
  4. CLI / orchestrator wiring is end-to-end — N/A (no new ISourceScraper).
  5. Tests assert behavior, not just structure — yield-order test fixture has 3 machines in a deterministic order; failure-isolation fixture inserts the failing product in the middle so the assertion Equal(2, games.Count) distinguishes "kept going" from "aborted on first failure"; PolitenessException tests pin Empty(handler.Requests) so a regression that catches and swallows can't pass.
  6. Build is zero-warningdotnet build0 Warning(s) / 0 Error(s).
  7. Identity checkgit log -1 --format='%an <%ae>'Jim Keeley <94459922+jkeeley2073@users.noreply.github.com> (personal noreply).

Test counts: 430 → 435 (+5).

Test plan

  • dotnet build — clean, 0 warnings.
  • dotnet test --filter FullyQualifiedName~BofProductScraperTests — 5 passed.
  • Full suite — 435 passed / 0 failed / 0 skipped.

🤖 Generated with Claude Code

@jkeeley2073 jkeeley2073 added the claude-code Generated with Claude Code label May 3, 2026
Backfill of the PR #41 template across the family. 5 tests covering
happy-path with full provenance + gate-vs-wire URL equality, per-page
failure isolation, discovery failure, PolitenessException on both Acquire
and Report. Single-yield (.Game only) WooCommerce storefront via
/product-category/machines/.

Pre-push audit: 7-item mechanical (all pass). /local-review deferred to
human reviewer at merge time.
@jkeeley2073 jkeeley2073 force-pushed the Dev-BofScraperTestInfra branch from 6fb268c to f1ba40f Compare May 3, 2026 11:51
@jkeeley2073 jkeeley2073 merged commit 2cd757a into main May 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-code Generated with Claude Code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant