Skip to content

docs(agents): add Fail Loud, Never Fake error-handling rule#101

Open
don-petry wants to merge 6 commits intomainfrom
feat/agents-fail-loud
Open

docs(agents): add Fail Loud, Never Fake error-handling rule#101
don-petry wants to merge 6 commits intomainfrom
feat/agents-fail-loud

Conversation

@don-petry
Copy link
Copy Markdown
Contributor

@don-petry don-petry commented Apr 9, 2026

Summary

Adds a new Fail Loud, Never Fake subsection to AGENTS.md under Coding Standards & Principles, immediately after Defensive Coding at System Boundaries. The new rule codifies a long-standing failure mode: agents (and humans) hiding errors behind silent fallbacks, placeholder data, or false success reports — turning what should be a 5-minute stack-trace fix into a day lost chasing wrong data downstream.

The section establishes:

  • A priority ladder for failure handling: works → disclosed fallback → clear error → never silent
  • Forbidden patterns (bare except: swallowing, synthetic data substitution, false "I've set up X" reports, weakening tests to make them pass)
  • A clear carve-out that disclosed fallbacks are fine — the rule is against hidden fallbacks, not all fallbacks
  • Agentic directives mirroring the style of the existing Breaking Changes section

Why

This complements the existing Fail fast bullet, which covers invariant violations, by addressing the adjacent (and more insidious) problem of work that appears to succeed but didn't. The two rules reinforce each other.

Test plan

  • Markdown lint passes (.markdownlint-cli2.yaml)
  • Section renders correctly on github.com (heading hierarchy, tables, code formatting)
  • No broken cross-references — this section is self-contained and adds no new links

Summary by CodeRabbit

  • Documentation
    • Introduced a “Fail Loud, Never Fake” standard to development guidelines: require surfacing, logging, or propagating exceptions; forbid silent degradation, swallowing errors, undisclosed synthetic or hardcoded fallback data, and reporting success after errors; mandate observable fallbacks (response flags/fields, logs/metrics, or UI indicators); and require fixing failing tests rather than weakening them to hide failures.

Adds a new subsection under Coding Standards & Principles establishing
that visible failure is preferred over silent fallbacks. Codifies a
priority ladder (works → disclosed fallback → clear error → never
silent), forbidden patterns, and agentic directives forbidding swallowed
exceptions, fake/placeholder data substitution, and false success
reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 9, 2026 00:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new documentation rule to the agent/human contribution guidelines to prevent silent fallbacks and “fake success” outcomes, complementing the existing defensive coding and fail-fast principles.

Changes:

  • Introduces a “Fail Loud, Never Fake” subsection with a clear priority ladder for handling failures.
  • Documents explicitly forbidden error-handling patterns (swallowed exceptions, synthetic substitutions, false success reporting, test weakening).
  • Clarifies that disclosed fallbacks are acceptable when observability is maintained (logs/flags/metrics/UI).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 9, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 678cc495-54be-47b7-baab-e19e5d19f837

📥 Commits

Reviewing files that changed from the base of the PR and between f247b89 and 9dd9174.

📒 Files selected for processing (1)
  • AGENTS.md

📝 Walkthrough

Walkthrough

Adds a "Fail Loud, Never Fake" section to AGENTS.md defining ordered failure-handling preferences, enumerating forbidden failure-hiding patterns, and adding agent directives to surface/log/propagate exceptions and make any fallbacks observable.

Changes

Cohort / File(s) Summary
Error Handling Guidelines
AGENTS.md
Added "Fail Loud, Never Fake" section: explicit failure-handling priority (real correct data → observable degraded fallback → clear error → no silent degradation); enumerated forbidden behaviors (swallowed exceptions, synthetic/placeholder data, hardcoded success after errors, undisclosed fallback success, weakened tests); agentic directives requiring exceptions be surfaced/logged/propagated and fallbacks be observable.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title accurately describes the main change: adding a new 'Fail Loud, Never Fake' error-handling rule to the AGENTS.md documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/agents-fail-loud

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Two new lines exceeded the 200-char markdownlint limit. Wrap them at
sentence boundaries to match the style used in the surrounding sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@don-petry
Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 9, 2026

✅ Actions performed

Full review triggered.

@don-petry
Copy link
Copy Markdown
Contributor Author

Automated review — APPROVED

Risk: LOW
Reviewed commit: f247b896c91336b74de79c8b4b6434a1e5771e95
Cascade: triage → deep (see triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6 for models)

Summary

Documentation-only change adding a 'Fail Loud, Never Fake' error-handling guideline to AGENTS.md. All CI checks pass (Lint, ShellCheck, CodeQL, SonarCloud, AgentShield), SonarCloud reports zero issues and zero security hotspots, and CodeRabbit has already approved. The content is well-structured, internally consistent, and complementary to the existing 'Fail fast' and defensive-coding sections with no security, correctness, or maintainability concerns.

Findings

Info

  • AGENTS.md:288 — The 'Fail Loud, Never Fake' section is coherent and well-scoped: priority ladder, forbidden patterns, disclosed-fallback carve-out, and agentic directives all align with the surrounding AGENTS.md style. No broken cross-references; section is self-contained.
  • Triage escalated due to 'triage-output-invalid' (triage tier crashed), not because of actual risk signals. Deep review confirms no code, config, or security-sensitive files were modified.

CI status

All status checks passed: Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud (0 new issues, 0 security hotspots), CodeRabbit (APPROVED). No failures or warnings.


Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6). Reply with @don-petry if you need a human.

@don-petry don-petry enabled auto-merge (squash) April 16, 2026 12:11
@don-petry
Copy link
Copy Markdown
Contributor Author

Automated review — APPROVED

Risk: LOW
Reviewed commit: d82732af0cc1ab8142c365fdb6e39b4196fb5790
Cascade: triage → deep (see triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6 for models)

Note: Formal approval skipped — reviewer is the PR author. Verdict: APPROVE.

Summary

Documentation-only PR adding a 'Fail Loud, Never Fake' error-handling section to AGENTS.md. Triage escalated due to a triage-tier crash (triage-output-invalid), not any actual risk signal — the prior review at f247b89 already confirmed this is LOW risk. All CI checks pass (Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud with 0 issues/0 hotspots, CodeRabbit APPROVED), no code or config files modified, no new dependencies.

Findings

Info

  • [info] AGENTS.md:289 — 'Fail Loud, Never Fake' section is well-structured with a clear priority ladder (works correctly → visible fallback → clear error → never silent), explicit forbidden patterns, a carve-out for disclosed fallbacks, and five numbered agentic directives. Style is consistent with surrounding sections (e.g., Defensive Coding, Breaking Changes). Section is self-contained with no new cross-references to break.
  • [info] Triage escalated with signal 'triage-output-invalid' — the triage tier crashed rather than detecting a genuine risk. This deep review confirms no security, correctness, or maintainability concerns exist.
  • [info] All status checks passed at head SHA d82732a: Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL (actions), SonarCloud (0 new issues, 0 security hotspots), CodeRabbit (APPROVED). Dependency audit jobs skipped correctly (no dependency files changed).
  • [info] Only file changed: AGENTS.md (+32/-0 lines). No code, config, secrets, CI workflow, schema, or dependency files were modified. Risk classification LOW is appropriate.

CI status

All CI checks passed at head SHA d82732af0cc1ab8142c365fdb6e39b4196fb5790: Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud (0 new issues, 0 security hotspots), CodeRabbit (APPROVED).


Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6). Reply with @don-petry if you need a human.

@don-petry don-petry disabled auto-merge April 16, 2026 17:08
@don-petry don-petry enabled auto-merge (squash) April 16, 2026 17:09
@sonarqubecloud
Copy link
Copy Markdown

@don-petry
Copy link
Copy Markdown
Contributor Author

Automated review — APPROVED

Risk: LOW
Reviewed commit: 9dd91748e727a346dbeb7660e0291d923fd7b118
Cascade: triage → deep (see triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6 for models)

Summary

Documentation-only PR adding a 'Fail Loud, Never Fake' error-handling section to AGENTS.md (+32/-0). Triage escalated due to a triage-tier crash (triage-output-invalid), not any genuine risk signal. All CI checks pass (Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud with 0 new issues and 0 security hotspots, CodeRabbit APPROVED), no code/config/secrets/CI-workflow files were modified, and no new dependencies introduced.

Findings

Info

  • AGENTS.md:289 — New 'Fail Loud, Never Fake' section is well-structured: priority ladder (works correctly → visible fallback → clear error → never silent), explicit forbidden patterns, a disclosed-fallback carve-out, and five numbered agentic directives. Style and heading hierarchy are consistent with surrounding AGENTS.md sections (Defensive Coding, Breaking Changes). Section is self-contained with no new cross-references.
  • (triage-signal) Triage escalated with signal 'triage-output-invalid' — the triage tier crashed rather than detecting a real risk. This deep review (and the two prior deep reviews at f247b89 and d82732a) confirm no security, correctness, or maintainability concerns exist. Head SHA 9dd9174 is a merge-from-main commit with no substantive diff beyond the already-reviewed documentation.

CI status

All status checks passed at head SHA 9dd91748e727a346dbeb7660e0291d923fd7b118: Lint (SUCCESS), ShellCheck (SUCCESS), AgentShield (SUCCESS), Agent Security Scan (SUCCESS), CodeQL/actions (SUCCESS), SonarCloud (SUCCESS, 0 new issues, 0 security hotspots), CodeRabbit (APPROVED). Dependency audit jobs correctly skipped — no dependency files changed.


Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6). Reply with @don-petry if you need a human.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants