Skip to content

test(parsers): annotate every test with its CASE.* in TEST_CASES.md#8729

Closed
keivenchang wants to merge 3 commits into
mainfrom
keivenchang/DIS-1842__parser-test-taxonomy_index
Closed

test(parsers): annotate every test with its CASE.* in TEST_CASES.md#8729
keivenchang wants to merge 3 commits into
mainfrom
keivenchang/DIS-1842__parser-test-taxonomy_index

Conversation

@keivenchang
Copy link
Copy Markdown
Contributor

@keivenchang keivenchang commented Apr 25, 2026

Closed — duplicate of #8725. See #8725 for the canonical taxonomy-annotation PR.

Cross-index existing inline parser tests with the universal test-case
taxonomy in lib/parsers/TESTING.md. Each test gets a one-line trailer
on its #[test] / #[rstest] attribute (e.g. `#[test] // CASE.5, CASE.16`)
so a reader of TESTING.md can grep `CASE.5` and find every test that
exercises missing-end-token recovery, across every parser.

Coverage:
- 232 inline tests annotated across 9 tool-call parsers + 5 reasoning
  parsers (lib/parsers/src/tool_calling/ + lib/parsers/src/reasoning/).
- 5 new categories added to TESTING.md (CASE.20-24) for tests that
  didn't fit the existing 19 entries: detection helpers, function-name
  conventions, whitespace tolerance, token-format variants, empty wrappers.
- Multi-category tests carry all relevant CASEs on one line.

No code change, just comments + TESTING.md edits. 389 existing tests
still pass.

Deferred to follow-up: annotating tests under lib/llm/tests/ (integration
suites like tool_choice.rs, tool_choice_finish_reasons.rs,
test_streaming_tool_parsers.rs).

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Annotated tests in the parser sources are the index now (one comment per
test). TESTING.md should describe what each CASE covers, not duplicate
that index. Removed all Ex: ... / Example: ... lines.

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
@keivenchang keivenchang requested a review from a team as a code owner April 25, 2026 16:59
@keivenchang keivenchang self-assigned this Apr 25, 2026
@github-actions github-actions Bot added test documentation Improvements or additions to documentation labels Apr 25, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 25, 2026

Walkthrough

Test functions across parser modules are annotated with inline case identifiers (e.g., // CASE.10), and documentation references updated from TESTING.md to TEST_CASES.md. Test case taxonomy expanded with new categories (CASE.20CASE.24). No functional logic or public APIs modified.

Changes

Cohort / File(s) Summary
Documentation
lib/parsers/README.md, lib/parsers/TEST_CASES.md
Updated documentation file references from TESTING.md to TEST_CASES.md; expanded test case taxonomy with new parser-internal categories and stripped inline examples from existing case definitions.
Reasoning Parsers
lib/parsers/src/reasoning/base_parser.rs, lib/parsers/src/reasoning/gpt_oss_parser.rs, lib/parsers/src/reasoning/granite_parser.rs, lib/parsers/src/reasoning/minimax_append_think_parser.rs, lib/parsers/src/reasoning/mod.rs
Added inline // CASE.* comment annotations to test attributes mapping each test to corresponding case identifiers.
Tool Calling - DSML
lib/parsers/src/tool_calling/dsml/parser.rs
Added inline // CASE.* annotations to test attributes and updated coverage reference to TEST_CASES.md.
Tool Calling - Harmony
lib/parsers/src/tool_calling/harmony/harmony_parser.rs
Added inline // CASE.* annotations to #[tokio::test] and #[test] attributes.
Tool Calling - JSON
lib/parsers/src/tool_calling/json/base_json_parser.rs, lib/parsers/src/tool_calling/json/deepseek_v3_1_parser.rs, lib/parsers/src/tool_calling/json/deepseek_v3_parser.rs, lib/parsers/src/tool_calling/json/mod.rs
Added inline // CASE.* annotations to test attributes for detector and parser test functions.
Tool Calling - Pythonic
lib/parsers/src/tool_calling/pythonic/pythonic_parser.rs
Added inline // CASE.* annotations to test attributes across multiple test functions.
Tool Calling - XML
lib/parsers/src/tool_calling/xml/glm47_parser.rs, lib/parsers/src/tool_calling/xml/kimi_k2_parser.rs, lib/parsers/src/tool_calling/xml/parser.rs
Added inline // CASE.* annotations to #[test] and #[rstest] attributes mapping tests to case identifiers.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 6.15% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the main change: annotating every parser test with its corresponding CASE identifier in TEST_CASES.md.
Description check ✅ Passed The description comprehensively covers all required template sections with clear detail on changes, structure, coverage distribution, and review guidance.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/parsers/src/tool_calling/dsml/parser.rs (1)

324-327: ⚠️ Potential issue | 🟡 Minor

Replace internal Linear ticket reference in source comments.

pre-DIS-1765 introduces an internal ticket ID in added Rust lines. Please replace it with a public GitHub issue/PR reference (or neutral wording) to keep source comments externally safe and compliant. As per coding guidelines, "**/*.{py,rs,go,ts,tsx,js,mjs,cjs,sh,yaml,yml,toml}: Flag internal Linear ticket references (e.g. DIS-XXXX, DYN-XXXX, OPS-XXXX, DEP-XXXX, or any -NNNN ID) in the added lines; see .ai/linear-ticket-refs.md for what to recommend."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/parsers/src/tool_calling/dsml/parser.rs` around lines 324 - 327, The
source comment in parser.rs contains an internal Linear ticket reference
"pre-DIS-1765" inside the CASE.5 comment; replace that token with either a
public GitHub issue/PR URL or neutral wording (e.g., "prior internal ticket" or
"pre-release issue") so no internal IDs remain. Update the comment text near the
CASE.5 block (the lines mentioning "Fix mid-stream truncation" and
"pre-DIS-1765") to use the neutral phrasing or a public reference, keeping the
rest of the comment intact and preserving the recovery pattern description and
surrounding wording.
🧹 Nitpick comments (2)
lib/parsers/TEST_CASES.md (2)

46-52: Clarify applicability of CASE.20CASE.24 in summary sections.

These new categories are well-defined, but their required/optional scope is not reflected in the existing “Category layout” and “Applicability summary,” which may create ambiguity for new parser authors. A short summary row or note would close the loop.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/parsers/TEST_CASES.md` around lines 46 - 52, Add a short, explicit
applicability note for CASE.20–CASE.24 to the “Category layout” and
“Applicability summary” sections: state which categories are required vs
optional for parsers (e.g., CASE.20 detection helpers required for streaming
entry points like detect_tool_call_start_* / find_tool_call_end_position_*,
CASE.21–CASE.23 rules optional but recommended, and CASE.24
must-be-handled/edge-case behavior), and include a single-row summary or brief
bulleted note that maps CASE.20–CASE.24 to their scope (parser-internal vs
full-pipeline) and relationship to CASE.8 so new author readers can immediately
see applicability.

35-35: Prefer explicit GitHub issue/PR references over generic “ticket ID”.

To align with the markdown guideline, this line should explicitly require a GitHub issue/PR reference (and optionally Linear as secondary context when paired).

Suggested wording
-- **`CASE.16`** Regression for a specific customer bug (ticket ID referenced in test name or body).
+- **`CASE.16`** Regression for a specific customer bug (GitHub issue/PR referenced in test name or body; optional Linear ID only when paired with the matching GitHub link).

As per coding guidelines "**/*.md: prefer to reference GitHub issues instead of Linear tickets, but may include Linear ticket references for internal context when a matching GitHub link also exists."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/parsers/TEST_CASES.md` at line 35, Update the TEST_CASES.md entry for
"CASE.16" so it requires an explicit GitHub issue or PR link instead of a
generic "ticket ID": locate the line containing "**`CASE.16`** Regression for a
specific customer bug (ticket ID referenced in test name or body)" and change
the wording to demand a GitHub issue/PR reference (you may allow an optional
Linear ticket for internal context only when paired with the GitHub link);
ensure the new text clearly instructs contributors to include the GitHub URL in
the test name or body.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@lib/parsers/src/tool_calling/dsml/parser.rs`:
- Around line 324-327: The source comment in parser.rs contains an internal
Linear ticket reference "pre-DIS-1765" inside the CASE.5 comment; replace that
token with either a public GitHub issue/PR URL or neutral wording (e.g., "prior
internal ticket" or "pre-release issue") so no internal IDs remain. Update the
comment text near the CASE.5 block (the lines mentioning "Fix mid-stream
truncation" and "pre-DIS-1765") to use the neutral phrasing or a public
reference, keeping the rest of the comment intact and preserving the recovery
pattern description and surrounding wording.

---

Nitpick comments:
In `@lib/parsers/TEST_CASES.md`:
- Around line 46-52: Add a short, explicit applicability note for
CASE.20–CASE.24 to the “Category layout” and “Applicability summary” sections:
state which categories are required vs optional for parsers (e.g., CASE.20
detection helpers required for streaming entry points like
detect_tool_call_start_* / find_tool_call_end_position_*, CASE.21–CASE.23 rules
optional but recommended, and CASE.24 must-be-handled/edge-case behavior), and
include a single-row summary or brief bulleted note that maps CASE.20–CASE.24 to
their scope (parser-internal vs full-pipeline) and relationship to CASE.8 so new
author readers can immediately see applicability.
- Line 35: Update the TEST_CASES.md entry for "CASE.16" so it requires an
explicit GitHub issue or PR link instead of a generic "ticket ID": locate the
line containing "**`CASE.16`** Regression for a specific customer bug (ticket ID
referenced in test name or body)" and change the wording to demand a GitHub
issue/PR reference (you may allow an optional Linear ticket for internal context
only when paired with the GitHub link); ensure the new text clearly instructs
contributors to include the GitHub URL in the test name or body.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b99ee381-61b3-4649-9295-73706b4ebc51

📥 Commits

Reviewing files that changed from the base of the PR and between 5f213df and 5c5bb7b.

📒 Files selected for processing (17)
  • lib/parsers/README.md
  • lib/parsers/TEST_CASES.md
  • lib/parsers/src/reasoning/base_parser.rs
  • lib/parsers/src/reasoning/gpt_oss_parser.rs
  • lib/parsers/src/reasoning/granite_parser.rs
  • lib/parsers/src/reasoning/minimax_append_think_parser.rs
  • lib/parsers/src/reasoning/mod.rs
  • lib/parsers/src/tool_calling/dsml/parser.rs
  • lib/parsers/src/tool_calling/harmony/harmony_parser.rs
  • lib/parsers/src/tool_calling/json/base_json_parser.rs
  • lib/parsers/src/tool_calling/json/deepseek_v3_1_parser.rs
  • lib/parsers/src/tool_calling/json/deepseek_v3_parser.rs
  • lib/parsers/src/tool_calling/json/mod.rs
  • lib/parsers/src/tool_calling/pythonic/pythonic_parser.rs
  • lib/parsers/src/tool_calling/xml/glm47_parser.rs
  • lib/parsers/src/tool_calling/xml/kimi_k2_parser.rs
  • lib/parsers/src/tool_calling/xml/parser.rs

@keivenchang keivenchang force-pushed the keivenchang/DIS-1842__parser-test-taxonomy_index branch from 05e0837 to 5c5bb7b Compare April 25, 2026 17:13
@keivenchang
Copy link
Copy Markdown
Contributor Author

closing — duplicate of #8725, which is the canonical taxonomy-annotation PR. work continues there.

@keivenchang keivenchang deleted the keivenchang/DIS-1842__parser-test-taxonomy_index branch May 9, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/XL test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant