Skip to content

Unify fence tracking across modules#236

Merged
leynos merged 5 commits intomainfrom
codex/extract-shared-iterator-for-fence-detection
Sep 28, 2025
Merged

Unify fence tracking across modules#236
leynos merged 5 commits intomainfrom
codex/extract-shared-iterator-for-fence-detection

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Sep 27, 2025

Summary

  • add a shared FenceTracker helper in wrap::fence and re-export it for downstream callers
  • update stream processing, list renumbering, thematic break formatting, and footnote detection to use the tracker instead of bespoke toggles
  • cover the tracker behaviour with dedicated unit tests

Testing

  • make fmt
  • make lint
  • make test

https://chatgpt.com/codex/tasks/task_e_68d81d0b061c832283fd3f68d4d9c09a

Summary by Sourcery

Unify fenced code block tracking across multiple modules by introducing a shared FenceTracker, updating all fence-sensitive logic to use it, and covering its behavior with dedicated tests

New Features:

  • Add FenceTracker helper to centralise fenced code block detection
  • Re-export FenceTracker from the wrap module for downstream callers

Enhancements:

  • Replace bespoke fence toggles with FenceTracker in wrap_text, process_stream, thematic break formatting, list renumbering, and footnote detection
  • Simplify handle_fence_line to delegate fence matching logic to FenceTracker

Tests:

  • Add unit tests covering FenceTracker open/close semantics
  • Add behavioural tests to verify fence-aware text wrapping suspends and resumes formatting correctly

- Introduce a shared `FenceTracker` in `wrap::fence` so all features reuse consistent fence semantics.\n- Replace bespoke toggles in stream processing, list renumbering, thematic break formatting, and footnote detection with the tracker.\n- Exercise the new tracker behaviour with dedicated unit tests.
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Sep 27, 2025

Reviewer's Guide

This PR centralizes fenced code block state management by introducing a shared FenceTracker helper, refactors existing fence detection across multiple modules to use it instead of custom toggles, and adds focused unit and behavioral tests to validate its behavior.

Sequence diagram for fence line processing with FenceTracker

sequenceDiagram
    participant ProcessStream
    participant FenceTracker
    participant OutputBuffer
    ProcessStream->>FenceTracker: observe(line)
    alt line is fence marker
        FenceTracker-->>ProcessStream: true
        ProcessStream->>OutputBuffer: flush_buffer()
        ProcessStream->>OutputBuffer: push(line)
    else line is not fence marker
        FenceTracker-->>ProcessStream: false
        alt FenceTracker.in_fence()
            ProcessStream->>OutputBuffer: push(line)
        end
    end
Loading

File-Level Changes

Change Details Files
Introduce a shared FenceTracker helper and re-export it in the wrap module
  • Define the FenceTracker struct with state and implement new(), observe(), and in_fence()
  • Add documentation and examples for the tracker in wrap::fence
  • Re-export FenceTracker alongside is_fence in wrap.rs
src/wrap/fence.rs
src/wrap.rs
Simplify handle_fence_line to delegate fence logic to the tracker
  • Remove in_code and fence_state parameters
  • Call tracker.observe(line) to detect fence markers
  • Flush buffer and emit fence lines based on tracker results
src/wrap/fence.rs
Refactor downstream modules to use FenceTracker instead of bespoke toggles
  • Import and instantiate FenceTracker in wrap_text, process_stream_inner, format_breaks, has_existing_footnote_block, and renumber_lists
  • Replace manual in_code or in_fence boolean flags with tracker.observe() and tracker.in_fence()
  • Adjust function signatures and conditional logic accordingly
src/process.rs
src/wrap.rs
src/breaks.rs
src/footnotes/lists.rs
src/lists.rs
Add dedicated unit and behavior tests for fence tracking
  • Add unit tests covering new tracker behavior in src/wrap/tests.rs
  • Introduce a behavioral integration test for fence-aware wrapping in tests/wrap/fence_behaviour.rs
src/wrap/tests.rs
tests/wrap/fence_behaviour.rs

Possibly linked issues

  • Implement markdown table fixer #1: The PR introduces FenceTracker to centralize fence detection, removing duplicate logic in src/wrap.rs.
  • Add fences option #100: The PR introduces FenceTracker to centralize fence state management, which directly addresses the issue's goal of reducing complexity in fence-related state transitions.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Sep 27, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Summary by CodeRabbit

  • New Features
    • Exposed FenceTracker for more robust, fence-aware processing in integrations.
  • Bug Fixes
    • More reliable detection of fenced code blocks across wrapping, lists, and footnotes.
    • Prevents wrapping/renumbering inside fences; handles info strings, indentation, and varying fence lengths.
    • Malformed fence endings no longer disrupt list numbering.
    • Thematic breaks are only altered outside fences.
  • Refactor
    • Unified fence handling across modules via FenceTracker for consistent behaviour.
  • Tests
    • Added comprehensive tests for fence behaviour and list renumbering with fenced blocks.

Walkthrough

Replace ad-hoc in-fence booleans with a shared FenceTracker. Thread FenceTracker through wrapping, processing, list, break and footnote logic; re-export FenceTracker from wrap. Detect fences with tracker.observe(line) and gate logic with tracker.in_fence().

Changes

Cohort / File(s) Summary of changes
Fence tracking core
src/wrap/fence.rs
Add public FenceTracker struct with new, observe, in_fence. Refactor handle_fence_line to accept &mut FenceTracker and delegate fence detection/state. Retain is_fence helper.
Wrap integration and exports
src/wrap.rs, src/wrap/tests.rs
Re-export FenceTracker alongside is_fence. Replace internal in_code/fence state with FenceTracker in wrap_text and related flows. Add unit tests for FenceTracker behaviour.
Processing pipeline
src/process.rs
Import and initialise FenceTracker; thread the tracker through fence handling and replace boolean in_code checks with tracker.in_fence() and tracker.observe(...).
Lists and related features
src/lists.rs, src/breaks.rs, src/footnotes/lists.rs
Replace manual fence toggling with FenceTracker; call observe(line) per line and guard list/break/footnote logic with !tracker.in_fence(). Update imports; preserve public signatures.
Integration tests
tests/wrap/fence_behaviour.rs, tests/wrap/mod.rs, tests/lists.rs
Add behavioural and unit tests asserting wrapping and list-renumbering respect fence boundaries; include test modules.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Src as Input Lines
  participant Proc as process_stream_inner
  participant FT as FenceTracker
  participant Core as wrap/lists/breaks/footnotes
  participant Out as Output

  loop For each line
    Src->>Proc: line
    Proc->>FT: observe(line)
    alt Line is fence marker
      Proc->>Out: flush buffers and emit fence line
      Note right of FT: Tracker records marker char & length
    else Inside fence
      FT-->>Proc: in_fence = true
      Proc->>Out: emit line unchanged
    else Normal text
      Proc->>Core: apply wrapping/renumbering/footnote checks
      Core-->>Proc: transformed line
      Proc->>Out: emit line
    end
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

Poem

Rails of backticks rise and fall,
A tracker keeps the tidy thrall;
Lists renumber and paragraphs stay true,
Fences open, close — the stream flows through.

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Title Check ✅ Passed Emphasise that the title clearly summarises a shared fence-tracking mechanism across modules and accurately reflects the pull request’s main change.
Description Check ✅ Passed Confirm that the description outlines the addition of the FenceTracker helper, the refactoring of various modules to use it, and the inclusion of tests, demonstrating its clear relevance to the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch codex/extract-shared-iterator-for-fence-detection

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f5c8307 and 5eb0731.

📒 Files selected for processing (1)
  • tests/wrap/fence_behaviour.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.

Files:

  • tests/wrap/fence_behaviour.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in...

Files:

  • tests/wrap/fence_behaviour.rs
🧬 Code graph analysis (1)
tests/wrap/fence_behaviour.rs (1)
src/process.rs (1)
  • process_stream (215-223)
🔍 Remote MCP

Additional Context for Fence Handling

  • CommonMark’s specification defines a fenced code block as opening with at least three consecutive backticks (```) or tildes (~~~), optionally followed by an “info string” (trimmed of leading/trailing spaces, first word used as language, and not containing backticks), and closing with a matching fence of the same character and at least the same length. The opening fence may be indented up to three spaces; content lines remove up to that many leading spaces; the closing fence may also be indented up to three spaces and may only be followed by spaces. (spec.commonmark.org)

  • GitHub Flavored Markdown likewise supports fenced code blocks and recommends trimming leading whitespace to detect fences reliably, placing blank lines before/after for readability, and handling both backtick- and tilde-style fences consistently. Any implementation of fence detection must therefore inspect line.trim_start().starts_with("```") or line.trim_start().starts_with("~~~") to handle fenced blocks inside lists or indented contexts correctly. (docs.github.com)

These rules underline why centralizing fence state in a FenceTracker (which observes trimmed lines and tracks marker length and type) is an appropriate refactoring: it encapsulates the CommonMark/GFM fence detection logic in one place, ensures consistency across modules (stream processing, list renumbering, wrapping, etc.), and avoids ad-hoc boolean toggles scattered through the code.

⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Sourcery review

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and found some issues that need to be addressed.

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `src/wrap/fence.rs:66` </location>
<code_context>
+///
+/// The tracker centralises fence matching logic so that callers share the
+/// same semantics for opening and closing blocks.
+#[derive(Default)]
+pub struct FenceTracker {
+    state: Option<(char, usize)>,
</code_context>

<issue_to_address>
**issue (review_instructions):** Add behavioural and unit tests for the new FenceTracker struct and its methods.

The FenceTracker struct and its methods (new, observe, in_fence) are new features. You must provide both behavioural and unit tests to demonstrate their correctness and expected behaviour.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*`

**Instructions:**
For any new feature or change to an existing feature, both behavioural *and* unit tests are required.

</details>
</issue_to_address>

### Comment 2
<location> `src/wrap/fence.rs:66` </location>
<code_context>
+///
+/// The tracker centralises fence matching logic so that callers share the
+/// same semantics for opening and closing blocks.
+#[derive(Default)]
+pub struct FenceTracker {
+    state: Option<(char, usize)>,
</code_context>

<issue_to_address>
**issue (review_instructions):** The #[derive(Default)] attribute should be placed after the doc comment for FenceTracker.

Rust convention and the review instruction require that attributes like #[derive(Default)] appear after the doc comment, not before. Please move the attribute below the doc comment for FenceTracker.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*.rs`

**Instructions:**
Place function attributes **after** doc comments.

</details>
</issue_to_address>

### Comment 3
<location> `src/wrap/fence.rs:66` </location>
<code_context>
+///
+/// The tracker centralises fence matching logic so that callers share the
+/// same semantics for opening and closing blocks.
+#[derive(Default)]
+pub struct FenceTracker {
+    state: Option<(char, usize)>,
</code_context>

<issue_to_address>
**issue (review_instructions):** src/wrap/fence.rs does not begin with a //! module-level comment.

Please add a //! comment at the top of src/wrap/fence.rs to describe the module's purpose, as required by the review instructions.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*.rs`

**Instructions:**
Every module **must** begin with a `//!` comment.

</details>
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Document FenceTracker usage and cover its behaviour with new unit and integration tests.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2236277 and 93d3b66.

📒 Files selected for processing (4)
  • src/wrap/fence.rs (1 hunks)
  • src/wrap/tests.rs (2 hunks)
  • tests/wrap/fence_behaviour.rs (1 hunks)
  • tests/wrap/mod.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.

Files:

  • tests/wrap/fence_behaviour.rs
  • src/wrap/tests.rs
  • src/wrap/fence.rs
  • tests/wrap/mod.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in...

Files:

  • tests/wrap/fence_behaviour.rs
  • src/wrap/tests.rs
  • src/wrap/fence.rs
  • tests/wrap/mod.rs
🧬 Code graph analysis (3)
tests/wrap/fence_behaviour.rs (1)
src/process.rs (1)
  • process_stream (215-223)
src/wrap/tests.rs (2)
src/wrap.rs (1)
  • wrap_text (327-441)
src/wrap/fence.rs (3)
  • new (87-89)
  • in_fence (120-122)
  • observe (96-116)
src/wrap/fence.rs (1)
src/wrap.rs (1)
  • flush_paragraph (251-273)
⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Sourcery review
🔇 Additional comments (4)
tests/wrap/mod.rs (1)

18-18: Wire up the new fence behaviour test module. LGTM.

The additional module import is correct and keeps the wrap integration suite cohesive.

src/wrap/fence.rs (2)

49-60: handle_fence_line now delegates detection to the tracker. LGTM.

The buffer flush before emitting the fence marker avoids interleaving prose with fence lines and matches the caller’s expectations.


118-123: Expose intent with a tiny helper.

Keep, but a named helper improves readability when scanning call sites.

Apply:

-    pub fn in_fence(&self) -> bool {
-        self.state.is_some()
-    }
+    pub fn in_fence(&self) -> bool { self.state.is_some() }

Likely an incorrect or invalid review comment.

src/wrap/tests.rs (1)

12-12: Confirm re-export exists; no changes needed
FenceTracker is publicly re-exported in src/wrap.rs at line 18, so the test import remains valid.

Comment thread src/wrap/fence.rs
Comment thread src/wrap/fence.rs
Comment thread src/wrap/tests.rs
Comment thread tests/wrap/fence_behaviour.rs
@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Sep 28, 2025

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and found some issues that need to be addressed.

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `src/breaks.rs:49-50` </location>
<code_context>
+    let mut fences = FenceTracker::default();

     for line in lines {
-        if is_fence(line).is_some() {
-            in_code = !in_code;
+        if fences.observe(line) {
             out.push(Cow::Borrowed(line.as_str()));
             continue;
</code_context>

<issue_to_address>
**suggestion:** Switching to FenceTracker may change how thematic breaks are detected inside code blocks.

Please verify that FenceTracker correctly handles edge cases, such as fences opened and closed on the same line or with atypical spacing, to ensure compliance with Markdown semantics.

Suggested implementation:

```rust
    for line in lines {
        // Handle multiple fences on the same line and atypical spacing.
        if fences.observe(line) {
            out.push(Cow::Borrowed(line.as_str()));
            continue;
        }

```

```rust
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_fence_tracker_edge_cases() {
        let lines = vec![
            "```rust code fence on one line```".to_string(),
            "   ```   ".to_string(),
            "text outside fence".to_string(),
            "```".to_string(),
            "text inside fence".to_string(),
            "```".to_string(),
            "   ```".to_string(),
            "text after fence".to_string(),
        ];
        let mut fences = FenceTracker::default();
        let mut results = Vec::new();
        for line in &lines {
            results.push(fences.observe(line));
        }
        // Fence opened and closed on same line should be detected
        assert!(results[0]);
        // Fence with atypical spacing should be detected
        assert!(results[1]);
        // Text outside fence should not be detected as fence
        assert!(!results[2]);
        // Fence should be detected
        assert!(results[3]);
        // Text inside fence should not be detected as fence
        assert!(!results[4]);
        // Closing fence should be detected
        assert!(results[5]);
        // Fence with leading spaces should be detected
        assert!(results[6]);
        // Text after fence should not be detected as fence
        assert!(!results[7]);
    }
}

```

If `FenceTracker::observe` does not already handle fences with leading/trailing spaces or multiple fences on the same line, you will need to update its implementation to normalize whitespace and scan for multiple fence patterns per line. Ensure it toggles the fence state correctly for each detected fence.
</issue_to_address>

### Comment 2
<location> `src/lists.rs:94-95` </location>
<code_context>
+    let mut fences = FenceTracker::default();

     for line in lines {
-        if is_fence(line).is_some() {
-            in_code = !in_code;
+        if fences.observe(line) {
             out.push(Cow::Borrowed(line.as_str()));
             continue;
</code_context>

<issue_to_address>
**suggestion:** List renumbering now respects FenceTracker, which may affect list handling inside code blocks.

Please test with nested lists and malformed fences to ensure FenceTracker handles these scenarios correctly.

Suggested implementation:

```rust
use super::*;
use std::borrow::Cow;

#[test]
fn test_nested_lists_with_fence_tracker() {
    let input = vec![
        "1. Outer list",
        "   ```",
        "   1. Code block list",
        "   ```",
        "2. Outer list continued",
        "   1. Nested list",
        "      ```",
        "      - Malformed fence",
        "      ```",
        "   2. Nested list continued",
    ].into_iter().map(|s| s.to_string()).collect::<Vec<_>>();

    let output = renumber_lists(&input);

    // Check that list renumbering is correct outside code blocks
    assert_eq!(output[0], Cow::Borrowed("1. Outer list"));
    assert_eq!(output[4], Cow::Borrowed("2. Outer list continued"));
    assert_eq!(output[5], Cow::Borrowed("   1. Nested list"));
    assert_eq!(output[8], Cow::Borrowed("   2. Nested list continued"));

    // Check that code block lines are preserved
    assert_eq!(output[1], Cow::Borrowed("   ```"));
    assert_eq!(output[2], Cow::Borrowed("   1. Code block list"));
    assert_eq!(output[3], Cow::Borrowed("   ```"));
    assert_eq!(output[6], Cow::Borrowed("      ```"));
    assert_eq!(output[7], Cow::Borrowed("      - Malformed fence"));
}

#[test]
fn test_malformed_fences() {
    let input = vec![
        "1. List before fence",
        "   ```",
        "   1. Inside code block",
        "   ``", // malformed fence
        "2. List after fence",
    ].into_iter().map(|s| s.to_string()).collect::<Vec<_>>();

    let output = renumber_lists(&input);

    // List renumbering should not occur inside code block
    assert_eq!(output[0], Cow::Borrowed("1. List before fence"));
    assert_eq!(output[1], Cow::Borrowed("   ```"));
    assert_eq!(output[2], Cow::Borrowed("   1. Inside code block"));
    assert_eq!(output[3], Cow::Borrowed("   ``"));
    assert_eq!(output[4], Cow::Borrowed("2. List after fence"));
}

```

- If your test runner does not automatically pick up `tests/lists.rs`, ensure it is included in your test suite.
- If `renumber_lists` or `FenceTracker` are not public, you may need to adjust their visibility (`pub`) for testing.
- If your project uses a different test organization, move these tests to the appropriate location.
</issue_to_address>

### Comment 3
<location> `src/wrap/tests.rs:347` </location>
<code_context>
+#[test]
+fn fence_tracker_ignores_shorter_closing_marker() {
</code_context>

<issue_to_address>
**suggestion (testing):** Suggestion to add tests for non-standard fence markers and malformed input.

Please add tests for malformed or non-standard fence markers, such as single backticks/ tildes or mixed markers, and for lines resembling fences that should not be recognized. This will improve robustness against edge cases.
</issue_to_address>

### Comment 4
<location> `tests/wrap/fence_behaviour.rs:20-22` </location>
<code_context>
+        "fn demonstrate() { println!(\"This code line intentionally exceeds eighty characters ",
+        "to ensure the wrapping logic would normally split it if fences were not honoured.\"); }",
+    );
+    let input = lines_vec![first_paragraph, "```", code_line, "```", closing_paragraph];
+    let output = process_stream(&input);
+    assert!(!output.iter().any(|l| l.starts_with("``") && l.len() == 2), "no false 2-tick fences");
+
+    let fence_positions: Vec<usize> = output
</code_context>

<issue_to_address>
**suggestion (testing):** Suggestion to add behavioral tests for edge cases with info strings and whitespace.

Add tests for fence lines with different info strings, trailing whitespace, and indentation to verify correct handling of all Markdown fence cases.
</issue_to_address>

### Comment 5
<location> `src/wrap/fence.rs:67` </location>
<code_context>
+/// The tracker centralises fence matching logic so that callers share the
+/// same semantics for opening and closing blocks.
+///
+/// # Examples
+///
+/// ```
</code_context>

<issue_to_address>
**issue (review_instructions):** Add unit tests for FenceTracker to verify edge cases and correctness.

The FenceTracker struct is a new feature. While an example is provided in the docstring, you must add dedicated unit tests to src/wrap/tests.rs to verify its behaviour, including edge cases such as mismatched fence markers and multiple consecutive fences.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*`

**Instructions:**
For any new feature or change to an existing feature, both behavioural *and* unit tests are required.

</details>
</issue_to_address>

### Comment 6
<location> `src/wrap/fence.rs:101` </location>
<code_context>
+    /// Panics when the fence regular expression yields an empty marker, which
+    /// would indicate the regex is inconsistent with Markdown fence rules.
+    #[must_use]
+    pub fn observe(&mut self, line: &str) -> bool {
+        let Some((_indent, fence, _info)) = is_fence(line) else {
+            return false;
</code_context>

<issue_to_address>
**issue (review_instructions):** Demonstrate bug fixes in fence handling with a test.

If this refactor fixes any previously existing bugs in fence handling, you must add a test that demonstrates the bug and its resolution.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*`

**Instructions:**
Bug fixes *must* be demonstrated by a test.

</details>
</issue_to_address>

### Comment 7
<location> `src/wrap/fence.rs:62` </location>
<code_context>
-                    *in_code = false;
-                    *fence_state = None;
-                }
+/// Tracks Markdown fenced code block state across lines.
+///
+/// The tracker centralises fence matching logic so that callers share the
</code_context>

<issue_to_address>
**issue (review_instructions):** The module is missing a `//!` doc comment at the top as required.

Please add a `//!` doc comment at the very top of this module to describe its purpose, as per the review instructions.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*.rs`

**Instructions:**
Every module **must** begin with a `//!` comment.

</details>
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread src/breaks.rs
Comment thread src/lists.rs
Comment thread src/wrap/tests.rs
Comment thread tests/wrap/fence_behaviour.rs
Add unit coverage for inline markers, tilde fences, and malformed sequences while reinforcing list renumbering and wrap behaviour with info string and short-marker scenarios.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8906b79 and f5c8307.

📒 Files selected for processing (3)
  • src/wrap/tests.rs (2 hunks)
  • tests/lists.rs (1 hunks)
  • tests/wrap/fence_behaviour.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.

Files:

  • src/wrap/tests.rs
  • tests/wrap/fence_behaviour.rs
  • tests/lists.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in...

Files:

  • src/wrap/tests.rs
  • tests/wrap/fence_behaviour.rs
  • tests/lists.rs
🧬 Code graph analysis (3)
src/wrap/tests.rs (2)
src/wrap.rs (1)
  • wrap_text (327-441)
src/wrap/fence.rs (3)
  • new (87-89)
  • in_fence (125-127)
  • observe (101-121)
tests/wrap/fence_behaviour.rs (1)
src/process.rs (1)
  • process_stream (215-223)
tests/lists.rs (1)
src/lists.rs (1)
  • renumber_lists (85-144)
⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Sourcery review

Comment thread src/wrap/tests.rs
Comment on lines +336 to +427
#[test]
fn fence_tracker_new_starts_outside_fence() {
let tracker = FenceTracker::new();
assert!(!tracker.in_fence());
}

#[test]
fn fence_tracker_closes_matching_markers() {
let mut tracker = FenceTracker::default();
assert!(!tracker.in_fence());
assert!(tracker.observe("```rust"));
assert!(tracker.in_fence());
assert!(tracker.observe("```"));
assert!(!tracker.in_fence());
}

#[test]
fn fence_tracker_closes_with_info_string() {
let mut tracker = FenceTracker::new();
assert!(tracker.observe("```rust"));
assert!(tracker.in_fence());
assert!(tracker.observe("``` "));
assert!(!tracker.in_fence());
}

#[test]
fn fence_tracker_ignores_shorter_closing_marker() {
let mut tracker = FenceTracker::new();
assert!(tracker.observe("````"));
assert!(tracker.in_fence());
assert!(tracker.observe("```"));
assert!(tracker.in_fence());
}

#[test]
fn fence_tracker_requires_matching_marker_to_close() {
let mut tracker = FenceTracker::default();
assert!(tracker.observe("```"));
assert!(tracker.in_fence());
assert!(tracker.observe("~~~"));
assert!(tracker.in_fence());
assert!(tracker.observe("````"));
assert!(!tracker.in_fence());
}

#[test]
fn fence_tracker_handles_inline_and_indented_markers() {
let lines = [
"```rust code fence on one line```",
" ``` ",
"text outside fence",
"```",
concat!(
"text inside fence that should remain intact even if it exceeds the usual width ",
"limit when wrapping is enabled."
),
"``` ",
"text after fence",
];
let mut tracker = FenceTracker::default();
let results: Vec<bool> = lines.iter().map(|line| tracker.observe(line)).collect();
assert_eq!(
results,
vec![true, true, false, true, false, true, false],
"expected fences to be recognised with inline markers and atypical spacing"
);
assert!(
!tracker.in_fence(),
"tracker should end outside of a fence after matching closures"
);
}

#[test]
fn fence_tracker_handles_tilde_fences() {
let mut tracker = FenceTracker::new();
assert!(tracker.observe("~~~~rust"));
assert!(tracker.in_fence());
assert!(tracker.observe("~~~~"));
assert!(!tracker.in_fence());
}

#[rstest]
#[case("`")]
#[case("``")]
#[case("`~~`")]
#[case("~~`")]
#[case("`` ~~")]
fn fence_tracker_rejects_short_or_mixed_markers(#[case] line: &str) {
let mut tracker = FenceTracker::default();
assert!(!tracker.observe(line));
assert!(!tracker.in_fence());
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Enforce the 400-line ceiling for this module. Move the new FenceTracker suite into a dedicated submodule (for example, tests::fence_tracker) so that src/wrap/tests.rs shrinks back below 400 lines; this file is now 428 lines long and violates the repository rule that test modules stay under 400 lines. As per coding guidelines.

🤖 Prompt for AI Agents
In src/wrap/tests.rs around lines 336 to 427, the new FenceTracker test suite
must be moved into its own test submodule to keep this file under the 400-line
limit; extract the entire block of FenceTracker tests into a new test file
(e.g., src/wrap/tests/fence_tracker.rs) and replace the removed block in
src/wrap/tests.rs with a mod declaration that pulls in the new tests (e.g., pub
mod fence_tracker or #[cfg(test)] mod fence_tracker as appropriate). Ensure the
new file has the same use/imports needed by the tests, preserves attributes
(#[test] / #[rstest]) and visibility, and update Cargo or module paths if
necessary so the test build still discovers and runs them.

Comment thread tests/wrap/fence_behaviour.rs Outdated
@leynos leynos merged commit e300d7f into main Sep 28, 2025
3 checks passed
@leynos leynos deleted the codex/extract-shared-iterator-for-fence-detection branch September 28, 2025 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant