Skip to content

Improve wrapping logic for hyphenated words#58

Merged
leynos merged 2 commits intomainfrom
codex/update-word-wrapping-for-hyphenated-words
Jul 13, 2025
Merged

Improve wrapping logic for hyphenated words#58
leynos merged 2 commits intomainfrom
codex/update-word-wrapping-for-hyphenated-words

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Jul 13, 2025

Summary

  • keep hyphenated words intact during wrapping using NoHyphenation
  • update README to mention the new behaviour
  • add unit test for wrap_text
  • add integration test for hyphenated wrapping

Testing

  • cargo clippy -- -D warnings
  • RUSTFLAGS="-D warnings" cargo test
  • markdownlint README.md docs/html-table-support.md docs/release-process.md docs/rust-testing-with-rstest-fixtures.md

https://chatgpt.com/codex/tasks/task_e_6873ae41e18c8322950cb5e37d57086b

Summary by Sourcery

Improve text wrapping logic to keep hyphenated words intact by introducing the NoHyphenation word splitter, update documentation, and add corresponding unit and integration tests.

New Features:

  • Preserve hyphenated words as single units during text wrapping

Enhancements:

  • Use the NoHyphenation WordSplitter in paragraph and text wrapping functions

Documentation:

  • Update README to document the new hyphenated word wrapping behavior

Tests:

  • Add unit test for wrap_text to ensure hyphenated words are not split
  • Add integration test for processing streams containing hyphenated words

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Jul 13, 2025

Reviewer's Guide

Wrap routines now use a NoHyphenation word splitter to preserve hyphenated words during wrapping, supported by new unit and integration tests and an updated README.

Class diagram for updated wrapping logic with NoHyphenation

classDiagram
    class Options {
        +new(width: usize)
        +word_splitter(splitter: WordSplitter) Options
    }
    class WordSplitter {
    }
    class NoHyphenation {
    }
    WordSplitter <|-- NoHyphenation
    Options o-- WordSplitter

    class fill {
        +fill(text: &str, opts: &Options) -> String
    }

    class wrap_text {
        +wrap_text(lines: &[String], width: usize) -> Vec<String>
    }

    wrap_text ..> Options : uses
    wrap_text ..> fill : uses
    fill ..> Options : uses
    Options ..> WordSplitter : configures
    WordSplitter <|-- NoHyphenation : new splitter
Loading

File-Level Changes

Change Details Files
Introduce NoHyphenation splitter in wrap routines
  • Instantiate Options::new(...).word_splitter(WordSplitter::NoHyphenation) for wrapping
  • Replace fill(&segment, width) calls in flush_paragraph with the new opts
  • Apply the same opts pattern in wrap_text for indented lines
src/lib.rs
Add tests verifying non-splitting of hyphenated words
  • Add wrap_text_preserves_hyphenated_words unit test
  • Add test_wrap_hyphenated_word integration test
src/lib.rs
tests/integration.rs
Document new hyphenation behavior in README
  • Update --wrap description to note that hyphenated words move as single units
  • Adjust surrounding text to mention preservation of hyphenated words
README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jul 13, 2025

Summary by CodeRabbit

  • Documentation

    • Clarified in the README that hyphenated words are treated as single units during wrapping and will not be split at the hyphen.
  • Bug Fixes

    • Improved text wrapping so that hyphenated words are kept intact and not split across lines.
  • Tests

    • Added integration tests to verify correct handling of hyphenated words and long words during wrapping.

Summary by CodeRabbit

  • Bug Fixes

    • Improved text wrapping so that hyphenated words are kept intact and are not split at hyphens when wrapping lines.
  • Documentation

    • Updated the README to clarify that hyphenated words are treated as single units during text wrapping.
  • Tests

    • Added tests to verify that hyphenated words are preserved and not broken across lines during wrapping.

Walkthrough

Update the README to clarify that hyphenated words are not split during wrapping. Adjust the core text wrapping logic in the source to disable hyphenation, ensuring hyphenated words move as single units. Add new tests in both unit and integration suites to verify this behaviour.

Changes

File(s) Change Summary
README.md Clarify --wrap option: hyphenated words are treated as single units and not split at hyphens.
src/lib.rs Update wrapping logic to use WordSplitter::NoHyphenation; add a unit test for hyphenated words.
tests/integration.rs Add integration tests verifying hyphenated words are not split during wrapping in various cases.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant WrappingLogic

    User->>CLI: Provide text with hyphenated words and --wrap option
    CLI->>WrappingLogic: Pass text and wrap settings
    WrappingLogic->>WrappingLogic: Process text, avoid splitting hyphenated words
    WrappingLogic-->>CLI: Return wrapped text
    CLI-->>User: Output wrapped text with hyphenated words intact
Loading

Possibly related PRs

  • Add text wrapping to mdtablefix #32: Refines the same text wrapping feature, focusing on preventing hyphenation of hyphenated words and modifying related functions and tests.
  • Add optional wrapping flag #38: Introduces an optional CLI flag to enable or disable wrapping, related to wrapping behaviour but focusing on different aspects.

Poem

Hyphens now hold hands,
No splitting at their bands—
Words march whole, not cleft,
In wrapping, none are left
Broken on the line—
A fix both neat and fine!
📝✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9d7eaab and 0977280.

📒 Files selected for processing (1)
  • tests/integration.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • AGENTS.md

⚙️ CodeRabbit Configuration File

🧬 Code Graph Analysis (1)
tests/integration.rs (1)
src/lib.rs (1)
  • process_stream (545-545)
⏰ Context from checks skipped due to timeout of 240000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build-test
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch codex/update-word-wrapping-for-hyphenated-words

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai auto-generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @leynos - I've reviewed your changes and they look great!

Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments

### Comment 1
<location> `tests/integration.rs:755` </location>
<code_context>
 }
+
+#[test]
+fn test_wrap_hyphenated_word() {
+    let line = format!("{} extremely-very-long-word end", "A".repeat(60));
+    let output = process_stream(&[line]);
+    assert_eq!(
+        output,
+        vec![
+            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
+            "extremely-very-long-word end".to_string(),
+        ]
+    );
+}
</code_context>

<issue_to_address>
Add integration tests for multiple hyphenated words and edge cases.

Please add tests for multiple hyphenated words in a line, hyphenated words at line boundaries, words longer than the wrap width, and lines without hyphenated words to ensure comprehensive coverage.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
#[test]
fn test_wrap_hyphenated_word() {
    let line = format!("{} extremely-very-long-word end", "A".repeat(60));
    let output = process_stream(&[line]);
    assert_eq!(
        output,
        vec![
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
            "extremely-very-long-word end".to_string(),
        ]
    );
}
=======
#[test]
fn test_wrap_hyphenated_word() {
    let line = format!("{} extremely-very-long-word end", "A".repeat(60));
    let output = process_stream(&[line]);
    assert_eq!(
        output,
        vec![
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
            "extremely-very-long-word end".to_string(),
        ]
    );
}

#[test]
fn test_wrap_multiple_hyphenated_words() {
    let line = format!("{} foo-bar baz-qux end", "A".repeat(60));
    let output = process_stream(&[line]);
    assert_eq!(
        output,
        vec![
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
            "foo-bar baz-qux end".to_string(),
        ]
    );
}

#[test]
fn test_wrap_hyphenated_word_at_boundary() {
    // The hyphenated word starts at the wrap boundary
    let line = format!("{} extremely-very-long-word", "A".repeat(60));
    let output = process_stream(&[line]);
    assert_eq!(
        output,
        vec![
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
            "extremely-very-long-word".to_string(),
        ]
    );
}

#[test]
fn test_wrap_word_longer_than_width() {
    // A single word longer than the wrap width (assuming 60)
    let long_word = "a".repeat(70);
    let output = process_stream(&[long_word.clone()]);
    assert_eq!(
        output,
        vec![
            long_word,
        ]
    );
}

#[test]
fn test_wrap_line_without_hyphenated_words() {
    let line = format!("{} end of line", "A".repeat(60));
    let output = process_stream(&[line]);
    assert_eq!(
        output,
        vec![
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
            "end of line".to_string(),
        ]
    );
}
>>>>>>> REPLACE

</suggested_fix>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread tests/integration.rs
Comment on lines +754 to +765
#[test]
fn test_wrap_hyphenated_word() {
let line = format!("{} extremely-very-long-word end", "A".repeat(60));
let output = process_stream(&[line]);
assert_eq!(
output,
vec![
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
"extremely-very-long-word end".to_string(),
]
);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add integration tests for multiple hyphenated words and edge cases.

Please add tests for multiple hyphenated words in a line, hyphenated words at line boundaries, words longer than the wrap width, and lines without hyphenated words to ensure comprehensive coverage.

Suggested change
#[test]
fn test_wrap_hyphenated_word() {
let line = format!("{} extremely-very-long-word end", "A".repeat(60));
let output = process_stream(&[line]);
assert_eq!(
output,
vec![
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
"extremely-very-long-word end".to_string(),
]
);
}
#[test]
fn test_wrap_hyphenated_word() {
let line = format!("{} extremely-very-long-word end", "A".repeat(60));
let output = process_stream(&[line]);
assert_eq!(
output,
vec![
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
"extremely-very-long-word end".to_string(),
]
);
}
#[test]
fn test_wrap_multiple_hyphenated_words() {
let line = format!("{} foo-bar baz-qux end", "A".repeat(60));
let output = process_stream(&[line]);
assert_eq!(
output,
vec![
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
"foo-bar baz-qux end".to_string(),
]
);
}
#[test]
fn test_wrap_hyphenated_word_at_boundary() {
// The hyphenated word starts at the wrap boundary
let line = format!("{} extremely-very-long-word", "A".repeat(60));
let output = process_stream(&[line]);
assert_eq!(
output,
vec![
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
"extremely-very-long-word".to_string(),
]
);
}
#[test]
fn test_wrap_word_longer_than_width() {
// A single word longer than the wrap width (assuming 60)
let long_word = "a".repeat(70);
let output = process_stream(&[long_word.clone()]);
assert_eq!(
output,
vec![
long_word,
]
);
}
#[test]
fn test_wrap_line_without_hyphenated_words() {
let line = format!("{} end of line", "A".repeat(60));
let output = process_stream(&[line]);
assert_eq!(
output,
vec![
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA".to_string(),
"end of line".to_string(),
]
);
}

@leynos leynos merged commit 22b3780 into main Jul 13, 2025
0 of 2 checks passed
@leynos leynos deleted the codex/update-word-wrapping-for-hyphenated-words branch July 13, 2025 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant