Conversation
Reviewer's GuideExpose the tokenize_markdown API in the wrap module and add a regression integration test (with fixtures) to validate combined wrapping and list renumbering behavior. Class diagram for wrap module API changesclassDiagram
class wrap {
+Token
+tokenize_markdown()
}
class tokenize {
+Token
+tokenize_markdown()
}
wrap ..> tokenize : re-exports
Class diagram for new integration test fileclassDiagram
class wrap_renumber {
+test_wrap_and_renumber_regression()
}
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Summary by CodeRabbit
WalkthroughExpand the public API by re-exporting Changes
Sequence Diagram(s)sequenceDiagram
participant Test as Regression Test
participant FS as File System
participant Mdtablefix as mdtablefix crate
Test->>FS: Read wrap_renumber_regression_input.txt
Test->>Mdtablefix: process_stream(input_lines)
Mdtablefix-->>Test: wrapped_lines
Test->>Mdtablefix: renumber_lists(wrapped_lines)
Mdtablefix-->>Test: renumbered_lines
Test->>FS: Read wrap_renumber_regression_expected.txt
Test->>Test: Compare renumbered_lines with expected_lines
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)**/*.rs📄 CodeRabbit Inference Engine (AGENTS.md)
Files:
⚙️ CodeRabbit Configuration File
Files:
🧬 Code Graph Analysis (1)src/wrap.rs (1)
🔇 Additional comments (1)
✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🔭 Outside diff range comments (1)
src/wrap.rs (1)
18-20: Avoidunwrap()in static initialiser.
Regex::newcan only fail on an invalid pattern; however,expectaligns with the project coding-guideline preference. Swapunwrap()forexpect("valid fence regex")to silence clippy’sexpect_usedfalse-positive without a suppression.- std::sync::LazyLock::new(|| Regex::new(r"^\s*(```|~~~).*").unwrap()); + std::sync::LazyLock::new(|| Regex::new(r"^\s*(```|~~~).*") + .expect("valid fence regex"));
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (6)
docs/architecture.md(1 hunks)src/lib.rs(1 hunks)src/wrap.rs(1 hunks)tests/data/wrap_renumber_regression_expected.txt(1 hunks)tests/data/wrap_renumber_regression_input.txt(1 hunks)tests/wrap_renumber.rs(1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.rs
📄 CodeRabbit Inference Engine (AGENTS.md)
**/*.rs: Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider usingArcto reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Prefer immutable data and avoid unnecessarymutbindings.
Handle errors with theResulttype instead of panicking where feasible.
Avoidunsafecode unless absolutely necessary and document any usage clearly.
Place function attributes after doc comments.
Do not usereturnin single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Preferexpectoverallow.
Prefer.expect()over.unwrap().
Useconcat!()to combine long string literals rather than escaping newlines with a backslash.
Prefer semantic error enums: Derivestd::error::Error(via thethiserrorcrate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Useeyre::Reportfor human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and toeyreonly in the mainmain()entrypoint or top-level async task.
Files:
src/lib.rssrc/wrap.rstests/wrap_renumber.rs
⚙️ CodeRabbit Configuration File
**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.
Adhere to single responsibility and CQRS
Place function attributes after doc comments.
Do not use
returnin single-line functions.Move conditionals with >2 branches into a predicate function.
Avoid
unsafeunless absolutely necessary.Every module must begin with a
//!doc comment that explains the module's purpose and utility.Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar
Lints must not be silenced except as a last resort.
#[allow]is forbidden.- Only narrowly scoped
#[expect(lint, reason = "...")]is allowed.- No lint groups, no blanket or file-wide suppression.
- Include
FIXME:with link if a fix is expected.Use
rstestfixtures for shared setup and to avoid repetition between tests.Replace duplicated tests with
#[rstest(...)]parameterised cases.Prefer
mockallfor mocks/stubs.Prefer
.expect()over.unwrap()Ensure that any API or behavioural changes are reflected in the documentation in
docs/Ensure that any completed roadmap steps are recorded in the appropriate roadmap in
docs/Files must not exceed 400 lines in length
- Large modules must be decomposed
- Long match statements or dispatch tables should be decomposed by domain and collocated with targets
- Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
Files:
src/lib.rssrc/wrap.rstests/wrap_renumber.rs
docs/**/*.md
📄 CodeRabbit Inference Engine (AGENTS.md)
docs/**/*.md: Reference: Use the markdown files within thedocs/directory as a knowledge base and source of truth for project requirements, dependency choices, and architectural decisions.
Update: When new decisions are made, requirements change, libraries are added/removed, or architectural patterns evolve, proactively update the relevant file(s) in thedocs/directory to reflect the latest state. Ensure the documentation remains accurate and current.
Files:
docs/architecture.md
**/*.md
📄 CodeRabbit Inference Engine (AGENTS.md)
**/*.md: Documentation must use en-GB-oxendict spelling and grammar. (EXCEPTION: the naming of the "LICENSE" file, which is to be left unchanged for community consistency.)
Validate Markdown files usingmake markdownlint.
Runmake fmtafter any documentation changes to format all Markdown files and fix table markup.
Validate Mermaid diagrams in Markdown files by runningmake nixie.
Markdown paragraphs and bullet points must be wrapped at 80 columns.
Code blocks must be wrapped at 120 columns.
Tables and headings must not be wrapped.
Use dashes (-) for list bullets.
Use GitHub-flavoured Markdown footnotes ([^1]) for references and footnotes.
Files:
docs/architecture.md
⚙️ CodeRabbit Configuration File
**/*.md: * Avoid 2nd person or 1st person pronouns ("I", "you", "we")
- Use en-GB-oxendict (-ize / -our) spelling and grammar
- Paragraphs and bullets must be wrapped to 80 columns, except where a long URL would prevent this (in which case, silence MD013 for that line)
- Code blocks should be wrapped to 120 columns.
- Headings must not be wrapped.
- Documents must start with a level 1 heading
- Headings must correctly increase or decrease by no more than one level at a time
- Use GitHub-flavoured Markdown style for footnotes and endnotes.
- Numbered footnotes must be numbered by order of appearance in the document.
Files:
docs/architecture.md
🧬 Code Graph Analysis (3)
src/lib.rs (2)
src/wrap.rs (2)
is_fence(191-191)wrap_text(275-376)src/wrap/tokenize.rs (1)
tokenize_markdown(150-184)
src/wrap.rs (1)
src/wrap/tokenize.rs (1)
tokenize_markdown(150-184)
tests/wrap_renumber.rs (2)
src/process.rs (1)
process_stream(206-214)src/lists.rs (1)
renumber_lists(78-143)
🪛 LanguageTool
tests/data/wrap_renumber_regression_input.txt
[typographical] ~3-~3: To join two clauses or introduce examples, consider using an em dash.
Context: ...and texts in a rust terminal application - GitHub, accessed on July 20, 2025, <http...
(DASH_RULE)
[typographical] ~5-~5: To join two clauses or introduce examples, consider using an em dash.
Context: ...play rich text in a terminal application - Rust Users Forum, accessed on July 20, 2...
(DASH_RULE)
[typographical] ~7-~7: To join two clauses or introduce examples, consider using an em dash.
Context: ...terminal-application/29386> 4. termimad - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~7-~7: To join two clauses or introduce examples, consider using an em dash.
Context: ...l-application/29386> 4. termimad - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~9-~9: To join two clauses or introduce examples, consider using an em dash.
Context: ...'s Guide to E2E Testing | by Tally Barak - Medium, accessed on July 20, 2025, <http...
(DASH_RULE)
[typographical] ~11-~11: To join two clauses or introduce examples, consider using an em dash.
Context: ...ing-b2a9eebeeb27> 6. How to Write Tests - The Rust Programming Language - Rust Doc...
(DASH_RULE)
[typographical] ~11-~11: To join two clauses or introduce examples, consider using an em dash.
Context: ...te Tests - The Rust Programming Language - Rust Documentation, accessed on July 20,...
(DASH_RULE)
[typographical] ~13-~13: To join two clauses or introduce examples, consider using an em dash.
Context: ...ch11-01-writing-tests.html> 7. termimad - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~15-~15: To join two clauses or introduce examples, consider using an em dash.
Context: ...io/crates/termimad/0.9.7> 8. assert_cmd - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~15-~15: To join two clauses or introduce examples, consider using an em dash.
Context: ...es/termimad/0.9.7> 8. assert_cmd - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~17-~17: To join two clauses or introduce examples, consider using an em dash.
Context: ...tps://docs.rs/assert_cmd> 9. assert_cmd - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~19-~19: To join two clauses or introduce examples, consider using an em dash.
Context: ...es/assert_cmd> 10. assert-rs/assert_cmd - Command - GitHub, accessed on July 20, 2...
(DASH_RULE)
[typographical] ~19-~19: To join two clauses or introduce examples, consider using an em dash.
Context: ...cmd> 10. assert-rs/assert_cmd - Command - GitHub, accessed on July 20, 2025, <http...
(DASH_RULE)
[typographical] ~21-~21: To join two clauses or introduce examples, consider using an em dash.
Context: ...f a man-in-the-middle proxy for whatever - GitHub, accessed on July 20, 2025, <http...
(DASH_RULE)
[typographical] ~25-~25: To join two clauses or introduce examples, consider using an em dash.
Context: ...025, https://insta.rs/docs/ 13. insta - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~25-~25: To join two clauses or introduce examples, consider using an em dash.
Context: ...ttps://insta.rs/docs/> 13. insta - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~29-~29: To join two clauses or introduce examples, consider using an em dash.
Context: ..., 2025, https://insta.rs/ 15. Testing - Command Line Applications in Rust, acces...
(DASH_RULE)
[typographical] ~31-~31: To join two clauses or introduce examples, consider using an em dash.
Context: ...ial/testing.html> 16. Test Organization - The Rust Programming Language, accessed ...
(DASH_RULE)
[typographical] ~33-~33: To join two clauses or introduce examples, consider using an em dash.
Context: ...11-03-test-organization.html> 17. insta - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~35-~35: To join two clauses or introduce examples, consider using an em dash.
Context: ...crates.io/crates/insta> 18. third-wheel - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~37-~37: To join two clauses or introduce examples, consider using an em dash.
Context: ...tes.io/crates/third-wheel> 19. tempfile - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~37-~37: To join two clauses or introduce examples, consider using an em dash.
Context: ...crates/third-wheel> 19. tempfile - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~39-~39: Consider using an em dash in dialogues and enumerations.
Context: ... be put in the same file as the source? - Rust Users Forum, accessed on July 20, ...
(DASH_RULE)
[typographical] ~43-~43: To join two clauses or introduce examples, consider using an em dash.
Context: ...-suite/> 22. Command in assert_cmd::cmd - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~43-~43: To join two clauses or introduce examples, consider using an em dash.
Context: ...> 22. Command in assert_cmd::cmd - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~45-~45: To join two clauses or introduce examples, consider using an em dash.
Context: ...t Rust command-line apps with assert_cmd - alexwlchan, accessed on July 20, 2025, <...
(DASH_RULE)
[typographical] ~47-~47: To join two clauses or introduce examples, consider using an em dash.
Context: ...cmd/> 24. assert_cmd for n00bs : r/rust - Reddit, accessed on July 20, 2025, <http...
(DASH_RULE)
[typographical] ~49-~49: To join two clauses or introduce examples, consider using an em dash.
Context: ...rt_cmd_for_n00bs/> 25. Snapshot Testing - Rust Project Primer, accessed on July 20...
(DASH_RULE)
[typographical] ~51-~51: To join two clauses or introduce examples, consider using an em dash.
Context: ...ing/snapshot.html> 26. Snapshot testing - Advanced Rust testing - Rust Exercises, ...
(DASH_RULE)
[typographical] ~51-~51: To join two clauses or introduce examples, consider using an em dash.
Context: ...Snapshot testing - Advanced Rust testing - Rust Exercises, accessed on July 20, 202...
(DASH_RULE)
[typographical] ~53-~53: To join two clauses or introduce examples, consider using an em dash.
Context: ...g/02_snapshots/00_intro.html> 27. insta - Rust, accessed on July 20, 2025, <https:...
(DASH_RULE)
[typographical] ~55-~55: To join two clauses or introduce examples, consider using an em dash.
Context: ...ines/doc/insta/index.html> 28. tempfile - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~55-~55: To join two clauses or introduce examples, consider using an em dash.
Context: ...c/insta/index.html> 28. tempfile - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~59-~59: To join two clauses or introduce examples, consider using an em dash.
Context: ...o Testing and Debugging Rust Code | 2024 - Rapid Innovation, accessed on July 20, 2...
(DASH_RULE)
tests/data/wrap_renumber_regression_expected.txt
[typographical] ~5-~5: To join two clauses or introduce examples, consider using an em dash.
Context: ... texts in a rust terminal application - GitHub, accessed on July 20, 2025, <h...
(DASH_RULE)
[typographical] ~8-~8: To join two clauses or introduce examples, consider using an em dash.
Context: ...play rich text in a terminal application - Rust Users Forum, accessed on July 20...
(DASH_RULE)
[typographical] ~12-~12: To join two clauses or introduce examples, consider using an em dash.
Context: ...terminal-application/29386> 4. termimad - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~12-~12: To join two clauses or introduce examples, consider using an em dash.
Context: ...l-application/29386> 4. termimad - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~15-~15: To join two clauses or introduce examples, consider using an em dash.
Context: ...'s Guide to E2E Testing | by Tally Barak - Medium, accessed on July 20, 2025, ...
(DASH_RULE)
[typographical] ~19-~19: To join two clauses or introduce examples, consider using an em dash.
Context: ...ing-b2a9eebeeb27> 6. How to Write Tests - The Rust Programming Language - Rust Doc...
(DASH_RULE)
[typographical] ~19-~19: To join two clauses or introduce examples, consider using an em dash.
Context: ...te Tests - The Rust Programming Language - Rust Documentation, accessed on July ...
(DASH_RULE)
[typographical] ~23-~23: To join two clauses or introduce examples, consider using an em dash.
Context: ...ch11-01-writing-tests.html> 7. termimad - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~26-~26: To join two clauses or introduce examples, consider using an em dash.
Context: ...io/crates/termimad/0.9.7> 8. assert_cmd - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~26-~26: To join two clauses or introduce examples, consider using an em dash.
Context: ...es/termimad/0.9.7> 8. assert_cmd - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~29-~29: To join two clauses or introduce examples, consider using an em dash.
Context: ...tps://docs.rs/assert_cmd> 9. assert_cmd - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~32-~32: To join two clauses or introduce examples, consider using an em dash.
Context: ...es/assert_cmd> 10. assert-rs/assert_cmd - Command - GitHub, accessed on July 20, 2...
(DASH_RULE)
[typographical] ~32-~32: To join two clauses or introduce examples, consider using an em dash.
Context: ...cmd> 10. assert-rs/assert_cmd - Command - GitHub, accessed on July 20, 2025, <...
(DASH_RULE)
[typographical] ~36-~36: To join two clauses or introduce examples, consider using an em dash.
Context: ...man-in-the-middle proxy for whatever - GitHub, accessed on July 20, 2025, <...
(DASH_RULE)
[typographical] ~42-~42: To join two clauses or introduce examples, consider using an em dash.
Context: ... https://insta.rs/docs/ 13. insta - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~42-~42: To join two clauses or introduce examples, consider using an em dash.
Context: ...ttps://insta.rs/docs/> 13. insta - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~47-~47: To join two clauses or introduce examples, consider using an em dash.
Context: ..., 2025, https://insta.rs/ 15. Testing - Command Line Applications in Rust, acces...
(DASH_RULE)
[typographical] ~50-~50: To join two clauses or introduce examples, consider using an em dash.
Context: ...ial/testing.html> 16. Test Organization - The Rust Programming Language, accessed ...
(DASH_RULE)
[typographical] ~53-~53: To join two clauses or introduce examples, consider using an em dash.
Context: ...11-03-test-organization.html> 17. insta - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~56-~56: To join two clauses or introduce examples, consider using an em dash.
Context: ...crates.io/crates/insta> 18. third-wheel - crates.io: Rust Pack...
(DASH_RULE)
[typographical] ~59-~59: To join two clauses or introduce examples, consider using an em dash.
Context: ...tes.io/crates/third-wheel> 19. tempfile - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~59-~59: To join two clauses or introduce examples, consider using an em dash.
Context: ...crates/third-wheel> 19. tempfile - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~62-~62: Consider using an em dash in dialogues and enumerations.
Context: ... be put in the same file as the source? - Rust Users Forum, accessed on July ...
(DASH_RULE)
[typographical] ~70-~70: To join two clauses or introduce examples, consider using an em dash.
Context: ...-suite/> 22. Command in assert_cmd::cmd - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~70-~70: To join two clauses or introduce examples, consider using an em dash.
Context: ...> 22. Command in assert_cmd::cmd - Rust - Docs.rs, accessed on ...
(DASH_RULE)
[typographical] ~74-~74: To join two clauses or introduce examples, consider using an em dash.
Context: ...t Rust command-line apps with assert_cmd - alexwlchan, accessed on July 20, 202...
(DASH_RULE)
[typographical] ~78-~78: To join two clauses or introduce examples, consider using an em dash.
Context: ...cmd/> 24. assert_cmd for n00bs : r/rust - Reddit, accessed on July 20, 2025, <...
(DASH_RULE)
[typographical] ~81-~81: To join two clauses or introduce examples, consider using an em dash.
Context: ...rt_cmd_for_n00bs/> 25. Snapshot Testing - Rust Project Primer, accessed on July 20...
(DASH_RULE)
[typographical] ~84-~84: To join two clauses or introduce examples, consider using an em dash.
Context: ...ing/snapshot.html> 26. Snapshot testing - Advanced Rust testing - Rust Exercises, ...
(DASH_RULE)
[typographical] ~84-~84: To join two clauses or introduce examples, consider using an em dash.
Context: ...Snapshot testing - Advanced Rust testing - Rust Exercises, accessed on July 20,...
(DASH_RULE)
[typographical] ~88-~88: To join two clauses or introduce examples, consider using an em dash.
Context: ...g/02_snapshots/00_intro.html> 27. insta - Rust, accessed on July 20, 2025, <ht...
(DASH_RULE)
[typographical] ~91-~91: To join two clauses or introduce examples, consider using an em dash.
Context: ...ines/doc/insta/index.html> 28. tempfile - Rust - Docs.rs, access...
(DASH_RULE)
[typographical] ~91-~91: To join two clauses or introduce examples, consider using an em dash.
Context: ...c/insta/index.html> 28. tempfile - Rust - Docs.rs, accessed on J...
(DASH_RULE)
[typographical] ~98-~98: To join two clauses or introduce examples, consider using an em dash.
Context: ...o Testing and Debugging Rust Code | 2024 - Rapid Innovation, accessed on July 2...
(DASH_RULE)
docs/architecture.md
[misspelling] ~271-~271: This word is normally spelled with a hyphen.
Context: ...oth Token and tokenize_markdown are re- exported by the crate root for use by downstream...
(EN_COMPOUNDS_RE_EXPORTED)
🔇 Additional comments (5)
tests/data/wrap_renumber_regression_input.txt (1)
1-60: No action required – test fixture only.tests/data/wrap_renumber_regression_expected.txt (1)
1-101: No action required – test fixture only.tests/wrap_renumber.rs (3)
1-1: Excellent module documentation.The module-level doc comment clearly explains the purpose and follows the coding guidelines requirement for module documentation.
3-3: Clean and appropriate imports.The imports are focused and bring in exactly what's needed for the test.
5-6: Confirminclude_lines!Macro Implementation
- Verify that
tests/common/mod.rsdefinesinclude_lines!usinginclude_str!($path), which triggers a compile-time error if the file is missing.- Note that it collects each line into a
Vec<String>as intended by the tests.- No additional runtime error handling is required; this matches Rust conventions for compile-time file inclusion.
| mod tokenize; | ||
| /// Token emitted by [`tokenize::segment_inline`] and used by higher-level wrappers. | ||
| /// Token emitted by [`tokenize::segment_inline`] and used by higher-level | ||
| /// wrappers. | ||
| /// | ||
| /// Re-export this so callers of [`crate::textproc`] can implement custom | ||
| /// Re-export these so callers of [`crate::textproc`] can implement custom | ||
| /// transformations without depending on internal modules. | ||
| pub use tokenize::Token; | ||
| pub use tokenize::{Token, tokenize_markdown}; | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Strengthen the Token docs and cite the lifetime.
The current comment tells what emits Token but not why a downstream caller would use it. Expand with a one-sentence purpose and mention the 'a lifetime parameter so docs render correctly:
-/// Token emitted by [`tokenize::segment_inline`] and used by higher-level
-/// wrappers.
-///
-/// Re-export these so callers of [`crate::textproc`] can implement custom
-/// transformations without depending on internal modules.
+/// Lightweight, zero-copy view into a slice of the source Markdown used by
+/// [`tokenize::segment_inline`]. Downstream crates can pattern-match on
+/// `Token<'a>` values to implement custom transforms without touching private
+/// internals.
+///
+/// `Token` and [`tokenize_markdown`] are re-exported at crate root to avoid
+/// deep paths like `mdtablefix::wrap::tokenize::Token`.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| mod tokenize; | |
| /// Token emitted by [`tokenize::segment_inline`] and used by higher-level wrappers. | |
| /// Token emitted by [`tokenize::segment_inline`] and used by higher-level | |
| /// wrappers. | |
| /// | |
| /// Re-export this so callers of [`crate::textproc`] can implement custom | |
| /// Re-export these so callers of [`crate::textproc`] can implement custom | |
| /// transformations without depending on internal modules. | |
| pub use tokenize::Token; | |
| pub use tokenize::{Token, tokenize_markdown}; | |
| mod tokenize; | |
| /// Lightweight, zero-copy view into a slice of the source Markdown used by | |
| /// [`tokenize::segment_inline`]. Downstream crates can pattern-match on | |
| /// `Token<'a>` values to implement custom transforms without touching private | |
| /// internals. | |
| /// | |
| /// `Token` and [`tokenize_markdown`] are re-exported at crate root to avoid | |
| /// deep paths like `mdtablefix::wrap::tokenize::Token`. | |
| pub use tokenize::{Token, tokenize_markdown}; |
🤖 Prompt for AI Agents
In src/wrap.rs around lines 10 to 17, the documentation for the Token type lacks
explanation of its purpose and does not mention the 'a lifetime parameter.
Update the doc comment to include a brief sentence describing why downstream
callers would use Token, and explicitly cite the 'a lifetime parameter to ensure
proper rendering in the documentation.
| #[test] | ||
| fn wrap_then_renumber_preserves_order() { | ||
| let input: Vec<String> = include_lines!("data/wrap_renumber_regression_input.txt"); | ||
| let expected: Vec<String> = include_lines!("data/wrap_renumber_regression_expected.txt"); | ||
|
|
||
| let mut out = process_stream(&input); | ||
| out = renumber_lists(&out); | ||
|
|
||
| assert_eq!(out, expected); | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Well-structured regression test with room for improvement.
The test correctly validates the combined behaviour of wrapping and renumbering functions. However, improve error diagnostics by adding more detailed assertion messages.
Apply this diff to improve test diagnostics:
- assert_eq!(out, expected);
+ assert_eq!(
+ out, expected,
+ "Combined wrap and renumber operations produced unexpected output"
+ );Additionally, consider extracting the file paths as constants for better maintainability:
+const INPUT_FILE: &str = "data/wrap_renumber_regression_input.txt";
+const EXPECTED_FILE: &str = "data/wrap_renumber_regression_expected.txt";
+
#[test]
fn wrap_then_renumber_preserves_order() {
- let input: Vec<String> = include_lines!("data/wrap_renumber_regression_input.txt");
- let expected: Vec<String> = include_lines!("data/wrap_renumber_regression_expected.txt");
+ let input: Vec<String> = include_lines!(INPUT_FILE);
+ let expected: Vec<String> = include_lines!(EXPECTED_FILE);Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In tests/wrap_renumber.rs around lines 8 to 17, the test lacks detailed
assertion messages which would improve error diagnostics. Modify the assert_eq!
macro to include a descriptive message showing the expected and actual outputs
on failure. Also, extract the input and expected file paths into constants at
the top of the file to enhance maintainability and readability.
1 participant
Summary
tokenize_markdownalongsideTokenTesting
make lintmake testhttps://chatgpt.com/codex/tasks/task_e_688c885a5a948322a2dfa9bb28debabc
Summary by Sourcery
Expose the markdown tokenization function in the text processing API and add a regression integration test to verify combined wrapping and list renumbering.
Enhancements:
Tests: