Skip to content

Add rayon concurrency note #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leynos merged 3 commits into from
Jul 20, 2025
Merged

Add rayon concurrency note #101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7815dfd
Clarify rayon usage in roadmap
leynos Jul 19, 2025
9ddb933
Update README.md
leynos Jul 20, 2025
cc2196b
Merge branch 'main' into codex/create-roadmap-for-parallel-file-proce…
leynos Jul 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ cargo install --path .

## Command-line usage


```bash
mdtablefix [--wrap] [--renumber] [--breaks] [--ellipsis] [--fences] [--footnotes] [--in-place] [FILE...]
```
Expand Down Expand Up @@ -110,8 +111,8 @@ A brief intermission for pizza.

## Library usage

The crate provides helper functions for embedding the table reflow logic in
your own Rust project:
The crate exposes helper functions for embedding the table-reflow logic in
Rust projects:

```rust
use mdtablefix::{process_stream_opts, rewrite, Options};
Expand Down
23 changes: 23 additions & 0 deletions docs/parallel-processing-roadmap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Roadmap for Parallel File Processing

The command-line tool currently processes input files sequentially. The steps
below outline the work required to allow concurrent processing while preserving
serial output order.

- [ ] **Adopt `rayon` for concurrency**
- Use `rayon` thread pools to spawn work for each file path.
- Ensure the approach integrates cleanly with existing modules.
- [ ] **Add chosen crate to `Cargo.toml`**
- Pin an explicit version and document the decision in `docs/`.
- [ ] **Refactor `main.rs` to launch parallel tasks**
- Spawn a worker for each file path using the concurrency crate.
- Maintain a list of handles so outputs can be gathered in order.
- [ ] **Collect results sequentially**
Comment on lines +14 to +15
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Insert the missing comma after “handles”.

The comma before “so” prevents the sentence from reading as a fused clause.

-  - Maintain a list of handles so outputs can be gathered in order.
+  - Maintain a list of handles, so outputs can be gathered in order.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Maintain a list of handles so outputs can be gathered in order.
- [ ] **Collect results sequentially**
- Maintain a list of handles, so outputs can be gathered in order.
- [ ] **Collect results sequentially**
🧰 Tools
🪛 LanguageTool

[uncategorized] ~14-~14: Use a comma before ‘so’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...cy crate. - Maintain a list of handles so outputs can be gathered in order. - [ ]...

(COMMA_COMPOUND_SENTENCE_2)

🤖 Prompt for AI Agents 
In docs/parallel-processing-roadmap.md around lines 14 to 15, insert a comma
after the word "handles" in the sentence "Maintain a list of handles so outputs
can be gathered in order." to correctly separate the clauses and improve
readability.

- Await or join handles in the same order the files were supplied.
- Print each processed file or error message before moving to the next.
- [ ] **Extend tests for parallel execution**
- Use `rstest` to verify that processing many files yields correct results.
- Add tests exercising error handling when some paths are invalid.
- [ ] **Update documentation**
- Document the new flags or behaviour in `README.md` and module docs.
- Note any concurrency caveats or performance implications.
Comment on lines +22 to +23
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Replace “caveats” with plainer British English.

“Caveats” is legalistic. Prefer “warnings” to keep the prose clear.

-  - Note any concurrency caveats or performance implications.
+  - Note any concurrency warnings or performance implications.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Document the new flags or behaviour in `README.md` and module docs.
- Note any concurrency caveats or performance implications.
- Document the new flags or behaviour in `README.md` and module docs.
- Note any concurrency warnings or performance implications.
🧰 Tools
🪛 LanguageTool

[style] ~23-~23: The word ‘caveats’ is a legal term. To make your text as clear as possible to all readers, do not use this foreign term unless it is used with its legal meaning. Possible alternatives are “cautions” or “warnings”.
Context: ...d module docs. - Note any concurrency caveats or performance implications.

(CAVEAT)

🤖 Prompt for AI Agents 
In docs/parallel-processing-roadmap.md around lines 22 to 23, replace the word
"caveats" with a plainer British English term such as "warnings" to make the
prose clearer and less legalistic. Update the sentence to reflect this change
while maintaining the original meaning about concurrency and performance
considerations.