Compiletest: Fix compare-output-by-lines directive#153797
Merged
rust-bors[bot] merged 1 commit intorust-lang:mainfrom Mar 13, 2026
Merged
Compiletest: Fix compare-output-by-lines directive#153797rust-bors[bot] merged 1 commit intorust-lang:mainfrom
rust-bors[bot] merged 1 commit intorust-lang:mainfrom
Conversation
Fixes: rust-lang#148235 This was checking that the blessed stderr was a subset of the stderr. Additionally, when re-blessing it would re-bless the subset that matched. So, when an extra line or comment was added and the line numbers changed, it would bless it without the line numbers.
Collaborator
|
Some changes occurred in src/tools/compiletest cc @jieyouxu |
rustbot
added
A-compiletest
jieyouxu
reviewed
Mar 13, 2026
Member
|
@rustbot author |
rustbot
added
S-waiting-on-author
Collaborator
|
Reminder, once the PR becomes ready for a review, use |
teor2345
reviewed
Mar 13, 2026
Contributor
Author
|
@rustbot ready |
rustbot
added
S-waiting-on-review
Member
|
I think that reblessing-can-swap-order is not an ideal property, but don't want to block this PR on that. Thanks. |
Contributor
rust-bors
bot
added
S-waiting-on-bors
Contributor
Author
|
Note they will only swap order if you re-bless when there is an actual change. If the diagnostics only change order (due to -Zthreads), they will match by lines and won't be re-blessed. |
rust-bors bot
pushed a commit
that referenced
this pull request
Mar 13, 2026
…uwer Rollup of 7 pull requests Successful merges: - #153650 (Streamline active job collection.) - #153707 (Remove `CycleErrorHandling`.) - #153384 (Add missing safety doc for CString::from_vec_unchecked and async_drop_in_place) - #153752 (fix(delegation): Filter Out Module Segments in Generic Args Inheritance) - #153797 (Compiletest: Fix compare-output-by-lines directive) - #153810 (compiletest: Use PYTHONPATH for lldb too, not only gdb) - #153820 (Rename `opt_span_diag_lint` into `opt_span_lint` and remove `emit_diag_lint`)
Contributor
It seems like a stable process would be:
Can we add that to the tooling (or comments, or documentation)? |
teor2345
reviewed
Mar 15, 2026
Zalathar
added a commit
to Zalathar/rust
that referenced
this pull request
Mar 17, 2026
Add the option to run UI tests with the parallel frontend This PR adds two arguments for tests: 1. `--parallel-frontend-threads`: specify `-Zthread` to compile test case (currently UI tests only) 2. `--iteration-count`: the number of times to run each test Also, due to the non-deterministic diagnostic orders and cycle errors, this PR adds the directive `//@ ignore-parallel-frontend` to ignore tests with cycle error when the parallel-frontend is enabled (by `--parallel-frontend-threads`) and enables `//@ compare-output-by-lines` by default. Context: [#t-compiler/parallel-rustc > Add the parallel front-end test suite @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fparallel-rustc/topic/Add.20the.20parallel.20front-end.20test.20suite/near/578781369) This PR should work with rust-lang#153797 together.
JonathanBrouwer
added a commit
to JonathanBrouwer/rust
that referenced
this pull request
Mar 17, 2026
Add the option to run UI tests with the parallel frontend This PR adds two arguments for tests: 1. `--parallel-frontend-threads`: specify `-Zthread` to compile test case (currently UI tests only) 2. `--iteration-count`: the number of times to run each test Also, due to the non-deterministic diagnostic orders and cycle errors, this PR adds the directive `//@ ignore-parallel-frontend` to ignore tests with cycle error when the parallel-frontend is enabled (by `--parallel-frontend-threads`) and enables `//@ compare-output-by-lines` by default. Context: [#t-compiler/parallel-rustc > Add the parallel front-end test suite @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fparallel-rustc/topic/Add.20the.20parallel.20front-end.20test.20suite/near/578781369) This PR should work with rust-lang#153797 together.
github-actions bot
pushed a commit
to rust-lang/rustc-dev-guide
that referenced
this pull request
Mar 18, 2026
Add the option to run UI tests with the parallel frontend This PR adds two arguments for tests: 1. `--parallel-frontend-threads`: specify `-Zthread` to compile test case (currently UI tests only) 2. `--iteration-count`: the number of times to run each test Also, due to the non-deterministic diagnostic orders and cycle errors, this PR adds the directive `//@ ignore-parallel-frontend` to ignore tests with cycle error when the parallel-frontend is enabled (by `--parallel-frontend-threads`) and enables `//@ compare-output-by-lines` by default. Context: [#t-compiler/parallel-rustc > Add the parallel front-end test suite @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fparallel-rustc/topic/Add.20the.20parallel.20front-end.20test.20suite/near/578781369) This PR should work with rust-lang/rust#153797 together.
github-actions bot
pushed a commit
to rust-lang/miri
that referenced
this pull request
Mar 18, 2026
Add the option to run UI tests with the parallel frontend This PR adds two arguments for tests: 1. `--parallel-frontend-threads`: specify `-Zthread` to compile test case (currently UI tests only) 2. `--iteration-count`: the number of times to run each test Also, due to the non-deterministic diagnostic orders and cycle errors, this PR adds the directive `//@ ignore-parallel-frontend` to ignore tests with cycle error when the parallel-frontend is enabled (by `--parallel-frontend-threads`) and enables `//@ compare-output-by-lines` by default. Context: [#t-compiler/parallel-rustc > Add the parallel front-end test suite @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fparallel-rustc/topic/Add.20the.20parallel.20front-end.20test.20suite/near/578781369) This PR should work with rust-lang/rust#153797 together.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #148235
This was checking that the blessed stderr was a subset of the stderr. Additionally, when re-blessing it would re-bless the subset that matched. So, when an extra line or comment was added and the line numbers changed, it would bless it without the line numbers.
I hope this is the intended behaviour now, my only concern is if the parallel frontend has cases where it sometimes emits less diagnostics than other times and so re-blessing the minimum was the intended behaviour? There used to be a flag called "compare-output-by-lines-subset" for this which used this logic.
r? @jieyouxu