Add hygiene annotations for tokens in `macro_rules!` bodies by aytey · Pull Request #153308 · rust-lang/rust

aytey · 2026-03-02T20:28:53Z

-Zunpretty=expanded,hygiene was not printing syntax context annotations for identifiers and lifetimes inside macro_rules! bodies. These tokens are printed via print_tt() → token_to_string_ext(), which converts tokens to strings without calling ann_post(). This meant that macro-generated macro_rules! definitions with hygienic metavar parameters (e.g. multiple $marg distinguished only by hygiene) were printed with no way to tell them apart.

This was fixed by adding a match on token.kind in print_tt() to call ann_post() for Ident, NtIdent, Lifetime, and NtLifetime tokens, matching how print_ident() and print_lifetime() already handle AST-level identifiers and lifetimes.

`-Zunpretty=expanded,hygiene` was not printing syntax context annotations for identifiers and lifetimes inside `macro_rules!` bodies. These tokens are printed via `print_tt()` → `token_to_string_ext()`, which converts tokens to strings without calling `ann_post()`. This meant that macro-generated `macro_rules!` definitions with hygienic metavar parameters (e.g. multiple `$marg` distinguished only by hygiene) were printed with no way to tell them apart. This was fixed by adding a match on `token.kind` in `print_tt()` to call `ann_post()` for `Ident`, `NtIdent`, `Lifetime`, and `NtLifetime` tokens, matching how `print_ident()` and `print_lifetime()` already handle AST-level identifiers and lifetimes. Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>

rustbot · 2026-03-02T20:28:57Z

r? @jackh726

rustbot has assigned @jackh726.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: compiler
compiler expanded to 69 candidates
Random selection from 15 candidates

aytey · 2026-03-02T20:30:33Z

r? @jdonszelmann

(I hope this appropriate; I only selected @jdonszelmann as she helped me with my previous commit in the same area)

Add `unpretty-debug-shadow` test covering macro body tokens that reference a shadowed variable, and simplify the `unpretty-debug-metavars` test macro. Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>

Kivooeo · 2026-03-06T13:02:07Z

tests/ui/hygiene/unpretty-debug-metavars.rs

+//@ check-pass
+//@ compile-flags: -Zunpretty=expanded,hygiene
+
+// Regression test for token hygiene annotations in -Zunpretty=expanded,hygiene


is there an issue about it?

@Kivooeo no, mainly because I discovered this bug while looking through the hygiene output -- I thought about making an issue, but I realised it would be just as quick to fix the issue ... hence this PR!

I guess two follow-ons:

Is it the word "regression" that suggests there should be an issue? Should I change this just to "test"?

Would you prefer it if I created an issue that this PR could then close?

aytey · 2026-03-11T15:39:39Z

Any thoughts @jdonszelmann / @Kivooeo? 😊

aytey · 2026-03-16T21:54:29Z

@fmease I don't suppose you'd be interested in taking a look at this?

jdonszelmann · 2026-03-18T14:34:14Z

This makes the output very verbose, but idk, that's maybe what you want from this unstable feature. I don't necessarily need to see an issue, unless this is part of a larger improvement of the unstable flag which idk if it is. If this just fixes a bug I'm ok. Regardless, this is not stabilized behavior, it seems more complete to me than it was before, and the behavior is well documented in the test, so I'm going to approve.

@bors r+ rollup

rust-bors · 2026-03-18T14:34:19Z

📌 Commit bf6db4f has been approved by jdonszelmann

It is now in the queue for this repository.

…elmann Add hygiene annotations for tokens in `macro_rules!` bodies `-Zunpretty=expanded,hygiene` was not printing syntax context annotations for identifiers and lifetimes inside `macro_rules!` bodies. These tokens are printed via `print_tt()` → `token_to_string_ext()`, which converts tokens to strings without calling `ann_post()`. This meant that macro-generated `macro_rules!` definitions with hygienic metavar parameters (e.g. multiple `$marg` distinguished only by hygiene) were printed with no way to tell them apart. This was fixed by adding a match on `token.kind` in `print_tt()` to call `ann_post()` for `Ident`, `NtIdent`, `Lifetime`, and `NtLifetime` tokens, matching how `print_ident()` and `print_lifetime()` already handle AST-level identifiers and lifetimes.

…uwer Rollup of 5 pull requests Successful merges: - #153308 (Add hygiene annotations for tokens in `macro_rules!` bodies) - #153557 (fix inference variables leaking into HIR const literal lowering logic) - #153913 (Fix some suggestions of the `for-loops-over-fallibles` lint) - #153987 (mGCA: Lower const generic args to infer when needed) - #153992 (bootstrap: Optionally print a backtrace if a command fails)

…elmann Add hygiene annotations for tokens in `macro_rules!` bodies `-Zunpretty=expanded,hygiene` was not printing syntax context annotations for identifiers and lifetimes inside `macro_rules!` bodies. These tokens are printed via `print_tt()` → `token_to_string_ext()`, which converts tokens to strings without calling `ann_post()`. This meant that macro-generated `macro_rules!` definitions with hygienic metavar parameters (e.g. multiple `$marg` distinguished only by hygiene) were printed with no way to tell them apart. This was fixed by adding a match on `token.kind` in `print_tt()` to call `ann_post()` for `Ident`, `NtIdent`, `Lifetime`, and `NtLifetime` tokens, matching how `print_ident()` and `print_lifetime()` already handle AST-level identifiers and lifetimes.

…uwer Rollup of 9 pull requests Successful merges: - #153957 (Add bootstrap step for stdarch-verify) - #153727 (When single impl can satisfy inference error, suggest type) - #153308 (Add hygiene annotations for tokens in `macro_rules!` bodies) - #153557 (fix inference variables leaking into HIR const literal lowering logic) - #153913 (Fix some suggestions of the `for-loops-over-fallibles` lint) - #153987 (mGCA: Lower const generic args to infer when needed) - #153992 (bootstrap: Optionally print a backtrace if a command fails) - #154036 (borrowck/type_check: remove helper left-over from unsized locals) - #154038 (merge `regions-outlives-nominal-type-*` tests into one file)

…elmann Add hygiene annotations for tokens in `macro_rules!` bodies `-Zunpretty=expanded,hygiene` was not printing syntax context annotations for identifiers and lifetimes inside `macro_rules!` bodies. These tokens are printed via `print_tt()` → `token_to_string_ext()`, which converts tokens to strings without calling `ann_post()`. This meant that macro-generated `macro_rules!` definitions with hygienic metavar parameters (e.g. multiple `$marg` distinguished only by hygiene) were printed with no way to tell them apart. This was fixed by adding a match on `token.kind` in `print_tt()` to call `ann_post()` for `Ident`, `NtIdent`, `Lifetime`, and `NtLifetime` tokens, matching how `print_ident()` and `print_lifetime()` already handle AST-level identifiers and lifetimes.

…uwer Rollup of 9 pull requests Successful merges: - #153957 (Add bootstrap step for stdarch-verify) - #153727 (When single impl can satisfy inference error, suggest type) - #153308 (Add hygiene annotations for tokens in `macro_rules!` bodies) - #153557 (fix inference variables leaking into HIR const literal lowering logic) - #153913 (Fix some suggestions of the `for-loops-over-fallibles` lint) - #153987 (mGCA: Lower const generic args to infer when needed) - #153992 (bootstrap: Optionally print a backtrace if a command fails) - #154036 (borrowck/type_check: remove helper left-over from unsized locals) - #154038 (merge `regions-outlives-nominal-type-*` tests into one file)

…elmann Add hygiene annotations for tokens in `macro_rules!` bodies `-Zunpretty=expanded,hygiene` was not printing syntax context annotations for identifiers and lifetimes inside `macro_rules!` bodies. These tokens are printed via `print_tt()` → `token_to_string_ext()`, which converts tokens to strings without calling `ann_post()`. This meant that macro-generated `macro_rules!` definitions with hygienic metavar parameters (e.g. multiple `$marg` distinguished only by hygiene) were printed with no way to tell them apart. This was fixed by adding a match on `token.kind` in `print_tt()` to call `ann_post()` for `Ident`, `NtIdent`, `Lifetime`, and `NtLifetime` tokens, matching how `print_ident()` and `print_lifetime()` already handle AST-level identifiers and lifetimes.

…uwer Rollup of 19 pull requests Successful merges: - #152870 (Do not enable split debuginfo for windows-gnu) - #153333 (Fix bootstrap rust build failure for vxworks) - #153681 (ci: add runners for vanilla LLVM 22) - #153727 (When single impl can satisfy inference error, suggest type) - #153824 (Add `Wake` diagnostic item for `alloc::task::Wake`) - #154077 (Optimize 128-bit integer formatting) - #154078 (`define_callbacks` tweaks) - #151905 (Split the `dec2flt::RawFloat` trait for easier reuse) - #153170 (Add is_disconnected functions to mpsc and mpmc channels) - #153308 (Add hygiene annotations for tokens in `macro_rules!` bodies) - #153557 (fix inference variables leaking into HIR const literal lowering logic) - #153804 (Derive Macro Eq: link to more detailed documentation) - #153913 (Fix some suggestions of the `for-loops-over-fallibles` lint) - #153974 (Point at return type when it is the source of the type expectation) - #153987 (mGCA: Lower const generic args to infer when needed) - #154018 (simplify and remove `tests/ui/kindck` and related tests) - #154036 (borrowck/type_check: remove helper left-over from unsized locals) - #154038 (merge `regions-outlives-nominal-type-*` tests into one file) - #154041 (bootstrap: Allow `--bless`ing changes to editor settings files)

rustbot assigned jackh726 Mar 2, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 2, 2026

aytey changed the title ~~Add hygiene annotations for tokens in macro_rules! bodies~~ Add hygiene annotations for tokens in macro_rules! bodies Mar 2, 2026

rustbot assigned jdonszelmann and unassigned jackh726 Mar 2, 2026

Add regression tests for token hygiene annotations in macro bodies

bf6db4f

Add `unpretty-debug-shadow` test covering macro body tokens that reference a shadowed variable, and simplify the `unpretty-debug-metavars` test macro. Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>

Kivooeo reviewed Mar 6, 2026

View reviewed changes

rust-bors bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 18, 2026

JonathanBrouwer mentioned this pull request Mar 18, 2026

Rollup of 5 pull requests #154047

Closed

JonathanBrouwer mentioned this pull request Mar 18, 2026

Rollup of 9 pull requests #154062

Closed

Zalathar mentioned this pull request Mar 19, 2026

Rollup of 20 pull requests #154079

Closed

JonathanBrouwer mentioned this pull request Mar 19, 2026

Rollup of 20 pull requests #154080

Closed

This was referenced Mar 19, 2026

Rollup of 20 pull requests #154091

Closed

Rollup of 19 pull requests #154092

Merged

rust-bors bot merged commit e103550 into rust-lang:main Mar 19, 2026
11 checks passed

rustbot added this to the 1.96.0 milestone Mar 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add hygiene annotations for tokens in `macro_rules!` bodies#153308

Add hygiene annotations for tokens in `macro_rules!` bodies#153308
rust-bors[bot] merged 2 commits intorust-lang:mainfrom
aytey:macro_meta_hygiene

aytey commented Mar 2, 2026

Uh oh!

rustbot commented Mar 2, 2026

Uh oh!

aytey commented Mar 2, 2026

Uh oh!

Kivooeo Mar 6, 2026

Uh oh!

aytey Mar 6, 2026

Uh oh!

aytey Mar 6, 2026

Uh oh!

aytey commented Mar 11, 2026

Uh oh!

aytey commented Mar 16, 2026

Uh oh!

jdonszelmann commented Mar 18, 2026

Uh oh!

rust-bors bot commented Mar 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

aytey commented Mar 2, 2026

Uh oh!

rustbot commented Mar 2, 2026

Uh oh!

aytey commented Mar 2, 2026

Uh oh!

Kivooeo Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

aytey Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

aytey Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

aytey commented Mar 11, 2026

Uh oh!

aytey commented Mar 16, 2026

Uh oh!

jdonszelmann commented Mar 18, 2026

Uh oh!

rust-bors bot commented Mar 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants