Skip to content

internal: Define rules for LLMs#21732

Merged
Veykril merged 1 commit intorust-lang:masterfrom
Wilfred:add_claude_rules
Mar 5, 2026
Merged

internal: Define rules for LLMs#21732
Veykril merged 1 commit intorust-lang:masterfrom
Wilfred:add_claude_rules

Conversation

@Wilfred
Copy link
Contributor

@Wilfred Wilfred commented Mar 2, 2026

rust-analyzer allows AI usage (see #21314), but requires contributors to declare usage. This adds a rule file that improves LLM output quality and instructs the LLM to declare usage in commit messages.

I've written the rules in CLAUDE.md, but also symlinked it to AGENTS.md so other LLM tools pick it up.

Rules file contents

(1) Instructions for both humans and AIs to declare AI usage.

(2) Relevant commands for testing, linting and codegen.

Note that I deliberately didn't include an overview of the project structure on a folder-by-folder basis. This can go stale, and there's some evidence that project structure can hurt LLM output quality overall.

See the following paper:

Evaluating AGENTS.md:
Are Repository-Level Context Files Helpful for Coding Agents?
https://arxiv.org/pdf/2602.11988

Testing

I exercised this change with the following contrived prompt. Note that in practice rust-analyzer is hitting review scaling limits for new code actions, but it was easy to test end-to-end.

Add a new code action that replaces the content of a string literal
with the text "banana".
...
commit it

This produced a functional code action with both Codex and Claude, and in both cases the commit message mentioned that it was AI generated. Example commit message:

Add "Replace string with banana" code action

Add a new assist that replaces a string literal's content with "banana"
when the cursor is on a STRING token.

AI: Generated with Claude Code (claude-opus-4-6).

I confirmed that the code action worked by testing a rust-analyzer build in Emacs, and also confirmed that the generated tests looked sensible.

AI Usage Disclosures

I wrote the first draft of the rules file with Opus 4.6, manually reviewed everything.

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 2, 2026
@rustbot

This comment has been minimized.

Copy link
Member

@Veykril Veykril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For any readers of this PR, given the PR used an assist as an example. Assists are very very unlikely to get merged these days. that goes even more so for ai generated ones

View changes since this review


Heavy/slow tests only run when `RUN_SLOW_TESTS` env var is set.

## Key Architectural Invariants
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can likely be improved more (in the future), its not really much stuff

rust-analyzer allows AI usage (see rust-lang#21314), but requires contributors
to declare usage. This adds a rule file that improves LLM output
quality and instructs the LLM to declare usage in commit messages.

I've written the rules in CLAUDE.md, but also symlinked it to
AGENTS.md so other LLM tools pick it up.

## Rules file contents

(1) Instructions for both humans and AIs to declare AI usage.

(2) Relevant commands for testing, linting and codegen.

Note that I deliberately didn't include an overview of the project
structure on a folder-by-folder basis. This can go stale, and there's
some evidence that project structure can hurt LLM output quality
overall.

See the following paper:

> Evaluating AGENTS.md:
> Are Repository-Level Context Files Helpful for Coding Agents?
> https://arxiv.org/pdf/2602.11988

## Testing

I exercised this change with the following contrived prompt. Note that
in practice rust-analyzer is hitting review scaling limits for new
code actions, but it was easy to test end-to-end.

> Add a new code action that replaces the content of a string literal
> with the text "banana".
...
> commit it

This produced a functional code action with both Codex and Claude, and
in both cases the commit message mentioned that it was AI
generated. Example commit message:

    Add "Replace string with banana" code action

    Add a new assist that replaces a string literal's content with "banana"
    when the cursor is on a STRING token.

    AI: Generated with Claude Code (claude-opus-4-6).

I confirmed that the code action worked by testing a rust-analyzer
build in Emacs, and also confirmed that the generated tests looked
sensible.

## AI Usage Disclosures

I wrote the first draft of the rules file with Opus 4.6, manually
reviewed everything.
@Wilfred Wilfred force-pushed the add_claude_rules branch from caae213 to ef3ab9a Compare March 3, 2026 11:23
@rustbot
Copy link
Collaborator

rustbot commented Mar 3, 2026

⚠️ Warning ⚠️

  • There are issue links (such as #123) in the commit messages of the following commits.
    Please move them to the PR description, to avoid spamming the issues with references to the commit, and so this bot can automatically canonicalize them to avoid issues with subtree.

@Veykril Veykril added this pull request to the merge queue Mar 5, 2026
Merged via the queue into rust-lang:master with commit 8f36fad Mar 5, 2026
16 checks passed
@rustbot rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 5, 2026
@Wilfred Wilfred deleted the add_claude_rules branch March 5, 2026 15:15
@lnicola
Copy link
Member

lnicola commented Mar 11, 2026

@Wilfred the bot can rewrite issue links in the PR descriptions, but not the commit messages, which kind of defeats the purpose of rewriting.

@Wilfred
Copy link
Contributor Author

Wilfred commented Mar 11, 2026

@lnicola sorry about this. All my PRs since #t-compiler/rust-analyzer > Warning: Issue links in the commit message should have proper fully qualified issue references, I hadn't understood before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants