internal: Define rules for LLMs#21732
Merged
Veykril merged 1 commit intorust-lang:masterfrom Mar 5, 2026
Merged
Conversation
This comment has been minimized.
This comment has been minimized.
Veykril
reviewed
Mar 3, 2026
|
|
||
| Heavy/slow tests only run when `RUN_SLOW_TESTS` env var is set. | ||
|
|
||
| ## Key Architectural Invariants |
Member
There was a problem hiding this comment.
This can likely be improved more (in the future), its not really much stuff
rust-analyzer allows AI usage (see rust-lang#21314), but requires contributors to declare usage. This adds a rule file that improves LLM output quality and instructs the LLM to declare usage in commit messages. I've written the rules in CLAUDE.md, but also symlinked it to AGENTS.md so other LLM tools pick it up. ## Rules file contents (1) Instructions for both humans and AIs to declare AI usage. (2) Relevant commands for testing, linting and codegen. Note that I deliberately didn't include an overview of the project structure on a folder-by-folder basis. This can go stale, and there's some evidence that project structure can hurt LLM output quality overall. See the following paper: > Evaluating AGENTS.md: > Are Repository-Level Context Files Helpful for Coding Agents? > https://arxiv.org/pdf/2602.11988 ## Testing I exercised this change with the following contrived prompt. Note that in practice rust-analyzer is hitting review scaling limits for new code actions, but it was easy to test end-to-end. > Add a new code action that replaces the content of a string literal > with the text "banana". ... > commit it This produced a functional code action with both Codex and Claude, and in both cases the commit message mentioned that it was AI generated. Example commit message: Add "Replace string with banana" code action Add a new assist that replaces a string literal's content with "banana" when the cursor is on a STRING token. AI: Generated with Claude Code (claude-opus-4-6). I confirmed that the code action worked by testing a rust-analyzer build in Emacs, and also confirmed that the generated tests looked sensible. ## AI Usage Disclosures I wrote the first draft of the rules file with Opus 4.6, manually reviewed everything.
caae213 to
ef3ab9a
Compare
Collaborator
|
Veykril
approved these changes
Mar 5, 2026
Member
|
@Wilfred the bot can rewrite issue links in the PR descriptions, but not the commit messages, which kind of defeats the purpose of rewriting. |
Contributor
Author
|
@lnicola sorry about this. All my PRs since #t-compiler/rust-analyzer > Warning: Issue links in the commit message should have proper fully qualified issue references, I hadn't understood before. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
rust-analyzer allows AI usage (see #21314), but requires contributors to declare usage. This adds a rule file that improves LLM output quality and instructs the LLM to declare usage in commit messages.
I've written the rules in CLAUDE.md, but also symlinked it to AGENTS.md so other LLM tools pick it up.
Rules file contents
(1) Instructions for both humans and AIs to declare AI usage.
(2) Relevant commands for testing, linting and codegen.
Note that I deliberately didn't include an overview of the project structure on a folder-by-folder basis. This can go stale, and there's some evidence that project structure can hurt LLM output quality overall.
See the following paper:
Testing
I exercised this change with the following contrived prompt. Note that in practice rust-analyzer is hitting review scaling limits for new code actions, but it was easy to test end-to-end.
This produced a functional code action with both Codex and Claude, and in both cases the commit message mentioned that it was AI generated. Example commit message:
I confirmed that the code action worked by testing a rust-analyzer build in Emacs, and also confirmed that the generated tests looked sensible.
AI Usage Disclosures
I wrote the first draft of the rules file with Opus 4.6, manually reviewed everything.