Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ cargo insta test --review

Will run the tests, generate the new snapshot references and let you review them.

### Updating grammars
Have a look at
<a href="https://mozilla.github.io/rust-code-analysis/developers/update-grammars.html" target="_blank">Update grammars guide</a>
to learn how to update languages grammars.

# Contributing

If you want to contribute to the development of this software, have a look at the
Expand Down
1 change: 1 addition & 0 deletions rust-code-analysis-book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@
- [Developers Guide](developers/README.md)
- [How-to: Add a new language](developers/new-language.md)
- [How-to: Implement LoC](developers/loc.md)
- [How-to: Update grammars](developers/update-grammars.md)
60 changes: 60 additions & 0 deletions rust-code-analysis-book/src/developers/update-grammars.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Update grammars

Each programming language needs to be parsed in order to extract its syntax and semantic: the so-called grammar of a language.
In `rust-code-analysis`, we use [tree-sitter](https://github.com/tree-sitter) as parsing library since it provides a set of distinct grammars for each of our
supported programming languages. But a grammar is not a static monolith, it changes over time, and it can also be affected by bugs,
hence it is necessary to update it every now and then.

As now, since we have used `bash` scripts to automate the operations, grammars can be updated natively **only** on `Linux` and `MacOS` systems, but these scripts can also run on `Windows` using `WSL`.

In `rust-code-analysis` we use both **third-party** and **internal** grammars.
The first ones are published on `crates.io` and maintained by external developers,
while the second ones have been thought and defined inside the project to manage variant of some languages
used in `Firefox`.
We are going to explain how to update both of them in the following sections.

## Third-party grammars

Update the grammar version in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-java` grammar

```toml
tree-sitter-java = "x.xx.x"
```
where `x` represents a digit.

Run `./recreate-grammars` to recreate and refresh all grammars structures and data

```bash
./recreate-grammars
```

Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems
introduced by changes in the grammars.

Commit your changes and create a new pull request

## Internal grammars

Update dependency `version` field in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-ccomment` grammar

```toml
tree-sitter-ccomment = { path = "./tree-sitter-ccomment", version = "=x.xx.x" }
```
where `x` represents a digit.

Open the `Cargo.toml` file of the chosen grammar and:
- Set its version to the **same** value present in the main `Cargo.toml` file
- Increase the `tree-sitter` version to the most recent one

Run `./generate-grammars/generate-grammar.sh` which updates the grammar recreating and refreshing every file and script.
This script requires the name of the grammar as mandatory argument.
Below an example always using the `tree-sitter-ccomment` grammar

```bash
./generate-grammars/generate-grammar.sh tree-sitter-ccomment
```

Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems
introduced by changes in the grammars.

Commit your changes and create a new pull request