Unexpected inclusion of uv.lock under toml

[bn-andrew]

The latest release of identify (2.6.17) now includes uv.lock as a toml type file.

See this PR / comment: #569 (comment)

Because it is a generated file rather than a user editable one, I am not sure this is desirable behaviour.

Our ruff pre-commit hook looks like this:

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.14.5
    hooks:
      - id: ruff-check
        name: ruff
        args:
          - --fix
          - --show-fixes
          - --exit-non-zero-on-fix
        types_or: [python, pyi, toml]
        # This will also validate pyproject.toml, note ruff doesn't have rules for any other toml file.

Due to this change, uv.lock is now being passed in to ruff also and causing it to raise 3000+ errors.

It took a while to debug this as I wasn't expecting a .lock file to be included with a toml flag (it wasn't till I found the source code / the above PR till I actually read the docs and realised it was toml)

I should be able to use some exclude rules to prevent this error but I am not sure this behaviour makes sense as the default.

I guess one option could be a generated or managed tag for files like this, so that hooks targeting manually edited files can ignore it?
Currently identify-cli uv.lock returns ["file", "non-executable", "text"] which other than the name / extension isn't overly easy to filter against.

I suspect there will be further instances of uv users running into variations of this issue.

Thanks

[asottile]

types_or: [..., toml] for ruff is nonsense. any other toml file in your code base would also match that.

[asottile]

also idk where you cargo culted --exit-non-zero-on-fix that's also nonsense since pre-commit will already fail any hook which makes modifications

[bn-andrew]

Thanks for that feedback.

any other toml file in your code base would also match that.
pyproject.toml is the only toml file (other than apparently uv.lock) in our repos and if it's not, ruff doesn't hate the others.

--exit-non-zero-on-fix I suspect that is a leftover from either an older version of ruff or a usage outside of pre-commit and it wasn't noticed that it wasn't strictly necessary here..

What is the recommended way to include pyproject.toml and not other toml files? Or does it need to live in it's own hook?

[asottile]

why are you even including pyproject.toml in the first place?

[bn-andrew]

Because we have had issues with pyproject getting broken by both humans and scripts in subtle ways.
Other tools we have attempted to use have introduced sorting which broke our commenting and other negative side effects.

So at the present point in time, this is currently our best option to have some validation without breaking our current setup.

[asottile]

I'm still not seeing it. ruff pyproject.toml doesn't do anything useful you kinda need python files for it to validate

[bn-andrew]

https://docs.astral.sh/ruff/rules/invalid-pyproject-toml/#invalid-pyproject-toml-ruf200
ruff has a specific rule that applies to pyproject.toml and checks for several schema and PEP related issues.

Also detects basic issues like duplicated keys:

ruff.....................................................................Failed
- hook id: ruff
- exit code: 2

ruff failed
  Cause: Failed to parse <.>/pyproject.toml
  Cause: TOML parse error at line 40, column 1
   |
40 | authors = [ { name = "Company", email = "user@company.com" } ]
   | ^
duplicate key `authors` in table `project`

[asottile]

ok then

have you read the pre-commit docs on how to use types properly?

[bn-andrew]

Yes I did read the documentation here: https://pre-commit.com/#filtering-files-with-types
That was how I found the link to this file https://github.com/pre-commit/identify/blob/main/identify/extensions.py (tags by extension / naming convention)
It seems like I can't really use types with a specific extension together in the same hook practically.

The last example in the types section is the closest thing I can see but then types is just everything and I am just using a regex..

I considered just excluding uv.lock in the top level exclude, but then it would be excluded in the one hook I do need to touch this file. So that doesn't seem like an option.
I am sure I have seen it before somewhere but I also couldn't find a clear indication of whether exclude or include took priority when combining these rules.

I am sure there is a way to do it but right now I can't see it.

After having spent several hours on trying to figure out why something that was working fine last week and didn't work today, it seems like the fastest fix, so I can get back to other priorities is ruff's file selection rules.
https://docs.astral.sh/ruff/configuration/#full-command-line-interface

There is probably very little if any difference as to which application does the filtering.

[mxr]

@bn-andrew you can mix types_or and files. So you can achieve what you want with

    types_or: ['python', 'pyi', 'toml']
    files: (pyproject.toml|\.pyi?)$

Alternatively, if you don't want to hardcode python file extensions, you could instead add a second ruff-check hook definition that only targets pyproject.toml

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.14.5
    hooks:
      - id: ruff-check
        name: ruff
        args:
          - --fix
          - --show-fixes
      - id: ruff-check (pyproject.toml)
        name: ruff
        args:
          - --fix
          - --show-fixes
        types_or: [toml]
        files: pyproject.toml

You can troubleshoot file/type matching with a local hook like this

- repo: local
  hooks:
  - id: print-passed-files
    name: print-passed-files
    language: system
    entry: sh -c 'printf "%s\n" "$@"' --
    types_or: ['python', 'pyi', 'toml']
    files: (pyproject.toml|\.pyi?)$

$ touch example.py example.pyi pyproject.toml whatever.toml uv.lock 
$ for f (example.py example.pyi pyproject.toml whatever.toml uv.lock); do printf '%s: %s\n' "$f" "$(identify-cli "$f" | head -n1)"; done
example.py: ["file", "non-executable", "python", "text"]
example.pyi: ["file", "non-executable", "pyi", "text"]
pyproject.toml: ["file", "non-executable", "text", "toml"]
whatever.toml: ["file", "non-executable", "text", "toml"]
uv.lock: ["file", "non-executable", "text"]
$ pre-commit run print-passed-files --verbose --files example.py example.pyi pyproject.toml whatever.toml uv.lock                                
print-passed-files.......................................................Passed
- hook id: print-passed-files
- duration: 0.01s

example.py
example.pyi
pyproject.toml

[bn-andrew]

Thanks @mxr !
That's exceptionally helpful, I will experiment some more with that when I can 👍🏻

[asottile]

I think what you actually want is

    - id: ruff
    - id: ruff
      types_or: [file]
      files: ^pyproject\.toml$