Consistent formatting/linting of Python code and pre-commit hooks

[atteggiani]

Formatting and linting of Python code for the released packages (and potentially for all our Python scripts) should be consistent across the Organisation.
This could be a first item in the agenda for the Python Standards Group shortly to be started.

This issue is a starting point for a discussion on the modalities to ensure Python Formatting/Liniting standards for our Python scripts, such as adding pre-commit hooks to do the job.

Please reply to this issue with your ideas and thoughts about the matter, and feel free to share suggested implementations.

[atteggiani]

Ruff to rule them all

I propose using ruff for both linting and formatting.

For the rules, I think we can have a discussion about how strict we want to be and if all scripts should have the same formatting/linting rules, or a more general set of rules should be applied as a base point, and then any repo could include other rules if they feel they want to be more restrictive (possibly no "basic" rules should be ignored by any repo).

ruff.toml

A starting point for a ruff.toml file (discussion can be raised also on whether we should embed ruff configs within the pyproject.toml instead of having a standalone ruff.toml) can be the following (maybe too restrictive so more rules can be ignored):

# Eclude common paths
exclude = [
    ".bzr",
    ".direnv",
    ".eggs",
    ".git",
    ".git-rewrite",
    ".hg",
    ".ipynb_checkpoints",
    ".mypy_cache",
    ".nox",
    ".pants.d",
    ".pyenv",
    ".pytest_cache",
    ".pytype",
    ".ruff_cache",
    ".svn",
    ".tox",
    ".venv",
    ".vscode",
    "__pypackages__",
    "_build",
    "buck-out",
    "build",
    "dist",
    "node_modules",
    "site-packages",
    "venv",
    "__init__.py",
]

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

line-length = 120
indent-width = 4

[format]
quote-style = "double"  
indent-style = "space"

[lint]
select = [
    "ALL", # include all the rules, including new ones
]
ignore = [
    #### modules
    "ANN", # flake8-annotations
    "COM", # flake8-commas
    "DJ",  # django
    "EXE", # flake8-executable
    "T10", # debugger
    "TID", # flake8-tidy-imports

    #### specific rules
    "D401", # non-imperative-mood
    "TD002", # missing-todo-author
    "TD003", # missing-todo-link    
    "FIX002", # line-contains-todo
]

pre-commit-config.yaml

For the pre-commit hooks, I think a starting point for the pre-commit-config.yaml file could be the following:

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
    -   id: check-yaml
    -   id: check-toml
    -   id: name-tests-test
    -   id: debug-statements
    -   id: requirements-txt-fixer
# Run ruff linter and formatter.
-   repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: 'v0.7.2'
    hooks:
    -   id: ruff
        args: ['--fix']
    -   id: ruff-format
# Run the type checker.
-   repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.13.0
    hooks:
    -   id: mypy

[atteggiani]

Related to #7