Skip to content

[CVAT] [Exchange Oracle] Add ruff#2395

Merged
zhiltsov-max merged 69 commits into
developfrom
ba/add-ruff
Aug 27, 2024
Merged

[CVAT] [Exchange Oracle] Add ruff#2395
zhiltsov-max merged 69 commits into
developfrom
ba/add-ruff

Conversation

@Bobronium
Copy link
Copy Markdown
Contributor

@Bobronium Bobronium commented Aug 16, 2024

Description

Adds ruff and integrates linting into the CI pipeline.

Ruff combines features from flake8 (with plugins), pylint, isort, black, autopep8, flynt, etc., into one extremely fast and well-maintained package.

One of the main features is availability of automatic fixes for some violations.

Relying on a tool like Ruff helps to reduce cognitive load when writing/reading the code by enforcing consistency.

Summary of changes

I've adopted the following strategy for integrating Ruff:

  • Reformatted the codebase.
  • Configured Ruff to report all violations.
  • Applied safe fixes each in its own commit.
  • Applied unsafe fixes each in its own commit.
  • Handled the most trivial violations manually.
  • Configured Ruff to ignore the noisiest/questionable violations.
  • Used # noqa for remaining violations where necessary.

Each commit with autofix has a comment explaining the reasoning behind the changes.

How to test the changes

The code should be semantically equivalent, with only stylistic and formatting changes. I've ensured that all tests pass.

References

After this PR and #2396 will get merged, I'll introduce a PR adding .git-blame-ignore-revs with revisions from merge commits, so these changes will be ignored by git blame for better history overview.

@vercel
Copy link
Copy Markdown

vercel Bot commented Aug 16, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
escrow-dashboard ⬜️ Ignored (Inspect) Visit Preview Aug 26, 2024 10:47am
faucet-server ⬜️ Ignored (Inspect) Visit Preview Aug 26, 2024 10:47am

Checks that "special" methods, like `__init__`, `__new__`, and `__call__`, have
return type annotations.

Type annotations are a good way to document the return types of functions. They also
help catch bugs, when used alongside a type checker, by ensuring that the types of
any returned values, and the types expected by callers, match expectation.

Note that type checkers often allow you to omit the return type annotation for
`__init__` methods, as long as at least one argument has a type annotation. To
opt in to this behavior, use the `mypy-init-return` setting in your `pyproject.toml`
or `ruff.toml` file:

```toml
[tool.ruff.lint.flake8-annotations]
mypy-init-return = true
```

```python
class Foo:
    def __init__(self, x: int):
        self.x = x
```

Use instead:
```python
class Foo:
    def __init__(self, x: int) -> None:
        self.x = x
```
Checks for membership tests using `not {element} in {collection}`.

Testing membership with `{element} not in {collection}` is more readable.

```python
Z = not X in Y
if not X.B in Y:
    pass
```

Use instead:
```python
Z = X not in Y
if X.B not in Y:
    pass
```
Checks for f-strings that do not contain any placeholder expressions.

f-strings are a convenient way to format strings, but they are not
necessary if there are no placeholder expressions to format. In this
case, a regular string should be used instead, as an f-string without
placeholders can be confusing for readers, who may expect such a
placeholder to be present.

An f-string without any placeholders could also indicate that the
author forgot to add a placeholder expression.

```python
f"Hello, world!"
```

Use instead:
```python
"Hello, world!"
```

**Note:** to maintain compatibility with PyFlakes, this rule only flags
f-strings that are part of an implicit concatenation if _none_ of the
f-string segments contain placeholder expressions.

For example:

```python
(
    f"Hello,"
    f" {name}!"
)

(
    f"Hello,"
    f" World!"
)
```

See [#10885](astral-sh/ruff#10885) for more.

- [PEP 498](https://www.python.org/dev/peps/pep-0498/)
Checks for the presence of unused variables in function scopes.

A variable that is defined but not used is likely a mistake, and should
be removed to avoid confusion.

If a variable is intentionally defined-but-not-used, it should be
prefixed with an underscore, or some other value that adheres to the
[`lint.dummy-variable-rgx`] pattern.

Under [preview mode](https://docs.astral.sh/ruff/preview), this rule also
triggers on unused unpacked assignments (for example, `x, y = foo()`).

```python
def foo():
    x = 1
    y = 2
    return x
```

Use instead:
```python
def foo():
    x = 1
    return x
```

- `lint.dummy-variable-rgx`
Checks for the presence of a `return None` statement when `None` is the only
possible return value.

Python implicitly assumes `return None` if an explicit `return` value is
omitted. Therefore, explicitly returning `None` is redundant and should be
avoided when it is the only possible `return` value across all code paths
in a given function.

```python
def foo(bar):
    if not bar:
        return
    return None
```

Use instead:
```python
def foo(bar):
    if not bar:
        return
    return
```
Checks for `else` statements with a `return` statement in the preceding
`if` block.

The `else` statement is not needed as the `return` statement will always
break out of the enclosing function. Removing the `else` will reduce
nesting and make the code more readable.

```python
def foo(bar, baz):
    if bar:
        return 1
    else:
        return baz
```

Use instead:
```python
def foo(bar, baz):
    if bar:
        return 1
    return baz
```
Checks for `else` statements with a `raise` statement in the preceding `if`
block.

The `else` statement is not needed as the `raise` statement will always
break out of the current scope. Removing the `else` will reduce nesting
and make the code more readable.

```python
def foo(bar, baz):
    if bar == "Specific Error":
        raise Exception(bar)
    else:
        raise Exception(baz)
```

Use instead:
```python
def foo(bar, baz):
    if bar == "Specific Error":
        raise Exception(bar)
    raise Exception(baz)
```
Checks for `else` statements with a `continue` statement in the preceding
`if` block.

The `else` statement is not needed, as the `continue` statement will always
continue onto the next iteration of a loop. Removing the `else` will reduce
nesting and make the code more readable.

```python
def foo(bar, baz):
    for i in bar:
        if i < baz:
            continue
        else:
            x = 0
```

Use instead:
```python
def foo(bar, baz):
    for i in bar:
        if i < baz:
            continue
        x = 0
```
Checks for negated `==` operators.

Negated `==` operators are less readable than `!=` operators. When testing
for non-equality, it is more common to use `!=` than `==`.

```python
not a == b
```

Use instead:
```python
a != b
```

- [Python documentation: Comparisons](https://docs.python.org/3/reference/expressions.html#comparisons)
Checks for `dict.get()` calls that pass `None` as the default value.

`None` is the default value for `dict.get()`, so it is redundant to pass it
explicitly.

```python
ages = {"Tom": 23, "Maria": 23, "Dog": 11}
age = ages.get("Cat", None)
```

Use instead:
```python
ages = {"Tom": 23, "Maria": 23, "Dog": 11}
age = ages.get("Cat")
```

- [Python documentation: `dict.get`](https://docs.python.org/3/library/stdtypes.html#dict.get)
Checks for uses of deprecated methods from the `unittest` module.

The `unittest` module has deprecated aliases for some of its methods.
The aliases may be removed in future versions of Python. Instead,
use their non-deprecated counterparts.

```python
from unittest import TestCase

class SomeTest(TestCase):
    def test_something(self):
        self.assertEquals(1, 1)
```

Use instead:
```python
from unittest import TestCase

class SomeTest(TestCase):
    def test_something(self):
        self.assertEqual(1, 1)
```

- [Python documentation: Deprecated aliases](https://docs.python.org/3/library/unittest.html#deprecated-aliases)
Checks for the use of generics that can be replaced with standard library
variants based on [PEP 585].

[PEP 585] enabled collections in the Python standard library (like `list`)
to be used as generics directly, instead of importing analogous members
from the `typing` module (like `typing.List`).

When available, the [PEP 585] syntax should be used instead of importing
members from the `typing` module, as it's more concise and readable.
Importing those members from `typing` is considered deprecated as of [PEP
585].

This rule is enabled when targeting Python 3.9 or later (see:
[`target-version`]). By default, it's _also_ enabled for earlier Python
versions if `from __future__ import annotations` is present, as
`__future__` annotations are not evaluated at runtime. If your code relies
on runtime type annotations (either directly or via a library like
Pydantic), you can disable this behavior for Python versions prior to 3.9
by setting [`lint.pyupgrade.keep-runtime-typing`] to `true`.

```python
from typing import List

foo: List[int] = [1, 2, 3]
```

Use instead:
```python
foo: list[int] = [1, 2, 3]
```

This rule's fix is marked as unsafe, as it may lead to runtime errors when
alongside libraries that rely on runtime type annotations, like Pydantic,
on Python versions prior to Python 3.9.

- `target-version`
- `lint.pyupgrade.keep-runtime-typing`

[PEP 585]: https://peps.python.org/pep-0585/
Check for type annotations that can be rewritten based on [PEP 604] syntax.

[PEP 604] introduced a new syntax for union type annotations based on the
`|` operator. This syntax is more concise and readable than the previous
`typing.Union` and `typing.Optional` syntaxes.

This rule is enabled when targeting Python 3.10 or later (see:
[`target-version`]). By default, it's _also_ enabled for earlier Python
versions if `from __future__ import annotations` is present, as
`__future__` annotations are not evaluated at runtime. If your code relies
on runtime type annotations (either directly or via a library like
Pydantic), you can disable this behavior for Python versions prior to 3.10
by setting [`lint.pyupgrade.keep-runtime-typing`] to `true`.

```python
from typing import Union

foo: Union[int, str] = 1
```

Use instead:
```python
foo: int | str = 1
```

This rule's fix is marked as unsafe, as it may lead to runtime errors when
alongside libraries that rely on runtime type annotations, like Pydantic,
on Python versions prior to Python 3.10. It may also lead to runtime errors
in unusual and likely incorrect type annotations where the type does not
support the `|` operator.

- `target-version`
- `lint.pyupgrade.keep-runtime-typing`

[PEP 604]: https://peps.python.org/pep-0604/
Checks for `str.format` calls that can be replaced with f-strings.

f-strings are more readable and generally preferred over `str.format`
calls.

```python
"{}".format(foo)
```

Use instead:
```python
f"{foo}"
```

- [Python documentation: f-strings](https://docs.python.org/3/reference/lexical_analysis.html#f-strings)
Checks for uses of deprecated imports based on the minimum supported
Python version.

Deprecated imports may be removed in future versions of Python, and
should be replaced with their new equivalents.

Note that, in some cases, it may be preferable to continue importing
members from `typing_extensions` even after they're added to the Python
standard library, as `typing_extensions` can backport bugfixes and
optimizations from later Python versions. This rule thus avoids flagging
imports from `typing_extensions` in such cases.

```python
from collections import Sequence
```

Use instead:
```python
from collections.abc import Sequence
```
Checks for the presence of unnecessary quotes in type annotations.

In Python, type annotations can be quoted to avoid forward references.

However, if `from __future__ import annotations` is present, Python
will always evaluate type annotations in a deferred manner, making
the quotes unnecessary.

Similarly, if the annotation is located in a typing-only context and
won't be evaluated by Python at runtime, the quotes will also be
considered unnecessary. For example, Python does not evaluate type
annotations on assignments in function bodies.

Given:

```python
from __future__ import annotations

def foo(bar: "Bar") -> "Bar": ...
```

Use instead:

```python
from __future__ import annotations

def foo(bar: Bar) -> Bar: ...
```

Given:

```python
def foo() -> None:
    bar: "Bar"
```

Use instead:

```python
def foo() -> None:
    bar: Bar
```

- [PEP 563](https://peps.python.org/pep-0563/)
- [Python documentation: `__future__`](https://docs.python.org/3/library/__future__.html#module-__future__)
…method

Checks that "special" methods, like `__init__`, `__new__`, and `__call__`, have
return type annotations.

Type annotations are a good way to document the return types of functions. They also
help catch bugs, when used alongside a type checker, by ensuring that the types of
any returned values, and the types expected by callers, match expectation.

Note that type checkers often allow you to omit the return type annotation for
`__init__` methods, as long as at least one argument has a type annotation. To
opt in to this behavior, use the `mypy-init-return` setting in your `pyproject.toml`
or `ruff.toml` file:

```toml
[tool.ruff.lint.flake8-annotations]
mypy-init-return = true
```

```python
class Foo:
    def __init__(self, x: int):
        self.x = x
```

Use instead:
```python
class Foo:
    def __init__(self, x: int) -> None:
        self.x = x
```
…ethod

Checks that static methods have return type annotations.

Type annotations are a good way to document the return types of functions. They also
help catch bugs, when used alongside a type checker, by ensuring that the types of
any returned values, and the types expected by callers, match expectation.

```python
class Foo:
    @staticmethod
    def bar():
        return 1
```

Use instead:
```python
class Foo:
    @staticmethod
    def bar() -> int:
        return 1
```
…thod

Checks that class methods have return type annotations.

Type annotations are a good way to document the return types of functions. They also
help catch bugs, when used alongside a type checker, by ensuring that the types of
any returned values, and the types expected by callers, match expectation.

```python
class Foo:
    @classmethod
    def bar(cls):
        return 1
```

Use instead:
```python
class Foo:
    @classmethod
    def bar(cls) -> int:
        return 1
```
Checks for uses of mutable objects as function argument defaults.

Function defaults are evaluated once, when the function is defined.

The same mutable object is then shared across all calls to the function.
If the object is modified, those modifications will persist across calls,
which can lead to unexpected behavior.

Instead, prefer to use immutable data structures, or take `None` as a
default, and initialize a new mutable object inside the function body
for each call.

Arguments with immutable type annotations will be ignored by this rule.
Types outside of the standard library can be marked as immutable with the
[`lint.flake8-bugbear.extend-immutable-calls`] configuration option.

Mutable argument defaults can be used intentionally to cache computation
results. Replacing the default with `None` or an immutable data structure
does not work for such usages. Instead, prefer the `@functools.lru_cache`
decorator from the standard library.

```python
def add_to_list(item, some_list=[]):
    some_list.append(item)
    return some_list

l1 = add_to_list(0)  # [0]
l2 = add_to_list(1)  # [0, 1]
```

Use instead:
```python
def add_to_list(item, some_list=None):
    if some_list is None:
        some_list = []
    some_list.append(item)
    return some_list

l1 = add_to_list(0)  # [0]
l2 = add_to_list(1)  # [1]
```

- `lint.flake8-bugbear.extend-immutable-calls`

- [Python documentation: Default Argument Values](https://docs.python.org/3/tutorial/controlflow.html#default-argument-values)
Checks for unused variables in loops (e.g., `for` and `while` statements).

Defining a variable in a loop statement that is never used can confuse
readers.

If the variable is intended to be unused (e.g., to facilitate
destructuring of a tuple or other object), prefix it with an underscore
to indicate the intent. Otherwise, remove the variable entirely.

```python
for i, j in foo:
    bar(i)
```

Use instead:
```python
for i, _j in foo:
    bar(i)
```

- [PEP 8: Naming Conventions](https://peps.python.org/pep-0008/#naming-conventions)
Checks for uses of `assert False`.

Python removes `assert` statements when running in optimized mode
(`python -O`), making `assert False` an unreliable means of
raising an `AssertionError`.

Instead, raise an `AssertionError` directly.

```python
assert False
```

Use instead:
```python
raise AssertionError
```

This rule's fix is marked as unsafe, as changing an `assert` to a
`raise` will change the behavior of your program when running in
optimized mode (`python -O`).

- [Python documentation: `assert`](https://docs.python.org/3/reference/simple_stmts.html#the-assert-statement)
Checks for unnecessary generators that can be rewritten as `list`
comprehensions (or with `list` directly).

It is unnecessary to use `list` around a generator expression, since
there are equivalent comprehensions for these types. Using a
comprehension is clearer and more idiomatic.

Further, if the comprehension can be removed entirely, as in the case of
`list(x for x in foo)`, it's better to use `list(foo)` directly, since it's
even more direct.

```python
list(f(x) for x in foo)
list(x for x in foo)
list((x for x in foo))
```

Use instead:
```python
[f(x) for x in foo]
list(foo)
list(foo)
```

This rule's fix is marked as unsafe, as it may occasionally drop comments
when rewriting the call. In most cases, though, comments will be preserved.
Checks for unnecessary generators that can be rewritten as `set`
comprehensions (or with `set` directly).

It is unnecessary to use `set` around a generator expression, since
there are equivalent comprehensions for these types. Using a
comprehension is clearer and more idiomatic.

Further, if the comprehension can be removed entirely, as in the case of
`set(x for x in foo)`, it's better to use `set(foo)` directly, since it's
even more direct.

```python
set(f(x) for x in foo)
set(x for x in foo)
```

Use instead:
```python
{f(x) for x in foo}
set(foo)
```

This rule's fix is marked as unsafe, as it may occasionally drop comments
when rewriting the call. In most cases, though, comments will be preserved.
Checks for unnecessary `dict`, `list` or `tuple` calls that can be
rewritten as empty literals.

It's unnecessary to call, e.g., `dict()` as opposed to using an empty
literal (`{}`). The former is slower because the name `dict` must be
looked up in the global scope in case it has been rebound.

```python
dict()
dict(a=1, b=2)
list()
tuple()
```

Use instead:
```python
{}
{"a": 1, "b": 2}
[]
()
```

This rule's fix is marked as unsafe, as it may occasionally drop comments
when rewriting the call. In most cases, though, comments will be preserved.

- `lint.flake8-comprehensions.allow-dict-calls-with-keyword-arguments`
Checks for lambda expressions which are assigned to a variable.

Per PEP 8, you should "Always use a def statement instead of an assignment
statement that binds a lambda expression directly to an identifier."

Using a `def` statement leads to better tracebacks, and the assignment
itself negates the primary benefit of using a `lambda` expression (i.e.,
that it can be embedded inside another expression).

```python
f = lambda x: 2 * x
```

Use instead:
```python
def f(x):
    return 2 * x
```

[PEP 8]: https://peps.python.org/pep-0008/#programming-recommendations
Checks for the presence of unused variables in function scopes.

A variable that is defined but not used is likely a mistake, and should
be removed to avoid confusion.

If a variable is intentionally defined-but-not-used, it should be
prefixed with an underscore, or some other value that adheres to the
[`lint.dummy-variable-rgx`] pattern.

Under [preview mode](https://docs.astral.sh/ruff/preview), this rule also
triggers on unused unpacked assignments (for example, `x, y = foo()`).

```python
def foo():
    x = 1
    y = 2
    return x
```

Use instead:
```python
def foo():
    x = 1
    return x
```

- `lint.dummy-variable-rgx`
Comment thread packages/examples/cvat/exchange-oracle/pyproject.toml Outdated
Comment thread packages/examples/cvat/exchange-oracle/src/cvat/api_calls.py
Comment thread packages/examples/cvat/exchange-oracle/src/cvat/api_calls.py Outdated
Comment thread packages/examples/cvat/exchange-oracle/src/db/errors.py
Comment thread packages/examples/cvat/exchange-oracle/src/endpoints/middleware.py Outdated
Comment thread packages/examples/cvat/exchange-oracle/src/handlers/job_creation.py
Comment thread packages/examples/cvat/exchange-oracle/src/handlers/job_creation.py
Comment thread packages/examples/cvat/exchange-oracle/src/services/cloud/types.py
Comment thread packages/examples/cvat/exchange-oracle/src/utils/annotations.py
Comment thread packages/examples/cvat/exchange-oracle/tests/api/test_exchange_api.py Outdated
@zhiltsov-max zhiltsov-max merged commit e9ad176 into develop Aug 27, 2024
@zhiltsov-max zhiltsov-max changed the title [Exchange Oracle] Add ruff [CVAT] [Exchange Oracle] Add ruff Aug 27, 2024
@flopez7 flopez7 deleted the ba/add-ruff branch November 5, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants