Misc fixes to allow make, make fmt and make fmt-tip to pass#6507
Conversation
When `make` was run on a non-release branch, the `check_version` target
failed because `tools/read-version` compares the static version from
`cloudinit/version.py` with a dynamically generated version from `git
describe`. The `git describe` output for development branches includes
additional details (e.g., `25.3-6-gc6a1d1f1`) that do not match the
static base version (`25.3`), resulting the error:
$ make
Error: read-version version '25.3-6-gc6a1d1f1' not equal to code version '25.3'
make: *** [Makefile:57: check_version] Error 2
Fix this by forcing the `long_match` variable to `None` in
`get_version_details`. This ensures `tools/read-version` only parses the
base version number from `git describe` output, making it consistent
with the static `cloudinit/version.py` version.
Release builds and CI environments have dedicated version handling and
are not affected by this change.
…bs (canonical#6507) The `make fmt` command was failing due to `mypy` errors: $ make fmt ... Cannot find implementation or library stub for module named "crypt" cloudinit/distros/netbsd.py:15: error: Cannot find implementation or library stub for module named "crypt" cloudinit/sources/DataSourceAzure.py:53: error: Cannot find implementation or library stub for module named "crypt" Resolve this by adding `type: ignore` comments to instructs `mypy` to skip type checking for this specific import, allowing the format check to pass without suppressing other type-checking. Detected on latest Debian unstable running `python3-mypy-extensions` version 1.1.0-1.
Correct the `check_format_tip` target to run Pylint only on the
project's source directories: `cloudinit/`, `tests/`, and `tools/`.
When Pylint was run on the entire current directory (`.`), it attempted
to lint all installed third-party libraries within the `.tox` virtual
environment. This led to a fatal `astroid.exceptions.AstroidError`
crash, preventing the format check from completing:
$ make fmt-tip
...
.tox/check_format_tip/lib/python3.13/site-packages/appdirs.py:538: [E1101(no-member), _get_win_folder_with_jna] Module 'array' has no 'zeros' member
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 831, in _check_file
check_astroid_module(ast_node)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1020, in check_astroid_module
retval = self._check_astroid_module(
ast_node, walker, rawcheckers, tokencheckers
)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1072, in _check_astroid_module
walker.walk(node)
~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
[Previous line repeated 2 more times]
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 87, in walk
callback(astroid)
~~~~~~~~^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1701, in visit_name
self._undefined_and_used_before_checker(node, stmt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1742, in _undefined_and_used_before_checker
action, nodes_to_consume = self._check_consumer(
~~~~~~~~~~~~~~~~~~~~^
node, stmt, frame, current_consumer, base_scope_type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1858, in _check_consumer
self._check_late_binding_closure(node)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 2961, in _check_late_binding_closure
assign_scope, stmts = node.lookup(node.name)
~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/_base_nodes.py", line 277, in lookup
return self.scope().scope_lookup(self, name)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 1706, in scope_lookup
return frame._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 91, in _scope_lookup
return pscope.scope_lookup(node, name)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 328, in scope_lookup
return self._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 80, in _scope_lookup
stmts = _filter_stmts(node, self.locals[name], self, offset)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/filter_statements.py", line 193, in _filter_stmts
if not (optional_assign or nodes.are_exclusive(_stmts[pindex], node)):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_classes.py", line 149, in are_exclusive
c2attr, c2node = node.locate_child(previous)
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_ng.py", line 391, in locate_child
raise AstroidError(msg % (repr(child), repr(self)))
astroid.exceptions.AstroidError: Could not find <ClassDef.TypedDict l.1340 at 0x7f2b3e3d5310> in <If l.1044 at 0x7f2b3ef8e8f0>'s children
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.13/concurrent/futures/process.py", line 254, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3.13/concurrent/futures/process.py", line 203, in _process_chunk
return [fn(*args) for args in chunk]
~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 79, in _worker_check_single_file
_worker_linter.check_single_file_item(file_item)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 740, in check_single_file_item
self._check_file(self.get_ast, check_astroid_module, file)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 833, in _check_file
raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__main__.py", line 10, in <module>
pylint.run_pylint()
~~~~~~~~~~~~~~~~~^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__init__.py", line 34, in run_pylint
PylintRun(argv or sys.argv[1:])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/run.py", line 240, in __init__
linter.check(args)
~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 679, in check
check_parallel(
~~~~~~~~~~~~~~^
self,
^^^^^
...<2 lines>...
extra_packages_paths,
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 162, in check_parallel
) in executor.map(_worker_check_single_file, files):
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/process.py", line 617, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
~~~~~~~~~~^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
astroid.exceptions.AstroidError
check_format_tip: exit 1 (126.06 seconds) /debcraft/source> .tox/check_format_tip/bin/python -m pylint . pid=6861
check_format_tip: FAIL code 1 (183.94=setup[57.76]+cmd[0.12,126.06] seconds)
evaluation failed :( (184.04 seconds)
make: *** [Makefile:119: fmt-tip] Error 1
When `make` was run on a non-release branch, the `check_version` target
failed because `tools/read-version` compares the static version from
`cloudinit/version.py` with a dynamically generated version from `git
describe`. The `git describe` output for development branches includes
additional details (e.g., `25.3-6-gc6a1d1f1`) that do not match the
static base version (`25.3`), resulting the error:
$ make
Error: read-version version '25.3-6-gc6a1d1f1' not equal to code version '25.3'
make: *** [Makefile:57: check_version] Error 2
Fix this by forcing the `long_match` variable to `None` in
`get_version_details`. This ensures `tools/read-version` only parses the
base version number from `git describe` output, making it consistent
with the static `cloudinit/version.py` version.
Release builds and CI environments have dedicated version handling and
are not affected by this change.
…bs (canonical#6507) The `make fmt` command was failing due to `mypy` errors: $ make fmt ... Cannot find implementation or library stub for module named "crypt" cloudinit/distros/netbsd.py:15: error: Cannot find implementation or library stub for module named "crypt" cloudinit/sources/DataSourceAzure.py:53: error: Cannot find implementation or library stub for module named "crypt" Resolve this by adding `type: ignore` comments to instructs `mypy` to skip type checking for this specific import, allowing the format check to pass without suppressing other type-checking. Detected on latest Debian unstable running `python3-mypy-extensions` version 1.1.0-1.
Correct the `check_format_tip` target to run Pylint only on the
project's source directories: `cloudinit/`, `tests/`, and `tools/`.
When Pylint was run on the entire current directory (`.`), it attempted
to lint all installed third-party libraries within the `.tox` virtual
environment. This led to a fatal `astroid.exceptions.AstroidError`
crash, preventing the format check from completing:
$ make fmt-tip
...
.tox/check_format_tip/lib/python3.13/site-packages/appdirs.py:538: [E1101(no-member), _get_win_folder_with_jna] Module 'array' has no 'zeros' member
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 831, in _check_file
check_astroid_module(ast_node)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1020, in check_astroid_module
retval = self._check_astroid_module(
ast_node, walker, rawcheckers, tokencheckers
)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1072, in _check_astroid_module
walker.walk(node)
~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
[Previous line repeated 2 more times]
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 87, in walk
callback(astroid)
~~~~~~~~^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1701, in visit_name
self._undefined_and_used_before_checker(node, stmt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1742, in _undefined_and_used_before_checker
action, nodes_to_consume = self._check_consumer(
~~~~~~~~~~~~~~~~~~~~^
node, stmt, frame, current_consumer, base_scope_type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1858, in _check_consumer
self._check_late_binding_closure(node)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 2961, in _check_late_binding_closure
assign_scope, stmts = node.lookup(node.name)
~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/_base_nodes.py", line 277, in lookup
return self.scope().scope_lookup(self, name)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 1706, in scope_lookup
return frame._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 91, in _scope_lookup
return pscope.scope_lookup(node, name)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 328, in scope_lookup
return self._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 80, in _scope_lookup
stmts = _filter_stmts(node, self.locals[name], self, offset)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/filter_statements.py", line 193, in _filter_stmts
if not (optional_assign or nodes.are_exclusive(_stmts[pindex], node)):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_classes.py", line 149, in are_exclusive
c2attr, c2node = node.locate_child(previous)
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_ng.py", line 391, in locate_child
raise AstroidError(msg % (repr(child), repr(self)))
astroid.exceptions.AstroidError: Could not find <ClassDef.TypedDict l.1340 at 0x7f2b3e3d5310> in <If l.1044 at 0x7f2b3ef8e8f0>'s children
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.13/concurrent/futures/process.py", line 254, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3.13/concurrent/futures/process.py", line 203, in _process_chunk
return [fn(*args) for args in chunk]
~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 79, in _worker_check_single_file
_worker_linter.check_single_file_item(file_item)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 740, in check_single_file_item
self._check_file(self.get_ast, check_astroid_module, file)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 833, in _check_file
raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__main__.py", line 10, in <module>
pylint.run_pylint()
~~~~~~~~~~~~~~~~~^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__init__.py", line 34, in run_pylint
PylintRun(argv or sys.argv[1:])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/run.py", line 240, in __init__
linter.check(args)
~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 679, in check
check_parallel(
~~~~~~~~~~~~~~^
self,
^^^^^
...<2 lines>...
extra_packages_paths,
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 162, in check_parallel
) in executor.map(_worker_check_single_file, files):
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/process.py", line 617, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
~~~~~~~~~~^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
astroid.exceptions.AstroidError
check_format_tip: exit 1 (126.06 seconds) /debcraft/source> .tox/check_format_tip/bin/python -m pylint . pid=6861
check_format_tip: FAIL code 1 (183.94=setup[57.76]+cmd[0.12,126.06] seconds)
evaluation failed :( (184.04 seconds)
make: *** [Makefile:119: fmt-tip] Error 1
|
@blackboxsw Can you review this one too? |
When `make` was run on a non-release branch, the `check_version` target
failed because `tools/read-version` compares the static version from
`cloudinit/version.py` with a dynamically generated version from `git
describe`. The `git describe` output for development branches includes
additional details (e.g., `25.3-6-gc6a1d1f1`) that do not match the
static base version (`25.3`), resulting the error:
$ make
Error: read-version version '25.3-6-gc6a1d1f1' not equal to code version '25.3'
make: *** [Makefile:57: check_version] Error 2
Fix this by forcing the `long_match` variable to `None` in
`get_version_details`. This ensures `tools/read-version` only parses the
base version number from `git describe` output, making it consistent
with the static `cloudinit/version.py` version.
Release builds and CI environments have dedicated version handling and
are not affected by this change.
…bs (canonical#6507) The `make fmt` command was failing due to `mypy` errors: $ make fmt ... Cannot find implementation or library stub for module named "crypt" cloudinit/distros/netbsd.py:15: error: Cannot find implementation or library stub for module named "crypt" cloudinit/sources/DataSourceAzure.py:53: error: Cannot find implementation or library stub for module named "crypt" Resolve this by adding `type: ignore` comments to instructs `mypy` to skip type checking for this specific import, allowing the format check to pass without suppressing other type-checking. Detected on latest Debian unstable running `python3-mypy-extensions` version 1.1.0-1.
Correct the `check_format_tip` target to run Pylint only on the
project's source directories: `cloudinit/`, `tests/`, and `tools/`.
When Pylint was run on the entire current directory (`.`), it attempted
to lint all installed third-party libraries within the `.tox` virtual
environment. This led to a fatal `astroid.exceptions.AstroidError`
crash, preventing the format check from completing:
$ make fmt-tip
...
.tox/check_format_tip/lib/python3.13/site-packages/appdirs.py:538: [E1101(no-member), _get_win_folder_with_jna] Module 'array' has no 'zeros' member
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 831, in _check_file
check_astroid_module(ast_node)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1020, in check_astroid_module
retval = self._check_astroid_module(
ast_node, walker, rawcheckers, tokencheckers
)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1072, in _check_astroid_module
walker.walk(node)
~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
[Previous line repeated 2 more times]
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 87, in walk
callback(astroid)
~~~~~~~~^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1701, in visit_name
self._undefined_and_used_before_checker(node, stmt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1742, in _undefined_and_used_before_checker
action, nodes_to_consume = self._check_consumer(
~~~~~~~~~~~~~~~~~~~~^
node, stmt, frame, current_consumer, base_scope_type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1858, in _check_consumer
self._check_late_binding_closure(node)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 2961, in _check_late_binding_closure
assign_scope, stmts = node.lookup(node.name)
~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/_base_nodes.py", line 277, in lookup
return self.scope().scope_lookup(self, name)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 1706, in scope_lookup
return frame._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 91, in _scope_lookup
return pscope.scope_lookup(node, name)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 328, in scope_lookup
return self._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 80, in _scope_lookup
stmts = _filter_stmts(node, self.locals[name], self, offset)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/filter_statements.py", line 193, in _filter_stmts
if not (optional_assign or nodes.are_exclusive(_stmts[pindex], node)):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_classes.py", line 149, in are_exclusive
c2attr, c2node = node.locate_child(previous)
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_ng.py", line 391, in locate_child
raise AstroidError(msg % (repr(child), repr(self)))
astroid.exceptions.AstroidError: Could not find <ClassDef.TypedDict l.1340 at 0x7f2b3e3d5310> in <If l.1044 at 0x7f2b3ef8e8f0>'s children
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.13/concurrent/futures/process.py", line 254, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3.13/concurrent/futures/process.py", line 203, in _process_chunk
return [fn(*args) for args in chunk]
~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 79, in _worker_check_single_file
_worker_linter.check_single_file_item(file_item)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 740, in check_single_file_item
self._check_file(self.get_ast, check_astroid_module, file)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 833, in _check_file
raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__main__.py", line 10, in <module>
pylint.run_pylint()
~~~~~~~~~~~~~~~~~^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__init__.py", line 34, in run_pylint
PylintRun(argv or sys.argv[1:])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/run.py", line 240, in __init__
linter.check(args)
~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 679, in check
check_parallel(
~~~~~~~~~~~~~~^
self,
^^^^^
...<2 lines>...
extra_packages_paths,
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 162, in check_parallel
) in executor.map(_worker_check_single_file, files):
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/process.py", line 617, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
~~~~~~~~~~^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
astroid.exceptions.AstroidError
check_format_tip: exit 1 (126.06 seconds) /debcraft/source> .tox/check_format_tip/bin/python -m pylint . pid=6861
check_format_tip: FAIL code 1 (183.94=setup[57.76]+cmd[0.12,126.06] seconds)
evaluation failed :( (184.04 seconds)
make: *** [Makefile:119: fmt-tip] Error 1
…bs (canonical#6507) The `make fmt` command was failing due to `mypy` errors: $ make fmt ... Cannot find implementation or library stub for module named "crypt" cloudinit/distros/netbsd.py:15: error: Cannot find implementation or library stub for module named "crypt" cloudinit/sources/DataSourceAzure.py:53: error: Cannot find implementation or library stub for module named "crypt" Resolve this by adding `type: ignore` comments to instructs `mypy` to skip type checking for this specific import, allowing the format check to pass without suppressing other type-checking. Detected on latest Debian unstable running `python3-mypy-extensions` version 1.1.0-1.
Correct the `check_format_tip` target to run Pylint only on the
project's source directories: `cloudinit/`, `tests/`, and `tools/`.
When Pylint was run on the entire current directory (`.`), it attempted
to lint all installed third-party libraries within the `.tox` virtual
environment. This led to a fatal `astroid.exceptions.AstroidError`
crash, preventing the format check from completing:
$ make fmt-tip
...
.tox/check_format_tip/lib/python3.13/site-packages/appdirs.py:538: [E1101(no-member), _get_win_folder_with_jna] Module 'array' has no 'zeros' member
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 831, in _check_file
check_astroid_module(ast_node)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1020, in check_astroid_module
retval = self._check_astroid_module(
ast_node, walker, rawcheckers, tokencheckers
)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 1072, in _check_astroid_module
walker.walk(node)
~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 90, in walk
self.walk(child)
~~~~~~~~~^^^^^^^
[Previous line repeated 2 more times]
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/utils/ast_walker.py", line 87, in walk
callback(astroid)
~~~~~~~~^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1701, in visit_name
self._undefined_and_used_before_checker(node, stmt)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1742, in _undefined_and_used_before_checker
action, nodes_to_consume = self._check_consumer(
~~~~~~~~~~~~~~~~~~~~^
node, stmt, frame, current_consumer, base_scope_type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 1858, in _check_consumer
self._check_late_binding_closure(node)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/checkers/variables.py", line 2961, in _check_late_binding_closure
assign_scope, stmts = node.lookup(node.name)
~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/_base_nodes.py", line 277, in lookup
return self.scope().scope_lookup(self, name)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 1706, in scope_lookup
return frame._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 91, in _scope_lookup
return pscope.scope_lookup(node, name)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/scoped_nodes.py", line 328, in scope_lookup
return self._scope_lookup(node, name, offset)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/scoped_nodes/mixin.py", line 80, in _scope_lookup
stmts = _filter_stmts(node, self.locals[name], self, offset)
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/filter_statements.py", line 193, in _filter_stmts
if not (optional_assign or nodes.are_exclusive(_stmts[pindex], node)):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_classes.py", line 149, in are_exclusive
c2attr, c2node = node.locate_child(previous)
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/astroid/nodes/node_ng.py", line 391, in locate_child
raise AstroidError(msg % (repr(child), repr(self)))
astroid.exceptions.AstroidError: Could not find <ClassDef.TypedDict l.1340 at 0x7f2b3e3d5310> in <If l.1044 at 0x7f2b3ef8e8f0>'s children
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.13/concurrent/futures/process.py", line 254, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3.13/concurrent/futures/process.py", line 203, in _process_chunk
return [fn(*args) for args in chunk]
~~^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 79, in _worker_check_single_file
_worker_linter.check_single_file_item(file_item)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 740, in check_single_file_item
self._check_file(self.get_ast, check_astroid_module, file)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 833, in _check_file
raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__main__.py", line 10, in <module>
pylint.run_pylint()
~~~~~~~~~~~~~~~~~^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/__init__.py", line 34, in run_pylint
PylintRun(argv or sys.argv[1:])
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/run.py", line 240, in __init__
linter.check(args)
~~~~~~~~~~~~^^^^^^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/pylinter.py", line 679, in check
check_parallel(
~~~~~~~~~~~~~~^
self,
^^^^^
...<2 lines>...
extra_packages_paths,
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/debcraft/source/.tox/check_format_tip/lib/python3.13/site-packages/pylint/lint/parallel.py", line 162, in check_parallel
) in executor.map(_worker_check_single_file, files):
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/process.py", line 617, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
~~~~~~~~~~^^^^^^^^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
astroid.exceptions.AstroidError
check_format_tip: exit 1 (126.06 seconds) /debcraft/source> .tox/check_format_tip/bin/python -m pylint . pid=6861
check_format_tip: FAIL code 1 (183.94=setup[57.76]+cmd[0.12,126.06] seconds)
evaluation failed :( (184.04 seconds)
make: *** [Makefile:119: fmt-tip] Error 1
When `make` was run on a non-release branch, and `git` was installed,
the `check_version` target failed because `tools/read-version` compares
the static version from `cloudinit/version.py` with a dynamically
generated version from `git describe`. The `git describe` output for
development branches includes additional details (e.g.,
`25.3-6-gc6a1d1f1`) that do not match the static base version (`25.3`),
resulting the error:
$ make
Error: read-version version '25.3-6-gc6a1d1f1' not equal to code version '25.3'
make: *** [Makefile:57: check_version] Error 2
Fix this by skipping the version check when not running in CI, so
develpers can now run `make`. Release builds and CI environments have
dedicated version handling and are not affected by this change.
When `make` was run on a non-release branch, and `git` was installed,
the `check_version` target failed because `tools/read-version` compares
the static version from `cloudinit/version.py` with a dynamically
generated version from `git describe`. The `git describe` output for
development branches includes additional details (e.g.,
`25.3-6-gc6a1d1f1`) that do not match the static base version (`25.3`),
resulting the error:
$ make
Error: read-version version '25.3-6-gc6a1d1f1' not equal to code version '25.3'
make: *** [Makefile:57: check_version] Error 2
Fix this by skipping the version check when not running in CI, so
develpers can now run `make`. Release builds and CI environments have
dedicated version handling and are not affected by this change.
|
Hello! Thank you for this proposed change to cloud-init. This pull request is now marked as stale as it has not seen any activity in 14 days. If no activity occurs within the next 7 days, this pull request will automatically close. If you are waiting for code review and you are seeing this message, apologies! Please reply, tagging blackboxsw, and he will ensure that someone takes a look soon. (If the pull request is closed and you would like to continue working on it, please do tag blackboxsw to reopen it.) |
holmanb
left a comment
There was a problem hiding this comment.
Which tests do you see failing? Please include failure output. There are a couple of reasons that we may not want this change.
The Makefile doesn't really add real value - it is basically just a wrapper of other things. Unless we really need these targets, I think we can probably drop them.
Similarly, we are considering dropping the read-version file (although we need to do some work to enable that first).
|
I guess the missing piece here is that I don't understand how you develop
on this project? Just doing git clone + make does not work.
Please read each git commit message, they all have the error messages they
fix (locally).
|
Did you check the docs? |
|
The page you linked has only two tox command examples. If does not say
anything about running full build and tests as the Makefile does or CI
scripts do. Other documentation pages like
https://docs.cloud-init.io/en/latest/development/integration_tests.html
don't explain it either.
So what command do you run locally to build everything to be able to run
and test the binaries? Not 'make'?
|
|
I'm sure that the contributing docs could use some improvements, I appreciate your feedback.
Correct, tox is used for running tests.
What makes you think that the Makefile does full build and test? The Github actions configs show how tests run in CI.
It isn't clear to me what you are expecting to be explained. Do you want to run all of the integration tests yourself?
It seems that there is a mismatch between your expectations and what is documented. What you are expecting to see? Cloud-init is not compiled, so no binaries are involved. This makes me wonder if perhaps you are referring to distro packages? |
|
Thanks for opening #6566, the contribution docs and process probably needs some general improvements. I think I will close this as it seems that there is no configuration where the Makefile targets work without breaking the CI, giving the impression that the developers in this project are actually not using the Makefile, and trying to "fix" it is moot. |
Proposed Commit Message
All commit messages are following Conventional Commits style. Each change is atomic, so this branch is best to be merged without squashing anything. The commits don't reference any issues, as discovering that various
makecommands don't work happened simply while trying to contribute to this proejct.Additional Context
This is a #hacktoberfest contribution.
Test Steps
Run before and after in a clean Debian unstable (or Ubuntu questing) container:
Without these commits they don't pass, and with these commits they pass.
Merge type