Skip to content

ubuntu/plucky: new_upstream_snapshot and manual refresh of retain-setuptools.patch#6611

Merged
blackboxsw merged 68 commits into
canonical:ubuntu/pluckyfrom
blackboxsw:ubuntu/plucky
Dec 17, 2025
Merged

ubuntu/plucky: new_upstream_snapshot and manual refresh of retain-setuptools.patch#6611
blackboxsw merged 68 commits into
canonical:ubuntu/pluckyfrom
blackboxsw:ubuntu/plucky

Conversation

@blackboxsw
Copy link
Copy Markdown
Collaborator

To ease the process of reviewing your PR, do make sure to complete the following checklist before submitting a pull request.

  • I have signed the CLA: https://ubuntu.com/legal/contributors
  • I have included a comprehensive commit message using the guide below
  • I have added unit tests to cover the new behavior under tests/unittests/
    • Test files should map to source files i.e. a source file cloudinit/example.py should be tested by tests/unittests/test_example.py
    • Run unit tests with tox -e py3
  • I have kept the change small, avoiding unnecessary whitespace or non-functional changes.
  • I have added a reference to issues that this PR relates to in the PR message (Refs integration: do not LXD bind mount /etc/cloud/cloud.cfg.d #1234, Fixes integration: do not LXD bind mount /etc/cloud/cloud.cfg.d #1234)
  • I have updated the documentation with the changed behavior.
    • If the change doesn't change the user interface and is trivial, this step may be skipped.
    • Cloud-config documentation is generated from the jsonschema.
    • Generate docs with tox -e doc.

Proposed Commit Message

pay special attention to manually created e96a554 for retain-setuptools.patch refresh

Manual steps performed:

# obtain most recent patched tools/test_tools.py
git checkout upstream/ubuntu/plucky
quilt push -a
cp tools/test_tools.py ..
quilt pop -a
gh co <PR_NUM>
new_upstream_snapshot.py
# note quilt patch error ..
quilt push -f
quilt add tools/test_tools.py
cp ../test_tools.py tools/
quilt refresh
quilt pop -a

Additional Context

Test Steps

quilt push -a
tox -e py3
tox -e doc
quilt pop -a

Merge type

  • Squash merge using "Proposed Commit Message"
  • Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)
  • git push upstream from commandline

szepeviktor and others added 30 commits September 29, 2025 11:37
Add sysvinit_freebsd support to meson.build to enable generating
rc.d scripts from templates and installing into the proper
/usr/local/etc/rc.d directory.

Update meson_options.txt to limit init_system choices to
sysvinit_openrc, systemd or sysvinit_freebsd.
Also, update tools/build-on-freebsd to use meson build operations.

Update tools/read-dependencies and pkg-deps.json for freebsd support.
…nical#6501)

The Meson-based installation does not install these text files whereas
the pre-Meson installation did, so reinstate them.
…cal#6488)

Signed-off-by: paulober <paul.oberosler@raspberrypi.com>
Refactored tests/unittests/distros/test_resolv.py to use pytest instead of
unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance
- Converted self.assert* methods to bare assert statements
- Replaced self.assertRaises with pytest.raises context manager
- Maintained all original test functionality

Related: canonical#6427
…al#6483)

This commit fixes keymap configuration on
Trixie-based Raspberry Pi OS images by aligning
behavior with raspi-config.

It also removes growpart and resizefs from the
cloud-init configuration for Raspberry Pi OS,
as these functions are already provided by the
system and caused unnecessary duplication.
…anonical#6466)

feat(cc_raspberry_pi): Replace rpi-connect with usb-gadget support

This commit replaces the rpi connect enablement with new
usb-gadget support because rpi-connect setup will be
moved outside cloud-init and having the option
to enable the usb gadget mode right in the cloud-init
config does simplify the setup for many.

Also move raspberry_pi config module out of
cloud-init-init boot stage and into cloud-init-network
boot stage to ensure it is only run after D-BUS is
available.
…cal#6512)

Refactored tests/unittests/sources/test_common.py to use pytest instead of
unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance from both test classes
- Renamed ExpectedDataSources to TestExpectedDataSources for pytest discovery
- Converted self.assertEqual() methods to bare assert statements
- Converted self.assertTrue() to assert statement
- Converted self.assertNotEqual() to assert statement
- Converted self.assertIsNotNone() to assert statement
- Removed unused test_helpers import
- Maintained all original test functionality

Related: canonical#6427
…nical#6511)

Refactored tests/unittests/distros/test_hostname.py to use
pytest instead of unittest.TestCase as part of the pytest migration effort.

- Removed unittest import
- Removed TestCase inheritance
- Converted self.assertEqual() methods to bare assert statements
- Maintained all original test functionality

Related: canonical#6427
…onical#6510)

Refactored tests/unittests/distros/test_sysconfig.py to use pytest instead of
unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance and unused py 2.6 method
- Converted self.assert* methods to bare assert statements
- Maintained all original test functionality

Related: canonical#6427
Refactored tests/unittests/distros/test_alpine.py to use pytest instead of
unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance
- Maintained all original test functionality

Related: canonical#6427
…anonical#6515)

Refactored tests/unittests/config/test_cc_spacewalk.py to use pytest
instead of unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance
- Removed unused helpers import
- Converted self.assertFalse() to strict boolean check (is False)
- Converted self.assertTrue() to strict boolean check (is True)
- Maintained all original test functionality

Related: canonical#6427
…nonical#6516)

Refactored tests/unittests/config/test_cc_ca_certs.py to use pytest
instead of unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance from all test classes
- Converted setUp methods to pytest fixtures using tmp_path and
   mocker
- Replaced self.assertEqual/assertTrue/assertFalse with bare assert
  statements
- Removed ExitStack usage and CAMocks.reset() method
- Converted all looping tests to use @pytest.mark.parametrize for
  cleaner, more pytest-idiomatic code
- Each test now runs once per distro (244 total tests)
- Replaced self.add_patch() with mocker.patch() from pytest-mock
- Added type annotation for class variable to satisfy mypy
- Maintained all original test functionality

Related: canonical#6427
…nical#6517)

Refactored tests/unittests/distros/test_ifconfig.py to use pytest instead of
unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance
- Refactored tests to use pytest fixtures instead of unittest setUp methods
- updated tests to receive fixtures as function arguments
- Maintained all original test functionality

Related: canonical#6427
Correct various spelling errors across documentation, comments, and code
to improve readability and maintain consistency. These typos were
identified by running:

    codespell --write-changes --check-filenames --check-hidden *

This includes the following corrections:

- absense -> absence
- adresses -> addresses
- alot -> a lot
- appropiate -> appropriate
- astrisk -> asterisk
- atleast -> at least
- atttributes -> attributes
- begining -> beginning
- bootup -> boot up
- configuraiton -> configuration
- Coverts -> Converts
- defition -> definition
- delimeted -> delimited
- directoty -> directory
- exis -> exist
- existance -> existence
- Exepected -> Expected
- exeption -> exception
- formated -> formatted
- (I|i)ncase -> in case
- indepedent -> independent
- intefaces -> interfaces
- invididual -> individual
- involes -> involves
- maching -> matching
- Niether -> Neither
- non-existance -> non-existence
- non-existant -> non-existent
- Nothin -> Nothing
- occured -> occurred
- outputing -> outputting
- overriden -> overridden
- parms -> params
- perferred -> preferred
- porpose -> purpose
- precendence -> precedence
- preferrable -> preferable
- removeing -> removing
- representd -> represented
- resilent -> resilient
- setted -> set
- soures -> sources
- sript -> script
- teh -> the
- thats -> that's
- Theses -> These
- varables -> variables
- Verifiy -> Verify
- wit -> with

Additionally, re-flow some lines for the `ruff check` to not fail on
too long lines.
Refactored tests/unittests/config/test_schema.py to use
pytest instead of unittest.TestCase as part of the pytest
migration effort.

- Removed unittest.TestCase inheritance
- Maintained all original test functionality

Related: canonical#6427
…ytest (canonical#6518)

Refactored tests/unittests/distros/test_user_data_normalize.py
to use pytest instead of unittest.TestCase as part of the pytest
migration effort.

- Removed unittest.TestCase inheritance and `setUp` method
- Added pytest.mark.parametrize for repetitive user simple dict tests
- Replaced self.assert* calls with simple assert statements
- Maintained all original test functionality

Related: canonical#6427

* Fix assertions for pytest style
  - Replaced `assert True == value` with `assert value`
  - fixed incorrect assert "zetta", users with assert "zetta" in users

* test: Be strict about boolean assertions to avoid passing unexpected truthy values
…anonical#6520)

Refactored tests/unittests/config/test_apt_conf_v1.py to
use pytest instead of unittest.TestCase as part of the pytest
migration effort.

- Removed unittest.TestCase inheritance
- Converted setUp() to pytest fixture for temporary files
- Parameterized proxy-related tests for cleaner duplication removal
- Replaced unittest assertions with plain pytest `assert` statements
- Replaced `_search_apt_config` as standalone helper function
- Maintained all original test functionality

Related: canonical#6427
…rsions

Allow for differentiation of downstream versions from upstream
cloud-init if necessary.

Meson builds will generate cloudinit/meson_version.py from
meson_version.py.in in order to create two python variables:

- UPSTREAM_VERSION which is set by meson.project_version in meson.build
- DOWNSTREAM_VERSION which is optionally defined by downstream packages
  to specialize runtime versions reported by cloud-init in logs and at
  the command line.

Add meson option downstream_version which can be set to express
a unique downstream package version of cloud-init if needed:

 meson setup ... -Ddownstream_version=25.1~3g89001234-0ubuntu2

Drop ./tools/read-version utility script from upstream and Makefile
target check_verison related to runtime versus packaged version check.
Downstreams can provide such package version checks in build tooling
such as SPEC files, debian/tests or other build scripts if necessary.

Drop tools/test_tools.py which was only used to validate read-version
behavior for setuptools needs.

Fixes canonicalGH-6389
…tes (canonical#6500)

Will be fixed in upstream rust-coreutils per
uutils/coreutils#8789. But, this may take a
bit to SRU to Ubuntu Questing.
Refactored tests/unittests/test_util.py to use pytest instead
of unittest.TestCase as part of the pytest migration effort.

- Removed TestCase inheritance
- Refactored tests to use pytest tmp_path fixture for temp
  directory handling
- Maintained all original test functionality

Related: canonical#6427
holmanb and others added 4 commits December 3, 2025 08:28
…deps (canonical#6459)

Align systemd network ordering with current Raspberry Pi OS behavior.
The previous sequencing was designed around older cloud-init and
setup-wizard requirements, but recent upstream and downstream
changes make those workarounds unnecessary. The updated ordering
matches default expectations and works reliably with newer
NetworkManager/Netplan fixes.

Remove the no-longer-needed Before=avahi-daemon.service dependency,
simplifying the unit and bringing it closer to other distros.

Run the network units before sysinit.target, which is consistent with how
RPi OS boots and has been validated in current downstream images.

Signed-off-by: paulober <paul.oberosler@raspberrypi.com>
canonical#6482)

Disable the fallback netcfg in the cloud.cfg template for
Raspberry Pi OS and remove the apt mirror config because
RPi OS ships as already installed images and there is no
scenario where users are expected to rely on cloud-init
for providing the apt config.

Signed-off-by: paulober <paul.oberosler@raspberrypi.com>
@github-actions github-actions Bot added the packaging Supplemental package review requested label Dec 9, 2025
Comment thread debian/rules Outdated
install -m 0644 -D debian/apport-general-hook.py debian/cloud-init-base/usr/share/apport/general-hooks/cloud-init.py
install -m 0644 -D debian/apport-launcher.py debian/cloud-init-base/usr/share/apport/package-hooks/cloud-init.py
flist=$$(find $(CURDIR)/debian/ -type f -name version.py) && sed -i 's,@@PACKAGED_VERSION@@,$(DEB_VERSION),' $${flist:-did-not-find-version-py-for-replacement}
version_dir=$$(find $(CURDIR)/debian/ -type f -name version.py | xargs dirname) && echo -e "# Generated by debian/rules\nUPSTREAM_VERSION = 'IGNORED'\n"DOWNSTREAM_VERSION = '$(DEB_VERSION)'" > $((version_dir)/meson_versions.py
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please use find <args> -exec dirname {} + rather than piping to xargs?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, now I'm confused - why is version.py in the debian/ directory?

Copy link
Copy Markdown
Member

@holmanb holmanb Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is just used to discover the version file, but we already know where the input file is, I think we can just use $(CURDIR)/cloudinit/version.py and skip the complexity, so something like:

printf "# Generated by debian/rules\nUPSTREAM_VERSION = 'IGNORED'\nDOWNSTREAM_VERSION = '$(DEB_VERSION)'" > $(CURDIR)/cloudinit/meson_versions.py

Copy link
Copy Markdown
Collaborator Author

@blackboxsw blackboxsw Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had wanted to initially avoid adding artifacts outside to debian directory which would be created by running debuild -b as it would leave the source repository under cloudinit/ dirty which would show untracked cloudinit/meson_versions.py from git status. But, we can easily move this up before our call to dh_auto_install --destdir=debian/cloud-init-base which will then automatically install the meson_versions.py with all other py modules to the right packaging debian/cloud-init-base subdirectory. We can then remove this build artifact from cloudinit/meson_versions.py after that dh_auto_install call so we don't dirty our source repo by calling debuild.

The following delta uses this approach of seeding the file we want to automatically install and cleaning up after the install takes place

--- a/debian/rules
+++ b/debian/rules
@@ -19,10 +19,11 @@ override_dh_installsystemd:
        dh_installsystemd --no-restart-on-upgrade --no-start
 
 override_dh_auto_install:
+       printf "# Generated by debian/rules\nUPSTREAM_VERSION = 'IGNORED'\nDOWNSTREAM_VERSION = '$(DEB_VERSION)'" > cloudinit/meson_versions.py
        dh_auto_install --destdir=debian/cloud-init-base
+       rm cloudinit/meson_versions.py  # avoid debuild artifacts in tree
        install -D -m 0644 ./tools/21-cloudinit.conf debian/cloud-init-base/etc/rsyslog.d/21-cloudinit.conf
        install -D ./tools/Z99-cloud-locale-test.sh debian/cloud-init-base/etc/profile.d/Z99-cloud-locale-test.sh
        install -D ./tools/Z99-cloudinit-warnings.sh debian/cloud-init-base/etc/profile.d/Z99-cloudinit-warnings.sh
        install -m 0644 -D debian/apport-general-hook.py debian/cloud-init-base/usr/share/apport/general-hooks/cloud-init.py
        install -m 0644 -D debian/apport-launcher.py debian/cloud-init-base/usr/share/apport/package-hooks/cloud-init.py
-       flist=$$(find $(CURDIR)/debian/ -type f -name version.py) && sed -i 's,@@PACKAGED_VERSION@@,$(DEB_VERSION),' $${flist:-did-not-find-version-py-for-replacement}

Copy link
Copy Markdown
Member

@holmanb holmanb Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had wanted to initially avoid adding artifacts outside to debian directory which would be created by running debuild -b as it would leave the source repository under cloudinit/ dirty which would show untracked cloudinit/meson_versions.py from git status. But, we can easily move this up before our call to dh_auto_install --destdir=debian/cloud-init-base which will then automatically install the meson_versions.py with all other py modules to the right packaging debian/cloud-init-base subdirectory. We can then remove this build artifact from cloudinit/meson_versions.py after that dh_auto_install call so we don't dirty our source repo by calling debuild.

Running debuild -b from this branch leaves git in a dirty state:

$ git status
On branch ubuntu/plucky
Your branch and 'origin/ubuntu/plucky' have diverged,
and have 64 and 2 different commits each, respectively.
  (use "git pull" if you want to integrate the remote branch with yours)

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .pybuild/                                   debian/cloud-init-base.preinst.debhelper    debian/cloud-init-smart-os.substvars
        debian/.debhelper/                          debian/cloud-init-base.prerm.debhelper      debian/cloud-init-smart-os/
        debian/cloud-init-azure.debhelper.log       debian/cloud-init-base.substvars            debian/cloud-init.debhelper.log
        debian/cloud-init-azure.substvars           debian/cloud-init-base/                     debian/cloud-init.substvars
        debian/cloud-init-azure/                    debian/cloud-init-cloud-sigma.debhelper.log debian/cloud-init/
        debian/cloud-init-base.debhelper.log        debian/cloud-init-cloud-sigma.substvars     debian/debhelper-build-stamp
        debian/cloud-init-base.postinst.debhelper   debian/cloud-init-cloud-sigma/              debian/files
        debian/cloud-init-base.postrm.debhelper     debian/cloud-init-smart-os.debhelper.log

nothing added to commit but untracked files present (use "git add" to track)

So why the extra complexity? It doesn't seem to buy us anything.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've now shifted the debian/rules changes in dh_auto_install to only create the meson_versions.py module in the package destination directory(dest-dir) under debian/cloud-init-base so it doesn't expose the cloudinit/meson_versions.py in the current $CURDIR/cloudinit root directory.

That dirty state comes from failed debuild -b due to earlier invalid variable substitution issues breaking the debian/rules

I've confirmed even on older releases debuild -b leaves uncleaned artifacts in debian/* for each package built.

Here's noble:

	.pybuild/
	debian/.debhelper/
	debian/cloud-init.debhelper.log
	debian/cloud-init.postinst.debhelper
	debian/cloud-init.postrm.debhelper
	debian/cloud-init.preinst.debhelper
	debian/cloud-init.prerm.debhelper
	debian/cloud-init.substvars
	debian/cloud-init/
	debian/debhelper-build-stamp
	debian/files

It's especially noticeable on plucky++ because we have the renamed cloud-init-(azure|base|cloud-sigma|smart-so) etc.

Copy link
Copy Markdown
Collaborator Author

@blackboxsw blackboxsw Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This debuiild -b behavior happens in other packages too. I just ran debuild -b on ubuntu-meta which left me with similar debian/<package-name> artifacts in the debian directory.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@holmanb: I've performed new_upstream_snapshot.py -c fcb98b03 on this branch and re-created debian/rules to inject meson_versions.py during both the dh_auto_clean stage to support python3 setup.py clean calling tools/read-version and dh_auto_install to place it in the package destination-dir debian/cloud-init-base/usr/lib/python3.X/dist-packages/cloudinit/meson_versions.py.

I also updated the quilt patch delta of debian/patch/retain-setuptools.patch:

  • purging anything not required in the downstream ubuntu branch: such as packages/* changes and tox.ini lowest-supported toxenv
  • including the dropped tools/read-version and tools/test_tools.py files which were dropped by upstream

I dput to a PPA to confirm successful package build with d/changelog fixed via sed -i 's/UNRELEASED/plucky/'
https://launchpad.net/~chad.smith/+archive/ubuntu/sru-25.4/+build/31606263

@blackboxsw blackboxsw changed the title ubuntu/plucky: new_upstream_changelog and manual refresh of retain-setuptools.patch ubuntu/plucky: new_upstream_snapshot and manual refresh of retain-setuptools.patch Dec 9, 2025
Copy link
Copy Markdown
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👆

@blackboxsw blackboxsw force-pushed the ubuntu/plucky branch 3 times, most recently from d434f3a to b2518ed Compare December 10, 2025 15:30
@blackboxsw blackboxsw requested a review from holmanb December 12, 2025 15:49
blackboxsw and others added 2 commits December 12, 2025 08:51
…anonical#6612)

Followup to baedc33 to support skipping azure ssh-keygen related
tests from running in environments without ssh-keygen installed.
Copy link
Copy Markdown
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment above.

Retain tools/test_tools.py as part of downstream non-meson package
validation.
Retain tools/read-version used by setuptools to set DOWNSTREAM_VERSION.

Drop unnecessary doc-requirements.txt and package/debian/rules
from patch because downstream workflows do not require building
docs or packages/bddeb using setuptools.
Comment thread debian/changelog Outdated
* d/rules: add meson_versions.py artifact and set DOWNSTREAM_VERSION
during package build.
* refresh patches:
- d/p/retain-setuptools.patch: retaini read-version tool for build.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit spelling: retain

Copy link
Copy Markdown
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One spelling issue in the changelog, otherwise looks good to me!

@blackboxsw blackboxsw merged commit d0ae5db into canonical:ubuntu/plucky Dec 17, 2025
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

packaging Supplemental package review requested

Projects

None yet

Development

Successfully merging this pull request may close these issues.