Skip to content

Ubuntu/devel#892

Merged
OddBloke merged 35 commits into
canonical:ubuntu/develfrom
TheRealFalcon:ubuntu/devel
May 11, 2021
Merged

Ubuntu/devel#892
OddBloke merged 35 commits into
canonical:ubuntu/develfrom
TheRealFalcon:ubuntu/devel

Conversation

@TheRealFalcon
Copy link
Copy Markdown
Contributor

@TheRealFalcon TheRealFalcon commented May 6, 2021

Have to manually readjust a ^M to be \r\n in debian/changelog (and now I'm wishing I had used a better commit message).

Also, is there a reason we use an annotated tag for upstream releases, but an unannotated one for Ubuntu releases, or is it unintentional?

timothegenzmer and others added 29 commits March 24, 2021 14:57
key is a property of source1 and not sources
…ical#842)

Ensure that the Azure helper's http handler sleeps a fixed duration
between retry failure attempts. The http handler will sleep a fixed
duration between failed attempts regardless of whether the attempt
failed due to (1) request timing out or (2) instant failure (no
timeout).

Due to certain platform issues, the http request to the Azure endpoint
may instantly fail without reaching the http timeout duration. Without
sleeping a fixed duration in between retry attempts, the http handler
will loop through the max retry attempts quickly. This causes the
communication between cloud-init and the Azure platform to be less
resilient due to the short total duration if there is no sleep in
between retries.
…anonical#817)

When output of SSH host keys and/or SSH fingerprints are disabled for
all keys do not display headers and footers.

Prevent risk of message text being interpreted as "logger" option by
appending "--" to logger options.

Correct syslog output that was tagged with "ec2" regardless of DataSource
in use. Now use "cloud-init" tag instead.

Various "shellcheck" corrections.

Add testcase for disabled output of SSH host keys.
…al#844)

klibc initramfs in debian allows the 'iscsi_target_ip=' cmdline
parameter to specify an iscsi device attachment. This can
cause cloud-init to mis-detect the cmdline paramter as a
networking config.

LP: #1919188
This patch adds support to resize a single partition of a VM if it's using an
LVM underneath. The patch detects if it's LVM if the given block device
is a device mapper by its name (e.g. `/dev/dm-1`) and if it has slave
devices under it on sysfs. After that syspath is updated to the real
block device and growpart will be called to resize it (and automatically
its Physical Volume).

The Volume Group will be updated automatically and a final call to
extend the rootfs to the remaining space available will be made.

Using the same growpart configuration, the user can specify only one
device to be resized when using LVM and growpart, otherwise cloud-init
won't know which one should be resized and will fail.

rhbz: #1810878
LP: #1799953

Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Signed-off-by: Scott Moser <smoser@brickies.net>
When bringing up DHCP-provided static routes, we check for "0.0.0.0/0"
to indicate an unspecified gateway. However, when parsing the static
route in `parse_static_routes`, the gateway is never specified with
a net length, so the "/0" will never happen.

This change updates the gateway check to check only for "0.0.0.0".
Update sysconfig configuration to use BONDING_MODULES_OPTS instead of
BONDING_OPTS when on a SUSE system. The sysconfig support requires use
of BONDING_MODULE_OPTS whereas the initscript support that rhel uses
requires BONDING_OPTS.
On the datasource class, we require the use of paths.run_dir to
perform some operations. On older cloud-init version, the
Paths class does not have the run_dir attribute. To fix that,
we are now manually adding that attribute in the Paths
object if doesn't exist in the unpickle operation.

LP: #1899299
This PR adds in support so that cloud-init can run on instances 
deployed on Vultr cloud. This was originally brought up in canonical#628.

Co-authored-by: Eric Benner <ebenner@vultr.com>
Invoking walinuxagent from within cloud-init is no longer
supported/necessary
…anonical#866)

git-grep showed a few more locations where we refer to a "user-scripts"
config module which is really cc_scripts_user module.  Replace these
references with slightly different language so as not to confuse
future me when looking for "user-scripts" vs. "scripts-user"
The current method of running a background sleep until travis is
finished is causing integration test runs to pass even when they should
be failing.

Instead, update the code to emit dots itself.
the above option allows the user to control the behavior of a distro
hostname selection if both short hostname and FQDN are supplied.
If `prefer_fqdn_over_hostname` is true the FQDN will be selected as
hostname; if false the hostname will be selected

LP: #1921004
)

In canonical#777, we added 'vendordata2' and 'vendordata2_raw' attributes to
the DataSource class, but didn't use the upgrade framework to deal
with an unpickle after upgrade. This commit adds the necessary
upgrade code.

Additionally, added a smaller-scope upgrade test to our integration
tests that will be run on every CI run so we catch these issues
immediately in the future.

LP: #1922739
The blkdev command is fragile re-reading partition tables if a
partition is mounted. This change instead uses the  partprobe if
it is available.

LP: #1920939
Due to hyper-v implementations, iso ejection is more efficient if performed
from within the guest. The code will attempt to perform a best-effort ejection.
Failure during ejection will not prevent reporting ready from happening. If iso
ejection is successful, later iso ejection from the platform will be a no-op.
In the event the iso ejection from the guest fails, iso ejection will still happen at
the platform level.
This change allows us to retrieve the username and hostname from
IMDS instead of having to rely on the mounted OVF.
…onical#878)

When network interfaces are hot-attached to the VM, attempting to get
network metadata might return 410 (or 500, 503 etc) because the info
is not yet available. In those cases, we retry getting the metadata
before giving up. The only case where we can move on to wait for more
nic attach events is if the call times out despite retries, which
means the interface is not likely a primary interface, and we should
try for more nic attach events.
Move chef_license from TPL_PATH_KEYS to TPL_KEYS as the chef license
setting is not a path but must be added to the client config template.
Fixes file or folder not found exception raised from ensure_dirs.
…nonical#883)

This allows us to use it when validating packages from -proposed (and
PPAs etc.).
…anonical#887)

This reverts commit 74fa008.

During pre-release testing, we discovered two issues with this commit.
Firstly, there's a typo in the udevadm command that causes a TypeError
for _all_ growpart executions.  Secondly, the LVM resizing does not
appear to successfully resize everything up to the LV, though some
things do get resized.

We certainly want this change, so we'll be happy to review and land it
alongside an integration test which confirms that it is working as
expected.

LP: #1922742
Bump the version in cloudinit/version.py to 21.2 and update ChangeLog.

LP: #1927254
drop the following cherry picks:
  cpick-83f6bbfb-Fix-unpickle-for-source-paths-missing-run_dir-863
  cpick-d132356c-fix-error-on-upgrade-caused-by-new-vendordata2
@TheRealFalcon TheRealFalcon changed the base branch from ubuntu/daily/devel to ubuntu/devel May 6, 2021 13:41
@TheRealFalcon TheRealFalcon requested a review from OddBloke May 6, 2021 13:47
@OddBloke
Copy link
Copy Markdown
Collaborator

OddBloke commented May 6, 2021

Have to manually readjust a ^M to be \r\n in debian/changelog (and now I'm wishing I had used a better commit message).

git presents it correctly as a 4 character sequence ("", "r", "", "n") so it's something in our tooling which is mangling it, I think: we certainly shouldn't have to avoid backslashes in commit messages.

Also, is there a reason we use an annotated tag for upstream releases, but an unannotated one for Ubuntu releases, or is it unintentional?

The canonical source for cloud-init upstream releases is those tags, so we want a GPG signature so people can verify that the tag was intentionally created by a person they trust. The canonical source for cloud-init in Ubuntu is the archive: that has its own trust path already, of course, so the tagging in our repo is more informational than anything else. So: I don't know if it's intentional, but I don't think it's a particular issue. (FAOD, using an annotated tag wouldn't be a problem, it's just not necessary.)

Comment thread debian/changelog Outdated
@OddBloke
Copy link
Copy Markdown
Collaborator

OddBloke commented May 6, 2021

I'm seeing a few test failures in my local package build:

=========================== short test summary info ============================
FAILED cloudinit/distros/tests/test_init.py::TestGetPackageMirrorInfo::test_valid_substitution[None-fk\tfake\t1-patterns21-expected21-True-ec2]
FAILED cloudinit/distros/tests/test_init.py::TestGetPackageMirrorInfo::test_valid_substitution[None-fk\nfake\n1-patterns22-expected22-True-ec2]
FAILED cloudinit/distros/tests/test_init.py::TestGetPackageMirrorInfo::test_valid_substitution[None-fk\rfake\r1-patterns25-expected25-True-ec2]
============ 3 failed, 2716 passed, 10 skipped, 1 warning in 33.01s ============

I can't reproduce in an impish daily container until I install packages from -proposed: at that point, it reproduces. My best from-the-hip guess is that this is a Python 3.9.4->3.9.5 difference, as that's available in -proposed ATM, but I haven't validated this.

@OddBloke
Copy link
Copy Markdown
Collaborator

OddBloke commented May 6, 2021

I can't reproduce in an impish daily container until I install packages from -proposed: at that point, it reproduces. My best from-the-hip guess is that this is a Python 3.9.4->3.9.5 difference, as that's available in -proposed ATM, but I haven't validated this.

https://docs.python.org/release/3.9.5/whatsnew/changelog.html indicates that https://bugs.python.org/issue43882 was part of 3.9.5: "[security] urllib.parse should sanitize urls containing ASCII newline and tabs". So that looks like a pretty clear match with these failures!

@TheRealFalcon
Copy link
Copy Markdown
Contributor Author

Thanks for finding this! I put up #893 to address this.

andrewlukoshko and others added 3 commits May 7, 2021 16:31
AlmaLinux OS is RHEL-compatible so all the changes needed are trivial.
…l#886)

UDEVADM_CMD is defined but not actually used in cc_disk_setup.py
so remove it.

Also modify the comment at top of read_parttbl function to remove the
reference to udevadm which implies it is used to scan the partition table.
In newer versions of python, when using urllib.parse, lines containing
newline or tab characters now get sanitized. This caused a unit test to
fail.

See https://bugs.python.org/issue43882
@TheRealFalcon
Copy link
Copy Markdown
Contributor Author

Updated to 21.2-3-g899bfaa9

Copy link
Copy Markdown
Collaborator

@OddBloke OddBloke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@OddBloke OddBloke merged commit 398f7e6 into canonical:ubuntu/devel May 11, 2021
@TheRealFalcon TheRealFalcon deleted the ubuntu/devel branch May 11, 2021 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.