Skip to content

Ubuntu/xenial add quilt patches to disable SRU_BLOCKERS#406

Merged
blackboxsw merged 15 commits into
canonical:ubuntu/xenialfrom
blackboxsw:ubuntu/xenial
Jun 4, 2020
Merged

Ubuntu/xenial add quilt patches to disable SRU_BLOCKERS#406
blackboxsw merged 15 commits into
canonical:ubuntu/xenialfrom
blackboxsw:ubuntu/xenial

Conversation

@blackboxsw
Copy link
Copy Markdown
Collaborator

@blackboxsw blackboxsw commented May 29, 2020

Add two quilt patches by following the ubuntu release quilt patch procedure.

Here are some manual steps to produce this;


# Setup patch files
cat > 1.patch <<EOF
diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
index 355b4e2f..ae72b0aa 100644
--- a/cloudinit/sources/DataSourceEc2.py
+++ b/cloudinit/sources/DataSourceEc2.py
@@ -408,13 +408,13 @@
         iface = self.fallback_interface
         net_md = self.metadata.get('network')
         if isinstance(net_md, dict):
-            # SRU_BLOCKER: xenial, bionic and eoan should default
+            # xenial, bionic and eoan now default
             # apply_full_imds_network_config to False to retain original
-            # behavior on those releases.
+            # behavior on those releases.  SRU_UPSTREAM_COMMIT: 6600c6
             result = convert_ec2_metadata_network_config(
                 net_md, fallback_nic=iface,
                 full_network_config=util.get_cfg_option_bool(
-                    self.ds_cfg, 'apply_full_imds_network_config', True))
+                    self.ds_cfg, 'apply_full_imds_network_config', False))
         else:
             LOG.warning("Metadata 'network' key not valid: %s.", net_md)
         self._network_config = result

diff --git a/tests/unittests/test_datasource/test_ec2.py b/tests/unittests/test_datasource/test_ec2.py
index ad1ea595..11abbdf4 100644
--- a/tests/unittests/test_datasource/test_ec2.py
+++ b/tests/unittests/test_datasource/test_ec2.py
@@ -447,7 +447,14 @@ class TestEc2(test_helpers.HttprettyTestCase):
         """
         ds = self._setup_ds(
             platform_data=self.valid_platform_data,
-            sys_cfg={'datasource': {'Ec2': {'strict_id': True}}},
+            sys_cfg={
+                'datasource': {
+                    'Ec2': {
+                        'strict_id': True,
+                        'apply_full_imds_network_config': True
+                    }
+                }
+            },
             md={'md': SECONDARY_IP_METADATA_2018_09_24})
         find_fallback_path = M_PATH_NET + 'find_fallback_nic'
         with mock.patch(find_fallback_path) as m_find_fallback:
EOF


cat 2.patch <<EOF
diff --git a/config/cloud.cfg.tmpl b/config/cloud.cfg.tmpl
index 1bb97f83..005efbc4 100644
--- a/config/cloud.cfg.tmpl
+++ b/config/cloud.cfg.tmpl
@@ -160,9 +160,7 @@ system_info:
      groups: [adm, audio, cdrom, dialout, dip, floppy, lxd, netdev, plugdev, sudo, video]
      sudo: ["ALL=(ALL) NOPASSWD:ALL"]
      shell: /bin/bash
-{# SRU_BLOCKER: do not ship network renderers on Xenial, Bionic or Eoan #}
-   network:
-     renderers: ['netplan', 'eni', 'sysconfig']
+{# SRU_UPSTREAM_COMMIT: 4fb6fd8a. Do not ship network renderers #}
    # Automatically discover the best ntp_client
    ntp_client: auto
    # Other config here will be given to the distro class and/or path classes
diff --git a/tests/unittests/test_render_cloudcfg.py b/tests/unittests/test_render_cloudcfg.py
index 8b1e6042..f4bffdd0 100644
--- a/tests/unittests/test_render_cloudcfg.py
+++ b/tests/unittests/test_render_cloudcfg.py
@@ -43,7 +43,7 @@ class TestRenderCloudCfg:

     @pytest.mark.parametrize('variant,renderers', (
         ('freebsd', ['freebsd']), ('netbsd', ['netbsd']),
-        ('openbsd', ['openbsd']), ('ubuntu', ['netplan', 'eni', 'sysconfig']))
+        ('openbsd', ['openbsd']))
     )
     def test_variant_sets_network_renderer_priority_in_cloud_cfg(
         self, variant, renderers, tmpdir
EOF


cat > 1.header <<EOF
Description: Ec2, do not render full network from IMDS by default
 Retain default network config behavior on Ec2, which is dhcp on eth0.
 If datasource config option apply_full_imds_network_config is set True,
 full network config for all interfaces and secondary addresses described in
 IMDS.
Author: Chad Smith <chad.smith@canonical.com>
Origin: backport
Forwarded: not-needed
Last-Update: 2020-06-01
EOF

cat > header.2 <<EOF
Description: Do not reprioritize netplan before eni on stable releases.
 Retain default network renderer behavior which prioritizes
 rendering /etc/network/interfaces before /etc/netplan.
Author: Chad Smith <chad.smith@canonical.com>
Origin: backport
Forwarded: not-needed
Last-Update: 2020-06-01
EOF

cat > 3.patch <<EOF
diff --git a/debian/changelog b/debian/changelog
index d851a97e..4d9ff2d8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,5 +1,12 @@
-cloud-init (20.2-45-g5f7825e2-0ubuntu1~16.04.1) UNRELEASED; urgency=medium
-
+cloud-init (20.2-45-g5f7825e2-0ubuntu1~16.04.1) xenial; urgency=medium
+
+  * d/control: drop python3-pep8, python3-pyflakes and python3-six from
+    Build-Depends
+  * d/cloud-init.templates: Add RbxCloud datasource support
+  * d/patches/ec2-dont-apply-full-imds-network-config.patch: by default only
+    configure DHCP on eth0 to retain original netwwork config behavior
+  * d/patches/renderer-do-not-prefer-netplan.patch: leave default renderer
+    order as ENI before netplan.
   * New upstream snapshot. (LP: #1881018)
     - test: fix all flake8 E241 (#403) [Joshua Powers]
     - test: ignore flake8 E402 errors in main.py (#402) [Joshua Powers]
diff --git a/debian/cloud-init.templates b/debian/cloud-init.templates
index 64cafbf9..734ba5ae 100644
--- a/debian/cloud-init.templates
+++ b/debian/cloud-init.templates
@@ -1,8 +1,8 @@
 Template: cloud-init/datasources
 Type: multiselect
-Default: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, None
-Choices-C: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, None
-Choices: NoCloud: Reads info from /var/lib/cloud/seed only, ConfigDrive: Reads data from Openstack Config Drive, OpenNebula: read from OpenNebula context disk, DigitalOcean: reads data from Droplet datasource, Azure: read from MS Azure cdrom. Requires walinux-agent, AltCloud: config disks for RHEVm and vSphere, OVF: Reads data from OVF Transports, MAAS: Reads data from Ubuntu MAAS, GCE: google compute metadata service, OpenStack: native openstack metadata service, CloudSigma: metadata over serial for cloudsigma.com,  SmartOS: Read from SmartOS metadata service, Bigstep: Bigstep metadata service, Scaleway: Scaleway metadata service, AliYun: Alibaba metadata service, Ec2: reads data from EC2 Metadata service, CloudStack: Read from CloudStack metadata service, Exoscale: Exoscale metadata service, None: Failsafe datasource
+Default: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, RbxCloud, None
+Choices-C: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, RbxCloud, None
+Choices: NoCloud: Reads info from /var/lib/cloud/seed only, ConfigDrive: Reads data from Openstack Config Drive, OpenNebula: read from OpenNebula context disk, DigitalOcean: reads data from Droplet datasource, Azure: read from MS Azure cdrom. Requires walinux-agent, AltCloud: config disks for RHEVm and vSphere, OVF: Reads data from OVF Transports, MAAS: Reads data from Ubuntu MAAS, GCE: google compute metadata service, OpenStack: native openstack metadata service, CloudSigma: metadata over serial for cloudsigma.com,  SmartOS: Read from SmartOS metadata service, Bigstep: Bigstep metadata service, Scaleway: Scaleway metadata service, AliYun: Alibaba metadata service, Ec2: reads data from EC2 Metadata service, CloudStack: Read from CloudStack metadata service, Exoscale: Exoscale metadata service, RbxCloud: HyperOne and Rootbox platforms, None: Failsafe datasource
 Description: Which data sources should be searched?
  Cloud-init supports searching different "Data Sources" for information
  that it uses to configure a cloud instance.
diff --git a/debian/control b/debian/control
index 6f919475..2155f693 100644
--- a/debian/control
+++ b/debian/control
@@ -15,13 +15,10 @@ Build-Depends: debhelper (>= 9),
                python3-jsonpatch,
                python3-mock,
                python3-oauthlib,
-               python3-pep8,
-               python3-pyflakes,
                python3-pytest,
                python3-requests,
                python3-serial,
                python3-setuptools,
-               python3-six,
                python3-yaml
 XS-Python-Version: all
 Vcs-Browser: https://git.launchpad.net/cloud-init/?h=ubuntu/devel
EOF

git fetch upstream
git checkout upstream/ubuntu/xenial -B ubuntu/xenial
new-upstream-snapshot 5f7825e
quilt push -a
quilt new ec2-dont-apply-full-imds-network-config.patch
cat 1.patch  | quilt fold -p1
quilt refresh
quilt header --dep3 -e   # use content from header.1
quilt pop -a
git add debian/patches
git commit -am 'd/patches: ec2 do not apply full network config from IMDS by default'
quilt push -a
quilt new renderer-do-not-prefer-netplan.patch
cat 2.patch  | quilt fold -p1
quilt refresh
quilt header --dep3 -e   # use content from header.2
quilt pop -a
git add debian/patches
git commit -am 'd/patches: network renderer default remains eni before netplan'
patch -p1 < 3.patch
git commit -m 'd/cloud-init.templates: add RbxCloud datasource support' debian/cloud-init.templates
git commit -m 'd/control: drop python3-pep8, python3-pyflakes and python3-six from build-deps' debian/control
git commit -m 'changelog: update 20.2-45-g5f7825e2-0ubuntu1~16.04.1 for release' debian/changelog 

To validate:

quilt push -a
tox -p auto
quilt pop -a
build-package
sbuild-it ../out/cloud-init_20.2-45-g5f7825e2-0ubuntu1~16.04.1.dsc


We don't merge this branch directly once approved, the author runs uss-tableflip/scripts/build-and-push to perform uploads and push tags.

Joshua Powers and others added 3 commits May 27, 2020 17:28
Instead of running pycodestyle and pyflakes seperately, use flake8 to
get the benefits of pyflakes and also stylistic checks as well as the
ability to configure the settings for the project.
Co-authored-by: Daniel Watkins <oddbloke@ubuntu.com>
Joshua Powers and others added 10 commits June 1, 2020 17:20
This removes the use of variables named ‘l’, ‘O’, or ‘I’. Generally
these are used in list comprehension to read the line of lines.
…anonical#358)

Replace the hardcoded list of devices with a more robust way of determining
the device which grub is installed to.

We use grub-probe to fetch the underlying disk the /boot directory is
located on, and attempt to match the disk with its /dev/disk/by-id value.
If no such /dev/disk/by-id/ value exists, we fallback to the plain disk
name.

The changes are robust to unstable kernel device names and ordering, and use
/dev/disk/by-id values to populate grub-pc/install_devices where possible.

LP: #1877491
This puts an ignore on the imports not at the top of the file errors.
The reason for the ignore instead of fix is that the file is using imp
to grab a lock and patch logging before further imports are completed.
Remove extra spaces after a ','
@blackboxsw blackboxsw force-pushed the ubuntu/xenial branch 2 times, most recently from 69a80f6 to 936088f Compare June 3, 2020 19:34
Copy link
Copy Markdown
Contributor

@lucasmoura lucasmoura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should edit the ec2-classic-dont-reapply-networking.patch to remove the following import:

from cloudinit.event import EventType

Otherwise we get the following flake8 warning:

cloudinit/sources/DataSourceEc2.py:22:1: F401 'cloudinit.event.EventType' imported but unused

@blackboxsw
Copy link
Copy Markdown
Collaborator Author

We should edit the ec2-classic-dont-reapply-networking.patch to remove the following import:

from cloudinit.event import EventType

Otherwise we get the following flake8 warning:

cloudinit/sources/DataSourceEc2.py:22:1: F401 'cloudinit.event.EventType' imported but unused

Thanks for this, I fixed by editing that patch directly:

quilt push debian/patches/ec2-classic-dont-reapply-networking.patch
quilt edit cloudinit/sources/DataSourceEc2.py
# deleete the EventType line
quilt refresh
quilt pop -a
git add debian/patches
git commit -m 'd/patches: fix flake error for ec2-classic-dont-reapply-networking.patch' debian/patches/
# then add changelog entry  for it

Copy link
Copy Markdown
Contributor

@lucasmoura lucasmoura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to reproduce the steps and build the packages.

But the package build reported the following lintian errors:

W: cloud-init source: binary-nmu-debian-revision-in-source 20.2-45-g5f7825e2-0ubuntu1~16.04.1                                                                                                                      
E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6                                                                                                                                     
W: cloud-init source: missing-file-from-potfiles-in grub.templates                                                                                                                                                 
W: cloud-init source: out-of-date-standards-version 3.9.6 (current is 3.9.7)                                                                                                                                       
W: cloud-init: binary-without-manpage usr/bin/cloud-id                                                                                                                                                             
W: cloud-init: binary-without-manpage usr/bin/cloud-init                                                                                                                                                           
W: cloud-init: binary-without-manpage usr/bin/cloud-init-per                                                                                                                                                       
W: cloud-init: command-with-path-in-maintainer-script postinst:141 /usr/sbin/grub-install                                                                                                                          
W: grub-legacy-ec2: binary-without-manpage usr/sbin/grub-set-default                                                                                                                                               
W: grub-legacy-ec2: binary-without-manpage usr/sbin/grub-set-default-legacy-ec2                                                                                                                                    
W: grub-legacy-ec2: binary-without-manpage usr/sbin/update-grub-legacy-ec2                                                                                                                                         
W: ec2-init: empty-binary-package

@blackboxsw
Copy link
Copy Markdown
Collaborator Author

I was able to reproduce the steps and build the packages.

But the package build reported the following lintian errors:

W: cloud-init source: binary-nmu-debian-revision-in-source 20.2-45-g5f7825e2-0ubuntu1~16.04.1                                                                                                                      
E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6                                                                                                                                     
W: cloud-init source: missing-file-from-potfiles-in grub.templates                                                                                                                                                 
W: cloud-init source: out-of-date-standards-version 3.9.6 (current is 3.9.7)                                                                                                                                       
W: cloud-init: binary-without-manpage usr/bin/cloud-id                                                                                                                                                             
W: cloud-init: binary-without-manpage usr/bin/cloud-init                                                                                                                                                           
W: cloud-init: binary-without-manpage usr/bin/cloud-init-per                                                                                                                                                       
W: cloud-init: command-with-path-in-maintainer-script postinst:141 /usr/sbin/grub-install                                                                                                                          
W: grub-legacy-ec2: binary-without-manpage usr/sbin/grub-set-default                                                                                                                                               
W: grub-legacy-ec2: binary-without-manpage usr/sbin/grub-set-default-legacy-ec2                                                                                                                                    
W: grub-legacy-ec2: binary-without-manpage usr/sbin/update-grub-legacy-ec2                                                                                                                                         
W: ec2-init: empty-binary-package

+1 thanks @lucasmoura we will track these lintian issues for our next SRU. the translatable templates have been fixed in ubuntu/focal && devel already by paride I believe.
Also the rest of the lintian errors are no regression from what we currently have. Let's add a trello card to the retrospective lane to address this post SRU for each series.

@blackboxsw blackboxsw merged commit d80d0fc into canonical:ubuntu/xenial Jun 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants