ds-identify cannot detect seed defined in cfg file

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1876375

Launchpad details

affected_projects = []
assignee = None
assignee_name = None
date_closed = 2022-05-18T16:51:28.242461+00:00
date_created = 2020-05-01T20:13:10.541732+00:00
date_fix_committed = 2022-04-25T20:44:20.318261+00:00
date_fix_released = 2022-05-18T16:51:28.242461+00:00
id = 1876375
importance = high
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1876375
milestone = None
owner = pwlars
owner_name = Paul Larson
private = False
status = fix_released
submitter = pwlars
submitter_name = Paul Larson
tags = []
duplicates = []

Launchpad user Paul Larson(pwlars) wrote on 2020-05-01T20:13:10.541732+00:00

Cloud-init version 20.1-10-g71af48df-0ubuntu5 from a core20 image
I added a file called 98_nocloud.cfg that looks like this:

#cloud-config
datasource_list: [ NoCloud, None ]
datasource:
NoCloud:
user-data: |
#cloud-config
users:
- name: ubuntu
ssh-authorized-keys:
- <MY_SSH_KEY>
homedir: /home/ubuntu
sudo: ALL=(ALL) NOPASSWD:ALL
shell: /bin/bash

meta-data: |
  instance_id: cloud-image

---

== Reproducing ==

1. create a usb stick, or kvm image for core20
2. place the above file in the ubuntu-seed partition under /data/etc/cloud/cloud.cfg.d/98_nocloud.cfg
   =================

When I boot, the datasource is not used and cloud-init does not process it unless I also include this line at the top:
datasource_list: [ NoCloud, None ]

This was surprising though, because another existing file in /etc/cloud/cloud.cfg.d called 90_dpkg.cfg has these contents:

to update this file, run dpkg-reconfigure cloud-init

datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, None ]

Here are the contents of /run/cloud-init/ds_identify.log when it fails:
[up 4.57s] ds-identify
policy loaded: mode=search report=false found=first maybe=none notfound=disabled
/etc/cloud/cloud.cfg.d/90_dpkg.cfg set datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, None ]
DMI_PRODUCT_NAME=Standard PC (i440FX + PIIX, 1996)
DMI_SYS_VENDOR=QEMU
DMI_PRODUCT_SERIAL=
DMI_PRODUCT_UUID=unavailable
PID_1_PRODUCT_NAME=unavailable
DMI_CHASSIS_ASSET_TAG=
FS_LABELS=ubuntu-seed,ubuntu-boot,ubuntu-data
ISO9660_DEVS=
KERNEL_CMDLINE=snapd_recovery_mode=run console=ttyS0 console=tty1 panic=-1
VIRT=kvm
UNAME_KERNEL_NAME=Linux
UNAME_KERNEL_RELEASE=5.4.0-26-generic
UNAME_KERNEL_VERSION=#30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020
UNAME_MACHINE=x86_64
UNAME_NODENAME=ubuntu
UNAME_OPERATING_SYSTEM=GNU/Linux
DSNAME=
DSLIST=NoCloud ConfigDrive OpenNebula DigitalOcean Azure AltCloud OVF MAAS GCE OpenStack CloudSigma SmartOS Bigstep Scaleway AliYun Ec2 CloudStack Hetzner IBMCloud Oracle Exoscale RbxCloud None
MODE=search
ON_FOUND=first
ON_MAYBE=none
ON_NOTFOUND=disabled
pid=607 ppid=591
is_container=false
is_ds_enabled(IBMCloud) = true.
is_ds_enabled(IBMCloud) = true.
ec2 platform is 'Unknown'.
No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1]
[up 4.76s] returning 1

[ubuntu-server-builder]

Launchpad user Ryan Harper(raharper) wrote on 2020-05-01T20:26:00.804740+00:00

Thanks for filing this.

I believe we can use check_config in ds-identify to see if datasource: NoCloud is present, just like we do for MAAS.

Optionally, we can decide if we should generally check for the dsname value in config as well.

[ubuntu-server-builder]

Launchpad user Paul Larson(pwlars) wrote on 2020-05-01T20:40:42.519657+00:00

Not sure if it's relevant, but one thing I found interesting is that as long as I include datasource_list at the top, I didn't even have to include a datasource: NoCloud: ... section.
For example, this works as long as I made sure it runs after 90_dpkg.cfg (mentioned in above comment)

#cloud-config
datasource_list: [ NoCloud, None ]
users:

• name: ubuntu
  ssh-authorized-keys:
  - <MY_SSY_KEY>
  homedir: /home/ubuntu
  sudo: ALL=(ALL) NOPASSWD:ALL
  shell: /bin/bash

[ubuntu-server-builder]

Launchpad user James Falcon(falcojr) wrote on 2022-04-15T02:24:06.838746+00:00

See #1381

Note that this has nothing to do with the reading of the config files. If the datasource list only contains one entry (or one + None), ds-identify skips datasource detection entirely and just uses the specified datasource. https://github.com/canonical/cloud-init/blob/main/tools/ds-identify#L1793-L1798 . The problem is that ds-identify just had no means of detecting this use case. It worked for you because you specified "datasource_list: [ NoCloud, None ]" whereas "90_dpkg.cfg" defined a much larger set.

[ubuntu-server-builder]

Launchpad user Brett Holman(holmanb) wrote on 2022-05-18T16:51:29.228991+00:00

This bug is believed to be fixed in cloud-init in version 22.2. If this is still a problem for you, please make a comment and set the state back to New

Thank you.