clout init error when running native scripts at start up

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1756083

Launchpad details

affected_projects = []
assignee = None
assignee_name = None
date_closed = 2018-05-01T18:12:06.501567+00:00
date_created = 2018-03-15T14:00:49.479817+00:00
date_fix_committed = None
date_fix_released = None
id = 1756083
importance = undecided
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1756083
milestone = None
owner = fewknow
owner_name = James Flint
private = False
status = invalid
submitter = fewknow
submitter_name = James Flint
tags = []
duplicates = []

Launchpad user James Flint(fewknow) wrote on 2018-03-15T14:00:49.479817+00:00

AWS AMI ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-*

ubuntu@ip-10-90-130-11:~$ cloud-init collect
usage: /usr/bin/cloud-init [-h] [--version] [--file FILES] [--debug] [--force]
{init,modules,single,dhclient-hook,features,analyze,devel,collect-logs,clean,status}
...
/usr/bin/cloud-init: error: argument subcommand: invalid choice: 'collect' (choose from 'init', 'modules', 'single', 'dhclient-hook', 'features', 'analyze', 'devel', 'collect-logs', 'clean', 'status')

When cloud init runs and hits my native scripts it err and there is no Reason, Stdout or Stderr.

These scripts are preforming some string replaces for config files using 'sed'/

[ubuntu-server-builder]

Launchpad user James Flint(fewknow) wrote on 2018-03-15T14:00:49.479817+00:00

Launchpad attachments: cloud-init.tar

[ubuntu-server-builder]

Launchpad user James Flint(fewknow) wrote on 2018-03-15T14:03:05.603416+00:00

Scripts that are trying to be ran.
Launchpad attachments: Native scripts for configuration string replace.

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2018-03-21T22:29:01.693348+00:00

Your scripts look like they are calling 'cloud-init collect' and not 'cloud-init collect-logs'

collect is not a valid command

/usr/bin/cloud-init: error: argument subcommand: invalid choice: 'collect' (choose from 'init', 'modules', 'single', 'dhclient-hook', 'features', 'analyze', 'devel', 'collect-logs', 'clean', 'status')

[ubuntu-server-builder]

Launchpad user James Flint(fewknow) wrote on 2018-03-21T22:44:10.251122+00:00

Chad,
That isn't my error...that was just what I got when collecting the logs. I have attached the error along with the logs. I was just follow the protocol for sending logs. The real error isn't invalid syntax. Please look at the attachments.

[ubuntu-server-builder]

Launchpad user James Flint(fewknow) wrote on 2018-03-21T22:46:17.217730+00:00

This is some of what is in the cloud-init.log

2018-03-15 13:09:23,723 - util.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/vault_update.sh'] with allowed return codes [0] (shell=False, capture=False)
2018-03-15 13:09:57,010 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2018-03-15 13:09:57,011 - handlers.py[DEBUG]: finish: modules-final/config-scripts-user: FAIL: running config-scripts-user with frequency once-per-instance
2018-03-15 13:09:57,011 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
2018-03-15 13:09:57,011 - util.py[DEBUG]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 793, in _run_modules
freq=freq)
File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run
return self._runners.run(name, functor, args, freq, clear_on_fail)
File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 188, in run
return (True, results)
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 76, in lock
yield self._acquire(name, freq)
File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 187, in run
results = functor(*args)
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py", line 45, in handle
util.runparts(runparts_path)
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 810, in runparts
% (len(failed), len(attempted)))

These are the error I am seeing when my native scripts are executed.
Thanks

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2018-05-01T18:11:58.794559+00:00

James, sorry I misunderstood because of your description on this bug. I had miswritten the protocol "cloud-init bug reporting guidelines:" when you clicked the original https://bugs.launchpad.net/cloud-init/+filebug.

It should have told you to run "sudo cloud-init collect-logs" instead of 'cloud-init collect'. I fixed that in Launchpad.

cloud-init-output.log will contain your stdout & stderr from scripts run if the traceback didn't give you enough into in /var/log/cloud-init.log

Looking in the cloud-init-output.log that you attached I see sed related errors reported:
sed: -e expression #1, char 37: unknown option to `s'

One quick way to diagnose and fix custom scripts you deliver is to run them in the deployed instance. they all get written to /var/lib/cloud/instance/scripts/.
You can look at those scripts to make sure they show the content you expected and iteratively run then and sort the errors with your scripts there to figure out what's wrong.

Setting this back in invalid, unless you have updates that you think indicate a cloud-init bug.

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2018-05-01T18:12:31.108074+00:00

If there is a bug that needs tracking, please feel free to update the description of this bug appropriately and set it back to New