Allow comments in runcmd and report failed commands correctly.

[holmanb]

Proposed Commit Message

summary: Allow comments in runcmd and report failed commands correctly.

A `runcmd` script may fail to parse properly, but does not mark `runcmd` as failed when that occurs.
Additionally `shellify()` fails to correctly parse scripts that contain a comment line.

Rectify both issues and add unit tests to verify correct behavior.

LP: #1853146

Test Steps

To reproduce these issues, use the following runcmd and check the logs after cloud-init runs.

runcmd:
   - echo start
   - # 
   - echo end

This PR contains four commits. The tests added in the first and third test will fail unless the second and fourth fix commits are applied. Rolling back to these commits from HEAD (git reset HEAD~3 && git reset HEAD~1) before running the new tests should demonstrate the failures. The tests should succeed from the second and fourth commit in this series.

Checklist:

• [ TODO: Sign the CLA ] My code follows the process laid out in the documentation
• [ x ] I have updated or added any unit tests accordingly
• [ x ] I have updated or added any documentation accordingly

[TheRealFalcon]

Thanks Brett. This is a great first PR.

I left some comments inline, but one other thing we need to deal with here is that schema validation still complains:

2021-10-07 15:38:43,471 - schema.py[WARNING]: Invalid config:
runcmd.1: None is not valid under any of the given schemas

We need to make sure that type 'null' is acceptable. You should be able to quickly validate the userdata schema with cloud-init devel schema --system

[TheRealFalcon]

Also, I technically broke the rules by giving you a review while still in Draft state. I'm guessing at this point this PR doesn't need to be Draft.

[holmanb]

Thanks Brett. This is a great first PR.

I left some comments inline, but one other thing we need to deal with here is that schema validation still complains:

2021-10-07 15:38:43,471 - schema.py[WARNING]: Invalid config:
runcmd.1: None is not valid under any of the given schemas

We need to make sure that type 'null' is acceptable. You should be able to quickly validate the userdata schema with cloud-init devel schema --system

Thanks for the review @TheRealFalcon! The latest commit addresses your comments (and it is rebased on main).

[TheRealFalcon]

LGTM, thanks!