user-data: only verify mime-types for TYPE_NEEDED and x-shellscript

[raharper]

Commit d00126c regressed cloud-init
handling in multipart MIME user-data. Specifically, cloud-init would
examine the payload of the MIME part to determine what the content
type and subsequently which handler to use. This meant that user-data
which had shellscript payloads (starts with #!) were always handled
as shellscripts, rather than their declared MIME type and affected
when the payload was handled.

One failing scenario was a MIME part with text/cloud-boothook type
declared and a shellscript payload. This was run at shellscript
processing time rather than boothook time resulting in an change in
behavior from previous cloud-init releases.

To continue to support known scenarios where clouds have specifed
a MIME type of text/x-shellscript but provided a payload of something
other than shellscripts, we're changing the lookup logic to check for
the TYPES_NEEDED (text/plain, text/x-not-multipart) and only
text/x-shellscript.

It is safe to check text/x-shellscript parts as all shellscripts must
include the #! marker and will be detected as text/x-shellscript types.
If the content is missing the #! marker, it will not be excuted. If
the content is detected as something cloud-init supports, such as
#cloud-config the appropriate cloud-init handler will be used.

This change will fix hanldling for parts which were shellscripts but
ran with the wrong handler due to ignoring of the provided mime-type.

LP: #1888822

[detiber]

These changes look good to me and I believe will resolve the issue we've been seeing where our boothook is being interpreted as 'text/x-shellscript'

[OddBloke]

Thanks!

[voor]

I'm still getting up to speed on the release cadence of cloud-init, do you know the ETA for this to make it live to an Ubuntu image pulling from the regular canonical repositories?

[mitechie]

It depends a bit on where the work is that needs to go into the next release. Right now we're finishing up a chunk of work to being a release this month so hopefully, it'll be in images in the next couple of weeks.

…

On Fri, Aug 14, 2020 at 1:55 PM Robert Van Voorhees < ***@***.***> wrote: I'm still getting up to speed on the release cadence of cloud-init, do you know the ETA for this to make it live to an Ubuntu image pulling from the regular canonical repositories? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#511 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAASRC7NVLVRH3ZVRLKB3U3SAV3ARANCNFSM4PK5L7NQ> .

[OddBloke]

On Fri, Aug 14, 2020 at 10:55:19AM -0700, Robert Van Voorhees wrote: I'm still getting up to speed on the release cadence of cloud-init, do you know the ETA for this to make it live to an Ubuntu image pulling from the regular canonical repositories?

This will go back to existing releases via the Stable Release Update (SRU) process (see https://wiki.ubuntu.com/StableReleaseUpdates for general background, and https://wiki.ubuntu.com/CloudinitUpdates for cloud-init's exception to the more general process). We perform SRUs as needed, on no particular cadence: as it happens, we intend to start one within the next week or so. We have to perform a fair amount of manual validation when backporting new cloud-init versions so, once started, the process generally takes a week or two. Once our validation is complete, we will release the cloud-init backport to -updates for each series (e.g. bionic-updates). At this point, it will be installable for instances pulling from the Ubuntu repos. The cloud image build process will pick that up within a day (they trigger builds on a daily cadence), and it should be promoted to a release cloud image once it is complete. So: we can't give you an exact ETA, but it should be on the order of weeks.