WebMIDI

[qdot]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: WebMIDI
• Specification or proposal URL: https://webaudio.github.io/web-midi-api/
• Caniuse.com URL (optional): https://caniuse.com/#feat=midi
• Bugzilla URL (optional): https://bugzilla.mozilla.org/show_bug.cgi?id=webmidi

Other information

WebMIDI has been in development in Firefox since 2015. The DOM API (https://bugzilla.mozilla.org/show_bug.cgi?id=1201590) recently landed, but is pref'd off on all branches, runs only in tests, and was mostly landed to curb bit-rot. Platform implementation bugs exist at:

• Mac: https://bugzilla.mozilla.org/show_bug.cgi?id=1201593
• Windows: https://bugzilla.mozilla.org/show_bug.cgi?id=1201596
• Linux: https://bugzilla.mozilla.org/show_bug.cgi?id=1201598

The major blocker right now is security issues related to device firmware hijacking. There's a public google doc available listing issues and ideas for mitigation at

https://docs.google.com/document/d/1SjYRmNvQKxOPHufWbx6n0NOeTKKd_O1WIiTBZAjVj5E/edit#heading=h.lavkfo6fb57g

[martinthomson]

Thanks for the extra context, that made reviewing this easy.

The big concern here is SysEx messages. Those have the same concerns as WebUSB. From the text in the specification, these appear to grant unrestricted access to the device. Simply asking for permission isn't a good answer to this class of problem: we can't expect someone to understand the full implications of this if the potential extent of capabilities they are extending to the origin includes rewriting firmware.

The mitigations doc says:

No firmware overwrite reports related to the Chrome implementation of WebMIDI have been mentioned in public so far.

That's not relevant. The same applies to the language assessing likelihood or difficulty of attack. What is relevant is that the possibility exists. And that includes rewriting the MIDI device to be a (remote controlled) keyboard, which has implications for not just browser security, but system security.

It would be irresponsible of us to ship this without addressing this concern. As I see it, the safe options are: disable SysEx, whitelist devices for SysEx, and don't ship. I don't see a huge amount of harm in shipping without SysEx or with a whitelist.

I'll let others speak to the benefits of the API. I don't do MIDI or know anyone who does, so don't have any investment.

[marcoscaceres]

I'm also not a MIDI person, so don't really know the implications of not including SysEx. The doc says the consequences are "severe" with respect to limiting functionality.

@qdot, should we ask some MIDI folks for opinions (e.g., Chris Wilson)?

I'll let others speak to the benefits of the API. I don't do MIDI or know anyone who does, so don't have any investment.

5 years ago, I spent a lot of time helping with editing, implementing a polyfill, and testing this specification. I've personally seen some pretty amazing things done with it at conferences and events I've attended: like, whole parties DJ'ed realtime using the API, and really impressive controller software written specifically for the API.

While I was looking into Web MIDI, I also investigated the iOS MIDI ecosystem... it's pretty vibrant (see example native apps). I mention iOS here because I believe we have an opportunity to tap into that community through the Web.

From the perspective that this allows the creation of really amazing musical experiences (and it makes the web a creative outlet for talented musicians) - I'd strongly support shipping this API. It would make a lot of people happy - and make more events pretty awesome 🎹💃🕺.

[marcoscaceres]

Loop Drop was the software I saw being used by @mmckegg at an event. Matt is a good person to talk to from a developer perspective, as he probably knows Web MIDI as well as anyone.

[stpeter]

I agree with @marcoscaceres on the possibilities here. I'm pretty sure @obensource & friends have used the Web MIDI API in the https://github.com/websound project - I'll check with Ben for details.

[stpeter]

@qdot With respect to whitelisting, is this something that we could co-own with Google or crowdsource with the MIDI community (including device vendors)?

[mmckegg]

Thanks for letting me know about this issue @marcoscaceres.

Some random thoughts about SysEx and Web Audio off the top of my head:

I think that (security issues aside), rewriting the firmware of a midi device is probably one of the coolest potential use cases for Web MIDI. Also a lot of devices use sysex to backup and restore presets (yet another potentially great web midi use case).

• Here is an example of a web page for updating the firmware of the RK-002 smart midi cable.
• Here's another page for updating the settings for the CV.OCD Midi -> CV converter. This doesn't use Web Midi, but instead allows you to download the sysex file for manual use - but this would be a great potential use case!

Besides system level stuff, SysEx can be vital for certain midi controllers to access advanced functionality. I use it personally in Loop Drop with the Launchpad MK2 for setting the colors of the RGB lights. Another case is for updating the LCD screen on the Ableton Push (original).

I'm not sure how standard this is, but I have noticed that it is quite common for devices to only accept firmware updates when they are booted in a special mode. Most likely exactly to avoid the case where some random software pushes malicious firmware updates to a device. Unfortunately this is not the case with all midi controllers.

So I think in combination with a security prompt "Would you like to allow full access to this midi device (SysEx)", this should be good enough. Maybe a warning in the message that this might allow the web page to update the firmware of the connected device?

---

We need to find a way to support sysex safely in Web Midi, otherwise we'll be missing out on a tonne of possibilities.

Still, I'd much prefer Web Midi API without SysEx than no Web Midi API at all!

[qdot]

@marcoscaceres I've already pinged Chris Wilson, Paul Adenot, and a couple of vendors who've implemented applications using WebMIDI that are currently running on Chrome. I hope that we'll hear from them over the next few days.

@stpeter Possibly! I'll let Chris Wilson speak to that, though I may try to loop in other Chrome engineers working on WebMIDI also.

[qdot]

@mmckegg In that gdocs file in the top comment, I listed firmware reflash instructions from a few devices from different manufacturers. From the instructions, it looks like some require user gesture to reflash, while others it appears can be reflashed by just having them connected. It's that second set that are the worrisome ones, unfortunately.

I should say that these firmware attack issues really only apply to USB MIDI devices too. If a device uses actual 5-pin MIDI, that'll be going through an adapter that most likely cannot be reflashed, so the potential there is less security and more annoyance.

[mmckegg]

Ah, thanks @qdot. That document is a good analysis!

The only addition I have would be to the Blacklist approach. Rather than blacklisting SysEx entirely on an device, could blacklist certain SysEx prefixes.

For example, the Launchpad MK2 is one of the devices that can be updated without being put into a special mode. Instead this mode is achieved by sending a particular sysex message (F0h 00h 20h 29h 00h 71h 00h 69h F7h). See Launchpad MK2 Programmers Reference Guide under "Other SysEx Messages" (page 16).

If the SysEx message itself was blacklisted for that device, there would no longer be a risk of the firmware being updated, and we can still have access to updating the LED colors, and scrolling text etc.

[dvhdr]

Thanks @qdot & co for the hard work and commentary! I just wanted to raise that sysex firmware upgrades are the reason we built our Components app. It's the single most important use case for our customers - getting new content onto their hardware with a minimum of fuss. Yes, this means firmware upgrades without gestural input, as @mmckegg describes above.

In fact, writing new third-party firmware to our devices is something we encourage (e.g. with open firmware development tools like https://github.com/dvhdr/launchpad-pro).

I'm no security expert and don't wish to downplay the concerns raised. I'd happily support any initiative that gets sysex-capable WebMIDI out into the wider world, even if that does compromise the user experience. But please - it's got to have sysex!

[ekr]

A few notes:
It's useful to have people explain why they want this, but ultimately our obligation is to the security of our users, so no matter how useful it is, if we can't make it secure we can't do it.

"So I think in combination with a security prompt "Would you like to allow full access to this midi device (SysEx)", this should be good enough. Maybe a warning in the message that this might allow the web page to update the firmware of the connected device?"

The problem here is that it's not obvious to users that this allows the site to quite likely take control of the user's computer.

The initial framing of this was: how do we avoid this mechanism being used for firmware updates, but if that's actually a feature you are trying to enable, I don't see how to make it safe.

[qdot]

There's still different levels of security inside of firmware updates too. For instance, are we /just/ worried about USB descriptor rewrite, which is what's required for the HID attack referenced? If so, there's nothing saying firmware rewrite would always enable that. Firmware is sometimes but not always responsible for USB descriptor handling.

If manufacturers are specifically open to third party firmware, it doesn't seem like the not-security-but-annoying issue of bricking is a thing we can reasonably worry about stopping. We'd have to expect the manufacturer to put in a way to always boot to bootloader, and to make sure the bootloader can't be overwritten by USB.

Not sure if this level of nuance matters in the argument or not, though, since we're mostly trying to cover unwanted firmware reflashes period.

[adamroach]

@ekr --

The initial framing of this was: how do we avoid this mechanism being used for firmware updates, but if that's actually a feature you are trying to enable, I don't see how to make it safe.

I think I'm reading that as "how do we avoid this mechanism being used for unwanted firmware updates?" For those devices that require the user to physically manipulate the device to initiate such an update, this seems to provide the right level of agency (unless I've overlooked something important). So whitelisting SysEx those devices that are known to have such a property would seem to get us to "safe," right?

[ekr]

@adamroach: again, I think the user has a lot of trouble meaningfully consenting to a firmware update, so "unwanted" seems difficult.

[martinthomson]

Is there any reason why SysEx access needs to be raw? Is it possible to identify common functions that are enabled by SysEx and build APIs specific to those functions?

My intuition is that the breadth of applications are large and the use of SysEx is sufficiently non-uniform as to make this untenable.

Thus, I might conclude that whitelisting would need to be contingent on some guarantee that either the device can't do bad things. Bad things definitely includes firmware updates, unless we have other guarantees that (for instance) this extends to a guarantee that a firmware update can't cause a device to change type or enable other new bad functions. That's a hard guarantee to extract, and a lot of effort to police effectively. I'm fairly sure that we don't want to be in that position.