As pointed out by @xlz, libusb has a new release 1.0.20. Luckily, this now does work with libfreenect2, at least on Intel controllers.
I've been so fed up with the whole libusb-multiple-version-RPATH mess that I quickly created an Ubuntu PPA for the new version: https://launchpad.net/~floe/+archive/ubuntu/libusb
Not sure how to handle this for Debian, though - suggestions?