Disable built-in TP to avoid interfering with this add-on

[fmarier]

We need to make sure that real TP is disabled to make sure that the breakages come from the add-on:

• privacy.trackingprotection.enabled = false
• privacy.trackingprotection.pbmode.enable = false

We should probably also detect Privacy Badger, uBlock, ABP, etc. and ask users to disable them.

[fmarier]

The concern here is that if you have both TP and blok enabled, then you need to whitelist a site twice for example if you want to exempt it from any blocking. For example, if a user disables blocking on cnn.com via blok, it still won't work because TP will be blocking the resources too.

The way it works is that add-ons go first and can decide whether or not to cancel network loads. then once the add-ons are all done, we do one last check in TP just before we open a network socket.

[groovecoder]

@lmorchard - can we add another broadcast channel for this? Maybe 'testpilot-aboutconfig' for web extensions to send messages for Test Pilot to set about:config values?

And/or, (how) could a web extension ask Test Pilot what other add-ons are installed?

[groovecoder]

@johngruen or @fmarier: do we have telemetry numbers on how many users are running with privacy.trackingprotection.enabled = false For the purposes of a test pilot experiment, might those numbers be low enough that we can simply ask the users who are reporting breakage if they are also running with built-in tracking protection?

[groovecoder]

@johngruen @fmarier: Actually, when we ask users for feedback, we will get a telemetry ping from them. That telemetry ping should include the metadata whether or not tracking protection is enabled, right?

[lmorchard]

WebExtensions are sandboxed for good security reasons, and we need to be really careful about how & what additional capabilities are granted through Test Pilot.

I think the BroadcastChannel scheme is reasonably locked down to individual add-on IDs, so we can restrict what messages Test Pilot honors. But, we need to be sure there's no way for other arbitrary WebExtensions to tap into those super-powers. Telemetry pings are one thing, but prefs changes are dangerous.

That said, I think we could add those capabilities via the BroadcastChannel mechanism. I'd say rather than create a new channel, we add a type property to the events sent to the existing BroadcastChannel to subdivide it between Telemetry pings and additional things like pref change & add-on list requests.

[groovecoder]

Yeah that makes sense - we should be extra careful about what messages Test Pilot can receive from web extensions.

Deny-by-default with a small enumerated list of specific types of messages/events we allow sounds like the best way.

[fmarier]

do we have telemetry numbers on how many users are running with privacy.trackingprotection.enabled = false

We do, it's the first one in here: https://wiki.mozilla.org/Security/Tracking_protection#Telemetry

[groovecoder]

So, approx. 1.45% of sessions have tracking protection enabled?

[fmarier]

It depends on the channel, it's 7% on Nightly and < 1% on Beta.

[fmarier]

Instead of disabling the built-in TP (i.e. changing a pref), another approach would be to detect it (i.e. reading a pref only) and then refuse to run the experiment until the user has turned it off.

[groovecoder]

@lmorchard - what do you think about adding Test Pilot code for these new types of messages:

• {type: 'prefValueRequest', pref: 'privacy.trackingprotection.enabled'}
• {type: 'addonListRequest'}

?

[lmorchard]

what do you think about adding Test Pilot code for these new types of messages

Those look just read-only, and so way less worrisome than changing prefs. Been swamped with the latest Test Pilot sprint this week, but can look into doing that stuff soon. If you get a minute, maybe file an issue on our repo and we can prioritize it?

[groovecoder]

Filed mozilla/testpilot#1068

[groovecoder]

@lmorchard: is there a way for Test Pilot (add-on and/or server) to only show the Blok experiment to users who:

• Have privacy.trackingprotection.enabled = false
  AND
• Do NOT have another blocking add-on

?

If so, maybe the best way to do this is to let Test Pilot filter users out of the Blok experiment altogether?