Attempt to speed up the computation of the auth_difference

[DMRobertson]

In matrix-org/matrix-spec#1118 I noted that one can ignore the events in the unconflicted state map when computing the auth difference, because their auth chains will never appear in the auth difference. This change changes Synapse to take advantage of this fact.

In highly unscientific testing, I reproduced the state resolution of an event with two parents whose conflicted state set contained 255 events (239 type/state-key pairs) and whose auth difference contains 250 events.

Before:

real	0m10.243s
user	0m2.117s
sys	0m0.106s

After:

real	0m7.869s
user	0m1.958s
sys	0m0.105s

I repeated these a handful of times and found those numbers (~10sec, ~8sec) to be fairly reproducible. But I'm afraid I don't have any rigorous statistics to justify this.

It's also worth nothing that I'm running this on my local machine connected remotely to a postgres database (circa 15 ms RTT) so that might be exaggerating time spent waiting for the DB.

I have no evidence to performance which shows what happens in a typical or average case. (Though see #13036.)

I checked my example that the outcome of state resolution was unchanged, but I'm relying on the unit test coverage to assert that is the case more broadly.

[DMRobertson]

I'm quite scared by this change. (Invoking Knuth, "Beware of bugs in the above code; I have only proved it correct, not tried it.") The one example is fine, and the tests are happy. But I wonder if there might be another way to build confidence... do you have any thoughts?

[erikjohnston]

I'm quite scared by this change. (Invoking Knuth, "Beware of bugs in the above code; I have only proved it correct, not tried it.") The one example is fine, and the tests are happy. But I wonder if there might be another way to build confidence... do you have any thoughts?

I have in the past taken a chunk of a large public room out of the DB and compared the resulting state at each event using both the old and new algorithm

[DMRobertson]

Maths was wrong, see the spec issue