unreliable benchmarks results

[pared]

Seems like results provided by asv are unreliable. Here are few latest runs from our StartupBench runs, compared to runs on my computer:

Both runs on the bottom of image are on my computer, one is the same number of runs as master (100) and other is 800.
Maximum disrepancy between those two runs is ~ 4%.

Working theory so far: I set number of processes to 2, and this is the number of processes of the runner, so it might be we are fighting for resources with GHA runner and other background processes.

I will try reducing the number of processes to 1. If it won't help, it might be another vote for using our own machines.

[pared]

#132 should fix it
Leaving open until we see the results.

[pared]

After #132, #159 and #154 Situation got a bit better, seems like results for windows and linux (real use cases, Add) are stable. #172 Will be a workaround that will silence less important benchmarks.

We still have problems with Darwin.

[pared]

After #211 situation got much better, though #221 and #220 show that issue is still not resolved.

[pared]

Closing, afterhttps://github.com//pull/296 we no longer use asv. We will reopen if we notice inconsistency in pytest based benchmarks.