[BUG] bytes_per_second depends on the timer

[krzikalla]

With

BENCHMARK(...)->Unit(kMillisecond)->Threads(12)->MeasureProcessCPUTime()->UseRealTime();
BENCHMARK(...)->Unit(kMillisecond)->Threads(12)->MeasureProcessCPUTime();

I get different bytes_per_second results:

...    Time             CPU   Iterations UserCounters...
----------------------------------------------------
... 42.6 ms          509 ms           12 bytes_per_second=26.2487G/s
... 43.4 ms          520 ms           12 bytes_per_second=2.14995G/s

The first line is reasonable. IMHO the bytes_per_second rate in the second line needs to be multiplied by 12 (I guess, the number of threads).

[LebedevRI]

#769/#946 could be related.

[dmah42]

it does seem like yet another instance where we're still not doing the right thing wrt threads.

[bryan-lunt]

It seems like when you use MeasureProcessCPUTime() you get items_per_second that are divided by the total CPU time, rather than the total walltime. (And I guess bytes_per_second are likewise devided by CPU time instead of walltime.)

Would it be possible to have items_per_second and also items_per_cpu_second ? Calculated against the walltime, and total cpu time, respectively.

(And the same for bytes_per_second)

--------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------------------------------------------------
BM_OpenMP_squares_serial/N:1048576/num_threads:1/process_time       453399 ns       453391 ns         1543 N=1048.58k items_per_second=2.31274G/s num_threads=1
BM_OpenMP_squares_parallel/N:1048576/num_threads:2/process_time     222271 ns       444513 ns         1388 N=1048.58k items_per_second=2.35893G/s num_threads=2
BM_OpenMP_squares_parallel/N:1048576/num_threads:4/process_time     171604 ns       686309 ns          826 N=1048.58k items_per_second=1.52785G/s num_threads=4
BM_OpenMP_primes_serial/N:1048576/num_threads:1/process_time    56356641648 ns   56355600233 ns            1 N=1048.58k items_per_second=18.6064k/s num_threads=1
BM_OpenMP_primes_parallel/N:1048576/num_threads:2/process_time  41748034121 ns   56860646201 ns            1 N=1048.58k items_per_second=18.4412k/s num_threads=2
BM_OpenMP_primes_parallel/N:1048576/num_threads:4/process_time  24696238448 ns   58096159629 ns            1 N=1048.58k items_per_second=18.049k/s num_threads=4

[bryan-lunt]

I guess my problem is fixed by just using UseRealTime() .

[LebedevRI]

Given that there are user counters, i think we shouldn't add more hadcoded counters.
It's good that it works for you, i assume the issue can be closed?

[bryan-lunt]

@LebedevRI I can only speak for myself, not for the original poster.

I guess it has to remain searchable. Even though I definitely knew this a couple of years ago when you added MeasureProcessCPUTime() , it is easy to forget.

Honestly, I don't think it's expected behaviour that the "items_per_second" would be by CPU time and not by walltime.
I mean, for someone who comes to this software as a firs time user (or having forgotten it), you expect items_per_second to be the number of items that were processed per second.

Is the reason to keep the current behaviour to prevent breaking old code that depends on this project?