MINOR: Make ByteUtilsBenchmark deterministic#13307
MINOR: Make ByteUtilsBenchmark deterministic#13307divijvaidya wants to merge 2 commits intoapache:trunkfrom
Conversation
There was a problem hiding this comment.
Thanks for the improvement! The first point makes sense to me. But for the second point:
We ensure that a new set of random numbers are generated per iteration, so that the bechmark calculation is performed over a diverse range of values
I didn't understand why we need this change. Could you elaborate more on it? From what I can see, it should make no difference if we use the same values or not for each iteration.
Thank you.
Having different values for each iteration gives us ability to benchmark over diverse sample set. This increases the probability that the algorithm we are testing is optimal over a larger set of values. |
Motivation
The current implementation of the benchmark may produce different set of integers across benchmarks and hence, may not provide apples to apples comparison.
Changes
Sample result of a benchmark run: