[BUG] Investigate false positives in ASV benchmarks #1142
Description
Bug Description
When using GitLab CI/CD runners, we occasionally see false positives reported in the ASV results (e.g. a benchmark gets reported as slowing down above the 10% threshold but no changes were made that would affect the quantity being measured).
This issue is used to keep track of the benchmarks with such sampling issues.
| Test Name | Measurement 1 | Measurement 2 | Variation | Observations |
|---|---|---|---|---|
codegen.mat22_augassign.CompileModule.time_cuda_codegen |
62.3±2ms | 77.9±2ms | 25% | 5 |
codegen.mat44_assignment.CompileModule.time_cuda_codegen |
58.9±2ms | 45.4±1ms | 30% | 2 |
bvh_build.BvhBuild.time_build('median', 'bunny') |
3.49±0.01ms | 4.25±0.6ms | 22% | 1 |
tile.load_store.LoadStore.time_cuda('register', 1024) |
10.3±0.5μs | 12.1±0.7μs | 17% | 1 |
codegen.array2d_augassign.RunForwardKernel.track_cuda |
7.73μs | 10.1μs | 30% | 2 |
codegen.array2d_augassign.RunBackwardKernel.track_cuda |
8.80μs | 10.1μs | 15% | 1 |
codegen.mat22_augassign.RunForwardKernel.track_cuda |
12.5μs | 13.9μs | 11% | 1 |
spatial_query.BvhRayQuery.time_bvh_ray_vs_aabb_query(480, 0, 'cpu') |
213±10ms | 163±20ms | 30% | 1 |
spatial_query.BvhRayQuery.time_mesh_ray_vs_aabb_query(480, 0, 'cpu') |
117±8ms | 89.8±4ms | 30% | 1 |
System Information
No response