Benchmarks for dotnet/runtime#127429 (for @EgorBo)

[EgorBot]

Processing benchmark request from PR dotnet/runtime#127429.

Targets: ubuntu24_azure_turin, ubuntu24_azure_emeraldrapids, macos15_helix_arm64
Commits: main;PR_127429

• ubuntu24_azure_turin: live logs
• ubuntu24_azure_emeraldrapids: live logs
• macos15_helix_arm64: live logs

Results will be posted here once they are ready.

[EgorBot]

Results for macos15_helix_arm64

@EgorBo

BenchmarkDotNet v0.15.8, macOS Sequoia 15.4.1 (24E263) [Darwin 24.4.0]
Apple M4, 1 CPU, 10 logical and 10 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), Arm64 RyuJIT armv8.0-a

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429**	4	0.9281 ns	0.0008 ns	1.00
SumInt	main	4	0.9439 ns	0.0012 ns	1.02
SumLong	PR #127429	4	0.6835 ns	0.0009 ns	1.00
SumLong	main	4	0.7048 ns	0.0006 ns	1.03
SumInt	PR #127429**	8	2.1232 ns	0.0392 ns	1.00
SumInt	main	8	2.1096 ns	0.0423 ns	0.99
SumLong	PR #127429	8	2.0281 ns	0.0521 ns	1.00
SumLong	main	8	2.0202 ns	0.0240 ns	1.00
SumInt	PR #127429**	16	6.8610 ns	0.0251 ns	1.00
SumInt	main	16	1.0952 ns	0.0010 ns	0.16
SumLong	PR #127429	16	4.1503 ns	0.0599 ns	1.00
SumLong	main	16	4.1184 ns	0.0420 ns	0.99
SumInt	PR #127429**	32	12.5983 ns	0.2187 ns	1.00
SumInt	main	32	2.7335 ns	0.0053 ns	0.22
SumLong	PR #127429	32	8.0132 ns	0.0491 ns	1.00
SumLong	main	32	7.9874 ns	0.0564 ns	1.00
SumInt	PR #127429**	50	17.2120 ns	0.3737 ns	1.00
SumInt	main	50	3.1731 ns	0.0024 ns	0.18
SumLong	PR #127429	50	12.2942 ns	0.2163 ns	1.00
SumLong	main	50	12.3459 ns	0.1080 ns	1.00
SumInt	PR #127429**	100	35.4789 ns	0.9136 ns	1.01
SumInt	main	100	7.3909 ns	0.0195 ns	0.21
SumLong	PR #127429	100	25.6157 ns	0.1280 ns	1.00
SumLong	main	100	25.7308 ns	0.1496 ns	1.00
SumInt	PR #127429**	256	99.7787 ns	3.4886 ns	1.01
SumInt	main	256	23.5085 ns	0.0665 ns	0.24
SumLong	PR #127429	256	70.0494 ns	0.3557 ns	1.00
SumLong	main	256	70.2869 ns	0.3917 ns	1.00
SumInt	PR #127429**	1024	453.7580 ns	24.3913 ns	1.03
SumInt	main	1024	114.0266 ns	0.2298 ns	0.26
SumLong	PR #127429	1024	247.9553 ns	1.5336 ns	1.00
SumLong	main	1024	247.7643 ns	1.5426 ns	1.00
SumInt	PR #127429**	4096	1,844.6548 ns	102.4325 ns	1.03
SumInt	main	4096	478.4417 ns	1.7430 ns	0.27
SumLong	PR #127429	4096	964.2839 ns	4.8786 ns	1.00
SumLong	main	4096	956.4189 ns	4.5831 ns	0.99
SumInt	PR #127429**	16384	7,588.7620 ns	365.7186 ns	1.02
SumInt	main	16384	1,935.6048 ns	7.2966 ns	0.26
SumLong	PR #127429	16384	3,937.4206 ns	42.2122 ns	1.00
SumLong	main	16384	3,930.1385 ns	31.5823 ns	1.00
SumInt	PR #127429**	65536	30,222.8071 ns	1,504.7136 ns	1.02
SumInt	main	65536	7,999.4535 ns	12.8818 ns	0.27
SumLong	PR #127429	65536	15,698.7462 ns	107.8341 ns	1.00
SumLong	main	65536	15,688.4569 ns	80.9624 ns	1.00
SumInt	PR #127429**	1000000	NA	NA	?
SumInt	main	1000000	NA	NA	?
SumLong	PR #127429	1000000	273,183.1764 ns	787.2693 ns	1.00
SumLong	main	1000000	273,298.4302 ns	927.8206 ns	1.00

Benchmarks with issues:
SumBench.SumInt: PR #127429) [N=1000000]
SumBench.SumInt: main) [N=1000000]

Full logs

[EgorBot]

Results for ubuntu24_azure_turin

@EgorBo

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 9V45 2.60GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429**	4	0.6473 ns	0.0340 ns	1.00
SumInt	main	4	0.6487 ns	0.0357 ns	1.01
SumLong	PR #127429	4	0.6161 ns	0.0256 ns	1.00
SumLong	main	4	0.6725 ns	0.0258 ns	1.09
SumInt	PR #127429**	8	1.2147 ns	0.0332 ns	1.00
SumInt	main	8	1.1947 ns	0.0127 ns	0.98
SumLong	PR #127429	8	1.1806 ns	0.0372 ns	1.00
SumLong	main	8	1.2010 ns	0.0427 ns	1.02
SumInt	PR #127429**	16	2.8457 ns	0.0326 ns	1.00
SumInt	main	16	2.8423 ns	0.0332 ns	1.00
SumLong	PR #127429	16	10.9844 ns	0.1135 ns	1.00
SumLong	main	16	1.5386 ns	0.0216 ns	0.14
SumInt	PR #127429**	32	14.0039 ns	0.1165 ns	1.00
SumInt	main	32	2.2288 ns	0.0267 ns	0.16
SumLong	PR #127429	32	18.2713 ns	0.1813 ns	1.00
SumLong	main	32	2.8218 ns	0.0625 ns	0.15
SumInt	PR #127429**	50	20.9519 ns	0.2441 ns	1.00
SumInt	main	50	3.1326 ns	0.0330 ns	0.15
SumLong	PR #127429	50	28.5260 ns	0.2374 ns	1.00
SumLong	main	50	4.0064 ns	0.0636 ns	0.14
SumInt	PR #127429**	100	29.1359 ns	0.4969 ns	1.00
SumInt	main	100	4.8660 ns	0.0780 ns	0.17
SumLong	PR #127429	100	54.5658 ns	0.7260 ns	1.00
SumLong	main	100	8.4862 ns	0.1311 ns	0.16
SumInt	PR #127429**	256	73.0571 ns	1.4565 ns	1.00
SumInt	main	256	13.8034 ns	0.2230 ns	0.19
SumLong	PR #127429	256	132.5139 ns	1.8190 ns	1.00
SumLong	main	256	30.6605 ns	0.4837 ns	0.23
SumInt	PR #127429**	1024	274.5089 ns	2.6021 ns	1.00
SumInt	main	1024	69.1246 ns	0.9341 ns	0.25
SumLong	PR #127429	1024	544.9742 ns	6.4863 ns	1.00
SumLong	main	1024	143.5087 ns	2.7018 ns	0.26
SumInt	PR #127429**	4096	1,088.2754 ns	16.1779 ns	1.00
SumInt	main	4096	296.4498 ns	5.2377 ns	0.27
SumLong	PR #127429	4096	2,162.8552 ns	20.7651 ns	1.00
SumLong	main	4096	592.1511 ns	6.9984 ns	0.27
SumInt	PR #127429**	16384	4,655.0153 ns	41.5753 ns	1.00
SumInt	main	16384	1,207.5198 ns	17.4927 ns	0.26
SumLong	PR #127429	16384	10,036.4658 ns	126.6689 ns	1.00
SumLong	main	16384	2,392.2698 ns	36.8227 ns	0.24
SumInt	PR #127429**	65536	20,186.5271 ns	270.3878 ns	1.00
SumInt	main	65536	4,809.2647 ns	67.1581 ns	0.24
SumLong	PR #127429	65536	40,122.3099 ns	569.8787 ns	1.00
SumLong	main	65536	9,516.0036 ns	125.8782 ns	0.24
SumInt	PR #127429**	1000000	NA	NA	?
SumInt	main	1000000	NA	NA	?
SumLong	PR #127429	1000000	618,118.0905 ns	10,079.4181 ns	1.00
SumLong	main	1000000	146,920.6685 ns	2,000.9993 ns	0.24

Benchmarks with issues:
SumBench.SumInt: PR #127429) [N=1000000]
SumBench.SumInt: main) [N=1000000]

Full logs

[EgorBot]

Results for ubuntu24_azure_emeraldrapids

@EgorBo

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
INTEL XEON PLATINUM 8573C 2.30GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429**	4	1.953 ns	0.0391 ns	1.00
SumInt	main	4	1.949 ns	0.0248 ns	1.00
SumLong	PR #127429	4	1.285 ns	0.0244 ns	1.00
SumLong	main	4	1.289 ns	0.0306 ns	1.00
SumInt	PR #127429**	8	3.227 ns	0.0420 ns	1.00
SumInt	main	8	3.232 ns	0.0333 ns	1.00
SumLong	PR #127429	8	2.681 ns	0.0286 ns	1.00
SumLong	main	8	2.673 ns	0.0459 ns	1.00
SumInt	PR #127429**	16	5.385 ns	0.0630 ns	1.00
SumInt	main	16	5.436 ns	0.0807 ns	1.01
SumLong	PR #127429	16	18.456 ns	0.1142 ns	1.00
SumLong	main	16	2.526 ns	0.0293 ns	0.14
SumInt	PR #127429**	32	17.156 ns	0.1552 ns	1.00
SumInt	main	32	4.538 ns	0.0336 ns	0.26
SumLong	PR #127429	32	25.523 ns	0.1855 ns	1.00
SumLong	main	32	5.645 ns	0.0784 ns	0.22
SumInt	PR #127429**	50	27.883 ns	0.3486 ns	1.00
SumInt	main	50	6.090 ns	0.0921 ns	0.22
SumLong	PR #127429	50	40.931 ns	0.3294 ns	1.00
SumLong	main	50	7.316 ns	0.0758 ns	0.18
SumInt	PR #127429**	100	39.841 ns	0.3709 ns	1.00
SumInt	main	100	9.086 ns	0.0500 ns	0.23
SumLong	PR #127429	100	68.984 ns	0.8711 ns	1.00
SumLong	main	100	13.095 ns	0.1135 ns	0.19
SumInt	PR #127429**	256	88.150 ns	0.5943 ns	1.00
SumInt	main	256	21.212 ns	0.1712 ns	0.24
SumLong	PR #127429	256	194.122 ns	1.0822 ns	1.00
SumLong	main	256	37.982 ns	0.2611 ns	0.20
SumInt	PR #127429**	1024	323.758 ns	3.1758 ns	1.00
SumInt	main	1024	76.895 ns	0.4550 ns	0.24
SumLong	PR #127429	1024	646.931 ns	5.4417 ns	1.00
SumLong	main	1024	149.810 ns	1.1818 ns	0.23
SumInt	PR #127429**	4096	1,257.051 ns	9.0563 ns	1.00
SumInt	main	4096	298.793 ns	1.4998 ns	0.24
SumLong	PR #127429	4096	2,536.524 ns	21.4747 ns	1.00
SumLong	main	4096	596.975 ns	3.9749 ns	0.24
SumInt	PR #127429**	16384	6,038.728 ns	43.7861 ns	1.00
SumInt	main	16384	1,213.239 ns	4.5616 ns	0.20
SumLong	PR #127429	16384	10,220.611 ns	62.0150 ns	1.00
SumLong	main	16384	2,404.500 ns	17.4311 ns	0.24
SumInt	PR #127429**	65536	22,610.842 ns	168.1575 ns	1.00
SumInt	main	65536	4,812.452 ns	23.6548 ns	0.21
SumLong	PR #127429	65536	48,073.664 ns	246.5240 ns	1.00
SumLong	main	65536	9,517.292 ns	54.8580 ns	0.20
SumInt	PR #127429**	1000000	NA	NA	?
SumInt	main	1000000	NA	NA	?
SumLong	PR #127429	1000000	633,158.729 ns	5,919.1747 ns	1.00
SumLong	main	1000000	269,228.205 ns	531.4264 ns	0.43

Benchmarks with issues:
SumBench.SumInt: PR #127429) [N=1000000]
SumBench.SumInt: main) [N=1000000]

Full logs

[EgorBo]

@EgorBot -amd -aws_amd -profiler

using System.Linq;
using BenchmarkDotNet.Attributes;

public class SumBench
{
    // Cover edge cases (below Vector<T>.Count*4 dispatch threshold), main-loop boundaries
    // around Vector<T>.Count multiples on V128/V256/V512 hosts, and L1/L2/RAM-resident sizes.
    [Params(4096)]
    public int N;

    private int[] _ints = null!;
    private long[] _longs = null!;

    [GlobalSetup]
    public void Setup()
    {
        _ints = Enumerable.Range(0, N).ToArray();
        _longs = Enumerable.Range(0, N).Select(i => (long)i).ToArray();
    }

    [Benchmark]
    public int SumInt() => _ints.Sum();
}

[EgorBot]

Benchmark job submitted. The results will be posted here once they are ready.

• ubuntu24_azure_turin: live logs
• ubuntu24_aws_genoa: live logs

[EgorBot]

Results for ubuntu24_azure_turin

@EgorBo

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 9V45 2.60GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429	4096	1,148.3 ns	3.90 ns	1.00
SumInt	main	4096	307.8 ns	0.78 ns	0.27

Profiling artifacts

SumBench.SumInt:

	main	PR_127429
flamegraph	link	link
functions	link	link
asm	link	link
stats	link	link
speedscope	link	link

Full logs

[EgorBot]

Results for ubuntu24_aws_genoa

@EgorBo

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.3 LTS (Noble Numbat)
AMD EPYC 9R14 2.60GHz, 1 CPU, 8 logical and 8 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429	4096	1,717.4 ns	1.49 ns	1.00
SumInt	main	4096	244.9 ns	0.18 ns	0.14

Profiling artifacts

SumBench.SumInt:

	main	PR_127429
flamegraph	link	link
functions	link	link
asm	link	link
stats	link	link
speedscope	link	link

Full logs

[EgorBo]

@EgorBot -arm --envvars DOTNET_TieredCompilation:0 DOTNET_ReadyToRun:0

using System.Linq;
using BenchmarkDotNet.Attributes;

public class SumBench
{
    // Cover edge cases (below Vector<T>.Count*4 dispatch threshold), main-loop boundaries
    // around Vector<T>.Count multiples on V128/V256/V512 hosts, and L1/L2/RAM-resident sizes.
    [Params(4096)]
    public int N;

    private int[] _ints = null!;
    private long[] _longs = null!;

    [GlobalSetup]
    public void Setup()
    {
        _ints = Enumerable.Range(0, N).ToArray();
        _longs = Enumerable.Range(0, N).Select(i => (long)i).ToArray();
    }

    [Benchmark]
    public int SumInt() => _ints.Sum();
}

[EgorBot]

Benchmark job submitted. The results will be posted here once they are ready.

• macos15_helix_arm64: live logs

[EgorBo]

@EgorBot -arm --envvars DOTNET_TieredCompilation:0 DOTNET_ReadyToRun:0

using System.Linq;
using BenchmarkDotNet.Attributes;

public class SumBench
{
    // Cover edge cases (below Vector<T>.Count*4 dispatch threshold), main-loop boundaries
    // around Vector<T>.Count multiples on V128/V256/V512 hosts, and L1/L2/RAM-resident sizes.
    [Params(4096)]
    public int N;

    private int[] _ints = null!;
    private long[] _longs = null!;

    [GlobalSetup]
    public void Setup()
    {
        _ints = Enumerable.Range(0, N).ToArray();
        _longs = Enumerable.Range(0, N).Select(i => (long)i).ToArray();
    }

    [Benchmark]
    public int SumInt() => _ints.Sum();
}

[EgorBot]

Benchmark job submitted. The results will be posted here once they are ready.

• macos15_helix_arm64: live logs

[EgorBot]

Results for macos15_helix_arm64

@EgorBo

BenchmarkDotNet v0.15.8, macOS Sequoia 15.4.1 (24E263) [Darwin 24.4.0]
Apple M4, 1 CPU, 10 logical and 10 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), Arm64 RyuJIT armv8.0-a

EnvironmentVariables=DOTNET_TieredCompilation=0,DOTNET_ReadyToRun=0  

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429	4096	490.5 ns	1.45 ns	1.00
SumInt	main	4096	500.8 ns	1.13 ns	1.02

Full logs

[EgorBot]

Results for macos15_helix_arm64

@EgorBo

BenchmarkDotNet v0.15.8, macOS Sequoia 15.4.1 (24E263) [Darwin 24.4.0]
Apple M2, 1 CPU, 8 logical and 8 physical cores
.NET SDK 11.0.100-preview.5.26224.123
  [Host]     : .NET 10.0.7 (10.0.7, 10.0.726.21808), Arm64 RyuJIT armv8.0-a

EnvironmentVariables=DOTNET_TieredCompilation=0,DOTNET_ReadyToRun=0  

Method	Toolchain	N	Mean	Error	Ratio
SumInt	PR #127429	4096	720.8 ns	0.54 ns	1.00
SumInt	main	4096	743.5 ns	0.42 ns	1.03

Full logs

[EgorBo]

@EgorBot -amd -intel -arm

using System.Linq;
using BenchmarkDotNet.Attributes;

public class SumBench
{
    // Cover edge cases (below Vector<T>.Count*4 dispatch threshold), main-loop boundaries
    // around Vector<T>.Count multiples on V128/V256/V512 hosts, and L1/L2/RAM-resident sizes.
    [Params(4, 8, 16, 32, 50, 100, 256, 1024, 4096, 16384, 65536, 1_000_000)]
    public int N;

    private int[] _ints = null!;
    private long[] _longs = null!;

    [GlobalSetup]
    public void Setup()
    {
        _ints = Enumerable.Range(0, N).ToArray();
        _longs = Enumerable.Range(0, N).Select(i => (long)i).ToArray();
    }

    [Benchmark]
    public int SumInt() => _ints.Sum();

    [Benchmark]
    public long SumLong() => _longs.Sum();
}

[EgorBot]

Benchmark job submitted. The results will be posted here once they are ready.

• ubuntu24_azure_turin: live logs
• ubuntu24_azure_emeraldrapids: live logs
• macos15_helix_arm64: live logs