System.Collections.Sort<BigStruct>.LinqQuery has regressed on all configs except Windows 64 bit

[adamsitnik]

This regression seems to be specific to all configs except of Windows 64 bit.

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter 'System.Collections.Sort<BigStruct>.LinqQuery'

Ubuntu Historical results

The diff points to #55604 (cc @alexcovington) and #59287 (cc @AndyAyersMS)

Windows Historical results

Details

Result	Base	Diff	Ratio	Operating System	Bit
Same	47068.40	46996.12	1.00	Windows 11	X64
Same	25061.00	25213.92	0.99	Windows 11	X64
Same	81332.07	82470.68	0.99	Windows 11	X64
Same	48471.02	49394.98	0.98	Windows 10	X64
Same	61753.97	65909.26	0.94	Windows 11	X64
Same	79322.94	78292.41	1.01	Windows 11	X64
Slower	33152.41	48551.85	0.68	ubuntu 18.04	X64
Slower	33670.35	49233.18	0.68	ubuntu 20.04	X64
Slower	65475.08	84542.75	0.77	ubuntu 18.04	X64
Same	102906.71	95691.63	1.08	ubuntu 18.04	X64
Slower	78941.99	99516.66	0.79	pop 20.04	X64
Slower	58025.14	76420.12	0.76	alpine 3.13	X64
Slower	58358.38	87952.92	0.66	debian 11	X64
Same	39738.00	38447.43	1.03	macOS Monterey 12.2.1	Arm64
Same	81077.12	83539.94	0.97	Windows 10	Arm64
Same	84261.45	85918.34	0.98	Windows 11	Arm64
Slower	51385.76	75022.36	0.68	Windows 11	X86
Slower	68915.32	91940.60	0.75	Windows 10	X86
Slower	61701.11	79972.87	0.77	Windows 10	X86
Slower	57559.08	70356.86	0.82	Windows 10	X86
Same	151162.22	145951.59	1.04	Windows 10	Arm
Slower	90819.89	108997.55	0.83	macOS Big Sur 11.6.3	X64
Slower	73211.06	98121.94	0.75	macOS Monterey 12.2.1	X64
Slower	79186.88	106613.19	0.74	macOS Monterey 12.2.1	X64

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This regression seems to be specific to all configs except of Windows 64 bit.

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter 'System.Collections.Sort<BigStruct>.LinqQuery'

Ubuntu Historical results

The diff points to #55604 (cc @alexcovington) and #59287 (cc @AndyAyersMS)

Windows Historical results

Details

Result	Base	Diff	Ratio	Operating System	Bit
Same	47068.40	46996.12	1.00	Windows 11	X64
Same	25061.00	25213.92	0.99	Windows 11	X64
Same	81332.07	82470.68	0.99	Windows 11	X64
Same	48471.02	49394.98	0.98	Windows 10	X64
Same	61753.97	65909.26	0.94	Windows 11	X64
Same	79322.94	78292.41	1.01	Windows 11	X64
Slower	33152.41	48551.85	0.68	ubuntu 18.04	X64
Slower	33670.35	49233.18	0.68	ubuntu 20.04	X64
Slower	65475.08	84542.75	0.77	ubuntu 18.04	X64
Same	102906.71	95691.63	1.08	ubuntu 18.04	X64
Slower	78941.99	99516.66	0.79	pop 20.04	X64
Slower	58025.14	76420.12	0.76	alpine 3.13	X64
Slower	58358.38	87952.92	0.66	debian 11	X64
Same	39738.00	38447.43	1.03	macOS Monterey 12.2.1	Arm64
Same	81077.12	83539.94	0.97	Windows 10	Arm64
Same	84261.45	85918.34	0.98	Windows 11	Arm64
Slower	51385.76	75022.36	0.68	Windows 11	X86
Slower	68915.32	91940.60	0.75	Windows 10	X86
Slower	61701.11	79972.87	0.77	Windows 10	X86
Slower	57559.08	70356.86	0.82	Windows 10	X86
Same	151162.22	145951.59	1.04	Windows 10	Arm
Slower	90819.89	108997.55	0.83	macOS Big Sur 11.6.3	X64
Slower	73211.06	98121.94	0.75	macOS Monterey 12.2.1	X64
Slower	79186.88	106613.19	0.74	macOS Monterey 12.2.1	X64

Author:	adamsitnik
Assignees:	-
Labels:	area-System.Collections, tenet-performance
Milestone:	-

[AndyAyersMS]

#59287 is locked so doesn't get cross linked. That seems unfortunate.

That change should have purely impacted jit diagnostics, so it's unlikely to have caused regressions.

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This regression seems to be specific to all configs except of Windows 64 bit.

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter 'System.Collections.Sort<BigStruct>.LinqQuery'

Ubuntu Historical results

The diff points to #55604 (cc @alexcovington) and #59287 (cc @AndyAyersMS)

Windows Historical results

Details

Result	Base	Diff	Ratio	Operating System	Bit
Same	47068.40	46996.12	1.00	Windows 11	X64
Same	25061.00	25213.92	0.99	Windows 11	X64
Same	81332.07	82470.68	0.99	Windows 11	X64
Same	48471.02	49394.98	0.98	Windows 10	X64
Same	61753.97	65909.26	0.94	Windows 11	X64
Same	79322.94	78292.41	1.01	Windows 11	X64
Slower	33152.41	48551.85	0.68	ubuntu 18.04	X64
Slower	33670.35	49233.18	0.68	ubuntu 20.04	X64
Slower	65475.08	84542.75	0.77	ubuntu 18.04	X64
Same	102906.71	95691.63	1.08	ubuntu 18.04	X64
Slower	78941.99	99516.66	0.79	pop 20.04	X64
Slower	58025.14	76420.12	0.76	alpine 3.13	X64
Slower	58358.38	87952.92	0.66	debian 11	X64
Same	39738.00	38447.43	1.03	macOS Monterey 12.2.1	Arm64
Same	81077.12	83539.94	0.97	Windows 10	Arm64
Same	84261.45	85918.34	0.98	Windows 11	Arm64
Slower	51385.76	75022.36	0.68	Windows 11	X86
Slower	68915.32	91940.60	0.75	Windows 10	X86
Slower	61701.11	79972.87	0.77	Windows 10	X86
Slower	57559.08	70356.86	0.82	Windows 10	X86
Same	151162.22	145951.59	1.04	Windows 10	Arm
Slower	90819.89	108997.55	0.83	macOS Big Sur 11.6.3	X64
Slower	73211.06	98121.94	0.75	macOS Monterey 12.2.1	X64
Slower	79186.88	106613.19	0.74	macOS Monterey 12.2.1	X64

Author:	adamsitnik
Assignees:	-
Labels:	tenet-performance, area-CodeGen-coreclr
Milestone:	7.0.0

[AndyAyersMS]

Digging through it looks like we expected this to be resolved -- see dotnet/perf-autofiling-issues#1501 (comment)

But that only fixed issues on Windows, Ubuntu did not benefit. So we still have a regression.

(Windows is slightly worse off too)

[AndyAyersMS]

Looks like this is still unassigned. I'll take it for now.

[AndyAyersMS]

Can reproduce running locally (via wsl2)

BenchmarkDotNet=v0.13.1.1823-nightly, OS=ubuntu 20.04
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK=7.0.100-rc.1.22408.1
  [Host]     : .NET 7.0.0 (7.0.22.40308), X64 RyuJIT
  Job-CFAJOE : .NET 5.0.1 (5.0.120.57516), X64 RyuJIT
  Job-JPHJBC : .NET 6.0.7 (6.0.722.32202), X64 RyuJIT
  Job-KPSCOL : .NET 7.0.0 (7.0.22.40308), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  InvocationCount=5000  IterationTime=250.0000 ms
MaxIterationCount=20  MinIterationCount=15  MinWarmupIterationCount=6
UnrollFactor=1  WarmupCount=-1

Method	Job	Runtime	Toolchain	Size	Mean	Error	StdDev	Median	Min	Max	Ratio	RatioSD	Gen 0	Gen 1	Allocated	Alloc Ratio
LinqQuery	Job-CFAJOE	.NET 5.0	net5.0	512	56.80 us	0.729 us	0.682 us	56.79 us	55.69 us	58.34 us	1.00	0.00	5.4000	0.4000	34.33 KB	1.00
LinqQuery	Job-JPHJBC	.NET 6.0	net6.0	512	58.25 us	0.707 us	0.662 us	57.97 us	57.41 us	59.44 us	1.03	0.02	5.4000	0.4000	34.33 KB	1.00
LinqQuery	Job-KPSCOL	.NET 7.0	net7.0	512	72.44 us	1.321 us	1.235 us	72.28 us	70.45 us	74.83 us	1.28	0.03	5.6000	0.6000	34.33 KB	1.00

[AndyAyersMS]

@adamsitnik is it expected that with -p EP I won't get cpu sample events? If so, any way to enable these via the command line?

[AndyAyersMS]

Hmm, I guess there are sample events but not ones that perfview recognizes?