Optimizations on System.Linq.Enumerable.ToArray

[manofstick]

3rd of n optimizations lifted from my exploration of an alternative System.Linq implementation Cistern.Linq. There are lots more goodies there, but only so many that will survive the transfer to System.Linq!

(Previous corefx optimization)

This is a bit more controversial, which although I think it is reasonable, may get kicked to the curb depending on you POV. So why do I think this is is controversial? Well it uses a Clayton's stackalloc (i.e. recursion) to store the first 64 elements (sounded like a reasonable number, but obviously up for discussion) to avoid temporary allocation buffers (can't use a "real" stackalloc due to not meeting the generic type constraints).

It then finishes off the process using recursion too. What this means is that you have got some deeper stack accesses than previously, but I feel that they are not unreasonable. So how much stack are we using?

So the maximum function depth is 16 + 25 = 41

max array size ~ 2^31

approx stack usage = 16*(4sizeof(T)+sizeof(T[])+sizeof(IEnumerable<>)+sizeof(int)+sizeof(Func<>)+sizeof(int)+sizeof(this)+function overhead) + (log2(2^31/64))(sizeof(object)+sizeof(ref int)+sizeof(Func<>)+sizeof(count)+function overhead).

which, for a sizeof(T) = 8 is ~ 2K

Anyway, so if that is unreasonable, then you can stop here and kill this PR. Otherwise, read on...

So I added some ToArray performance tests, but haven't pushed them to the Performance repository (as they are a bit excessive for what they are doing there), but you can view them here.

The tests are modifying 3 axes... Length, the type of predicate used in Where and the source type (i.e. a generator function, an array or a list). What you are not seeing here is the smaller amount of allocation that are occuring due to avoiding temporary buffers for small arrays.

Anyway, I'll add some more details - some memory usage, etc. (and I realised that I haven't actually included the case of the Generator without a Where) if this is actually going to be considered to be pulled into master

(oh, and the plan would also be to extend this to WhereSelect as well, but TBD...)

Slower	diff/base	Base Median (ns)	Diff Median (ns)	Modality
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: FirstHalf, Enum: Array)	1.15	15.34	17.64	several?
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: FirstHalf, Enum: Array)	1.08	32662.74	35416.31
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: Interleaved, Enum: Array)	1.05	16.57	17.32
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: False, Enum: Array)	1.04	1376.24	1429.36

Faster	base/diff	Base Median (ns)	Diff Median (ns)	Modality
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: True, Enum: Array)	2.33	182.54	78.49
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: True, Enum: List)	2.29	182.99	79.78
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: True, Enum: List)	2.29	282.44	123.43
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: FirstHalf, Enum: List)	2.27	101.11	44.63
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: FirstHalf, Enum: Array)	2.18	158.58	72.82
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: Interleaved, Enum: List)	2.18	103.74	47.68
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: FirstHalf, Enum: List)	2.16	104.79	48.56
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: True, Enum: Array)	2.15	265.49	123.35
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: FirstHalf, Enum: Array)	2.14	98.21	45.87
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: Interleaved, Enum: List)	2.12	106.41	50.29
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: FirstHalf, Enum: List)	2.08	154.72	74.26
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: True, Enum: Array)	2.06	114.62	55.68
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: Interleaved, Enum: Array)	2.04	112.34	55.01
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: FirstHalf, Enum: Array)	2.02	103.16	51.06	several?
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: Interleaved, Enum: List)	2.02	161.18	79.86
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: FirstHalf, Enum: List)	2.02	115.88	57.49
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: Interleaved, Enum: Array)	2.01	101.01	50.32
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: True, Enum: List)	1.99	118.19	59.34
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: Interleaved, Enum: List)	1.96	114.94	58.51
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: Interleaved, Enum: Array)	1.96	104.32	53.35
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: FirstHalf, Enum: Array)	1.95	112.09	57.41
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: True, Enum: Array)	1.94	79.82	41.24	several?
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: Interleaved, Enum: Generator)	1.91	130.29	68.07
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: True, Enum: List)	1.90	425.07	224.20
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: True, Enum: Generator)	1.84	121.67	66.02
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: True, Enum: Array)	1.83	76.45	41.66
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: Interleaved, Enum: List)	1.82	238.43	131.28
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: Interleaved, Enum: Array)	1.81	149.81	82.90
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: FirstHalf, Enum: List)	1.80	370.30	205.91
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: Interleaved, Enum: Generator)	1.79	145.91	81.32
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: True, Enum: List)	1.79	76.82	42.82
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: FirstHalf, Enum: Array)	1.79	225.20	125.55
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: Interleaved, Enum: List)	1.79	77.34	43.17
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: True, Enum: List)	1.79	81.51	45.59
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: True, Enum: Array)	1.78	83.51	46.84	several?
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: FirstHalf, Enum: Array)	1.78	344.55	193.87
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: True, Enum: Generator)	1.77	188.60	106.76
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: True, Enum: List)	1.76	85.55	48.71
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: Interleaved, Enum: Array)	1.75	217.37	124.39
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: True, Enum: Generator)	1.75	133.35	76.33
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: Interleaved, Enum: Generator)	1.75	128.81	73.80
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: FirstHalf, Enum: List)	1.74	221.55	127.22
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: Interleaved, Enum: List)	1.72	393.70	228.39
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: Interleaved, Enum: Array)	1.72	75.78	44.16
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: FirstHalf, Enum: Generator)	1.70	143.33	84.53
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: True, Enum: Array)	1.69	367.21	217.87
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: True, Enum: Generator)	1.68	462.87	275.08
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: True, Enum: Generator)	1.68	139.65	83.00
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: Interleaved, Enum: Array)	1.66	355.92	214.32
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: Interleaved, Enum: List)	1.64	643.61	393.09
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: FirstHalf, Enum: Generator)	1.62	130.37	80.43	bimodal
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: FirstHalf, Enum: Array)	1.59	549.81	346.58
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: FirstHalf, Enum: Generator)	1.58	164.40	104.17
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: True, Enum: List)	1.57	694.40	442.13
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: Interleaved, Enum: Generator)	1.56	246.04	157.64
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: False, Enum: Array)	1.55	57.20	36.89
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: False, Enum: Array)	1.54	59.82	38.74
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: Interleaved, Enum: Generator)	1.54	734.28	477.95
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: Interleaved, Enum: Generator)	1.53	166.31	108.52
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: Interleaved, Enum: Generator)	1.53	405.54	264.73
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: FirstHalf, Enum: List)	1.53	576.73	376.60
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: Interleaved, Enum: Array)	1.53	567.35	371.47
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: True, Enum: Generator)	1.52	757.28	496.94
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: True, Enum: Generator)	1.52	265.65	175.29
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: True, Enum: List)	1.51	51.22	33.88
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: False, Enum: List)	1.51	51.06	33.87
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: Interleaved, Enum: List)	1.50	51.32	34.11
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: True, Enum: Array)	1.49	622.51	417.38
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: False, Enum: List)	1.49	65.20	43.74
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: FirstHalf, Enum: Generator)	1.49	235.21	157.86
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: False, Enum: Generator)	1.49	80.73	54.25
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: FirstHalf, Enum: Generator)	1.48	77.13	52.17
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: FirstHalf, Enum: Array)	1.48	58.28	39.46
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: FirstHalf, Enum: List)	1.47	51.81	35.15
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: False, Enum: List)	1.45	58.91	40.73
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: FirstHalf, Enum: Generator)	1.40	668.98	477.35
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: Interleaved, Enum: Generator	1.39	1249.75	897.68
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: True, Enum: Generator)	1.38	73.75	53.36
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: FirstHalf, Enum: Generator)	1.38	80.73	58.46
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: False, Enum: List)	1.38	52.82	38.32
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: FirstHalf, Enum: List)	1.36	54.37	40.03
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: True, Enum: List)	1.36	2421.67	1783.85
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: Interleaved, Enum: List)	1.36	45226.34	33325.68
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: False, Enum: Array)	1.35	66.03	48.81
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: FirstHalf, Enum: Generator)	1.35	372.16	275.16
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: False, Enum: List)	1.35	71.74	53.25
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: Interleaved, Enum: List)	1.34	2370.59	1765.25
System.Linq.Tests.Perf_ToArray.Linq(Len: 1, Pred: False, Enum: Generator)	1.33	80.45	60.29
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: True, Enum: Generator)	1.33	1271.30	955.42
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: True, Enum: List)	1.32	4562.01	3455.84
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: False, Enum: List)	1.30	92.53	71.24
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: Interleaved, Enum: Generator)	1.30	77.81	60.04
System.Linq.Tests.Perf_ToArray.Linq(Len: 2, Pred: False, Enum: Generator)	1.27	87.50	68.98
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: False, Enum: Generator)	1.25	95.61	76.75
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: False, Enum: List)	1.24	148.23	119.17
System.Linq.Tests.Perf_ToArray.Linq(Len: 3, Pred: False, Enum: Array)	1.24	58.82	47.30
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: Interleaved, Enum: List)	1.24	4533.83	3666.57
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: True, Enum: Generator)	1.23	5217.96	4236.11
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: FirstHalf, Enum: List)	1.23	1965.41	1597.56
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: FirstHalf, Enum: Generator)	1.23	1090.96	890.22
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: FirstHalf, Enum: List)	1.23	3761.06	3069.51
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: True, Enum: List)	1.22	41794.84	34157.86
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: False, Enum: List)	1.21	416.73	344.04
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: False, Enum: List)	1.21	241.05	199.08
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: True, Enum: Generator)	1.20	10135.89	8451.64
System.Linq.Tests.Perf_ToArray.Linq(Len: 6, Pred: False, Enum: Generator)	1.20	120.48	100.47
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: True, Enum: Array)	1.20	1956.78	1636.89
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: FirstHalf, Enum: List)	1.18	35312.14	29887.81
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: True, Enum: Generator)	1.18	97575.66	83007.15
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: False, Enum: Generator)	1.18	179.29	152.55
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: True, Enum: Array)	1.17	16.59	14.13
System.Linq.Tests.Perf_ToArray.Linq(Len: 12, Pred: False, Enum: Array)	1.17	80.09	68.46
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: Interleaved, Enum: Generator	1.17	5185.25	4449.25
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: False, Enum: List)	1.16	3619.78	3123.49
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: False, Enum: List)	1.16	1833.52	1585.03
System.Linq.Tests.Perf_ToArray.Linq(Len: 0, Pred: False, Enum: Array)	1.15	19.89	17.23
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: False, Enum: Generator)	1.14	286.57	251.01
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: True, Enum: Array)	1.13	3616.96	3209.43
System.Linq.Tests.Perf_ToArray.Linq(Len: 25, Pred: False, Enum: Array)	1.12	127.86	113.83
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: False, Enum: List)	1.11	34534.83	30994.17
System.Linq.Tests.Perf_ToArray.Linq(Len: 51, Pred: False, Enum: Generator)	1.11	504.09	452.85
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: Interleaved, Enum: Array)	1.09	37439.77	34274.85
System.Linq.Tests.Perf_ToArray.Linq(Len: 100, Pred: False, Enum: Generator)	1.09	902.58	828.94
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: Interleaved, Enum: Generato	1.08	10062.68	9281.56	multimodal
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: Interleaved, Enum: Generat	1.08	93561.71	86407.35
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: FirstHalf, Enum: Array)	1.07	1831.45	1707.05
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: False, Enum: Generator)	1.07	4190.69	3919.56
System.Linq.Tests.Perf_ToArray.Linq(Len: 10000, Pred: False, Enum: Generator)	1.07	81893.55	76627.60
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: False, Enum: Generator)	1.06	8239.05	7752.35
System.Linq.Tests.Perf_ToArray.Linq(Len: 1000, Pred: Interleaved, Enum: Array)	1.05	3654.49	3472.22
System.Linq.Tests.Perf_ToArray.Linq(Len: 500, Pred: FirstHalf, Enum: Generator)	1.03	4318.80	4174.29

[manofstick]

...and some memory usage...

Method	Len	Pred	Enum	Allocated (this PR)	Allocated (original)	ratio
Linq	0	True	Array	-	-
Linq	1	True	Array	80 B	80 B	1.0
Linq	3	True	Array	88 B	88 B	1.0
Linq	9	True	Array	112 B	264 B	0.42
Linq	27	True	Array	184 B	464 B	0.40
Linq	81	True	Array	680 B	944 B	0.72
Linq	243	True	Array	1864 B	2264 B	0.82

[stephentoub]

Thanks, @manofstick.

I've not looked at the changes yet, but I am concerned by:

So why do I think this is is controversial? Well it uses a Clayton's stackalloc (i.e. recursion) to store the first 64 elements

This is effectively saying the code is doing stackalloc T[64] (even though you can't actually write that, which is why you're employing recursion). I don't think we'd ever choose to do that in a library like this, because we have no idea what T is or how large it is. It could be a very large struct, in which case this is a recipe for a stack overflow that kills the process, all because Enumerable.ToArray was used.

[manofstick]

@stephentoub

This is effectively saying the code is doing stackalloc T[64] (even though you can't actually write that, which is why you're employing recursion).

Yes, but worse (due to other overheads)! :-)

I don't think we'd ever choose to do that in a library like this, because we have no idea what T is or how large it is. It could be a very large struct, in which case this is a recipe for a stack overflow that kills the process, all because Enumerable.ToArray was used.

Given that we have a 1Mb stack, if were from the root then that would be structs of ~16K. Pretty big. OK, can't expect that we'd be from the root, so 1/2 Mb - already a hugely deep call stack, then we ~8 K? I think that's a stretch too... But obviosuly you have to be conservative in a library. I understand, I'm not trying to argue too much :-), and obviously you're call is the law, but is there any size that we can consider as reasonable?

[manofstick]

@stephentoub

OK, how's this for an idea...

First of all I can do (array|list).Where.ToArray() without storing T, as I can just store indexes. So these should work all fine.

Second I can do quick value type/ref type check (i.e default is null) so I can the recurse with ref types, which should be all fine.

Third for other types I just go through my recursive store only once - i.e. storing only the first 4 values. Now this is safe (ish) as the largest value type that can be used in an array is 128k (the Type Loader throws an exception for bigger sizes from my testing....), so at most I consume - in the exceedingly unlikely case - only half the stack. But really I can't see anyone realistically created Value Types anywhere near this, and if they did, they are likely blowing the stack elsewhere, or have increased it's size anyway...

Anyway, if you still think this is all just crazy talk, do feel free to shoot me down. I've got a pretty thick skin :-)

[stephentoub]

Thanks. I want to take an in-depth look at it, I just haven't had the time yet. I will :)

[stephentoub]

@manofstick, apologies for the long delay in getting back to this.

I spent some time looking at the change. If I can summarize what you're trying to do here, it's to cater to small ToArray inputs by unrolling the loop and using stack allocation for the temporary values before they're stored into the small result array, yes?

It's an interesting approach. I just don't think it's going to move the needle sufficiently to warrant its weight (it also drops some existing optimizations for things like e.g. Enumerable.Range(...).ToArray), especially once the recursion / stackoverflow concerns are addressed and the limit drops to four entries as you suggested. Using an approach like @jkotas did in #32365 (comment), I took a look at the overhead this incurs just in generic instantations, and for a simple case, it was increasing working set by ~2.5K per generic instantiation; even for the "speed optimized" build, that's hefty for such a commonly-used method as ToArray.

FWIW, I actually think we should be looking to go in the other direction and simplify the implementation of ToArray we already have. We accepted the contributions to get it to where it is, but all of the builder stuff that's trying to improve memory usage and throughput is adding a lot of complexity and also itself significantly increasing working set. I just tried replacing the three lines at

runtime/src/libraries/Common/src/System/Collections/Generic/EnumerableHelpers.Linq.cs

Lines 110 to 112 in 0bf721a

	var builder = new LargeArrayBuilder<T>(initialize: true);
	builder.AddRange(source);
	return builder.ToArray();

with a super simple "enumerate the source, store the items into a growing array, and return a resized array result" implementation, and it reduced working set by more than this PR is increasing it, appx 4.5K per generic instantiation!

I'll defer to @eiriktsarpalis and @adamsitnik, but I think we should close this PR, but keep some of your ideas in mind as we look at ways to simplify what's currently there; maybe we can have our cake and eat it, too. At the moment, though, I don't think this yields the right trade-offs.

Thanks!

[manofstick]

@stephentoub

Thanks for taking the time to peruse. This one always was a bit of a long-shot, but I'm glad I threw it out there and hopefully it will seed some ideas for you.

I'll close it off.