[WIP] [no merge] Support limiting the buffer size in LargeArrayBuilder.

[jamesqo]

There are certain Linq methods, such as Where and Take, where we cannot pinpoint the exact number of elements the iterator will contain, but we can establish an upper bound on how many elements there are. For example, list.Where(p) can have at most list.Count elements, and e.Take(5) can have at most 5 elements.

Currently, when we call ToArray on any of these iterators, the resizing pattern will be to allocate 4, then 8, then 16, etc. size buffers. This can be wasteful because e.g. if the source enumerable in Where contains 100 elements, we allocate space for 128 elements even though those last 28 slots will never be used.

This PR adds support to LargeArrayBuilder<T> to limit how many elements we allocate, so there is no extra space wasted. It should help drive down allocations in Where, Where.Select, & Take when the maximum count is far from the next power of 2.

Waiting on: #12703, #13628

[svick]

Since the old Add(T) was marked with AggressiveInlining, should this new version be marked too?

[jamesqo]

I think I found in my experiments a while ago that the JIT inlines methods sequentially; it will recognize the body of Add(T) is small, then inline it. Then it will recognize the body of Add(T, int) is small, and inline it. Consequently, you only need to apply AggressiveInlining once.

[svick]

Why is limit passed on every call to Add? Is it expected to change? If not, shouldn't it be passed in the constructor and then stored in a field? Or is the cost of one more field too high?

[jamesqo]

Well, the size of the struct is already 40 bytes. I didn't want to penalize other cases where we're not using this parameter, e.g. Select or Concat. No, it's not expected to change,

[stephentoub]

Well, the size of the struct is already 40 bytes

Why does the size of this struct matter? In all usage I can remember, it's only used on the stack, it's not passed around, etc., in which case the difference between 40 and 44 bytes doesn't matter.

[jamesqo]

@stephentoub Future-proofing. I have been thinking of using it in Reverse, for example, in which case it would need to be stored in a field.

[stephentoub]

I have been thinking of using it in Reverse, for example

Where would it need to be stored in a field? If you mean using it instead of using Buffer and storing the array, you're likely going to run into other complications.

Future-proofing

While it's good to think about future usage, from my perspective this change makes existing code harder to understand, unnecessarily. We should only do so if/when there's known good reason to do so.

[stephentoub]

This PR adds support to LargeArrayBuilder to limit how many elements we allocate

Let's only add such support at the same time such support is consumed, so that we can actually see what real impact it has, and determine whether it's really worth the added complexity.

[jamesqo]

@stephentoub Sure. Will close for now until the the other PRs are merged