Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

Optimize Enumerable.Range(...).Select(...)#37410

Merged
stephentoub merged 2 commits intodotnet:masterfrom
stephentoub:linqopt
May 6, 2019
Merged

Optimize Enumerable.Range(...).Select(...)#37410
stephentoub merged 2 commits intodotnet:masterfrom
stephentoub:linqopt

Conversation

@stephentoub
Copy link
Copy Markdown
Member

Looking at some large code indexes, by far the most common uses of Enumerable.Range are when it's either directly iterated with a foreach, converted to an array with ToArray, or probably the most common, succeeded by Select (and then iterated or ToArray'd). The first two are already decently optimized, with a custom range iterator returned from Enumerable.Range and that iterator already having a ToArray override. But it's missing the Select case. This commit adds that, by copying the existing SelectArrayIterator and then just tweaking it to use _start/_end rather than accessing a stored _source array.

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Collections.Generic;
using System.Linq;

[InProcess]
[MemoryDiagnoser]
public class Test
{
    public static void Main() => BenchmarkRunner.Run<Test>();

    [Params(1, 10, 100, 1000)]
    public int Size { get; set; }

    [Benchmark] public void RangeSelectIterate() { foreach (int item in Enumerable.Range(0, Size).Select(i => i)) { } }
    [Benchmark] public int[] RangeSelectToArray() => Enumerable.Range(0, Size).Select(i => i).ToArray();
    [Benchmark] public List<int> RangeSelectToList() => Enumerable.Range(0, Size).Select(i => i).ToList();
}
Method Size Mean Before Mean After Memory Before Memory After
RangeSelectIterate 1 83.65 ns 57.00 ns 96 B 88 B
RangeSelectToArray 1 72.15 ns 50.64 ns 128 B 120 B
RangeSelectToList 1 77.64 ns 58.61 ns 160 B 152 B
RangeSelectIterate 10 190.89 ns 123.68 ns 96 B 88 B
RangeSelectToArray 10 138.11 ns 71.17 ns 160 B 152 B
RangeSelectToList 10 151.71 ns 84.80 ns 192 B 184 B
RangeSelectIterate 100 1,136.12 ns 689.90 ns 96 B 88 B
RangeSelectToArray 100 741.59 ns 279.15 ns 520 B 512 B
RangeSelectToList 100 727.97 ns 336.18 ns 552 B 544 B
RangeSelectIterate 1000 10,603.56 ns 6,362.80 ns 96 B 88 B
RangeSelectToArray 1000 6,729.38 ns 2,371.04 ns 4120 B 4112 B
RangeSelectToList 1000 6,665.97 ns 2,799.10 ns 4152 B 4144 B

cc: @maryamariyan, @cston, @danmosemsft

Looking at some large code indexes, by far the most common uses of Enumerable.Range are when it's either directly iterated with a foreach, converted to an array with ToArray, or probably the most common, succeeded by Select.  The first two are already decently optimized, with a custom range iterator returned from Enumerable.Range and that iterator already having a ToArray override.  But it's missing the Select case.  This commit adds that, by copying the existing SelectArrayIterator and then just tweaking it to use _start/_end rather than accessing a stored _source array.
@ahsonkhan
Copy link
Copy Markdown

[Benchmark] public void RangeSelectIterate()

Add these to dotnet/performance?

cc @adamsitnik, @billwert

Copy link
Copy Markdown

@ahsonkhan ahsonkhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits/questions, but looks like you are following the existing code patterns used in other iterators.

Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Copy link
Copy Markdown
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I left some questions to get a better understanding.

Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
Comment thread src/System.Linq/src/System/Linq/Select.SpeedOpt.cs
@stephentoub
Copy link
Copy Markdown
Member Author

Thanks for reviewing, @ahsonkhan and @adamsitnik.

@stephentoub stephentoub merged commit 23f320b into dotnet:master May 6, 2019
@stephentoub stephentoub deleted the linqopt branch May 6, 2019 14:14
@timandy
Copy link
Copy Markdown
Contributor

timandy commented May 12, 2019

Does 'repeat.select' should be optimized also?

@stephentoub
Copy link
Copy Markdown
Member Author

I found very little usage of Repeat in general outside of test code, and very little of that used Select. What's the scenario where it's important that be a fast as possible?

public SelectRangeIterator(int start, int end, Func<int, TResult> selector)
{
Debug.Assert(start < end);
Debug.Assert((end - start) <= int.MaxValue);
Copy link
Copy Markdown
Contributor

@timandy timandy May 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This expression is always true,even overflow.

Does it should be?

Debug.Assert(unchecked((uint)(end - start) <= (uint)int.MaxValue));

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, it's missing the casts. We'd welcome a PR if you'd like to fix it. Thanks.

timandy added a commit to timandy/corefx that referenced this pull request May 17, 2019
@karelz karelz added this to the 3.0 milestone May 22, 2019
picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022
* Optimize Enumerable.Range(...).Select(...)

Looking at some large code indexes, by far the most common uses of Enumerable.Range are when it's either directly iterated with a foreach, converted to an array with ToArray, or probably the most common, succeeded by Select.  The first two are already decently optimized, with a custom range iterator returned from Enumerable.Range and that iterator already having a ToArray override.  But it's missing the Select case.  This commit adds that, by copying the existing SelectArrayIterator and then just tweaking it to use _start/_end rather than accessing a stored _source array.

* Add more Debug.Asserts


Commit migrated from dotnet/corefx@23f320b
picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants