Benchmarks for ReadOnlySequence.Slice. by pgovind · Pull Request #589 · dotnet/performance

pgovind · 2019-06-26T01:11:08Z

Contains hacks to get it working in VS

NOT to be merged yet. I'm just putting this up for dotnet/corefx#38431

Contains hacks to get it working in VS

ahsonkhan · 2019-06-26T03:41:57Z

Contains hacks to get it working in VS

cc @adamsitnik, @billwert - what can we do to make the experience within the perf repo better for testing latest?

ahsonkhan · 2019-06-26T03:49:39Z

+        public ReadOnlySequence<byte> MS_RepeatSlice_StartPosition_And_EndPosition()
+        {
+            var localSequence = _multiSegmentSequence.Slice(_ms_start, _ms_end);
+            localSequence = localSequence.Slice(_ms_start, localSequence.End);


It's probably better to chose different start/end positions for slicing rather than reusing the same one, but it probably doesn't matter.

ahsonkhan · 2019-06-26T03:50:44Z

+
+        // Add multiple calls for Slice(startPosition, endPosition)
+        [Benchmark]
+        public ReadOnlySequence<byte> MS_RepeatSlice_StartPosition_And_EndPosition()


Can you share the result of this benchmark (along with some of the other missing ones)? I only saw a few mentioned in dotnet/corefx#38431 (comment).

pgovind · 2019-06-26T21:42:03Z

So, after addressing @ahsonkhan's comments, I see this when I run the benchmarks.

C:\Users\prgovi\Desktop\Work\performance\src\tools\ResultsComparer>dotnet run --base "C:\Users\prgovi\Desktop\Work\Benchmarks\Before" --diff "C:\Users\prgovi\Desktop\Work\Benchmarks\After" --threshold 2%
summary:
better: 5, geomean: 1.057
worse: 3, geomean: 1.152
total diff: 8

Slower	diff/base	Base Median (ns)	Diff Median (ns)
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.MS_StartPosition	1.24	8.44	10.47
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.StartPosition	1.17	4.97	5.79
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.MS_StartPosition_And_EndPositi	1.06	11.02	11.65

Faster	base/diff	Base Median (ns)	Diff Median (ns)
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.RepeatSlice_StartPosition_And_	1.08	24.39	22.56
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.Start_And_EndPosition	1.07	6.55	6.10
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.StartPosition_And_EndPosition	1.07	5.99	5.61
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.RepeatSlice	1.03	66.89	64.63
MicroBenchmarks.corefx.ReadOnlySequenceBenchmarks.MS_RepeatSlice	1.03	81.18	79.00

@ahsonkhan

adamsitnik · 2019-07-01T19:01:58Z

Contains hacks to get it working in VS

what can we do to make the experience within the perf repo better for testing latest?

I have fixed that in #594 Now to get it working in VS you should just install the latest Core SDK from https://github.com/dotnet/core-sdk#installers-and-binaries

ahsonkhan · 2019-07-16T04:25:19Z

I have fixed that in #594

@pgovind - could you please verify this works in VS as you would expect now and update this PR so it's ready to be merged?

…Slice

…arks so the Slice benchmarks should contain keyword slice

adamsitnik · 2019-07-18T09:30:54Z

@pgovind please excuse me for such a delay in the reviewing process. I am now doing 2.2 vs 3.0 comparison and I really needed these benchmarks so I just pushed the fixes to your branch. PTAL at the changes that I've introduced.

adamsitnik · 2019-07-18T09:31:51Z

Sample results:

dotnet run -c Release -f netcoreapp3.0 --filter SequenceReader

Method	Mean	Error	StdDev	Median	Min	Max	Gen 0/1k Op	Gen 1/1k Op	Gen 2/1k Op	Allocated Memory/Op
TryReadTo	198.7 ns	1.471 ns	1.376 ns	198.4 ns	197.0 ns	201.5 ns	-	-	-	-

dotnet run -c Release -f netcoreapp3.0 --filter ReadOnlySequence

Method	Segment	Mean	Error	StdDev	Median	Min	Max	Gen 0/1k Op	Gen 1/1k Op	Gen 2/1k Op	Allocated Memory/Op
Slice_StartPosition	Single	6.171 ns	0.2860 ns	0.3294 ns	6.316 ns	5.673 ns	6.649 ns	-	-	-	-
Slice_Start	Single	10.097 ns	0.0748 ns	0.0663 ns	10.128 ns	9.955 ns	10.177 ns	-	-	-	-
Slice_Start_And_Length	Single	12.756 ns	0.0766 ns	0.0716 ns	12.747 ns	12.645 ns	12.860 ns	-	-	-	-
Slice_Start_And_EndPosition	Single	6.793 ns	0.0736 ns	0.0688 ns	6.763 ns	6.714 ns	6.918 ns	-	-	-	-
Slice_StartPosition_And_Length	Single	7.099 ns	0.0592 ns	0.0553 ns	7.092 ns	7.032 ns	7.211 ns	-	-	-	-
Slice_StartPosition_And_EndPosition	Single	8.407 ns	0.0549 ns	0.0514 ns	8.410 ns	8.344 ns	8.526 ns	-	-	-	-
Slice_Repeat	Single	70.908 ns	0.7022 ns	0.6568 ns	70.713 ns	70.046 ns	72.189 ns	-	-	-	-
Slice_Repeat_StartPosition_And_EndPosition	Single	25.973 ns	0.2128 ns	0.1991 ns	25.914 ns	25.719 ns	26.398 ns	-	-	-	-
Slice_StartPosition	Multiple	11.185 ns	0.0768 ns	0.0719 ns	11.167 ns	11.092 ns	11.345 ns	-	-	-	-
Slice_Start	Multiple	10.068 ns	0.0827 ns	0.0774 ns	10.104 ns	9.915 ns	10.140 ns	-	-	-	-
Slice_Start_And_Length	Multiple	18.785 ns	0.1607 ns	0.1503 ns	18.853 ns	18.528 ns	18.958 ns	-	-	-	-
Slice_Start_And_EndPosition	Multiple	13.287 ns	0.1012 ns	0.0947 ns	13.268 ns	13.154 ns	13.481 ns	-	-	-	-
Slice_StartPosition_And_Length	Multiple	13.363 ns	0.0951 ns	0.0889 ns	13.343 ns	13.260 ns	13.554 ns	-	-	-	-
Slice_StartPosition_And_EndPosition	Multiple	14.448 ns	0.0898 ns	0.0840 ns	14.495 ns	14.314 ns	14.546 ns	-	-	-	-
Slice_Repeat	Multiple	85.813 ns	1.6846 ns	1.5758 ns	85.161 ns	83.973 ns	88.796 ns	-	-	-	-
Slice_Repeat_StartPosition_And_EndPosition	Multiple	59.594 ns	0.3184 ns	0.2823 ns	59.531 ns	59.248 ns	60.268 ns	-	-	-	-

ahsonkhan · 2019-07-19T01:30:09Z

I am now doing 2.2 vs 3.0 comparison and I really needed these benchmarks so I just pushed the fixes to your branch. PTAL at the changes that I've introduced.

Great. It would be great to verify that we didn't significantly regress ROSequence.Slice between 2.2 and 3.0.

Benchmarks for ReadOnlySequence.Slice.

3bd4e53

Contains hacks to get it working in VS

pgovind mentioned this pull request Jun 26, 2019

Improve default handling in ReadOnlySequence.Slice dotnet/corefx#38431

Merged

ahsonkhan reviewed Jun 26, 2019

View reviewed changes

jorive added the *NO MERGE* label Jun 26, 2019

adamsitnik added 7 commits July 18, 2019 10:42

Merge remote-tracking branch 'upstream/master' into ReadOnlySequence_…

c6c1abd

…Slice

reset solution and project file changes

4483330

return the result to avoid dead code elimination

a196b93

use [Params] to avoid code duplication and "MS_" prefixes

f610973

fix namespaces and #if defines

77b41f3

perform more operations, return the result

8c18713

rename the benchmarks, in the future this type might have more benchm…

0595a88

…arks so the Slice benchmarks should contain keyword slice

adamsitnik merged commit dba17a2 into dotnet:master Jul 18, 2019

This was referenced Jul 18, 2019

Add benchmarks for (de)serializing dictionaries in System.Text.Json #530

Merged

Add benchmarks for ISet<T> (de)serialization and general framework fo… #540

Merged

Add benchmarks for (de)serializing non-generic IEnumerable, IList, ICollection #547

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmarks for ReadOnlySequence.Slice.#589

Benchmarks for ReadOnlySequence.Slice.#589
adamsitnik merged 8 commits intodotnet:masterfrom
pgovind:ReadOnlySequence_Slice

pgovind commented Jun 26, 2019

Uh oh!

ahsonkhan commented Jun 26, 2019 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ahsonkhan Jun 26, 2019

Uh oh!

ahsonkhan Jun 26, 2019

Uh oh!

pgovind commented Jun 26, 2019 •

edited

Loading

Uh oh!

adamsitnik commented Jul 1, 2019

Uh oh!

ahsonkhan commented Jul 16, 2019

Uh oh!

adamsitnik commented Jul 18, 2019

Uh oh!

adamsitnik commented Jul 18, 2019

Uh oh!

ahsonkhan commented Jul 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

pgovind commented Jun 26, 2019

Uh oh!

ahsonkhan commented Jun 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ahsonkhan Jun 26, 2019

Choose a reason for hiding this comment

Uh oh!

ahsonkhan Jun 26, 2019

Choose a reason for hiding this comment

Uh oh!

pgovind commented Jun 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamsitnik commented Jul 1, 2019

Uh oh!

ahsonkhan commented Jul 16, 2019

Uh oh!

adamsitnik commented Jul 18, 2019

Uh oh!

adamsitnik commented Jul 18, 2019

Uh oh!

ahsonkhan commented Jul 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ahsonkhan commented Jun 26, 2019 •

edited

Loading

pgovind commented Jun 26, 2019 •

edited

Loading