Skip to content

executor: revert mostly changes in #11678#12481

Merged
sre-bot merged 4 commits into
pingcap:masterfrom
SunRunAway:window-func2
Oct 16, 2019
Merged

executor: revert mostly changes in #11678#12481
sre-bot merged 4 commits into
pingcap:masterfrom
SunRunAway:window-func2

Conversation

@SunRunAway
Copy link
Copy Markdown
Contributor

@SunRunAway SunRunAway commented Sep 29, 2019 •
edited
Loading

What problem does this PR solve?

(a following PR for #12480)
After #12480, we do not need to copy row data in window functions.

What is changed and how it works?

Revert mostly changes in #11678, and get the performance loss back.

name                                                           old time/op    new time/op    delta
WindowRows/(func:row_number,_ndv:10,_rows:1000)-12               92.9µs ±13%    86.0µs ± 7%     ~     (p=0.310 n=5+5)
WindowRows/(func:row_number,_ndv:1000,_rows:1000)-12              202µs ± 5%     230µs ± 8%  +13.65%  (p=0.008 n=5+5)
WindowRows/(func:row_number,_ndv:10,_rows:100000)-12             8.24ms ± 1%    8.37ms ± 3%     ~     (p=0.310 n=5+5)
WindowRows/(func:row_number,_ndv:1000,_rows:100000)-12           8.53ms ± 1%    8.60ms ± 0%   +0.92%  (p=0.032 n=5+5)
WindowFunctions/(func:row_number,_ndv:1000,_rows:100000)-12      8.58ms ± 1%    8.59ms ± 2%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:rank,_ndv:1000,_rows:100000)-12            11.8ms ± 2%     9.2ms ± 1%  -22.03%  (p=0.008 n=5+5)
WindowFunctions/(func:dense_rank,_ndv:1000,_rows:100000)-12      11.8ms ± 1%     9.2ms ± 1%  -22.36%  (p=0.008 n=5+5)
WindowFunctions/(func:cume_dist,_ndv:1000,_rows:100000)-12       12.2ms ± 1%     9.4ms ± 1%  -23.20%  (p=0.008 n=5+5)
WindowFunctions/(func:percent_rank,_ndv:1000,_rows:100000)-12    11.9ms ± 1%     9.4ms ± 1%  -21.40%  (p=0.008 n=5+5)
WindowFunctions/(func:ntile,_ndv:1000,_rows:100000)-12           8.70ms ± 0%    8.72ms ± 1%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:lead,_ndv:1000,_rows:100000)-12            13.3ms ± 1%    10.5ms ± 1%  -20.79%  (p=0.008 n=5+5)
WindowFunctions/(func:lag,_ndv:1000,_rows:100000)-12             13.0ms ± 2%    10.4ms ± 1%  -19.48%  (p=0.008 n=5+5)
WindowFunctions/(func:first_value,_ndv:1000,_rows:100000)-12     8.97ms ± 1%    9.00ms ± 5%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:last_value,_ndv:1000,_rows:100000)-12      8.90ms ± 1%    9.04ms ± 2%   +1.59%  (p=0.008 n=5+5)
WindowFunctions/(func:nth_value,_ndv:1000,_rows:100000)-12       8.96ms ± 1%    9.14ms ± 2%     ~     (p=0.056 n=5+5)

name                                                           old alloc/op   new alloc/op   delta
WindowRows/(func:row_number,_ndv:10,_rows:1000)-12               82.5kB ± 0%    82.5kB ± 0%     ~     (all equal)
WindowRows/(func:row_number,_ndv:1000,_rows:1000)-12              105kB ± 1%     106kB ± 0%     ~     (p=0.151 n=5+5)
WindowRows/(func:row_number,_ndv:10,_rows:100000)-12             12.1MB ± 0%    12.1MB ± 0%     ~     (p=0.254 n=4+5)
WindowRows/(func:row_number,_ndv:1000,_rows:100000)-12           7.96MB ± 0%    7.97MB ± 0%     ~     (p=0.230 n=5+5)
WindowFunctions/(func:row_number,_ndv:1000,_rows:100000)-12      7.97MB ± 0%    7.97MB ± 0%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:rank,_ndv:1000,_rows:100000)-12            14.0MB ± 0%     8.0MB ± 0%  -43.08%  (p=0.008 n=5+5)
WindowFunctions/(func:dense_rank,_ndv:1000,_rows:100000)-12      14.0MB ± 0%     8.0MB ± 0%  -43.07%  (p=0.008 n=5+5)
WindowFunctions/(func:cume_dist,_ndv:1000,_rows:100000)-12       14.0MB ± 0%     8.0MB ± 0%  -43.07%  (p=0.008 n=5+5)
WindowFunctions/(func:percent_rank,_ndv:1000,_rows:100000)-12    14.0MB ± 0%     8.0MB ± 0%  -43.10%  (p=0.008 n=5+5)
WindowFunctions/(func:ntile,_ndv:1000,_rows:100000)-12           7.97MB ± 0%    7.97MB ± 0%     ~     (p=0.079 n=4+5)
WindowFunctions/(func:lead,_ndv:1000,_rows:100000)-12            9.57MB ± 0%    7.97MB ± 0%  -16.71%  (p=0.008 n=5+5)
WindowFunctions/(func:lag,_ndv:1000,_rows:100000)-12             9.60MB ± 0%    7.97MB ± 0%  -16.90%  (p=0.008 n=5+5)
WindowFunctions/(func:first_value,_ndv:1000,_rows:100000)-12     7.96MB ± 0%    7.97MB ± 0%     ~     (p=0.262 n=5+5)
WindowFunctions/(func:last_value,_ndv:1000,_rows:100000)-12      7.97MB ± 0%    7.96MB ± 0%     ~     (p=0.294 n=5+5)
WindowFunctions/(func:nth_value,_ndv:1000,_rows:100000)-12       7.97MB ± 0%    7.97MB ± 0%     ~     (p=0.452 n=5+5)

name                                                           old allocs/op  new allocs/op  delta
WindowRows/(func:row_number,_ndv:10,_rows:1000)-12                  162 ± 0%       162 ± 0%     ~     (all equal)
WindowRows/(func:row_number,_ndv:1000,_rows:1000)-12              1.68k ± 1%     1.68k ± 0%     ~     (p=1.000 n=5+5)
WindowRows/(func:row_number,_ndv:10,_rows:100000)-12              5.17k ± 0%     5.17k ± 0%     ~     (p=0.238 n=4+5)
WindowRows/(func:row_number,_ndv:1000,_rows:100000)-12            14.1k ± 0%     14.1k ± 0%     ~     (p=0.175 n=5+5)
WindowFunctions/(func:row_number,_ndv:1000,_rows:100000)-12       14.1k ± 0%     14.1k ± 0%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:rank,_ndv:1000,_rows:100000)-12             33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:dense_rank,_ndv:1000,_rows:100000)-12       33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:cume_dist,_ndv:1000,_rows:100000)-12        33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:percent_rank,_ndv:1000,_rows:100000)-12     33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:ntile,_ndv:1000,_rows:100000)-12            14.1k ± 0%     14.1k ± 0%     ~     (p=0.079 n=4+5)
WindowFunctions/(func:lead,_ndv:1000,_rows:100000)-12              114k ± 0%       14k ± 0%  -87.63%  (p=0.008 n=5+5)
WindowFunctions/(func:lag,_ndv:1000,_rows:100000)-12               115k ± 0%       14k ± 0%  -87.74%  (p=0.008 n=5+5)
WindowFunctions/(func:first_value,_ndv:1000,_rows:100000)-12      14.1k ± 0%     14.1k ± 0%     ~     (p=0.262 n=5+5)
WindowFunctions/(func:last_value,_ndv:1000,_rows:100000)-12       14.1k ± 0%     14.1k ± 0%     ~     (p=0.294 n=5+5)
WindowFunctions/(func:nth_value,_ndv:1000,_rows:100000)-12        14.1k ± 0%     14.1k ± 0%     ~     (p=0.452 n=5+5)

Check List

Tests

  • None

Code changes

  • None

Side effects

  • None

Related changes

  • None

Release note

  • None

@SunRunAway SunRunAway mentioned this pull request Sep 29, 2019
Merged
@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 29, 2019 •
edited
Loading

Codecov Report

Merging #12481 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #12481   +/-   ##
===========================================
  Coverage   80.0316%   80.0316%           
===========================================
  Files           462        462           
  Lines        105592     105592           
===========================================
  Hits          84507      84507           
  Misses        14884      14884           
  Partials       6201       6201

@SunRunAway
Copy link
Copy Markdown
Contributor Author

SunRunAway commented Sep 29, 2019

/run-all-tests

@SunRunAway
Copy link
Copy Markdown
Contributor Author

SunRunAway commented Sep 29, 2019

/run-unit-test

@SunRunAway SunRunAway changed the title [WIP] executor: revert mostly changes in #11678 executor: revert mostly changes in #11678 Oct 12, 2019
alivxxx
alivxxx reviewed Oct 14, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This was referenced Oct 14, 2019
Merged
Merged
lzmhhh123
lzmhhh123 reviewed Oct 14, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, shall we cherry-pick this PR in 3.0 and 3.1?

Comment thread executor/aggfuncs/func_cume_dist.go Outdated
@SunRunAway
Copy link
Copy Markdown
Contributor Author

SunRunAway commented Oct 15, 2019

/run-all-tests

@SunRunAway SunRunAway requested a review from lzmhhh123 October 15, 2019 12:12
lzmhhh123
lzmhhh123 approved these changes Oct 16, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lzmhhh123 lzmhhh123 added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. labels Oct 16, 2019
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

Your auto merge job has been accepted, waiting for 12724

@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

/run-all-tests

@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

@SunRunAway merge failed.

@SunRunAway
Copy link
Copy Markdown
Contributor Author

SunRunAway commented Oct 16, 2019

/merge

@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

Your auto merge job has been accepted, waiting for 12700

@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

/run-all-tests

@sre-bot sre-bot merged commit 609b529 into pingcap:master Oct 16, 2019
@sre-bot sre-bot mentioned this pull request Oct 16, 2019
Merged
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

cherry pick to release-3.0 in PR #12770

@sre-bot sre-bot mentioned this pull request Oct 16, 2019
Merged
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Oct 16, 2019

cherry pick to release-3.1 in PR #12771

@SunRunAway SunRunAway deleted the window-func2 branch October 16, 2019 18:00
sre-bot added a commit that referenced this pull request Oct 17, 2019
@sre-bot
executor: revert mostly changes in #11678 (#12481) (#12771)
1983b54
sre-bot added a commit that referenced this pull request Oct 17, 2019
@sre-bot
executor: revert mostly changes in #11678 (#12481) (#12770)
150dd34
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
@SunRunAway @XiaTianliang
executor: revert mostly changes in pingcap#11678 (pingcap#12481)
bcc4858
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@alivxxx alivxxx alivxxx left review comments

@lzmhhh123 lzmhhh123 lzmhhh123 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@SunRunAway @sre-bot @alivxxx @lzmhhh123