*: unify get store function everywhere by rleungx · Pull Request #1671 · tikv/pd

rleungx · 2019-08-08T08:36:55Z

What problem does this PR solve?

See #1654 (comment)

What is changed and how it works?

This PR unifies all signature of GetStore functions and remove TryGetStore.

Check List

Tests

Unit test

codecov-io · 2019-08-09T10:20:25Z

Codecov Report

Merging #1671 into master will increase coverage by <.01%.
The diff coverage is 52.43%.

@@            Coverage Diff             @@
##           master    #1671      +/-   ##
==========================================
+ Coverage   76.68%   76.68%   +<.01%     
==========================================
  Files         157      157              
  Lines       15462    15489      +27     
==========================================
+ Hits        11857    11878      +21     
  Misses       2592     2592              
- Partials     1013     1019       +6

Impacted Files	Coverage Δ
server/schedulers/shuffle_hot_region.go	`65.38% <0%> (+4.85%)`	⬆️
server/cluster.go	`82.76% <0%> (+0.35%)`	⬆️
server/schedulers/hot_region.go	`83.03% <0%> (-0.61%)`	⬇️
server/schedulers/balance_leader.go	`96.66% <100%> (+0.14%)`	⬆️
server/schedule/range_cluster.go	`90.47% <100%> (ø)`	⬆️
tests/cluster.go	`85.99% <100%> (ø)`	⬆️
pkg/mock/mockcluster/mockcluster.go	`91.66% <100%> (+0.06%)`	⬆️
server/heartbeat_streams.go	`69% <14.28%> (+0.68%)`	⬆️
server/schedulers/balance_region.go	`84.79% <33.33%> (-1.96%)`	⬇️
server/schedule/operator/operator.go	`87.38% <42.85%> (-0.49%)`	⬇️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ec155a...1206b86. Read the comment docs.

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx · 2019-08-16T03:53:32Z

PTAL. @disksing @shafreeck

CLAassistant · 2019-08-19T06:04:14Z

All committers have signed the CLA.

sre-bot · 2019-08-19T06:05:48Z

/run-all-tests

sre-bot · 2019-08-19T06:09:46Z

@rleungx merge failed.

disksing · 2019-08-19T06:14:51Z

/run-all-tests

rleungx · 2019-08-19T07:37:11Z

/merge

sre-bot · 2019-08-19T07:37:45Z

/run-all-tests

* add TopN * replace cache to TopN * change the threshold * modify parameters * add metrics of TopN * fix panic * change rand hot region for test * change RandHotRegion for test * remove verbose lock functions * change RandHotRegion back * change hot scheduler strategy * use denoised FlowBytes if possible * add stores stats for schedulers * use store flow bytes when select source and target stores * *: unify get store function everywhere (#1671) Signed-off-by: Ryan Leung <rleungx@gmail.com> * disable some failed tests

* *: unify get store function everywhere (#1671) Signed-off-by: Ryan Leung <rleungx@gmail.com> * server: use leader lease to determine tso service validity (#1676) Signed-off-by: disksing <i@disksing.com> * test: fix tests (#1696) * test: fix region syncer test Signed-off-by: disksing <i@disksing.com> * add config-check flag for pd-server (#1695) Signed-off-by: cwen0 <cwenyin0@gmail.com> * operator: rewrite move region related functions (#1667) * *: support setting endKey for ScanRange (#1700) Signed-off-by: disksing <i@disksing.com> * *: reduce some unnecessary parameters (#1698) Signed-off-by: Ryan Leung <rleungx@gmail.com> * schedule: Do not send an operator of a region wth a stale epoch (#1659) * schedule: Do not send an operator of a region wth a stale epoch Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: check the version changed by the operator self Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: fix unit test Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: fix to avoid dispatching a stale opstep Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: refactor "ConsumeConfVer() int" to "ExpectConfVerChange() bool" Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: fix typo in comment Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * fix typo Co-Authored-By: Ryan Leung <rleungx@gmail.com> * dispatch: fix unittest Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: refine format Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * server: fix the dead lock in scatter region (#1706) Signed-off-by: Ryan Leung <rleungx@gmail.com>

* *: unify get store function everywhere (#1671) Signed-off-by: Ryan Leung <rleungx@gmail.com> * remove unnecessary parentheses * server: use leader lease to determine tso service validity (#1676) Signed-off-by: disksing <i@disksing.com> * change internal stat values to float64 * add pending operator influence * add metrics of pending influence * fix metrics * fix panic * adjust pending influence of balanceHotWrite * change weight of pending influence * test: fix tests (#1696) * test: fix region syncer test Signed-off-by: disksing <i@disksing.com> * decrease region rolling window; store pending influence in scheduler * add config-check flag for pd-server (#1695) Signed-off-by: cwen0 <cwenyin0@gmail.com> * decrease possiblility transfer hot write leader * change pending influence weight * add unstarted op metrics * add logs for debug * add log for debug * add logs for debug * add logs for debug * add logs for debug * add logs for debug * add logs for debug * add logs for debug * Revert "add logs for debug" This reverts commit e74c7a9. * add metrics for hotspot operators * operator: rewrite move region related functions (#1667) * add metrics for pending operators * *: support setting endKey for ScanRange (#1700) Signed-off-by: disksing <i@disksing.com> * fix bug * fix bug * fix bug * fix metrics thread-safe bug * fix logic bug * *: reduce some unnecessary parameters (#1698) Signed-off-by: Ryan Leung <rleungx@gmail.com> * schedule: Do not send an operator of a region wth a stale epoch (#1659) * schedule: Do not send an operator of a region wth a stale epoch Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: check the version changed by the operator self Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: fix unit test Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: fix to avoid dispatching a stale opstep Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: refactor "ConsumeConfVer() int" to "ExpectConfVerChange() bool" Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: fix typo in comment Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * fix typo Co-Authored-By: Ryan Leung <rleungx@gmail.com> * dispatch: fix unittest Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: refine format Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * server: fix the dead lock in scatter region (#1706) Signed-off-by: Ryan Leung <rleungx@gmail.com> * add drop time for operator * use IsDropped to recognize canceled ops * try to fix trans leader burst * try to fix trans leader burst * add zombie influence * change select src dst strategy; improve op_controller * change select src strategy * fix bug * tools: fix set namespace in pd-ctl (#1701) Signed-off-by: Ryan Leung <rleungx@gmail.com> * tools: fix parse url without http prefix (#1703) Signed-off-by: Ryan Leung <rleungx@gmail.com> * tests: support deadlock detection in make test (#1704) Signed-off-by: Ryan Leung <rleungx@gmail.com> * Makefile: fix failpoint enable (#1722) Signed-off-by: nolouch <nolouch@gmail.com> * checker: fix the issue that a region does not merge to the sibling with smaller size (#1723) Signed-off-by: disksing <i@disksing.com> * tools: balance region simulator (#1708) * scheduler: do not remove the operator when the step does not finish (#1715) Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * operator: fix the AddLearner config version judgment (#1732) Signed-off-by: nolouch <nolouch@gmail.com> * tools: fix TLS in pd control (#1729) Signed-off-by: Ryan Leung <rleungx@gmail.com> * syncer: support TLS for region syncer (#1728) Signed-off-by: Ryan Leung <rleungx@gmail.com> * schedule: fix a thread-safe bug and improve code (#1719)

* *: unify get store function everywhere (#1671) Signed-off-by: Ryan Leung <rleungx@gmail.com> * server: use leader lease to determine tso service validity (#1676) Signed-off-by: disksing <i@disksing.com> * test: fix tests (#1696) * test: fix region syncer test Signed-off-by: disksing <i@disksing.com> * add config-check flag for pd-server (#1695) Signed-off-by: cwen0 <cwenyin0@gmail.com> * operator: rewrite move region related functions (#1667) * *: support setting endKey for ScanRange (#1700) Signed-off-by: disksing <i@disksing.com> * *: reduce some unnecessary parameters (#1698) Signed-off-by: Ryan Leung <rleungx@gmail.com> * schedule: Do not send an operator of a region wth a stale epoch (#1659) * schedule: Do not send an operator of a region wth a stale epoch Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: check the version changed by the operator self Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: fix unit test Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * schedule: fix to avoid dispatching a stale opstep Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: refactor "ConsumeConfVer() int" to "ExpectConfVerChange() bool" Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: fix typo in comment Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * fix typo Co-Authored-By: Ryan Leung <rleungx@gmail.com> * dispatch: fix unittest Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * dispatch: refine format Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * server: fix the dead lock in scatter region (#1706) Signed-off-by: Ryan Leung <rleungx@gmail.com> * tools: fix set namespace in pd-ctl (#1701) Signed-off-by: Ryan Leung <rleungx@gmail.com> * tools: fix parse url without http prefix (#1703) Signed-off-by: Ryan Leung <rleungx@gmail.com> * tests: support deadlock detection in make test (#1704) Signed-off-by: Ryan Leung <rleungx@gmail.com> * Makefile: fix failpoint enable (#1722) Signed-off-by: nolouch <nolouch@gmail.com> * checker: fix the issue that a region does not merge to the sibling with smaller size (#1723) Signed-off-by: disksing <i@disksing.com> * tools: balance region simulator (#1708) * scheduler: do not remove the operator when the step does not finish (#1715) Signed-off-by: Shafreeck Sea <shafreeck@gmail.com> * operator: fix the AddLearner config version judgment (#1732) Signed-off-by: nolouch <nolouch@gmail.com> * tools: fix TLS in pd control (#1729) Signed-off-by: Ryan Leung <rleungx@gmail.com> * syncer: support TLS for region syncer (#1728) Signed-off-by: Ryan Leung <rleungx@gmail.com> * schedule: fix a thread-safe bug and improve code (#1719) * statistics: fix region flow calculation (#1688) Signed-off-by: jiyingtk <jiyingtk@mail.ustc.edu.cn> * makefile: improve deadlock-enable/disable (#1736) * api: fix missing keys statistic in region information (#1741) Signed-off-by: nolouch <nolouch@gmail.com> * *: update go version to 1.13 (#1742) Signed-off-by: disksing <i@disksing.com> * coordinator: add the operator cost time in log field (#1748) Signed-off-by: nolouch <nolouch@gmail.com>

disksing added status/can-merge Indicates a PR has been approved by a committer. and removed status/can-merge Indicates a PR has been approved by a committer. labels Aug 12, 2019

disksing reviewed Aug 12, 2019

View reviewed changes

Comment thread server/api/label.go Outdated

rleungx requested a review from shafreeck August 14, 2019 08:04

rleungx added 4 commits August 16, 2019 10:22

refactor cluster related logic

5e755ae

Signed-off-by: Ryan Leung <rleungx@gmail.com>

address comments

1afd91b

Signed-off-by: Ryan Leung <rleungx@gmail.com>

resolve the conflict

108d5c9

Signed-off-by: Ryan Leung <rleungx@gmail.com>

unify the get store function everywhere

e8cbb66

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx force-pushed the unify-get-store branch from cf173a8 to fb6a375 Compare August 16, 2019 03:23

address comments

5cc31f0

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx force-pushed the unify-get-store branch from fb6a375 to 5cc31f0 Compare August 16, 2019 03:33

nolouch approved these changes Aug 16, 2019

View reviewed changes

nolouch added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 16, 2019

disksing approved these changes Aug 19, 2019

View reviewed changes

disksing added the status/can-merge Indicates a PR has been approved by a committer. label Aug 19, 2019

Merge branch 'master' into unify-get-store

9be52e4

Merge branch 'master' into unify-get-store

1206b86

sre-bot merged commit 561d004 into tikv:master Aug 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: unify get store function everywhere#1671

*: unify get store function everywhere#1671
sre-bot merged 7 commits into
tikv:masterfrom
rleungx:unify-get-store

rleungx commented Aug 8, 2019

Uh oh!

codecov-io commented Aug 9, 2019 •

edited

Loading

Uh oh!

Uh oh!

rleungx commented Aug 16, 2019

Uh oh!

CLAassistant commented Aug 19, 2019 •

edited

Loading

Uh oh!

sre-bot commented Aug 19, 2019

Uh oh!

sre-bot commented Aug 19, 2019

Uh oh!

disksing commented Aug 19, 2019

Uh oh!

rleungx commented Aug 19, 2019

Uh oh!

sre-bot commented Aug 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

rleungx commented Aug 8, 2019

What problem does this PR solve?

What is changed and how it works?

Check List

Uh oh!

codecov-io commented Aug 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

rleungx commented Aug 16, 2019

Uh oh!

CLAassistant commented Aug 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sre-bot commented Aug 19, 2019

Uh oh!

sre-bot commented Aug 19, 2019

Uh oh!

disksing commented Aug 19, 2019

Uh oh!

rleungx commented Aug 19, 2019

Uh oh!

sre-bot commented Aug 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-io commented Aug 9, 2019 •

edited

Loading

CLAassistant commented Aug 19, 2019 •

edited

Loading