Skip to content

statistics: add metrics for unneeded analyze table#54822

Merged
ti-chi-bot[bot] merged 4 commits into
pingcap:masterfrom
hawkingrei:add_health_for_uneligibleAnalyze
Jul 26, 2024
Merged

statistics: add metrics for unneeded analyze table#54822
ti-chi-bot[bot] merged 4 commits into
pingcap:masterfrom
hawkingrei:add_health_for_uneligibleAnalyze

Conversation

@hawkingrei
Copy link
Copy Markdown
Member

@hawkingrei hawkingrei commented Jul 23, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #54823

Problem Summary:

What changed and how does it work?

The primary changes in pkg/statistics/handle/cache/statscache.go file.
the rest are code migrations to avoid cyclic imports.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

I test in the 1m tables cluster. we can see this metrics in the grafana.

image
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 23, 2024
@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 23, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 90.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 56.9952%. Comparing base (f774ef6) to head (2a88ef3).
Report is 38 commits behind head on master.

Additional details and impacted files 
@@                Coverage Diff                @@
##             master     #54822         +/-   ##
=================================================
- Coverage   74.7207%   56.9952%   -17.7256%     
=================================================
  Files          1557       1684        +127     
  Lines        363500     631235     +267735     
=================================================
+ Hits         271610     359774      +88164     
- Misses        72240     247422     +175182     
- Partials      19650      24039       +4389
Flag Coverage Δ
integration 37.6885% <85.0000%> (?)
unit 72.1299% <90.0000%> (-1.5151%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.3749% <ø> (-1.8246%) ⬇️
parser ∅ <ø> (∅)
br 51.9165% <ø> (+4.2288%) ⬆️

@hawkingrei
statistics: add metrics for unneeded analyze table
fb56168 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
statistics: add metrics for unneeded analyze table
aa6287a 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei hawkingrei force-pushed the add_health_for_uneligibleAnalyze branch from 69d932a to aa6287a Compare July 23, 2024 05:41
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Jul 23, 2024

/retest

@hawkingrei hawkingrei requested a review from winoros July 23, 2024 09:55
@hawkingrei
update
0907f8a 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Jul 24, 2024

/retest

1 similar comment
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Jul 24, 2024

/retest

@ti-chi-bot ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 24, 2024
@hawkingrei hawkingrei requested a review from 0xPoe July 25, 2024 03:55
0xPoe
0xPoe reviewed Jul 25, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@0xPoe 0xPoe left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update the grafana json file?

Comment thread pkg/statistics/table.go Outdated
@hawkingrei @0xPoe
Update pkg/statistics/table.go
2a88ef3 
Co-authored-by: 二手掉包工程师 <rustin.liu@gmail.com>
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Jul 25, 2024 •
edited
Loading

Do we need to update the grafana json file?

No, because this type is generated dynamically by the grafana.

@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Jul 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hi-rustin, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 25, 2024
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Jul 25, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-07-24 12:41:39.274428107 +0000 UTC m=+1048921.265369577: ☑️ agreed by winoros.
  • 2024-07-25 07:53:23.497835995 +0000 UTC m=+1118025.488777466: ☑️ agreed by hi-rustin.

@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Jul 25, 2024

/retest

1 similar comment
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Jul 26, 2024

/retest

@ti-chi-bot ti-chi-bot Bot merged commit 7e73ddc into pingcap:master Jul 26, 2024
morgo added a commit to morgo/tidb that referenced this pull request Jul 28, 2024
@morgo
Merge remote-tracking branch 'upstream/master' into add-default-literal
4424521 
* upstream/master: (93 commits)
  disjoinset: add generic impl (pingcap#54917)
  planner: derive index filters for mv index paths (pingcap#54877)
  br: cli refactor backup error handling logic (pingcap#54697)
  expression: fix infinity loop in `timestampadd` (pingcap#54916)
  planner: import more expand test. (pingcap#54962)
  planner: use code-gen to generate CloneForPlanCache method for some operators (pingcap#54957)
  test: fix flaky test TestFailSchemaSyncer (pingcap#54958)
  planner: move logical show into logicalop pkg. (pingcap#54928)
  privilege: Remove TestAbnormalMySQLTable (pingcap#54925)
  resource_control: support unlimited keyword when setting the resource group (pingcap#54704)
  ddl: fix a data race on localRowCntListener Written() (pingcap#54484)
  lightning: fix SET SESSION on connection pool (pingcap#54927)
  planner: move logical mem-table to logicalop pkg. (pingcap#54903)
  table: Add `CachedTableSupport` and `TemporaryTableSupport` for `MutateContext` (pingcap#54900)
  executor: fix index_hash_join hang when context canceled (pingcap#54855)
  statistics: add metrics for unneeded analyze table (pingcap#54822)
  *: refine pipelined dml benchmarks (pingcap#54844)
  ddl: assign table IDs for jobs submitted to queue (pingcap#54880)
  *: avoid using Tables field of model.DBInfo, use API instead (pingcap#52302)
  planner: remove useless check (pingcap#54907)
  ...
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Aug 22, 2024

/cherrypick release-8.1
/cherrypick release-7.5

@ti-chi-bot
Copy link
Copy Markdown
Member

ti-chi-bot commented Aug 22, 2024

@hawkingrei: new pull request created to branch release-7.5: #55577.

Details

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
2848d33 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Aug 22, 2024
Merged
13 tasks
@ti-chi-bot
Copy link
Copy Markdown
Member

ti-chi-bot commented Aug 22, 2024

@hawkingrei: new pull request created to branch release-8.1: #55578.

Details

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
2f8e42c 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Aug 22, 2024
Closed
13 tasks
@hawkingrei hawkingrei deleted the add_health_for_uneligibleAnalyze branch August 22, 2024 03:36
ti-chi-bot Bot pushed a commit that referenced this pull request Aug 25, 2024
@ti-chi-bot
statistics: add metrics for unneeded analyze table (#54822) (#55577)
47279f5 
close #54823
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Nov 4, 2024

/cherrypick release-8.1

@ti-chi-bot
Copy link
Copy Markdown
Member

ti-chi-bot commented Nov 4, 2024

@hawkingrei: new pull request created to branch release-8.1: #57100.

Details

In response to this:

/cherrypick release-8.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Nov 4, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
ab6e036 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Nov 4, 2024
Merged
13 tasks
ti-chi-bot Bot pushed a commit that referenced this pull request Nov 4, 2024
@ti-chi-bot
statistics: add metrics for unneeded analyze table (#54822) (#57100)
152b102 
close #54823
@hawkingrei
Copy link
Copy Markdown
Member Author

hawkingrei commented Dec 4, 2024

/cherry-pick release-8.1-20241203-v8.1.1

@ti-chi-bot
Copy link
Copy Markdown
Member

ti-chi-bot commented Dec 4, 2024

@hawkingrei: new pull request created to branch release-8.1-20241203-v8.1.1: #57966.

Details

In response to this:

/cherry-pick release-8.1-20241203-v8.1.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Dec 4, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
3e9752d 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Dec 4, 2024
Merged
13 tasks
hawkingrei pushed a commit to ti-chi-bot/tidb that referenced this pull request Dec 4, 2024
@ti-chi-bot @hawkingrei
statistics: add metrics for unneeded analyze table (pingcap#54822) (p…
430072a 
…ingcap#57100)

close pingcap#54823
ti-chi-bot Bot pushed a commit that referenced this pull request Dec 4, 2024
@ti-chi-bot
statistics: add metrics for unneeded analyze table (#54822) (#57966)
238b0d8 
close #54823
@0xPoe 0xPoe mentioned this pull request Oct 21, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@winoros winoros winoros approved these changes

@0xPoe 0xPoe 0xPoe approved these changes

Assignees

No one assigned

Labels

approved lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

add metrics for unneeded analyze

4 participants

@hawkingrei @ti-chi-bot @winoros @0xPoe