[Profile] Visualize the query plan and query profile by morningman · Pull Request #5475 · apache/doris

morningman · 2021-03-06T02:13:27Z

Proposed changes

Add command:

EXPLAIN GRAPGH SELECT ...
SHOW QUERY PROFILE "..."

Document will be added in next PR.
Details can be seem in #5474

Types of changes

New feature (non-breaking change which adds functionality)

Other

I introduced a tree printer class from https://github.com/davidsusu/tree-printer
This is a greet tree printer, but it release version has a NPE bug.
So I can't just add maven dependency of it, but has to add all its source codes in
our repo and fix the bug.

I will also send a PR to davidsusu/tree-printer, and once it merged and release a new version.
I will remove the source code of the tree printer in Doris and use its maven dependency.

Update:
Using tree-printer 1.2 instead of source code

For Reviewer:

The file in package: fe/fe-common/src/main/java/org/apache/doris/common/treeprinter/
are from davidsusu/tree-printer, no need to review.

Add a new statement: SHOW QUERY PROFILE "/";

Files under fe/fe-core/src/main/java/org/apache/doris/common/profile/ are for tree construction
and tree printer.

EmmyMiao87 · 2021-03-10T02:30:36Z

Can't his tool directly reference jar? Do we have to buckle the code into our own code base?

morningman · 2021-03-10T02:32:56Z

Can't his tool directly reference jar? Do we have to buckle the code into our own code base?

OK, I will wait it to make to new release.

Add command: 1. EXPLAIN GRAPGH SELECT ... 2. SHOW QUERY PROFILE "..." Document will be added in next PR Change-Id: Ifd9365e10b1f9ff4fdf8ae0556343783d97545f0

morningman · 2021-03-11T15:03:53Z

Can't his tool directly reference jar? Do we have to buckle the code into our own code base?

OK, I will wait it to make to new release.

@EmmyMiao87 Done

EmmyMiao87 · 2021-03-15T11:29:04Z

fe/fe-core/src/main/cup/sql_parser.cup

+        stmt.getQueryStmt().setIsExplain(new ExplainOptions(true, false));
        RESULT = stmt;
    :}
    ;


Change KW_DESCRIBE to KW_EXPLAIN

KW_DESCRIBE means both "desc" and "explain",
see fe/fe-core/src/main/jflex/sql_scanner.flex

EmmyMiao87 · 2021-03-15T11:37:33Z

fe/fe-core/src/main/java/org/apache/doris/analysis/StatementBase.java

@@ -36,9 +36,7 @@ public abstract class StatementBase implements ParseNode {
    private String clusterName;

    // True if this QueryStmt is the top level query from an EXPLAIN <query>


Please update comment.

EmmyMiao87 · 2021-03-15T11:56:13Z

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java

+    }
+
+    private void buildForPlanNode(PlanNode planNode, PlanTreeNode parent) {
+        StringBuilder sb = new StringBuilder();


Why not print the plan node directly?

EmmyMiao87 · 2021-03-15T11:56:50Z

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java

+            PlanTreeNode sinkNode = null;
+            if (sink != null) {
+                StringBuilder sb = new StringBuilder();
+                sb.append("[").append(sink.getExchNodeId().asInt()).append(": DATA SINK]");


DATA SINK or Result Sink

Change to sink's class name

EmmyMiao87 · 2021-03-15T12:03:05Z

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java

+
+public class PlanTreeBuilder {
+
+    private static final String EXCHANGE_NODE = "EXCHANGE";


Use final params in Exchange Node.

Move this to ExchangeNode

EmmyMiao87 · 2021-03-15T12:03:13Z

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java

+public class PlanTreeBuilder {
+
+    private static final String EXCHANGE_NODE = "EXCHANGE";
+    private static final String MERGING_EXCHANGE_NODE = "MERGING-EXCHANGE";


Same as above

EmmyMiao87

LGTM

Add command: 1. EXPLAIN GRAPGH SELECT ... 2. SHOW QUERY PROFILE "..." Document will be added in next PR Change-Id: Ifd9365e10b1f9ff4fdf8ae0556343783d97545f0

morningman added the kind/improvement label Mar 6, 2021

morningman self-assigned this Mar 6, 2021

morningman force-pushed the view_profile branch 2 times, most recently from f198ad7 to 0e7cbd2 Compare March 8, 2021 01:28

morningman mentioned this pull request Mar 8, 2021

[Bug] Fix NPE when using BorderTreeNodeDecorator davidsusu/tree-printer#4

Merged

EmmyMiao87 self-assigned this Mar 10, 2021

morningman-cmy and others added 3 commits March 11, 2021 23:00

[Profile] Support to view query plan and profile in graph

c4f626a

Add command: 1. EXPLAIN GRAPGH SELECT ... 2. SHOW QUERY PROFILE "..." Document will be added in next PR Change-Id: Ifd9365e10b1f9ff4fdf8ae0556343783d97545f0

fix bug

f0de52d

add tree-printer-1.2

9b54958

morningman force-pushed the view_profile branch from c704929 to 9b54958 Compare March 11, 2021 15:01

EmmyMiao87 linked an issue Mar 15, 2021 that may be closed by this pull request

[Proposal] Visualize the query plan and query profile #5474

Closed

EmmyMiao87 reviewed Mar 15, 2021

View reviewed changes

fix by review

74e74b7

EmmyMiao87 approved these changes Mar 19, 2021

View reviewed changes

EmmyMiao87 added the area/profile Issues or PRs related to the profile label Mar 19, 2021

morningman merged commit b9d92e0 into apache:master Mar 21, 2021

morningman mentioned this pull request Oct 10, 2021

Release note 0.15.0 #6806

Closed

w41ter added a commit to w41ter/incubator-doris that referenced this pull request Nov 10, 2025

[chore](snapshot) Read only clone is not supported yet (apache#5475)

025ced3

		@@ -36,9 +36,7 @@ public abstract class StatementBase implements ParseNode {
		private String clusterName;

		// True if this QueryStmt is the top level query from an EXPLAIN <query>


		public class PlanTreeBuilder {

		private static final String EXCHANGE_NODE = "EXCHANGE";

Conversation

morningman commented Mar 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Types of changes

Other

For Reviewer:

Uh oh!

EmmyMiao87 commented Mar 10, 2021

Uh oh!

morningman commented Mar 10, 2021

Uh oh!

morningman commented Mar 11, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EmmyMiao87 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

morningman commented Mar 6, 2021 •

edited

Loading