[feature](mtmv) Support querying rewrite by materialized view when DML such as insert and insert overwrite #37561
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
Contributor
Author
|
run buildall |
2 similar comments
Contributor
Author
|
run buildall |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 40869 ms |
TPC-DS: Total hot run time: 175187 ms |
ClickBench: Total hot run time: 30.38 s |
Contributor
|
PR approved by anyone and no changes requested. |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 40186 ms |
TPC-DS: Total hot run time: 172714 ms |
ClickBench: Total hot run time: 30.31 s |
seawinde
force-pushed
the
ddl_enable_mv_rewrite
branch
from
46f9756 to
d0e868c
Compare
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 39985 ms |
TPC-DS: Total hot run time: 171118 ms |
ClickBench: Total hot run time: 30.77 s |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 39725 ms |
TPC-DS: Total hot run time: 174677 ms |
ClickBench: Total hot run time: 30.71 s |
zddr
reviewed
Jul 12, 2024
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/OptimizeGroupExpressionJob.java
Show resolved
Hide resolved
.../org/apache/doris/nereids/rules/exploration/mv/InitConsistentMaterializationContextHook.java
Show resolved
Hide resolved
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 40142 ms |
TPC-DS: Total hot run time: 174149 ms |
ClickBench: Total hot run time: 30.7 s |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 40096 ms |
Contributor
Author
|
run buildall |
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
1 similar comment
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
TPC-H: Total hot run time: 39702 ms |
TPC-DS: Total hot run time: 173964 ms |
ClickBench: Total hot run time: 31.09 s |
morrySnow
reviewed
Jul 17, 2024
| LOG.debug("Start analyze plan"); | ||
| } | ||
| keepOrShowPlanProcess(showPlanProcess, () -> cascadesContext.newAnalyzer().analyze()); | ||
| if (this.cascadesContext.getConnectContext().getSessionVariable().isEnableDmlMaterializedViewRewrite() |
Contributor
There was a problem hiding this comment.
u should add a analysis rule and check do this in rule. move hooks from NereidsPlanner into StatementContext
| = "enable_dml_materialized_view_rewrite"; | ||
|
|
||
| public static final String ENABLE_DML_MATERIALIZED_VIEW_REWRITE_WHEN_BASE_TABLE_UNAWARENESS | ||
| = "enable_dml_materialized_view_rewrite_when_base_table_unawareness"; |
Contributor
There was a problem hiding this comment.
unawareness? is it a standard phrases?
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 39975 ms |
TPC-DS: Total hot run time: 172991 ms |
ClickBench: Total hot run time: 30.88 s |
zddr
approved these changes
Jul 18, 2024
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 39795 ms |
TPC-DS: Total hot run time: 174140 ms |
ClickBench: Total hot run time: 30.64 s |
morrySnow
previously approved these changes
Jul 18, 2024
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
morrySnow
reviewed
Jul 18, 2024
| @Override | ||
| public List<Rule> buildRules() { | ||
| return ImmutableList.of( | ||
| RuleType.INIT_MATERIALIZATION_HOOK_FOR_FILE_SINK.build(logicalFileSink() |
Contributor
There was a problem hiding this comment.
could use any().when(LogicalTableSink.class::isInstanceOf)
…sert and insert overwrite dml
seawinde
force-pushed
the
ddl_enable_mv_rewrite
branch
from
ac8d705 to
ad9f822
Compare
Contributor
Author
|
this is replaced by #38115, this branch rebase wrongly, it doesn't modify the meta etc |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Support querying rewrite by materialized view when DML such as insert and insert overwrite , into outfile etc.
this is default enable, you can disable it by
set enable_dml_materialized_view_rewrite = falseand the
enable_materialized_view_rewriteis only control DQL.It would not rewrite by materialized view when query sql use external table.
If you want rewrite by materialized view when use external table in DML,
you should
set enable_dml_materialized_view_rewrite_when_base_table_unawareness = true;this is default false.
this pr is replaced by #38115