[improvement](pipeline) task group scan entity #19924 #27039
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Since the default column separator for tvf reading csv format has changed, these cases need to be fixed.
…pache#26106) add external_docker tag for some external case
…enclose or escape (apache#25816)
Fix select table tablet not effective, table distributed by random. If tabletID specified in query does not exist in this partition, skip scan partition.
…hema (apache#25844) There is FE config `infodb_support_ext_catalog`, the default is false. Which means that the tables in `information_schema` database will not return info of external catalog. Because if there are too many external catalogs in Doris with lots of db/tbl (like running p0 regression tests), querying infomation_schema db will take a long time and may causing rpc timeout. And there is an unresolved issue that if thrift rpc timeout, the BE may be crashed in ASAN mode. So to avoid this issue(not fix yet), this PR mainly changes: if `infodb_support_ext_catalog` is false, 1. query info of external catalog in information_schema db is not allowed, such as show database like "external_catalog"; show tables like "xxx" 2. select * from information_schema.tbl will not contains external catalogs' info 3. For external p0 regression test pipeline, set `infodb_support_ext_catalog` to true to run the tests related to external catalog
Add timeout config for sync
…timeunit is const (apache#25824) this PR apache#22602 have check function. only support date_trunc(column, const), so the second must be const literal and no need to check time unit every row.
…ce_Clickbench_ClickbenchNew apache#25827 (apache#26219)
…ould not be pushed down (apache#25901) (apache#26215)
…pache#25938) (apache#26222) we put bound expr into unbound group by list by mistake. This will lead to bind twice on some exprssion. Since binding is not idempotent, below exception will be thrown for sql ```sql select k5 / k5 as nu, sum(k1) from test group by nu order by nu nulls first ``` ``` Caused by: org.apache.doris.nereids.exceptions.AnalysisException: Input slot(s) not in child's output: k5#5 in plan: LogicalProject[176] ( distinct=false, projects=[(cast(k5#5 as DECIMALV3(16, 10)) / k5#5) AS `nu`apache#14, sum(k1)apache#15], excepts=[] ), child output is: [nu#16, sum(k1)apache#15] plan tree: LogicalProject[176] ( distinct=false, projects=[(cast(k5#5 as DECIMALV3(16, 10)) / k5#5) AS `nu`apache#14, sum(k1)apache#15], excepts=[] ) +--LogicalAggregate[168] ( groupByExpr=[nu#16], outputExpr=[nu#16, sum(k1#1) AS `sum(k1)`apache#15], hasRepeat=false ) +--LogicalProject[156] ( distinct=false, projects=[k1#1, (cast(k5#5 as DECIMALV3(16, 10)) / k5#5) AS `nu`apache#16], excepts=[] ) +--LogicalOlapScan ( qualified=default_cluster:regression_test_nereids_syntax_p0.test, indexName=test, selectedIndexId=503229, preAgg=OFF, Aggregate function sum(k1) contains key column k1. ) at org.apache.doris.nereids.rules.analysis.CheckAfterRewrite.checkAllSlotReferenceFromChildren(CheckAfterRewrite.java:108) ~[classes/:?] ```
…n load multi table (apache#25762) (apache#26223)
…contains complex where conditions apache#23874 (apache#26212)
…y time checkpoint mysql load (apache#26031) (apache#26139)
…an in limit with where scan (apache#25952) (apache#26815)
… nullable columns without default value apache#26776 (apache#26848)
… reporting correctly to TeamCity (apache#26606) (apache#26871)
…apache#26890) backport apache#26435 Improve the accuracy of sample stats collection. For non distribution columns, use `n*d / (n - f1 + f1*n/N)` where `f1` is the number of distinct values that occurred exactly once in our sample of n rows (from a total of N), and `d` is the total number of distinct values in the sample. For distribution columns, use `ndv(n) * fraction of tablets sampled` for NDV. For very large tablet to sample, use limit to control the total lines to scan (for non key column only, because key column is sorted and will be inaccurate using limit).
…tatement is a point query in OriginPlanner apache#26881 (apache#26900)
…pache#26926) Co-authored-by: zhangstar333 <87313068+zhangstar333@users.noreply.github.com>
…not in tablet schema apache#26990 (apache#26991)
…roovy (apache#26925) Co-authored-by: stephen <hello-stephen@qq.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#19924