[enhance](Cooldown) add SessionVariable for enableCooldownReplicaAffinity #41354
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
github-actions
bot
removed
the
approved
Merged
6 tasks
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
run buildall |
… rather than convert_block (apache#41293) ## Proposed changes the convert_block is empty firstly, after execute exprs will be filled. so convert_block maybe different with block. so when send data we still need use block. <!--Describe your changes.-->
…map from cache failed (apache#41309) ## Proposed changes Issue Number: close #xxx To accelerate the speed of sync latest delete bitmap, apache#35856 try to get the delete bitmap from `CloudTxnDeleteBitmapCache` first. In the following situation, compaction may get empty delete bitmap and cause duplicate key: 1. compaction started 2. several load succeed during the compaction 3. compaction finished data merging and start to calculate delete bitmap generated by latest load tasks 4. compaction try to sync rowset and delete bitmap, it get delete bitmap first from `CloudTxnDeleteBitmapCache` 5. `CloudTxnDeleteBitmapCache::get_delete_bitmap()` can get txn infos from it's inner map, but cache missed when it try to get delete bitmap from LRU cache, it don't report error but returned an empty delete bitmap 6. compaction used wrong delete bitmap, duplicate key occured.
…ile buffer memory usage (apache#40597) Track s3 file buffer memory usage with thread local tracker, so that memory usage will be specified to detail.
After apache#40924, `get_error_log_file_path` might be accessed concurrently, leading to error_log_file being closed multiple times. Therefore, a lock was added to solve this issue.
…ache#41319) In previous implementations, binlog files would be linked according to making snapshot requests. However, sometimes not all requests can be executed directly. For example, when a certain version in missing_version does not exist, it will fallback to the full snapshot. Therefore, it is correct to link binlog files according to consistent rowsets.
## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
…sumption. (apache#41203) For string type columns, use xxhash_64 to transfer column value to an integer, and then calculate the NDV based on the integer hash value. In this case, we can reduce the memory cost of sample analyze and improve the performance. For example, l_comment column of TPCH 100G lineitem table. The memory cost to calculate its NDV is reduced to 8GB from 22GB
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
list partitions support NULL columns as partition columns
Before:
```
mysql> select auto_partition_name('list','wyx','null');
+-------------------------------------------------------------+
| auto_partition_name('list','wyx','null') |
+-------------------------------------------------------------+
| NULL |
+-------------------------------------------------------------+
```
Now:
```
mysql> select auto_partition_name('list','wyx','null');
+-------------------------------------------------------------+
| auto_partition_name('list','wyx','null') |
+-------------------------------------------------------------+
| pwyx3X |
+-------------------------------------------------------------+
```
…auto_partition_name (apache#41372) Apply check in constant folding path for function auto_partition_name
before
```
mysql [(none)]>select sec_to_time(time_to_sec(cast('2024-09-24 16:00:00' as varchar)));
+---------------------------------------------------------------------------------------+
| sec_to_time(time_to_sec(cast(cast('2024-09-24 16:00:00' as VARCHAR(65533)) as TIME))) |
+---------------------------------------------------------------------------------------+
| 16:00:00 |
+---------------------------------------------------------------------------------------+
mysql [(none)]>select ifnull(sec_to_time(time_to_sec(cast('2024-09-24 16:00:00' as varchar))), cast(300 as time));
+--------------------------------------------------------------------------------------------------------------------------------------------------+
| ifnull(cast(sec_to_time(time_to_sec(cast(cast('2024-09-24 16:00:00' as VARCHAR(65533)) as TIME))) as DOUBLE), cast(cast(300 as TIME) as DOUBLE)) |
+--------------------------------------------------------------------------------------------------------------------------------------------------+
| 57600000000 |
+--------------------------------------------------------------------------------------------------------------------------------------------------+
```
now
```
mysql [(none)]>select ifnull(sec_to_time(time_to_sec(cast('2024-09-24 16:00:00' as varchar))), cast(300 as time));
+------------------------------------------------------------------------------------------------------------------+
| ifnull(sec_to_time(time_to_sec(cast(cast('2024-09-24 16:00:00' as VARCHAR(65533)) as TIME))), cast(300 as TIME)) |
+------------------------------------------------------------------------------------------------------------------+
| 16:00:00 |
+------------------------------------------------------------------------------------------------------------------+
```
Add copy into regression test case.
## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
…pache#40770) It's better to keep the same with other subnets as 168.x.x.x Co-authored-by: stephen <hello-stephen@qq.com>
## Proposed changes Issue Number: close #xxx <!--Describe your changes.--> Co-authored-by: stephen <hello-stephen@qq.com>
…41432) 1. Set enable_phrase_query_sequential_opt = true to optimize conjunction matching in sequential phrase queries. 2. For example, match_phrase "赵丽颖 中国 ~20+" ensures that "赵丽颖" appears consecutively, and "中国" also appears consecutively, while maintaining the semantics of sequential phrase queries.
…pache#41648) ## Proposed changes change bloom filter DCHECK to error status
## Proposed changes make some agg function restrict
set var hint need to be enable to use before analyze, so it need to be set when parsing sql now it would set twice when parse and begin of analyze
This pr refactors the PredicatePropagation module and adds support for predicate deduction, including: 1. Support for predicate deduction of like, not in, !=; 2. Support for predicate deduction of abs(b)=1 for a=b and abs(a)=1; 3. Support for transitive deduction of non-equivalent relations, for example, a>b b>1 leads to a>1. 4. Deleted useless predicates. But still has something to do in predicate inference: 1. support expr in infer predicate, e.g. abs(t1.c1)>abs(t2.c2) and abs(t1.c1)<1 2. need to add expr qualifier info, to determine whether abs(t1.c1) and abs(t2.c2) is from same table.
…d segment failed (apache#41608) ## Proposed changes 1. If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again. 2. Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached. --------- Co-authored-by: yiguolei <yiguolei@gmail.com>
this resize function called shrink function which defined in IColumn, and function set_num_rows in Block.cpp call shrink to make column rows changed, maybe less or more. In this situation if we would make less rows which like cut the rows, so we should also make nested data in array/map resize to the less number of rows , otherwise we will meet exception
## Proposed changes Issue Number: close #xxx <!--Describe your changes.--> Co-authored-by: stephen <hello-stephen@qq.com>
…pache#40528)" (apache#41652) Cause mem overflow and use more
…pache#41670) ## Proposed changes fix wrong check on BloomFilterFuncBase::merge introduced by apache#41648
…n happend in FE DeleteJob execution (apache#41672) ## Proposed changes Fail task should also count down for the count down latch to prevent job stuck.
…operators (apache#41602) ## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
cjj2010
requested review from
BiteTheDDDDt,
CalvinKirs,
gavinchou,
morningman and
platoneko
as code owners
Contributor
Author
|
run buildall |
Contributor
Author
|
Submission error, transfer to another PR #41741 |
dataroaring
pushed a commit
to apache/doris-website
that referenced
this pull request
Oct 29, 2024
…#1157) # Versions - [x] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 # Languages - [x] Chinese - [x] English # Functional PR [pr_link](apache/doris#41354)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add SessionVariable for enableCooldownReplicaAffinity, Configure document link
Proposed changes
Issue Number: close #xxx