2.1.9 Release Note

[yiguolei]

Behavior Changes

• The SQLHash in the audit log is now accurately calculated based on the currently executing SQL, instead of all SQLs in the same query request. [fix](audit) fix sql has same sqlhash in one request #48242
  The ColumnLabelName returned in the query now exactly matches the input in the SQL. [fix](Nereids) let anonymous alias same as user input #47093

• Now all variables set in user properties have a higher priority than those set at the session level. [enhance](auth)The priority of attributes is higher than session variable #47185

New Features

Storage Management

• Renaming partition columns is prohibited. [chore](rename) Forbid renaming partition columns since this is a buggy feature #47596

Others

• New metrics for the number of Catalogs, Databases, and Tables are added to the FE monitoring metrics. [feature](metrics) add catalog/db/table num metrics #47891

Improvements

Inverted Index

• Inverted index is supported for ARRAY within VARIANT. [InvertedIndex](Variant) support inverted index for array type in variant #47688
  Performance indicators of the inverted index for each filtering condition are displayed in the Profile. [feature](inverted index) Add profile statistics for each condition in inverted index filters #47504

Query Optimizer

• Select * is supported in aggregate queries if the lower-level relation only outputs aggregate key columns. branch-2.1: [opt](Nereids) support bind asterisk in aggregate #48006

Storage Management

• CCR optimizes the efficiency of binlog recycling, small file transfer efficiency, and enhances the robustness in a chaotic environment. [feat](binlog) Speed binlog gc by locked binlogs #47547 [fix](restore) Make the DirMoveTask idempotent. #47313 [feat](clone) Speed clone tablet via batch small file downloading #45061
• The error prompts for imports are improved to make them more specific. [improve](load) print error string in local fs error messages #47918 [improve](move-memtable) improve error log and message for "not enough streams" #47470

Bug Fixes

Lake-House Integration

• Fixed the issue that the krb5.conf path cannot be correctly configured on the BE side. [fix](jni) pass krb5 conf to jni #47679
• Retries of the SELECT OUFILE statement are prohibited to avoid repeated data export. [fix](Outfile) Set retryTime to one while executing outfile statement. #48095
• Fixed the issue that Paimon tables cannot be accessed via the JAVA API. [Fix](catalog)Fixes query failures for Paimon tables stored in Kerberized HDFS #47192
• Fixed the issue that data cannot be written to Hive tables with the storage location s3a://. [fix](hive) Incorrect location conversion for target path #47162
• Fixed the issue that the Comment field of Catalog is not persisted. [fix](external catalog) Persisting the External Catalog comment field #46946
• Fixed the issue of class loading leakage on the JDBC BE side in certain cases. [fix](jdbc catalog) Change BE jdbc Driver loading to Java code #46912
• Fixed the issue that the JDBC Catalog cannot use the high-version Clickhouse JDBC Driver. [opt](jdbc catalog) Compatible with higher ClickHouse JDBC Driver versions #46026
• Fixed the issue that reading Iceberg Position Delete in certain cases causes the BE to crash. [fix](iceberg) Don't prematurely erase DeleteRows in reading iceberg table with position delete #47977
• Fixed the issue of incorrect data reading from Max Compute tables in the case of multiple partition columns. [fix](maxcompute)fix maxcompute partition column schema order #48325
• Fixed the issue of incorrect reading of complex column types in Parquet in certain cases. [fix](parquet)Fix data column and null map column not equal when reading Parquet complex type cross-page data #47734

Inverted Index

• Fixed the issue of incorrect handling of null values in the inverted index of ARRAY type. [fix](array index) Fix index build failure when all array column data is null #48231
• Fixed the issue of exceptions when executing BUILD INDEX on a newly added column. [fix](index build) Correct inverted index behavior after dynamically adding a column #48389
• Fixed the issue that the index of special character UTF8 encoding is truncated, resulting in incorrect results. [fix] (inverted index) Fix UTF-8 4-byte truncation issue and add configuration to control correct term writing #48657

Semi-structured Data Types

• Fixed the issue that the array_agg function crashes in special cases. [fix](array_avg) fix core for array_avg #46927
• Fixed the issue that the stream load import of JSON type crashes due to incorrect setting of the chunk parameter. [fix](load) Fix import failure when the stream load parameter specifies Transfer-Encoding:chunked #48196

Query Optimizer

• Fixed the issue that keyword functions such as current_date nested within time functions cannot be subjected to constant folding. [fix](Nereids) fix fold constant of time acquired functions #47288
• Fixed the issue of incorrect results related to non-deterministic functions. [fix](nereids) fix merge project contains non foldable expression #48321
• Fixed the issue that create table like cannot be executed when the original table has an on update column attribute. [enhancement](nereids)remove dependence on old CreateTableStmt for CreateTableLikeCommand #48007
• Fixed the issue that the materialized view of a directly queried aggregate model table may generate unexpected planning errors. [fix](mv)Fix direct select mv pre agg bug. #47658
• Fixed the issue that PrepareStatement throws an exception due to internal ID overflow. [BugFix](PreparedStatement) fix stmtId overflow #47950

Query Execution Engine

• Fixed the issue that queries on system tables may get stuck or encounter null pointer exceptions. [fix](schema scan) Fix invalid pointer access #48370
• The lead/lag function now supports the double type. [enhance](function) add more signatures for lag/lead fucntion #47940
• Fixed the issue that queries report errors when the number of case when conditions exceeds 256. [Bug](function) fix wrong result when case when have more than 256 conditions and the… #47179
• Fixed the issue of incorrect results of the str_to_date function when there are spaces. [fix](function) fix error result when STR_TO_DATE input all space (#4… #48920
• Fixed the issue of incorrect results when the split_part function encounters || during constant folding. branch-2.1: [fix](Nereids) fix split part with regex not exist in source string #48895 #48910
• Fixed the issue of incorrect results of the log function. [fix](Nereids) fix log bug and add more numeric arithmetic function cases #47228
• Fixed the issue that using array/map functions within lambda expressions causes a core error. https://github.com/apache/doris/pull/49140/files

Storage Management

• Fixed the possible issue of dirty memory writes when importing aggregate tables. [Fix]Fix double-free when memtable agg #47523
• Fixed the occasional coredump issue during mow import when memory is tight. [Fix](memory) Add try catch block for Segment::load_pk_index_and_bf #47715
• Fixed the issue that repeated keys may occur in mow during BE restart and schemachange. [Fix](merge-on-write) should calculate delete bitmaps between segments before skip if tablet is in NOT_READY state in flush phase #48056 [Fix](merge-on-write) should re-calculate delete bitmaps between segments if BE restart before publish #48775
• Fixed the issues related to group commit, global column update enablement, and memtable forward movement. [fix](group commit) group commit failed if enable global enable_unique_key_partial_update #48120 [fix](group commit) replay wal failed when enable global enable_memtable_on_sink_node #47968

Permission Management

• When using LDAP, the PartialResultException will no longer be thrown. [fix](auth)ldap template ignore partial result exception #47858

[github-actions]

We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!