Release notes 2.1.1

[yiguolei]

Release Note 2.1.0

Behavior Changed

1. Change float type output format to improve float type serialization performance. [behavior change](output) change float output format  #32049
2. Change system table value functions active_queries(), workload_groups() to system tables. [Refactor](executor)Add information_schema.workload_groups (#32195) #32314
3. Disable show query/load profile stmt because there are not so many developers use it and the pipeline and pipelinex engine not support it. [opt](profile) Disable show query/load profile stmt #32467
4. Upgrade arrow flight version to 15.0.2 to fix some bugs, so that please use ADBC 15.0.2 version to access Doris. [branch-2.1](thirdparty) upgrade arrow to 15.0.2 #32827.

Upgrade Problem

1. BE will core when rolling pgrade problem from 2.0.x to 2.1.x [Fix](compress) Fix occasional crushes when serializing blocks  #32672 [Fix](smooth-upgrade) Fix right function incompatibility when upgrade from 2.0 to 2.1 #32444 [Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 #32162
2. JDBC Catalog will have query errors when rolling grade rom 2.0.x to 2.1.x. [fix](jdbc catalog) Fix query errors without jdbc pool default value on only BE upgrade #32618

New Feature

1. Enable column auth by default. [enhance](auth)enable col auth #32659
2. Get correct cores for pipeline and pipelinex engine when running within docker or k8s. [feature](cpu cores)get the cores when running within a cgroup. #32370
3. Support read parquet int96 type. [fix](ParquetReader) Fix Parquet Reader to read int96 parquet type problem #32394
4. Enable proxy protocol to support IP transparency. Using this protocol, IP transparency for load balancing can be achieved, so that after load balancing, Doris can still obtain the client's real IP and implement permission control such as whitelisting. https://github.com/apache/doris/pull/32338/files
5. Add workload group queue related columns for active_queries system table. Uses could use this system to monitor the workload queue usage. [Improvement](executor)Add queue columns for active_queries #32259
6. Add new system table backend_active_tasks to monitor the realtime query statics on every BE. [Improvement]Add schema table backend_active_tasks #31945
7. Add ipv4 and ipv6 support for spark-doris connector. [fix](ip) conversion of ipv4 or ipv6 to arrow type #32240
8. Add inverted index support for CCR. [improvement](binlog)Support inverted index in CCR (#31743) #32101
9. Support select experimental session variable. [chore](config) support select experimental session variable #31837
10. Support materialized view with bitmap_union(bitmap_from_array()) case. [Feature](materialized-view) support mv with bitmap_union(bitmap_from_array()) case #31962
11. Support partition prune for HIVE_DEFAULT_PARTITION. [fix](hive) support partition prune for _HIVE_DEFAULT_PARTITION_ #31736
12. Support function in set variable statement. [opt](variable) user variable support expression rather than literal #32492
13. Support arrow serialization for varint type. [Serde](Variant) support arrow serialization for varint type #32809

Optimization

1. Auto resume routine load when be restart or during upgrade. And keep the routine load stable. [opt](routine-load) enhance auto resume to keep routine load stable #32239
2. Routine Load: optimize allocate task to be algorithm for load balance. [opt](routine-load) optimize allocate task to be algorithm for load balance #32021
3. Spark Load: update spark version for spark load to resolve cve problem. [refactor](spark load) update spark version for spark load to resolve cve problem #30368
4. Skip cooldown if the tablet is dropped. [enhance](Cooldown) Skip cooldown if the tablet is dropped #32079
5. Support using workload group to manage routine load. [Improvement](executor)Routine load support workload group #31671
6. [MTMV ]Improve the performance for query rewritting by materialized view. [improvement](mtmv) Improve the performance for query rewritting by materialized view #31886
7. Reduce jvm heap memory consumed by profiles of BrokerLoadJob. [opt](fe) Reduce jvm heap memory consumed by profiles of BrokerLoadJob #31985
8. Imporve the high QPS query by speed up PartitionPrunner. [enhancement](Nereids) Speedup PartitionPrunner #31970
9. Reduce duplicated memory consumption for column name and column path for schema cache. [Improve](TabletSchemaCache) reduce duplicated memory consumption for column name and column path #31141
10. Support more join types for query rewriting by materialized view such as INNER JOIN、LEFT OUTER JOIN、RIGHT OUTER JOIN、FULL OUTER JOIN、LEFT SEMI JOIN、RIGHT SEMI JOIN、LEFT ANTI JOIN、RIGHT ANTI JOIN [improvement](mtmv) Support more join types for query rewriting by materialized view (#32685) #32909

Bugfix

1. Do not push down topn-filter through right/full outer join if the first orderkey is nulls first. [fix](Nereids) do not push down topn-filter through right/full outer join if the first orderkey is nulls first #32633
2. Fix memory leak in Java UDF [bug](udaf) fix memory leak in the java udaf #32630.
3. If some odbc tables use the same resource, and restore not all odbc tables, it will not retain the resource.
   and check some conf for backup/restore [fix](restore) fix restore odbc resource bug #31989
4. Fold constant will core for variant type. [bug](fold) fix fold constant core dump with variant type #32265
5. Routine load will pause when transaction fail in some cases. [fix](routine-load) avoid routine load pause for check transaction status fail #32638
6. the result of left semi join with empty right side should be false instead of null. [fix](join) the result of left semi join with empty right side should be false instead of null #32477
7. Fix core when build inverted index for a new column with no data. [fix](build index) fix core when build index for a new column which w… #32669
8. Fix be core caused by null-safe-equal join. [fix](join) core caused by null-safe-equal join #32623
9. Partial update: fix data correctness risk when load delete sign data into a table with sequence col. [fix](partial update) fix data correctness risk when load delete sign data into a table with sequence col #32574
10. Select outfile: Fix the column type mapping in the orc/parquet file format. [Fix](Outfile) Fix the column type mapping in the orc/parquet file format #32281
11. Fix BE core during restore stage. [fix](snapshot-loader) Fix be crash caused by deref end() iterator #32489
12. Use array_agg func after other agg func like count, sum, may make be core. [FIX](array_agg) fix array agg with other agg function #32387
13. Variant type should always nullable or there will some bugs. [Fix](Variant) fix variant with not null #32248
14. Fix the bug of handling empty blocks in schema change. [fix](schema-change) fix the bug of handling empty blocks in schema change #32396
15. Fix BE will core when use json_length() in some cases. [bugfix]json_length() BE crash fix #32145
16. [iceberg] Fix error when query iceberg table using date cast predicate [fix](iceberg) fix error when query iceberg table using date cast predicate #32194
17. Fix some bugs when build inverted index for variant type. [Fix](Variant) fix inverted index lost due to wrong column unique id #31992
18. Wrong result of two or more map_agg functions in query. [fix](agg) wrong result of two or more map_agg functions in query #31928
19. Fix wrong result of money_format function. [fix](money_format) fix money_format #31883
20. Fix connection hang after too many connections. [fix](connections) fix connection hang after too many connections #31594

[github-actions]

We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!