From a88c24322b2b3320418dc8390af0d563e3686d20 Mon Sep 17 00:00:00 2001 From: ti-srebot Date: Mon, 19 Jul 2021 10:31:13 +0800 Subject: [PATCH 01/24] update v4.0.14 release notes at 2021-07-19T10:31:12 Signed-off-by: ti-srebot --- releases/4.0.14.md | 148 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 releases/4.0.14.md diff --git a/releases/4.0.14.md b/releases/4.0.14.md new file mode 100644 index 0000000000000..698f0ff48b45a --- /dev/null +++ b/releases/4.0.14.md @@ -0,0 +1,148 @@ +--- +title: tidb v4.0.14 Release Notes +category: Releases +aliases: ['/docs/dev/releases/v4.0.14/'] +--- + +# tidb v4.0.14 Release Notes + +Release date: July 19, 2021 + +TiDB version: v4.0.14 + +## Bug Fixes + ++ TiDB + + - Generate correct number of rows when all agg funcs are pruned [#26039](https://github.com/pingcap/tidb/pull/26039) + - Executor: fix ifnull bug when arg is enum/set [#26035](https://github.com/pingcap/tidb/pull/26035) + - Fix wrong aggregate pruning for some cases [#26033](https://github.com/pingcap/tidb/pull/26033) + - Fix incorrect result of set type for merge join [#26032](https://github.com/pingcap/tidb/pull/26032) + - Expression: fix IN expr critical bug [#25665](https://github.com/pingcap/tidb/pull/25665) + - Fix panic when 'select ... for update' works on a join operation and the join uses partition table [#25501](https://github.com/pingcap/tidb/pull/25501) + - Fix the issue point get cached plan of prepared statement is incorrectly used by in transaction point get statement. [#24764](https://github.com/pingcap/tidb/pull/24764) + ++ TiFlash + + - Fix potential npe in executeTS during DAG compile [#2377](https://github.com/pingcap/tics/pull/2377) + - Fix the panic issue that occurs when the read load is heavy [#2280](https://github.com/pingcap/tics/pull/2280) + - Fix the issue that TiFlash nodes keep restart because of split failure. [#2218](https://github.com/pingcap/tics/pull/2218) + - Fix the bug that TiFlash can not GC delta data under rare case [#2183](https://github.com/pingcap/tics/pull/2183) + - Fix the potential concurrency problem when clone the shared delta index. [#2032](https://github.com/pingcap/tics/pull/2032) + - Fix the bug that incomplete data may make TiFlash fail to restart [#2002](https://github.com/pingcap/tics/pull/2002) + - Fix the problem that old dmfile is not removed atomically [#1924](https://github.com/pingcap/tics/pull/1924) + - Fix the problem that TiFlash will crash when executing SUBSTRING function with specific argument [#1914](https://github.com/pingcap/tics/pull/1914) + - Fix the issue that cast int as time function in TiFlash's coprocessor may produce wrong result [#1892](https://github.com/pingcap/tics/pull/1892) + ++ Tools + + - BR + + * Fix parquet parse when parse decimal type [#1276](https://github.com/pingcap/br/pull/1276) + * Fix the bug that lightning returns EOF error when CSV file without '\r\n' at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) + * Fix the bug that lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) + +## Compatibility Changes + ++ TiDB + + - For users upgrading from TiDB 4.0, the value of `tidb_multi_statement_mode` is now `OFF`. It is recommended to use the multi-statement feature of your client library instead, see the documentation on `tidb_multi_statement_mode` for additional details. [#25749](https://github.com/pingcap/tidb/pull/25749) + +## Improvements + ++ Tools + + - BR + + * Refactor storage.ExternalStorage interface to support compress [#1317](https://github.com/pingcap/br/pull/1317) + +## Others + ++ TiDB + + - Fix a bug which is caused by prior bug fix PR [#26274](https://github.com/pingcap/tidb/pull/26274) + - Change the lock record into put record for the index keys using point/batch point get for update read. [#26223](https://github.com/pingcap/tidb/pull/26223) + - Load: fix load data with non-utf8 can succeed [#26142](https://github.com/pingcap/tidb/pull/26142) + - TiDB now supports the mysql system variable `init_connect` and associated functionality. [#26031](https://github.com/pingcap/tidb/pull/26031) + - Planner: support stable result mode [#26003](https://github.com/pingcap/tidb/pull/26003) + - Enlarge the variable tidb_stmt_summary_max_stmt_count default value from 200 to 3000 [#25872](https://github.com/pingcap/tidb/pull/25872) + - Enable the pushdown of builtin function `json_unquote()` to TiKV. [#25721](https://github.com/pingcap/tidb/pull/25721) + - Planner: select distinct should bypass batchget [#25532](https://github.com/pingcap/tidb/pull/25532) + - Important security issue for handling ALTER USER statements [#25347](https://github.com/pingcap/tidb/pull/25347) + - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status. [#24918](https://github.com/pingcap/tidb/pull/24918) + - Handle a potential statistic object's memory leak when HTTP api is used [#24650](https://github.com/pingcap/tidb/pull/24650) + - Don't let SPM be affected by charset [#23295](https://github.com/pingcap/tidb/pull/23295) + - Time: parse datatime won't truncate the reluctant string. [#22260](https://github.com/pingcap/tidb/pull/22260) + - Fix a bug that `select into outfile` has no data with year column type [#22185](https://github.com/pingcap/tidb/pull/22185) + ++ TiKV + + - ```release-note [#10572](https://github.com/tikv/tikv/pull/10572) + - ```release-note [#10532](https://github.com/tikv/tikv/pull/10532) + - ```release-note [#10531](https://github.com/tikv/tikv/pull/10531) + - ```release-note [#10517](https://github.com/tikv/tikv/pull/10517) + - ```release-note [#10504](https://github.com/tikv/tikv/pull/10504) + - Ensure panic output is flushed to the log [#10488](https://github.com/tikv/tikv/pull/10488) + - ```release-note [#10462](https://github.com/tikv/tikv/pull/10462) + - ```release-note [#10433](https://github.com/tikv/tikv/pull/10433) + - Copr: fix the wrong arguments type of json_unquote [#10425](https://github.com/tikv/tikv/pull/10425) + - ```release-note [#10400](https://github.com/tikv/tikv/pull/10400) + - ```release-note [#10395](https://github.com/tikv/tikv/pull/10395) + - For release-5.0 [#10360](https://github.com/tikv/tikv/pull/10360) + - ```release-note [#10320](https://github.com/tikv/tikv/pull/10320) + - ```release-note [#10302](https://github.com/tikv/tikv/pull/10302) + - ```release-note [#10291](https://github.com/tikv/tikv/pull/10291) + - ```release-note [#10279](https://github.com/tikv/tikv/pull/10279) + - Approximate split range evenly [#10275](https://github.com/tikv/tikv/pull/10275) + - BR now supports S3-compatible storage using virtual-host addressing style. [#10242](https://github.com/tikv/tikv/pull/10242) + - ```release-note [#10147](https://github.com/tikv/tikv/pull/10147) + - Add metrics to see pending pd heartbeats number [#10008](https://github.com/tikv/tikv/pull/10008) + - Change the default merge-check-tick-interval to 2 to speed up the retry of merge process [#9676](https://github.com/tikv/tikv/pull/9676) + - Fix an issue that split may panic and corrupt the metadata if the split process is too slow and region merge is open. [#9584](https://github.com/tikv/tikv/pull/9584) + ++ PD + + - ```release-note [#3882](https://github.com/pingcap/pd/pull/3882) + - ```release-note [#3858](https://github.com/pingcap/pd/pull/3858) + - ```release-note [#3854](https://github.com/pingcap/pd/pull/3854) + - ```release-note [#3825](https://github.com/pingcap/pd/pull/3825) + - ```release-note [#3809](https://github.com/pingcap/pd/pull/3809) + - ```release-note [#3797](https://github.com/pingcap/pd/pull/3797) + - ```release-note [#3790](https://github.com/pingcap/pd/pull/3790) + - ```release-note [#3773](https://github.com/pingcap/pd/pull/3773) + - ```release-note [#3761](https://github.com/pingcap/pd/pull/3761) + - ```release-note [#3718](https://github.com/pingcap/pd/pull/3718) + - ```release-note [#3703](https://github.com/pingcap/pd/pull/3703) + - ```release-note [#3680](https://github.com/pingcap/pd/pull/3680) + ++ Tools + + - BR + + * Fix the issue that restore from `mysql` schema would failed. [#1142](https://github.com/pingcap/br/pull/1142) + * No realease note [#1131](https://github.com/pingcap/br/pull/1131) + * Fix the issue that lightning panic due to batch kv greater than 4.0g. [#1128](https://github.com/pingcap/br/pull/1128) + * Speed up restore by merging small backup files. [#655](https://github.com/pingcap/br/pull/655) + + - Dumpling + + * Always split TiDB v3.* tables through tidb rowid to save TiDB's memory. [#306](https://github.com/pingcap/dumpling/pull/306) + * When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself. [#287](https://github.com/pingcap/dumpling/pull/287) + + - TiCDC + + * ```release-note [#2258](https://github.com/pingcap/ticdc/pull/2258) + * ```release-note [#2215](https://github.com/pingcap/ticdc/pull/2215) + * Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) + * Better err msg when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) + * Add `capture-session-ttl` in cdc server config [#2169](https://github.com/pingcap/ticdc/pull/2169) + * Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) + * Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) + * Make sorter IO errors more user-friendly. [#1976](https://github.com/pingcap/ticdc/pull/1976) + * Fix Unified Sorter memory consumption when tables are many. [#1957](https://github.com/pingcap/ticdc/pull/1957) + * Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses. [#1945](https://github.com/pingcap/ticdc/pull/1945) + * Add concurrency limit to the region incremental scan in kv client. [#1926](https://github.com/pingcap/ticdc/pull/1926) + * Add metrics for table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) + * Owner: When tikv_gc_life_time is greater than gcttl, use tikv_gc_life_time to calculate the lower bound of gcSafePoint. [#1871](https://github.com/pingcap/ticdc/pull/1871) + * Reduce memory malloc in sort heap to avoid too much CPU overhead. [#1862](https://github.com/pingcap/ticdc/pull/1862) + * Fix a bug about resolved ts stopped when move a table. [#1827](https://github.com/pingcap/ticdc/pull/1827) From 727c29682a6ededf32061011950bf67f924c2168 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 10:48:10 +0800 Subject: [PATCH 02/24] add toc --- TOC.md | 1 + releases/{4.0.14.md => release-4.0.14.md} | 40 +++++++++++------------ releases/release-notes.md | 1 + 3 files changed, 21 insertions(+), 21 deletions(-) rename releases/{4.0.14.md => release-4.0.14.md} (95%) diff --git a/TOC.md b/TOC.md index c6f1eb4005d73..5fc74ac8cea9c 100644 --- a/TOC.md +++ b/TOC.md @@ -557,6 +557,7 @@ + [5.0 GA](/releases/release-5.0.0.md) + [5.0.0-rc](/releases/release-5.0.0-rc.md) + v4.0 + + [4.0.14](/releases/release-4.0.14.md) + [4.0.13](/releases/release-4.0.13.md) + [4.0.12](/releases/release-4.0.12.md) + [4.0.11](/releases/release-4.0.11.md) diff --git a/releases/4.0.14.md b/releases/release-4.0.14.md similarity index 95% rename from releases/4.0.14.md rename to releases/release-4.0.14.md index 698f0ff48b45a..e5a42beca8589 100644 --- a/releases/4.0.14.md +++ b/releases/release-4.0.14.md @@ -1,14 +1,26 @@ --- -title: tidb v4.0.14 Release Notes -category: Releases -aliases: ['/docs/dev/releases/v4.0.14/'] +title: TiDB 4.0.14 Release Notes --- -# tidb v4.0.14 Release Notes +# TiDB 4.0.14 Release Notes -Release date: July 19, 2021 +Release date: July 22, 2021 -TiDB version: v4.0.14 +TiDB version: 4.0.14 + +## Compatibility Changes + ++ TiDB + + - For users upgrading from TiDB 4.0, the value of `tidb_multi_statement_mode` is now `OFF`. It is recommended to use the multi-statement feature of your client library instead, see the documentation on `tidb_multi_statement_mode` for additional details. [#25749](https://github.com/pingcap/tidb/pull/25749) + +## Improvements + ++ Tools + + + Backup & Restore (BR) + + - Refactor storage.ExternalStorage interface to support compress [#1317](https://github.com/pingcap/br/pull/1317) ## Bug Fixes @@ -42,21 +54,7 @@ TiDB version: v4.0.14 * Fix the bug that lightning returns EOF error when CSV file without '\r\n' at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) * Fix the bug that lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) -## Compatibility Changes - -+ TiDB - - - For users upgrading from TiDB 4.0, the value of `tidb_multi_statement_mode` is now `OFF`. It is recommended to use the multi-statement feature of your client library instead, see the documentation on `tidb_multi_statement_mode` for additional details. [#25749](https://github.com/pingcap/tidb/pull/25749) - -## Improvements - -+ Tools - - - BR - - * Refactor storage.ExternalStorage interface to support compress [#1317](https://github.com/pingcap/br/pull/1317) - -## Others +## 以下 note 未分类。请将以下 note 进行分类 (New feature, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 + TiDB diff --git a/releases/release-notes.md b/releases/release-notes.md index ca02d43d472e3..e72c917b049f2 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -19,6 +19,7 @@ aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] ## 4.0 +- [4.0.14](/releases/release-4.0.14.md) - [4.0.13](/releases/release-4.0.13.md) - [4.0.12](/releases/release-4.0.12.md) - [4.0.11](/releases/release-4.0.11.md) From 3a703a8b44e579e734e74c7f7630103e955e7c5f Mon Sep 17 00:00:00 2001 From: MartinNose Date: Mon, 19 Jul 2021 12:00:10 +0800 Subject: [PATCH 03/24] update release-4.0.14.md --- releases/release-4.0.14.md | 58 +++++++++++++++++++------------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index e5a42beca8589..766c7a725c16a 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -75,43 +75,43 @@ TiDB version: 4.0.14 + TiKV - - ```release-note [#10572](https://github.com/tikv/tikv/pull/10572) - - ```release-note [#10532](https://github.com/tikv/tikv/pull/10532) - - ```release-note [#10531](https://github.com/tikv/tikv/pull/10531) - - ```release-note [#10517](https://github.com/tikv/tikv/pull/10517) - - ```release-note [#10504](https://github.com/tikv/tikv/pull/10504) + - Fix duration calculation panics on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) + - Fix wrong function cast double to double [#10532](https://github.com/tikv/tikv/pull/10532) + - ```release-note None [#10531](https://github.com/tikv/tikv/pull/10531) + - ```release-note None [#10517](https://github.com/tikv/tikv/pull/10517) + - ```release-note None [#10504](https://github.com/tikv/tikv/pull/10504) - Ensure panic output is flushed to the log [#10488](https://github.com/tikv/tikv/pull/10488) - - ```release-note [#10462](https://github.com/tikv/tikv/pull/10462) - - ```release-note [#10433](https://github.com/tikv/tikv/pull/10433) + - Avoid panic when building a snapshot twice if encryption enabled [#10462](https://github.com/tikv/tikv/pull/10462) + - ```release-note None [#10433](https://github.com/tikv/tikv/pull/10433) - Copr: fix the wrong arguments type of json_unquote [#10425](https://github.com/tikv/tikv/pull/10425) - - ```release-note [#10400](https://github.com/tikv/tikv/pull/10400) - - ```release-note [#10395](https://github.com/tikv/tikv/pull/10395) - - For release-5.0 [#10360](https://github.com/tikv/tikv/pull/10360) - - ```release-note [#10320](https://github.com/tikv/tikv/pull/10320) - - ```release-note [#10302](https://github.com/tikv/tikv/pull/10302) - - ```release-note [#10291](https://github.com/tikv/tikv/pull/10291) - - ```release-note [#10279](https://github.com/tikv/tikv/pull/10279) + - ```release-note None [#10400](https://github.com/tikv/tikv/pull/10400) + - skip clearing callback during gracefully shutdown to avoid breaking ACID in some cases [#10395](https://github.com/tikv/tikv/pull/10395) + - Fix backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) + - ```release-note None [#10320](https://github.com/tikv/tikv/pull/10320) + - ```release-note None [#10302](https://github.com/tikv/tikv/pull/10302) + - ```release-note None [#10291](https://github.com/tikv/tikv/pull/10291) + - ```release-note None [#10279](https://github.com/tikv/tikv/pull/10279) - Approximate split range evenly [#10275](https://github.com/tikv/tikv/pull/10275) - BR now supports S3-compatible storage using virtual-host addressing style. [#10242](https://github.com/tikv/tikv/pull/10242) - - ```release-note [#10147](https://github.com/tikv/tikv/pull/10147) + - Limit CDC sink memory consumption. [#10147](https://github.com/tikv/tikv/pull/10147) - Add metrics to see pending pd heartbeats number [#10008](https://github.com/tikv/tikv/pull/10008) - Change the default merge-check-tick-interval to 2 to speed up the retry of merge process [#9676](https://github.com/tikv/tikv/pull/9676) - Fix an issue that split may panic and corrupt the metadata if the split process is too slow and region merge is open. [#9584](https://github.com/tikv/tikv/pull/9584) + PD - - ```release-note [#3882](https://github.com/pingcap/pd/pull/3882) - - ```release-note [#3858](https://github.com/pingcap/pd/pull/3858) - - ```release-note [#3854](https://github.com/pingcap/pd/pull/3854) - - ```release-note [#3825](https://github.com/pingcap/pd/pull/3825) - - ```release-note [#3809](https://github.com/pingcap/pd/pull/3809) - - ```release-note [#3797](https://github.com/pingcap/pd/pull/3797) - - ```release-note [#3790](https://github.com/pingcap/pd/pull/3790) - - ```release-note [#3773](https://github.com/pingcap/pd/pull/3773) - - ```release-note [#3761](https://github.com/pingcap/pd/pull/3761) - - ```release-note [#3718](https://github.com/pingcap/pd/pull/3718) - - ```release-note [#3703](https://github.com/pingcap/pd/pull/3703) - - ```release-note [#3680](https://github.com/pingcap/pd/pull/3680) + - TiDB Dashboard: Add OIDC based SSO support [#3882](https://github.com/pingcap/pd/pull/3882) + - Reduce the conflict due to multiple scheduler running in same time [#3858](https://github.com/pingcap/pd/pull/3858) + - ```release-note None [#3854](https://github.com/pingcap/pd/pull/3854) + - Fix the issue that the scheduler may appear again even if we have already executed the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) + - ```release-note None [#3809](https://github.com/pingcap/pd/pull/3809) + - Update TiDB Dashboard to v2021.06.15.1 [#3797](https://github.com/pingcap/pd/pull/3797) + - ```release-note None [#3790](https://github.com/pingcap/pd/pull/3790) + - Fix the data race when the scheduler is started before loading TTL configurations [#3773](https://github.com/pingcap/pd/pull/3773) + - Fix pd may get panic during scattering region [#3761](https://github.com/pingcap/pd/pull/3761) + - Fix the issue that leader re-election be slow when there are many stores [#3718](https://github.com/pingcap/pd/pull/3718) + - Fix the priority problems for some kinds of operators [#3703](https://github.com/pingcap/pd/pull/3703) + - Fix the panic issue about remove evict leader scheduler from a nonexistent store [#3680](https://github.com/pingcap/pd/pull/3680) + Tools @@ -129,8 +129,8 @@ TiDB version: 4.0.14 - TiCDC - * ```release-note [#2258](https://github.com/pingcap/ticdc/pull/2258) - * ```release-note [#2215](https://github.com/pingcap/ticdc/pull/2215) + * ```release-note None [#2258](https://github.com/pingcap/ticdc/pull/2258) + * ```release-note None [#2215](https://github.com/pingcap/ticdc/pull/2215) * Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) * Better err msg when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) * Add `capture-session-ttl` in cdc server config [#2169](https://github.com/pingcap/ticdc/pull/2169) From ec9d64e7b1760108a639968d8a598605dabc2f5a Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 12:51:32 +0800 Subject: [PATCH 04/24] Apply suggestions from code review --- releases/release-4.0.14.md | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 766c7a725c16a..37c1d997aebb5 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -77,20 +77,11 @@ TiDB version: 4.0.14 - Fix duration calculation panics on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) - Fix wrong function cast double to double [#10532](https://github.com/tikv/tikv/pull/10532) - - ```release-note None [#10531](https://github.com/tikv/tikv/pull/10531) - - ```release-note None [#10517](https://github.com/tikv/tikv/pull/10517) - - ```release-note None [#10504](https://github.com/tikv/tikv/pull/10504) - Ensure panic output is flushed to the log [#10488](https://github.com/tikv/tikv/pull/10488) - Avoid panic when building a snapshot twice if encryption enabled [#10462](https://github.com/tikv/tikv/pull/10462) - - ```release-note None [#10433](https://github.com/tikv/tikv/pull/10433) - Copr: fix the wrong arguments type of json_unquote [#10425](https://github.com/tikv/tikv/pull/10425) - - ```release-note None [#10400](https://github.com/tikv/tikv/pull/10400) - skip clearing callback during gracefully shutdown to avoid breaking ACID in some cases [#10395](https://github.com/tikv/tikv/pull/10395) - Fix backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) - - ```release-note None [#10320](https://github.com/tikv/tikv/pull/10320) - - ```release-note None [#10302](https://github.com/tikv/tikv/pull/10302) - - ```release-note None [#10291](https://github.com/tikv/tikv/pull/10291) - - ```release-note None [#10279](https://github.com/tikv/tikv/pull/10279) - Approximate split range evenly [#10275](https://github.com/tikv/tikv/pull/10275) - BR now supports S3-compatible storage using virtual-host addressing style. [#10242](https://github.com/tikv/tikv/pull/10242) - Limit CDC sink memory consumption. [#10147](https://github.com/tikv/tikv/pull/10147) @@ -104,9 +95,7 @@ TiDB version: 4.0.14 - Reduce the conflict due to multiple scheduler running in same time [#3858](https://github.com/pingcap/pd/pull/3858) - ```release-note None [#3854](https://github.com/pingcap/pd/pull/3854) - Fix the issue that the scheduler may appear again even if we have already executed the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) - - ```release-note None [#3809](https://github.com/pingcap/pd/pull/3809) - Update TiDB Dashboard to v2021.06.15.1 [#3797](https://github.com/pingcap/pd/pull/3797) - - ```release-note None [#3790](https://github.com/pingcap/pd/pull/3790) - Fix the data race when the scheduler is started before loading TTL configurations [#3773](https://github.com/pingcap/pd/pull/3773) - Fix pd may get panic during scattering region [#3761](https://github.com/pingcap/pd/pull/3761) - Fix the issue that leader re-election be slow when there are many stores [#3718](https://github.com/pingcap/pd/pull/3718) @@ -129,8 +118,6 @@ TiDB version: 4.0.14 - TiCDC - * ```release-note None [#2258](https://github.com/pingcap/ticdc/pull/2258) - * ```release-note None [#2215](https://github.com/pingcap/ticdc/pull/2215) * Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) * Better err msg when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) * Add `capture-session-ttl` in cdc server config [#2169](https://github.com/pingcap/ticdc/pull/2169) From 24fc7bbf0a5ef99599f549ebb246d24b3e638b2f Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 12:51:46 +0800 Subject: [PATCH 05/24] Update releases/release-4.0.14.md --- releases/release-4.0.14.md | 1 - 1 file changed, 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 37c1d997aebb5..cfa9b7e4762e5 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -93,7 +93,6 @@ TiDB version: 4.0.14 - TiDB Dashboard: Add OIDC based SSO support [#3882](https://github.com/pingcap/pd/pull/3882) - Reduce the conflict due to multiple scheduler running in same time [#3858](https://github.com/pingcap/pd/pull/3858) - - ```release-note None [#3854](https://github.com/pingcap/pd/pull/3854) - Fix the issue that the scheduler may appear again even if we have already executed the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) - Update TiDB Dashboard to v2021.06.15.1 [#3797](https://github.com/pingcap/pd/pull/3797) - Fix the data race when the scheduler is started before loading TTL configurations [#3773](https://github.com/pingcap/pd/pull/3773) From c65400bb1d697312dd7ce278ee37da74dd56e02a Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 13:12:52 +0800 Subject: [PATCH 06/24] add grafana notes --- releases/release-4.0.14.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index cfa9b7e4762e5..1b986e39a900f 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -13,6 +13,7 @@ TiDB version: 4.0.14 + TiDB - For users upgrading from TiDB 4.0, the value of `tidb_multi_statement_mode` is now `OFF`. It is recommended to use the multi-statement feature of your client library instead, see the documentation on `tidb_multi_statement_mode` for additional details. [#25749](https://github.com/pingcap/tidb/pull/25749) + - Upgrade Grafana from version 6.1.16 to version 7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. ## Improvements From f17d27408d88b94734de55bdc9db0b8c280af109 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 13:44:30 +0800 Subject: [PATCH 07/24] refine compatibility notes --- releases/release-4.0.14.md | 49 +++++++++++++++++++------------------- 1 file changed, 24 insertions(+), 25 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 1b986e39a900f..1d25ccc364bb4 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -12,8 +12,8 @@ TiDB version: 4.0.14 + TiDB - - For users upgrading from TiDB 4.0, the value of `tidb_multi_statement_mode` is now `OFF`. It is recommended to use the multi-statement feature of your client library instead, see the documentation on `tidb_multi_statement_mode` for additional details. [#25749](https://github.com/pingcap/tidb/pull/25749) - - Upgrade Grafana from version 6.1.16 to version 7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. + - Change the default value of `tidb_multi_statement_mode` from `WARN` to `OFF` in v4.0. It is recommended to use the multi-statement feature of your client library instead. See [the documentation on `tidb_multi_statement_mode`](/system-variables.md#tidb_multi_statement_mode-new-in-v4011) for details. [#25749](https://github.com/pingcap/tidb/pull/25749) + - Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. ## Improvements @@ -51,9 +51,9 @@ TiDB version: 4.0.14 - BR - * Fix parquet parse when parse decimal type [#1276](https://github.com/pingcap/br/pull/1276) - * Fix the bug that lightning returns EOF error when CSV file without '\r\n' at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) - * Fix the bug that lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) + - Fix parquet parse when parse decimal type [#1276](https://github.com/pingcap/br/pull/1276) + - Fix the bug that lightning returns EOF error when CSV file without '\r\n' at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) + - Fix the bug that lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) ## 以下 note 未分类。请将以下 note 进行分类 (New feature, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 @@ -104,30 +104,29 @@ TiDB version: 4.0.14 + Tools - - BR + + Backup & Restore (BR) - * Fix the issue that restore from `mysql` schema would failed. [#1142](https://github.com/pingcap/br/pull/1142) - * No realease note [#1131](https://github.com/pingcap/br/pull/1131) - * Fix the issue that lightning panic due to batch kv greater than 4.0g. [#1128](https://github.com/pingcap/br/pull/1128) - * Speed up restore by merging small backup files. [#655](https://github.com/pingcap/br/pull/655) + - Fix the issue that restore from `mysql` schema would failed. [#1142](https://github.com/pingcap/br/pull/1142) + - Fix the issue that lightning panic due to batch kv greater than 4.0g. [#1128](https://github.com/pingcap/br/pull/1128) + - Speed up restore by merging small backup files. [#655](https://github.com/pingcap/br/pull/655) - Dumpling - * Always split TiDB v3.* tables through tidb rowid to save TiDB's memory. [#306](https://github.com/pingcap/dumpling/pull/306) - * When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself. [#287](https://github.com/pingcap/dumpling/pull/287) + - Always split TiDB v3.* tables through tidb rowid to save TiDB's memory. [#306](https://github.com/pingcap/dumpling/pull/306) + - When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself. [#287](https://github.com/pingcap/dumpling/pull/287) - TiCDC - * Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) - * Better err msg when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) - * Add `capture-session-ttl` in cdc server config [#2169](https://github.com/pingcap/ticdc/pull/2169) - * Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) - * Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) - * Make sorter IO errors more user-friendly. [#1976](https://github.com/pingcap/ticdc/pull/1976) - * Fix Unified Sorter memory consumption when tables are many. [#1957](https://github.com/pingcap/ticdc/pull/1957) - * Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses. [#1945](https://github.com/pingcap/ticdc/pull/1945) - * Add concurrency limit to the region incremental scan in kv client. [#1926](https://github.com/pingcap/ticdc/pull/1926) - * Add metrics for table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) - * Owner: When tikv_gc_life_time is greater than gcttl, use tikv_gc_life_time to calculate the lower bound of gcSafePoint. [#1871](https://github.com/pingcap/ticdc/pull/1871) - * Reduce memory malloc in sort heap to avoid too much CPU overhead. [#1862](https://github.com/pingcap/ticdc/pull/1862) - * Fix a bug about resolved ts stopped when move a table. [#1827](https://github.com/pingcap/ticdc/pull/1827) + - Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) + - Better err msg when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) + - Add `capture-session-ttl` in cdc server config [#2169](https://github.com/pingcap/ticdc/pull/2169) + - Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) + - Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) + - Make sorter IO errors more user-friendly. [#1976](https://github.com/pingcap/ticdc/pull/1976) + - Fix Unified Sorter memory consumption when tables are many. [#1957](https://github.com/pingcap/ticdc/pull/1957) + - Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses. [#1945](https://github.com/pingcap/ticdc/pull/1945) + - Add concurrency limit to the region incremental scan in kv client. [#1926](https://github.com/pingcap/ticdc/pull/1926) + - Add metrics for table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) + - Owner: When tikv_gc_life_time is greater than gcttl, use tikv_gc_life_time to calculate the lower bound of gcSafePoint. [#1871](https://github.com/pingcap/ticdc/pull/1871) + - Reduce memory malloc in sort heap to avoid too much CPU overhead. [#1862](https://github.com/pingcap/ticdc/pull/1862) + - Fix a bug about resolved ts stopped when move a table. [#1827](https://github.com/pingcap/ticdc/pull/1827) From 75c5a537d3f8895faab0df72e2833c3f23978d80 Mon Sep 17 00:00:00 2001 From: kennytm Date: Mon, 19 Jul 2021 15:00:28 +0800 Subject: [PATCH 08/24] release-4.0.14: update "tools" section --- releases/release-4.0.14.md | 69 ++++++++++++++++++++------------------ 1 file changed, 36 insertions(+), 33 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 1d25ccc364bb4..cc3848d515357 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -21,7 +21,18 @@ TiDB version: 4.0.14 + Backup & Restore (BR) - - Refactor storage.ExternalStorage interface to support compress [#1317](https://github.com/pingcap/br/pull/1317) + - Speed up restore by merging small backup files. [#655](https://github.com/pingcap/br/pull/655) + + + Dumpling + + - Always split TiDB v3.* tables through `_tidb_rowid` to reduce TiDB's memory use. [#306](https://github.com/pingcap/dumpling/pull/306) + + + TiCDC + + - Better error message when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) + - Make sorter IO errors more user-friendly. [#1976](https://github.com/pingcap/ticdc/pull/1976) + - Add concurrency limit to the region incremental scan in kv client. [#1926](https://github.com/pingcap/ticdc/pull/1926) + - Add metrics for table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) ## Bug Fixes @@ -49,11 +60,32 @@ TiDB version: 4.0.14 + Tools - - BR + + Backup & Restore (BR) + + - Support restoring tables in `mysql` schema when explicitly opt-in by including `'mysql.*'` into the `filter`. [#1142](https://github.com/pingcap/br/pull/1142) + + + TiDB Lightning - Fix parquet parse when parse decimal type [#1276](https://github.com/pingcap/br/pull/1276) - - Fix the bug that lightning returns EOF error when CSV file without '\r\n' at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) - - Fix the bug that lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) + - Fix the bug that Lightning returns EOF error when CSV file without `\r\n` at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) + - Fix the bug that Lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) + - Fix the issue that Lightning panics due to batching KV larger than 4 GB. [#1128](https://github.com/pingcap/br/pull/1128) + + + Dumpling + + - When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself. [#287](https://github.com/pingcap/dumpling/pull/287) + + + TiCDC + + - Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) + - Add `capture-session-ttl` in CDC server config [#2169](https://github.com/pingcap/ticdc/pull/2169) + - Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) + - Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) + - Fix Unified Sorter memory consumption when tables are many. [#1957](https://github.com/pingcap/ticdc/pull/1957) + - Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses. [#1945](https://github.com/pingcap/ticdc/pull/1945) + - Owner: When `tikv_gc_life_time` is greater than gcttl, use `tikv_gc_life_time` to calculate the lower bound of gcSafePoint. [#1871](https://github.com/pingcap/ticdc/pull/1871) + - Reduce memory malloc in sort heap to avoid too much CPU overhead. [#1862](https://github.com/pingcap/ticdc/pull/1862) + - Fix a bug about resolved ts stopped when move a table. [#1827](https://github.com/pingcap/ticdc/pull/1827) ## 以下 note 未分类。请将以下 note 进行分类 (New feature, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 @@ -101,32 +133,3 @@ TiDB version: 4.0.14 - Fix the issue that leader re-election be slow when there are many stores [#3718](https://github.com/pingcap/pd/pull/3718) - Fix the priority problems for some kinds of operators [#3703](https://github.com/pingcap/pd/pull/3703) - Fix the panic issue about remove evict leader scheduler from a nonexistent store [#3680](https://github.com/pingcap/pd/pull/3680) - -+ Tools - - + Backup & Restore (BR) - - - Fix the issue that restore from `mysql` schema would failed. [#1142](https://github.com/pingcap/br/pull/1142) - - Fix the issue that lightning panic due to batch kv greater than 4.0g. [#1128](https://github.com/pingcap/br/pull/1128) - - Speed up restore by merging small backup files. [#655](https://github.com/pingcap/br/pull/655) - - - Dumpling - - - Always split TiDB v3.* tables through tidb rowid to save TiDB's memory. [#306](https://github.com/pingcap/dumpling/pull/306) - - When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself. [#287](https://github.com/pingcap/dumpling/pull/287) - - - TiCDC - - - Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) - - Better err msg when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) - - Add `capture-session-ttl` in cdc server config [#2169](https://github.com/pingcap/ticdc/pull/2169) - - Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) - - Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) - - Make sorter IO errors more user-friendly. [#1976](https://github.com/pingcap/ticdc/pull/1976) - - Fix Unified Sorter memory consumption when tables are many. [#1957](https://github.com/pingcap/ticdc/pull/1957) - - Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses. [#1945](https://github.com/pingcap/ticdc/pull/1945) - - Add concurrency limit to the region incremental scan in kv client. [#1926](https://github.com/pingcap/ticdc/pull/1926) - - Add metrics for table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) - - Owner: When tikv_gc_life_time is greater than gcttl, use tikv_gc_life_time to calculate the lower bound of gcSafePoint. [#1871](https://github.com/pingcap/ticdc/pull/1871) - - Reduce memory malloc in sort heap to avoid too much CPU overhead. [#1862](https://github.com/pingcap/ticdc/pull/1862) - - Fix a bug about resolved ts stopped when move a table. [#1827](https://github.com/pingcap/ticdc/pull/1827) From c4c9a9b571cb9420bca5c199c19491b5207a30ea Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 16:56:10 +0800 Subject: [PATCH 09/24] add tiflash notes --- releases/release-4.0.14.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index cc3848d515357..3f46fc82e6694 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -48,15 +48,15 @@ TiDB version: 4.0.14 + TiFlash - - Fix potential npe in executeTS during DAG compile [#2377](https://github.com/pingcap/tics/pull/2377) - - Fix the panic issue that occurs when the read load is heavy [#2280](https://github.com/pingcap/tics/pull/2280) - - Fix the issue that TiFlash nodes keep restart because of split failure. [#2218](https://github.com/pingcap/tics/pull/2218) - - Fix the bug that TiFlash can not GC delta data under rare case [#2183](https://github.com/pingcap/tics/pull/2183) - - Fix the potential concurrency problem when clone the shared delta index. [#2032](https://github.com/pingcap/tics/pull/2032) - - Fix the bug that incomplete data may make TiFlash fail to restart [#2002](https://github.com/pingcap/tics/pull/2002) - - Fix the problem that old dmfile is not removed atomically [#1924](https://github.com/pingcap/tics/pull/1924) - - Fix the problem that TiFlash will crash when executing SUBSTRING function with specific argument [#1914](https://github.com/pingcap/tics/pull/1914) - - Fix the issue that cast int as time function in TiFlash's coprocessor may produce wrong result [#1892](https://github.com/pingcap/tics/pull/1892) + - Fix the potential panic issue that occurs while compiling DAG requests + - Fix the panic issue that occurs when read load is heavy + - Fix the issue that TiFlash keeps restarting because of split failure in column storage + - Fix a potential bug that TiFlash can not GC delta data + - Fix the issue of incorrect results when cloning shared delta index concurrently + - Fix a bug that TiFlash fails to restart because of incomplete data + - Fix the issue that old dm files are not removed automatically + - Fix the panic issue that occurs while executing `SUBSTRING` function with specific arguments + - Fix the issue of incorrect results when casting the `TIME` type to the `INTEGER` type + Tools @@ -67,13 +67,13 @@ TiDB version: 4.0.14 + TiDB Lightning - Fix parquet parse when parse decimal type [#1276](https://github.com/pingcap/br/pull/1276) - - Fix the bug that Lightning returns EOF error when CSV file without `\r\n` at the last line and `strict-format = true`. [#1188](https://github.com/pingcap/br/pull/1188) - - Fix the bug that Lightning rebase wrong auto_increment base when the auto_increment field type is float or double. [#1185](https://github.com/pingcap/br/pull/1185) - - Fix the issue that Lightning panics due to batching KV larger than 4 GB. [#1128](https://github.com/pingcap/br/pull/1128) + - Fix the bug that Lightning returns EOF error when CSV file without `\r\n` at the last line and `strict-format = true` [#1188](https://github.com/pingcap/br/pull/1188) + - Fix the bug that Lightning rebase wrong auto_increment base when the auto_increment field type is float or double [#1185](https://github.com/pingcap/br/pull/1185) + - Fix the issue that Lightning panics due to batching KV larger than 4 GB [#1128](https://github.com/pingcap/br/pull/1128) + Dumpling - - When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself. [#287](https://github.com/pingcap/dumpling/pull/287) + - When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself [#287](https://github.com/pingcap/dumpling/pull/287) + TiCDC From f1069b7fc5e99e6723a20e0f5969eb2af11bb76a Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 17:36:10 +0800 Subject: [PATCH 10/24] refine tools notes --- releases/release-4.0.14.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 3f46fc82e6694..7bfbd05a5d09a 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -62,7 +62,7 @@ TiDB version: 4.0.14 + Backup & Restore (BR) - - Support restoring tables in `mysql` schema when explicitly opt-in by including `'mysql.*'` into the `filter`. [#1142](https://github.com/pingcap/br/pull/1142) + - Fix the issue that data restore from the `mysql` schema might fail [#1142](https://github.com/pingcap/br/pull/1142) + TiDB Lightning @@ -77,15 +77,15 @@ TiDB version: 4.0.14 + TiCDC - - Fix extra partition dispatching when adding new table partition. [#2205](https://github.com/pingcap/ticdc/pull/2205) + - Fix extra partition dispatching when adding new table partition [#2205](https://github.com/pingcap/ticdc/pull/2205) - Add `capture-session-ttl` in CDC server config [#2169](https://github.com/pingcap/ticdc/pull/2169) - Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) - Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) - - Fix Unified Sorter memory consumption when tables are many. [#1957](https://github.com/pingcap/ticdc/pull/1957) - - Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses. [#1945](https://github.com/pingcap/ticdc/pull/1945) - - Owner: When `tikv_gc_life_time` is greater than gcttl, use `tikv_gc_life_time` to calculate the lower bound of gcSafePoint. [#1871](https://github.com/pingcap/ticdc/pull/1871) - - Reduce memory malloc in sort heap to avoid too much CPU overhead. [#1862](https://github.com/pingcap/ticdc/pull/1862) - - Fix a bug about resolved ts stopped when move a table. [#1827](https://github.com/pingcap/ticdc/pull/1827) + - Fix Unified Sorter memory consumption when tables are many [#1957](https://github.com/pingcap/ticdc/pull/1957) + - Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses [#1945](https://github.com/pingcap/ticdc/pull/1945) + - Fix the issue that TiCDC changefeed cannot be created when start ts is less than current ts - gcttl [#1871](https://github.com/pingcap/ticdc/pull/1871) + - Reduce memory malloc in sort heap to avoid too much CPU overhead [#1862](https://github.com/pingcap/ticdc/pull/1862) + - Fix a bug about resolved ts stopped when move a table [#1827](https://github.com/pingcap/ticdc/pull/1827) ## 以下 note 未分类。请将以下 note 进行分类 (New feature, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 From 8b738f434077d734871007270a88453ebbc8b2c4 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 19 Jul 2021 17:53:22 +0800 Subject: [PATCH 11/24] Update releases/release-4.0.14.md Co-authored-by: Zhigao Tong --- releases/release-4.0.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 7bfbd05a5d09a..342a68ab70976 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -56,7 +56,7 @@ TiDB version: 4.0.14 - Fix a bug that TiFlash fails to restart because of incomplete data - Fix the issue that old dm files are not removed automatically - Fix the panic issue that occurs while executing `SUBSTRING` function with specific arguments - - Fix the issue of incorrect results when casting the `TIME` type to the `INTEGER` type + - Fix the issue of incorrect results when casting the `INTEGER` type to the `TIME` type + Tools From 0b6719f8f8783a7109fd7a5cd9a50eee68720891 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 20 Jul 2021 19:05:19 +0800 Subject: [PATCH 12/24] add pd and tikv notes --- releases/release-4.0.14.md | 49 ++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 342a68ab70976..07b1a86a3885a 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -15,8 +15,25 @@ TiDB version: 4.0.14 - Change the default value of `tidb_multi_statement_mode` from `WARN` to `OFF` in v4.0. It is recommended to use the multi-statement feature of your client library instead. See [the documentation on `tidb_multi_statement_mode`](/system-variables.md#tidb_multi_statement_mode-new-in-v4011) for details. [#25749](https://github.com/pingcap/tidb/pull/25749) - Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. +## Feature Enhancements + ++ TiDB Dashboard + + - Add OIDC SSO support. Users can sign into TiDB Dashboard without entering the SQL password by using OIDC SSO services (like Okta, Auth0, etc). [#960](https://github.com/pingcap/tidb-dashboard/pull/960) + - Add Debug API UI, which can be used to invoke several common TiDB and PD internal APIs for advanced debugging. [#927](https://github.com/pingcap/tidb-dashboard/pull/927) + ## Improvements ++ PD + + - Update TiDB Dashboard to v2021.07.17.1 [#3882](https://github.com/pingcap/pd/pull/3882) + - Reduce the conflict due to multiple schedulers running at the same time [#3858](https://github.com/pingcap/pd/pull/3858) [#3854](https://github.com/tikv/pd/pull/3854) + - Fix the issue that leader re-election is slow when there are many stores [#3718](https://github.com/pingcap/pd/pull/3718) + ++ TiDB Dashboard + + - Support sharing session as read-only to avoid further modifications [#960](https://github.com/pingcap/tidb-dashboard/pull/960) + + Tools + Backup & Restore (BR) @@ -58,6 +75,20 @@ TiDB version: 4.0.14 - Fix the panic issue that occurs while executing `SUBSTRING` function with specific arguments - Fix the issue of incorrect results when casting the `INTEGER` type to the `TIME` type ++ PD + + - Fix the bug that the scheduler may reappear after executing the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) + - Fix the data race when the scheduler is started before loading TTL config [#3773](https://github.com/pingcap/pd/pull/3773) + - Fix the bug that PD may panic during the scattering region [#3761](https://github.com/pingcap/pd/pull/3761) + - Fix the issue that the priority of some operators is not set correctly [#3703](https://github.com/pingcap/pd/pull/3703) + - Fix the bug that PD may panic when deleting the evict leader scheduler from a non-existent store. [#3680](https://github.com/pingcap/pd/pull/3680) + ++ TiDB Dashboard + + - Fix profiling UI cannot profile all TiDB instances [#944](https://github.com/pingcap/tidb-dashboard/pull/944) + - Fix statement UI does not display "# Plans" [#939](https://github.com/pingcap/tidb-dashboard/pull/939) + - Fix slow query UI displays "unknown field" error after cluster upgrade [#930](https://github.com/pingcap/tidb-dashboard/pull/930) + + Tools + Backup & Restore (BR) @@ -121,15 +152,9 @@ TiDB version: 4.0.14 - Add metrics to see pending pd heartbeats number [#10008](https://github.com/tikv/tikv/pull/10008) - Change the default merge-check-tick-interval to 2 to speed up the retry of merge process [#9676](https://github.com/tikv/tikv/pull/9676) - Fix an issue that split may panic and corrupt the metadata if the split process is too slow and region merge is open. [#9584](https://github.com/tikv/tikv/pull/9584) - -+ PD - - - TiDB Dashboard: Add OIDC based SSO support [#3882](https://github.com/pingcap/pd/pull/3882) - - Reduce the conflict due to multiple scheduler running in same time [#3858](https://github.com/pingcap/pd/pull/3858) - - Fix the issue that the scheduler may appear again even if we have already executed the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) - - Update TiDB Dashboard to v2021.06.15.1 [#3797](https://github.com/pingcap/pd/pull/3797) - - Fix the data race when the scheduler is started before loading TTL configurations [#3773](https://github.com/pingcap/pd/pull/3773) - - Fix pd may get panic during scattering region [#3761](https://github.com/pingcap/pd/pull/3761) - - Fix the issue that leader re-election be slow when there are many stores [#3718](https://github.com/pingcap/pd/pull/3718) - - Fix the priority problems for some kinds of operators [#3703](https://github.com/pingcap/pd/pull/3703) - - Fix the panic issue about remove evict leader scheduler from a nonexistent store [#3680](https://github.com/pingcap/pd/pull/3680) + - Fix region heartbeat preventing from TiKV splitting large regions [#10274](https://github.com/tikv/tikv/pull/10274) + - Fix wrong statistics due to format inconsistency of CM Sketch between TiKV and TiDB [#10433](https://github.com/tikv/tikv/pull/10433) + - Fix wrong apply wait duration metrics [#9966](https://github.com/tikv/tikv/pull/9966) + - Fix "Missing Blob" error after using delete_files_in_range with titan [#10232](https://github.com/tikv/tikv/pull/10232) + - Shutdown status server first to report correct liveness to health check [#10504](https://github.com/tikv/tikv/pull/10504) + - Always respond to stale peer to destroy it quicker [#10400](https://github.com/tikv/tikv/pull/10400) From 7c8126cc75370a48599692c73da0b281035b8512 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 21 Jul 2021 10:33:31 +0800 Subject: [PATCH 13/24] sort tikv notes --- releases/release-4.0.14.md | 53 ++++++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 22 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 07b1a86a3885a..4b1f289b6f05c 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -15,8 +15,17 @@ TiDB version: 4.0.14 - Change the default value of `tidb_multi_statement_mode` from `WARN` to `OFF` in v4.0. It is recommended to use the multi-statement feature of your client library instead. See [the documentation on `tidb_multi_statement_mode`](/system-variables.md#tidb_multi_statement_mode-new-in-v4011) for details. [#25749](https://github.com/pingcap/tidb/pull/25749) - Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. ++ TiKV + + - Change the default merge-check-tick-interval to 2 to speed up the retry of merge process [#9676](https://github.com/tikv/tikv/pull/9676) + ## Feature Enhancements ++ TiKV + + - Add metrics to see pending pd heartbeats number [#10008](https://github.com/tikv/tikv/pull/10008) + - BR now supports S3-compatible storage using virtual-host addressing style. [#10242](https://github.com/tikv/tikv/pull/10242) + + TiDB Dashboard - Add OIDC SSO support. Users can sign into TiDB Dashboard without entering the SQL password by using OIDC SSO services (like Okta, Auth0, etc). [#960](https://github.com/pingcap/tidb-dashboard/pull/960) @@ -24,6 +33,13 @@ TiDB version: 4.0.14 ## Improvements ++ TiKV + + - Always respond to stale peer to destroy it quicker [#10400](https://github.com/tikv/tikv/pull/10400) + - Shutdown status server first to report correct liveness to health check [#10504](https://github.com/tikv/tikv/pull/10504) + - Limit CDC sink memory consumption. [#10147](https://github.com/tikv/tikv/pull/10147) + - Approximate split range evenly [#10275](https://github.com/tikv/tikv/pull/10275) + + PD - Update TiDB Dashboard to v2021.07.17.1 [#3882](https://github.com/pingcap/pd/pull/3882) @@ -63,6 +79,21 @@ TiDB version: 4.0.14 - Fix panic when 'select ... for update' works on a join operation and the join uses partition table [#25501](https://github.com/pingcap/tidb/pull/25501) - Fix the issue point get cached plan of prepared statement is incorrectly used by in transaction point get statement. [#24764](https://github.com/pingcap/tidb/pull/24764) ++ TiKV + + - Fix duration calculation panics on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) + - Fix wrong function cast double to double [#10532](https://github.com/tikv/tikv/pull/10532) + - Ensure panic output is flushed to the log [#10488](https://github.com/tikv/tikv/pull/10488) + - Avoid panic when building a snapshot twice if encryption enabled [#10462](https://github.com/tikv/tikv/pull/10462) + - Copr: fix the wrong arguments type of json_unquote [#10425](https://github.com/tikv/tikv/pull/10425) + - skip clearing callback during gracefully shutdown to avoid breaking ACID in some cases [#10395](https://github.com/tikv/tikv/pull/10395) + - Fix backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) + - Fix an issue that split may panic and corrupt the metadata if the split process is too slow and region merge is open. [#9584](https://github.com/tikv/tikv/pull/9584) + - Fix region heartbeat preventing from TiKV splitting large regions [#10274](https://github.com/tikv/tikv/pull/10274) + - Fix wrong statistics due to format inconsistency of CM Sketch between TiKV and TiDB [#10433](https://github.com/tikv/tikv/pull/10433) + - Fix wrong apply wait duration metrics [#9966](https://github.com/tikv/tikv/pull/9966) + - Fix "Missing Blob" error after using delete_files_in_range with titan [#10232](https://github.com/tikv/tikv/pull/10232) + + TiFlash - Fix the potential panic issue that occurs while compiling DAG requests @@ -136,25 +167,3 @@ TiDB version: 4.0.14 - Don't let SPM be affected by charset [#23295](https://github.com/pingcap/tidb/pull/23295) - Time: parse datatime won't truncate the reluctant string. [#22260](https://github.com/pingcap/tidb/pull/22260) - Fix a bug that `select into outfile` has no data with year column type [#22185](https://github.com/pingcap/tidb/pull/22185) - -+ TiKV - - - Fix duration calculation panics on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) - - Fix wrong function cast double to double [#10532](https://github.com/tikv/tikv/pull/10532) - - Ensure panic output is flushed to the log [#10488](https://github.com/tikv/tikv/pull/10488) - - Avoid panic when building a snapshot twice if encryption enabled [#10462](https://github.com/tikv/tikv/pull/10462) - - Copr: fix the wrong arguments type of json_unquote [#10425](https://github.com/tikv/tikv/pull/10425) - - skip clearing callback during gracefully shutdown to avoid breaking ACID in some cases [#10395](https://github.com/tikv/tikv/pull/10395) - - Fix backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) - - Approximate split range evenly [#10275](https://github.com/tikv/tikv/pull/10275) - - BR now supports S3-compatible storage using virtual-host addressing style. [#10242](https://github.com/tikv/tikv/pull/10242) - - Limit CDC sink memory consumption. [#10147](https://github.com/tikv/tikv/pull/10147) - - Add metrics to see pending pd heartbeats number [#10008](https://github.com/tikv/tikv/pull/10008) - - Change the default merge-check-tick-interval to 2 to speed up the retry of merge process [#9676](https://github.com/tikv/tikv/pull/9676) - - Fix an issue that split may panic and corrupt the metadata if the split process is too slow and region merge is open. [#9584](https://github.com/tikv/tikv/pull/9584) - - Fix region heartbeat preventing from TiKV splitting large regions [#10274](https://github.com/tikv/tikv/pull/10274) - - Fix wrong statistics due to format inconsistency of CM Sketch between TiKV and TiDB [#10433](https://github.com/tikv/tikv/pull/10433) - - Fix wrong apply wait duration metrics [#9966](https://github.com/tikv/tikv/pull/9966) - - Fix "Missing Blob" error after using delete_files_in_range with titan [#10232](https://github.com/tikv/tikv/pull/10232) - - Shutdown status server first to report correct liveness to health check [#10504](https://github.com/tikv/tikv/pull/10504) - - Always respond to stale peer to destroy it quicker [#10400](https://github.com/tikv/tikv/pull/10400) From e82886c2161a521ac5e77c0e8c340fd709e4cb09 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 21 Jul 2021 13:15:14 +0800 Subject: [PATCH 14/24] add tidb notes & review tikv notes (before bug fixes) --- releases/release-4.0.14.md | 80 ++++++++++++++++++-------------------- 1 file changed, 38 insertions(+), 42 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 4b1f289b6f05c..9340dcbe40307 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -14,17 +14,18 @@ TiDB version: 4.0.14 - Change the default value of `tidb_multi_statement_mode` from `WARN` to `OFF` in v4.0. It is recommended to use the multi-statement feature of your client library instead. See [the documentation on `tidb_multi_statement_mode`](/system-variables.md#tidb_multi_statement_mode-new-in-v4011) for details. [#25749](https://github.com/pingcap/tidb/pull/25749) - Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. + - Enlarge the variable `tidb_stmt_summary_max_stmt_count` default value from 200 to 3000 [#25872](https://github.com/pingcap/tidb/pull/25872) + TiKV - - Change the default merge-check-tick-interval to 2 to speed up the retry of merge process [#9676](https://github.com/tikv/tikv/pull/9676) + - Change the default value of `merge-check-tick-interval` from `10` to `2` to speed up the Region merge process [#9676](https://github.com/tikv/tikv/pull/9676) ## Feature Enhancements + TiKV - - Add metrics to see pending pd heartbeats number [#10008](https://github.com/tikv/tikv/pull/10008) - - BR now supports S3-compatible storage using virtual-host addressing style. [#10242](https://github.com/tikv/tikv/pull/10242) + - Add a metric to monitor the number of pending PD heartbeats, which helps locate the issue of slow PD threads [#10008](https://github.com/tikv/tikv/pull/10008) + - Support using the virtual-host addressing mode to make BR support the S3-compatible storage [#10242](https://github.com/tikv/tikv/pull/10242) + TiDB Dashboard @@ -33,12 +34,20 @@ TiDB version: 4.0.14 ## Improvements ++ TiDB + + - Change the lock record into put record for the index keys using point/batch point get for update read. [#26223](https://github.com/pingcap/tidb/pull/26223) + - Support the MySQL system variable `init_connect` and associated functionality. [#26031](https://github.com/pingcap/tidb/pull/26031) + - Support stable result mode [#26003](https://github.com/pingcap/tidb/pull/26003) + - Enable the pushdown of builtin function `json_unquote()` to TiKV. [#25721](https://github.com/pingcap/tidb/pull/25721) + - Make SPM not be affected by charset [#23295](https://github.com/pingcap/tidb/pull/23295) + + TiKV - - Always respond to stale peer to destroy it quicker [#10400](https://github.com/tikv/tikv/pull/10400) - - Shutdown status server first to report correct liveness to health check [#10504](https://github.com/tikv/tikv/pull/10504) - - Limit CDC sink memory consumption. [#10147](https://github.com/tikv/tikv/pull/10147) - - Approximate split range evenly [#10275](https://github.com/tikv/tikv/pull/10275) + - Shutdown the status server first to make sure that the client can correctly check the shutdown status [#10504](https://github.com/tikv/tikv/pull/10504) + - Always respond to stale peers to make sure that these peers are cleared quicker [#10400](https://github.com/tikv/tikv/pull/10400) + - Limit the TiCDC sink's memory consumption [#10147](https://github.com/tikv/tikv/pull/10147) + - When a Region is too large, use the even split to speed up the split process [#10275](https://github.com/tikv/tikv/pull/10275) + PD @@ -71,13 +80,19 @@ TiDB version: 4.0.14 + TiDB - - Generate correct number of rows when all agg funcs are pruned [#26039](https://github.com/pingcap/tidb/pull/26039) - - Executor: fix ifnull bug when arg is enum/set [#26035](https://github.com/pingcap/tidb/pull/26035) + - Generate a correct number of rows when all agg funcs are pruned [#26039](https://github.com/pingcap/tidb/pull/26039) + - Fix ifnull bug when arg is enum/set [#26035](https://github.com/pingcap/tidb/pull/26035) - Fix wrong aggregate pruning for some cases [#26033](https://github.com/pingcap/tidb/pull/26033) - Fix incorrect result of set type for merge join [#26032](https://github.com/pingcap/tidb/pull/26032) - - Expression: fix IN expr critical bug [#25665](https://github.com/pingcap/tidb/pull/25665) + - Fix IN expr critical bug [#25665](https://github.com/pingcap/tidb/pull/25665) - Fix panic when 'select ... for update' works on a join operation and the join uses partition table [#25501](https://github.com/pingcap/tidb/pull/25501) - - Fix the issue point get cached plan of prepared statement is incorrectly used by in transaction point get statement. [#24764](https://github.com/pingcap/tidb/pull/24764) + - Fix the issue point get cached plan of a prepared statement is incorrectly used by in transaction point get statement. [#24764](https://github.com/pingcap/tidb/pull/24764) + - Fix load data with non-utf8 can succeed [#26142](https://github.com/pingcap/tidb/pull/26142) + - Fix a potential statistic object's memory leak when HTTP api is used [#24650](https://github.com/pingcap/tidb/pull/24650) + - Fix the security issue for handling ALTER USER statements [#25347](https://github.com/pingcap/tidb/pull/25347) + - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status. [#24918](https://github.com/pingcap/tidb/pull/24918) + - Fix parsing DateTime not truncate the reluctant string. [#22260](https://github.com/pingcap/tidb/pull/22260) + - Fix `select into outfile` have no data with year column type [#22185](https://github.com/pingcap/tidb/pull/22185) + TiKV @@ -94,18 +109,6 @@ TiDB version: 4.0.14 - Fix wrong apply wait duration metrics [#9966](https://github.com/tikv/tikv/pull/9966) - Fix "Missing Blob" error after using delete_files_in_range with titan [#10232](https://github.com/tikv/tikv/pull/10232) -+ TiFlash - - - Fix the potential panic issue that occurs while compiling DAG requests - - Fix the panic issue that occurs when read load is heavy - - Fix the issue that TiFlash keeps restarting because of split failure in column storage - - Fix a potential bug that TiFlash can not GC delta data - - Fix the issue of incorrect results when cloning shared delta index concurrently - - Fix a bug that TiFlash fails to restart because of incomplete data - - Fix the issue that old dm files are not removed automatically - - Fix the panic issue that occurs while executing `SUBSTRING` function with specific arguments - - Fix the issue of incorrect results when casting the `INTEGER` type to the `TIME` type - + PD - Fix the bug that the scheduler may reappear after executing the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) @@ -120,6 +123,18 @@ TiDB version: 4.0.14 - Fix statement UI does not display "# Plans" [#939](https://github.com/pingcap/tidb-dashboard/pull/939) - Fix slow query UI displays "unknown field" error after cluster upgrade [#930](https://github.com/pingcap/tidb-dashboard/pull/930) ++ TiFlash + + - Fix the potential panic issue that occurs while compiling DAG requests + - Fix the panic issue that occurs when read load is heavy + - Fix the issue that TiFlash keeps restarting because of split failure in column storage + - Fix a potential bug that TiFlash can not GC delta data + - Fix the issue of incorrect results when cloning shared delta index concurrently + - Fix a bug that TiFlash fails to restart because of incomplete data + - Fix the issue that old dm files are not removed automatically + - Fix the panic issue that occurs while executing `SUBSTRING` function with specific arguments + - Fix the issue of incorrect results when casting the `INTEGER` type to the `TIME` type + + Tools + Backup & Restore (BR) @@ -148,22 +163,3 @@ TiDB version: 4.0.14 - Fix the issue that TiCDC changefeed cannot be created when start ts is less than current ts - gcttl [#1871](https://github.com/pingcap/ticdc/pull/1871) - Reduce memory malloc in sort heap to avoid too much CPU overhead [#1862](https://github.com/pingcap/ticdc/pull/1862) - Fix a bug about resolved ts stopped when move a table [#1827](https://github.com/pingcap/ticdc/pull/1827) - -## 以下 note 未分类。请将以下 note 进行分类 (New feature, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 - -+ TiDB - - - Fix a bug which is caused by prior bug fix PR [#26274](https://github.com/pingcap/tidb/pull/26274) - - Change the lock record into put record for the index keys using point/batch point get for update read. [#26223](https://github.com/pingcap/tidb/pull/26223) - - Load: fix load data with non-utf8 can succeed [#26142](https://github.com/pingcap/tidb/pull/26142) - - TiDB now supports the mysql system variable `init_connect` and associated functionality. [#26031](https://github.com/pingcap/tidb/pull/26031) - - Planner: support stable result mode [#26003](https://github.com/pingcap/tidb/pull/26003) - - Enlarge the variable tidb_stmt_summary_max_stmt_count default value from 200 to 3000 [#25872](https://github.com/pingcap/tidb/pull/25872) - - Enable the pushdown of builtin function `json_unquote()` to TiKV. [#25721](https://github.com/pingcap/tidb/pull/25721) - - Planner: select distinct should bypass batchget [#25532](https://github.com/pingcap/tidb/pull/25532) - - Important security issue for handling ALTER USER statements [#25347](https://github.com/pingcap/tidb/pull/25347) - - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status. [#24918](https://github.com/pingcap/tidb/pull/24918) - - Handle a potential statistic object's memory leak when HTTP api is used [#24650](https://github.com/pingcap/tidb/pull/24650) - - Don't let SPM be affected by charset [#23295](https://github.com/pingcap/tidb/pull/23295) - - Time: parse datatime won't truncate the reluctant string. [#22260](https://github.com/pingcap/tidb/pull/22260) - - Fix a bug that `select into outfile` has no data with year column type [#22185](https://github.com/pingcap/tidb/pull/22185) From 3afd254a001a1c7607dcf43b60e4b97b9758469a Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 21 Jul 2021 16:54:40 +0800 Subject: [PATCH 15/24] modify notes except Tools --- releases/release-4.0.14.md | 104 ++++++++++++++++++------------------- 1 file changed, 52 insertions(+), 52 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 9340dcbe40307..f0ae725771157 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -13,8 +13,8 @@ TiDB version: 4.0.14 + TiDB - Change the default value of `tidb_multi_statement_mode` from `WARN` to `OFF` in v4.0. It is recommended to use the multi-statement feature of your client library instead. See [the documentation on `tidb_multi_statement_mode`](/system-variables.md#tidb_multi_statement_mode-new-in-v4011) for details. [#25749](https://github.com/pingcap/tidb/pull/25749) - - Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the [Grafana post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. - - Enlarge the variable `tidb_stmt_summary_max_stmt_count` default value from 200 to 3000 [#25872](https://github.com/pingcap/tidb/pull/25872) + - Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the [Grafana blog post](https://grafana.com/blog/2020/06/03/grafana-6.7.4-and-7.0.2-released-with-important-security-fix/) for details. + - Change the default value of the `tidb_stmt_summary_max_stmt_count` variable from `200` to `3000` [#25872](https://github.com/pingcap/tidb/pull/25872) + TiKV @@ -29,18 +29,18 @@ TiDB version: 4.0.14 + TiDB Dashboard - - Add OIDC SSO support. Users can sign into TiDB Dashboard without entering the SQL password by using OIDC SSO services (like Okta, Auth0, etc). [#960](https://github.com/pingcap/tidb-dashboard/pull/960) - - Add Debug API UI, which can be used to invoke several common TiDB and PD internal APIs for advanced debugging. [#927](https://github.com/pingcap/tidb-dashboard/pull/927) + - Support OIDC SSO. By setting the OIDC-compatible SSO services (such as Okta and Auth0), users can log into TiDB Dashboard without entering the SQL password. [#960](https://github.com/pingcap/tidb-dashboard/pull/960) + - Add the **Debug API** UI, which is an alternative method to the command line to call several common TiDB and PD internal APIs for advanced debugging. [#927](https://github.com/pingcap/tidb-dashboard/pull/927) ## Improvements + TiDB - - Change the lock record into put record for the index keys using point/batch point get for update read. [#26223](https://github.com/pingcap/tidb/pull/26223) - - Support the MySQL system variable `init_connect` and associated functionality. [#26031](https://github.com/pingcap/tidb/pull/26031) - - Support stable result mode [#26003](https://github.com/pingcap/tidb/pull/26003) - - Enable the pushdown of builtin function `json_unquote()` to TiKV. [#25721](https://github.com/pingcap/tidb/pull/25721) - - Make SPM not be affected by charset [#23295](https://github.com/pingcap/tidb/pull/23295) + ?- Change the lock records into put record for the index keys using `point get` or `batch point get` for `UPDATE` reads [#26223](https://github.com/pingcap/tidb/pull/26223) + - Support the MySQL system variable `init_connect` and its associated features [#26031](https://github.com/pingcap/tidb/pull/26031) + - Support the stable result mode to make the query results more stable [#26003](https://github.com/pingcap/tidb/pull/26003) + - Support pushing down the built-in function `json_unquote()` to TiKV [#25721](https://github.com/pingcap/tidb/pull/25721) + - Make the SQL Plan Management (SPM) not affected by the character set [#23295](https://github.com/pingcap/tidb/pull/23295) + TiKV @@ -51,13 +51,12 @@ TiDB version: 4.0.14 + PD - - Update TiDB Dashboard to v2021.07.17.1 [#3882](https://github.com/pingcap/pd/pull/3882) - - Reduce the conflict due to multiple schedulers running at the same time [#3858](https://github.com/pingcap/pd/pull/3858) [#3854](https://github.com/tikv/pd/pull/3854) - - Fix the issue that leader re-election is slow when there are many stores [#3718](https://github.com/pingcap/pd/pull/3718) + - Reduce the conflicts among multiple schedulers that run at the same time [#3858](https://github.com/pingcap/pd/pull/3858) [#3854](https://github.com/tikv/pd/pull/3854) + TiDB Dashboard - - Support sharing session as read-only to avoid further modifications [#960](https://github.com/pingcap/tidb-dashboard/pull/960) + - Update TiDB Dashboard to v2021.07.17.1 [#3882](https://github.com/pingcap/pd/pull/3882) + - Support sharing the current session as a read-only session to avoid further modification to it [#960](https://github.com/pingcap/tidb-dashboard/pull/960) + Tools @@ -80,59 +79,60 @@ TiDB version: 4.0.14 + TiDB - - Generate a correct number of rows when all agg funcs are pruned [#26039](https://github.com/pingcap/tidb/pull/26039) - - Fix ifnull bug when arg is enum/set [#26035](https://github.com/pingcap/tidb/pull/26035) - - Fix wrong aggregate pruning for some cases [#26033](https://github.com/pingcap/tidb/pull/26033) - - Fix incorrect result of set type for merge join [#26032](https://github.com/pingcap/tidb/pull/26032) - - Fix IN expr critical bug [#25665](https://github.com/pingcap/tidb/pull/25665) - - Fix panic when 'select ... for update' works on a join operation and the join uses partition table [#25501](https://github.com/pingcap/tidb/pull/25501) - - Fix the issue point get cached plan of a prepared statement is incorrectly used by in transaction point get statement. [#24764](https://github.com/pingcap/tidb/pull/24764) - - Fix load data with non-utf8 can succeed [#26142](https://github.com/pingcap/tidb/pull/26142) - - Fix a potential statistic object's memory leak when HTTP api is used [#24650](https://github.com/pingcap/tidb/pull/24650) - - Fix the security issue for handling ALTER USER statements [#25347](https://github.com/pingcap/tidb/pull/25347) + - Fix the issue that the `SELECT` result is incompatible with MySQL when joining a subquery with a `WHERE` clause evaluated to `false` [#24865](https://github.com/pingcap/tidb/issues/24865) + - Fix the calculation error of the `ifnull` function that occurs when the argument is the `ENUM` or `SET` type [#24944](https://github.com/pingcap/tidb/issues/24944) + - Fix the wrong aggregate pruning in some cases [#26033](https://github.com/pingcap/tidb/pull/26033) + - Fix the incorrect result of the merge join operation that might occur when the column is the `SET` type [#26032](https://github.com/pingcap/tidb/pull/26032) + - Fix the issue that TiDB returns wrong results for cartesian join [#25665](https://github.com/pingcap/tidb/pull/25665) + - Fix the panic issue that occurs when `SELECT ... FOR UPDATE` works on a join operation and the join uses a partitioned table [#25501](https://github.com/pingcap/tidb/pull/25501) + - Fix the issue that the cached `prepared` plan is incorrectly used for `point get` [#24764](https://github.com/pingcap/tidb/pull/24764) + - Fix the issue that the `LOAD DATA` statement can abnormally import non-utf8 data [#26142](https://github.com/pingcap/tidb/pull/26142) + - Fix a potential memory leak issue that occurs when accessing the statistics via an HTTP API [#24650](https://github.com/pingcap/tidb/pull/24650) + - Fix a security issue that occurs when executing the `ALTER USER` statement [#25347](https://github.com/pingcap/tidb/pull/25347) - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status. [#24918](https://github.com/pingcap/tidb/pull/24918) - - Fix parsing DateTime not truncate the reluctant string. [#22260](https://github.com/pingcap/tidb/pull/22260) - - Fix `select into outfile` have no data with year column type [#22185](https://github.com/pingcap/tidb/pull/22185) + - Fix the issue that invalid strings are not truncated when parsing `DateTime` [#22260](https://github.com/pingcap/tidb/pull/22260) + - Fix the issue that the `select into outfile` statement might have no result when the column type is `YEAR` [#22185](https://github.com/pingcap/tidb/pull/22185) + TiKV - - Fix duration calculation panics on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) - - Fix wrong function cast double to double [#10532](https://github.com/tikv/tikv/pull/10532) - - Ensure panic output is flushed to the log [#10488](https://github.com/tikv/tikv/pull/10488) - - Avoid panic when building a snapshot twice if encryption enabled [#10462](https://github.com/tikv/tikv/pull/10462) - - Copr: fix the wrong arguments type of json_unquote [#10425](https://github.com/tikv/tikv/pull/10425) - - skip clearing callback during gracefully shutdown to avoid breaking ACID in some cases [#10395](https://github.com/tikv/tikv/pull/10395) - - Fix backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) - - Fix an issue that split may panic and corrupt the metadata if the split process is too slow and region merge is open. [#9584](https://github.com/tikv/tikv/pull/9584) - - Fix region heartbeat preventing from TiKV splitting large regions [#10274](https://github.com/tikv/tikv/pull/10274) - - Fix wrong statistics due to format inconsistency of CM Sketch between TiKV and TiDB [#10433](https://github.com/tikv/tikv/pull/10433) - - Fix wrong apply wait duration metrics [#9966](https://github.com/tikv/tikv/pull/9966) - - Fix "Missing Blob" error after using delete_files_in_range with titan [#10232](https://github.com/tikv/tikv/pull/10232) + - Fix the issue that the duration calculation might panic on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) + - Fix the wrong function that casts `DOUBLE` to `DOUBLE` [#25200](https://github.com/pingcap/tidb/issues/25200) + - Fix the issue that the panic log might be lost when using the async logger [#8998](https://github.com/tikv/tikv/issues/8998) + - Fix the panic issue that occurs when building a snapshot twice if encryption is enabled [#10462](https://github.com/tikv/tikv/pull/10462) + - Fix the wrong arguments type of the `json_unquote()` function in the coprocessor [#10176](https://github.com/tikv/tikv/issues/10176) + - Fix the issues of suspicious warnings during shutdown and the non-deterministic response from Raftstore [#10395](https://github.com/tikv/tikv/pull/10395) + - Fix the issue of backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) + - Fix the issue that Region split might panic and corrupt the metadata if the split process is too slow and Region merge is on-going [#9584](https://github.com/tikv/tikv/pull/9584) + - Fix the issue that the Region heartbeats prevent TiKV from splitting large Regions in some situation [#10274](https://github.com/tikv/tikv/pull/10274) + - Fix the wrong statistics caused by the format inconsistency of CM Sketch between TiKV and TiDB [#10433](https://github.com/tikv/tikv/pull/10433) + - Fix the wrong statistics of the `apply wait duration` metric [#9966](https://github.com/tikv/tikv/pull/9966) + - Fix the "Missing Blob" error after using `delete_files_in_range` in Titan [#10232](https://github.com/tikv/tikv/pull/10232) + PD - - Fix the bug that the scheduler may reappear after executing the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) - - Fix the data race when the scheduler is started before loading TTL config [#3773](https://github.com/pingcap/pd/pull/3773) - - Fix the bug that PD may panic during the scattering region [#3761](https://github.com/pingcap/pd/pull/3761) + - Fix a bug that the scheduler might reappear after executing the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) + - Fix the data race issue that might occur when the scheduler is started before the temporary configuration is loaded [#3773](https://github.com/pingcap/pd/pull/3773) + - Fix a PD panic issue that might occur during the Region scattering operation [#3761](https://github.com/pingcap/pd/pull/3761) - Fix the issue that the priority of some operators is not set correctly [#3703](https://github.com/pingcap/pd/pull/3703) - - Fix the bug that PD may panic when deleting the evict leader scheduler from a non-existent store. [#3680](https://github.com/pingcap/pd/pull/3680) + - Fix a PD panic issue that might occur when deleting the `evict-leader` scheduler from a non-existent store [#3680](https://github.com/pingcap/pd/pull/3680) + - Fix the issue that the PD Leader re-election is slow when there are many stores [#3697](https://github.com/tikv/pd/issues/3697) + TiDB Dashboard - - Fix profiling UI cannot profile all TiDB instances [#944](https://github.com/pingcap/tidb-dashboard/pull/944) - - Fix statement UI does not display "# Plans" [#939](https://github.com/pingcap/tidb-dashboard/pull/939) - - Fix slow query UI displays "unknown field" error after cluster upgrade [#930](https://github.com/pingcap/tidb-dashboard/pull/930) + - Fix the issue that the **Profiling** UI cannot profile all TiDB instances [#944](https://github.com/pingcap/tidb-dashboard/pull/944) + - Fix the issue that the **Statements** UI does not display "Plan Count" [#939](https://github.com/pingcap/tidb-dashboard/pull/939) + - Fix the issue that the **Slow Query** UI might display the "unknown field" error after cluster upgrade [#930](https://github.com/pingcap/tidb-dashboard/pull/930) + TiFlash - - Fix the potential panic issue that occurs while compiling DAG requests - - Fix the panic issue that occurs when read load is heavy - - Fix the issue that TiFlash keeps restarting because of split failure in column storage - - Fix a potential bug that TiFlash can not GC delta data - - Fix the issue of incorrect results when cloning shared delta index concurrently - - Fix a bug that TiFlash fails to restart because of incomplete data - - Fix the issue that old dm files are not removed automatically - - Fix the panic issue that occurs while executing `SUBSTRING` function with specific arguments + - Fix the potential panic issue that occurs when compiling DAG requests + - Fix the panic issue that occurs when the read load is heavy + - Fix the issue that TiFlash keeps restarting because of the split failure in column storage + - Fix a potential bug that TiFlash cannot delete the delta data + - Fix the incorrect results that occur when cloning the shared delta index concurrently + - Fix a bug that TiFlash fails to restart in the case of incomplete data + - Fix the issue that the old dm files cannot be removed automatically + - Fix the panic issue that occurs when executing the `SUBSTRING` function with specific arguments - Fix the issue of incorrect results when casting the `INTEGER` type to the `TIME` type + Tools From 4e75564b35aab5e0e9f4e62877355848480a7852 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 21 Jul 2021 19:45:49 +0800 Subject: [PATCH 16/24] modify tools notes --- releases/release-4.0.14.md | 40 +++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index f0ae725771157..a7b9e9ae63f1a 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -62,18 +62,19 @@ TiDB version: 4.0.14 + Backup & Restore (BR) - - Speed up restore by merging small backup files. [#655](https://github.com/pingcap/br/pull/655) + - Speed up restore by merging small backup files [#655](https://github.com/pingcap/br/pull/655) + Dumpling - - Always split TiDB v3.* tables through `_tidb_rowid` to reduce TiDB's memory use. [#306](https://github.com/pingcap/dumpling/pull/306) + - Always split tables using `_tidb_rowid` when the upstream is a TiDB v3.x cluster, which helps reduce TiDB's memory usage [#306](https://github.com/pingcap/dumpling/pull/306) + TiCDC - - Better error message when PD endpoint missing certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) - - Make sorter IO errors more user-friendly. [#1976](https://github.com/pingcap/ticdc/pull/1976) - - Add concurrency limit to the region incremental scan in kv client. [#1926](https://github.com/pingcap/ticdc/pull/1926) - - Add metrics for table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) + - Improve the error message returned when a PD endpoint misses the certificate [#2184](https://github.com/pingcap/ticdc/pull/2184) + - Make the sorter I/O errors more user-friendly [#1976](https://github.com/pingcap/ticdc/pull/1976) + - Add a concurrency limit on the Region incremental scan in the KV client to reduce the pressure of TiKV [#1926](https://github.com/pingcap/ticdc/pull/1926) + - Add metrics for the table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) + - Add `capture-session-ttl` to the TiCDC server configuration [#2169](https://github.com/pingcap/ticdc/pull/2169) ## Bug Fixes @@ -143,23 +144,22 @@ TiDB version: 4.0.14 + TiDB Lightning - - Fix parquet parse when parse decimal type [#1276](https://github.com/pingcap/br/pull/1276) - - Fix the bug that Lightning returns EOF error when CSV file without `\r\n` at the last line and `strict-format = true` [#1188](https://github.com/pingcap/br/pull/1188) - - Fix the bug that Lightning rebase wrong auto_increment base when the auto_increment field type is float or double [#1185](https://github.com/pingcap/br/pull/1185) - - Fix the issue that Lightning panics due to batching KV larger than 4 GB [#1128](https://github.com/pingcap/br/pull/1128) + - Fix the issue that TiDB fails to parse the `DECIMAL` type data in Parquet files [#1276](https://github.com/pingcap/br/pull/1276) + - Fix the EOF error reported when TiDB Lightning splits the imported large CSV files [#1133](https://github.com/pingcap/br/issues/1133) + - Fix a bug that an excessively large base value is generated when TiDB Lightning imports tables with the `auto_increment` column of the `FLOAT` or `DOUBLE` type [#1185](https://github.com/pingcap/br/pull/1185) + - Fix the issue of TiDB Lightning panic that occurs when generating KV data larger than 4 GB [#1128](https://github.com/pingcap/br/pull/1128) + Dumpling - - When using Dumpling to export to S3, we no longer require s3:ListBucket permission on the entire bucket, only the data source prefix itself [#287](https://github.com/pingcap/dumpling/pull/287) + - When using Dumpling to export to S3, the `s3:ListBucket` permission is no longer required on the entire bucket. Only the data source prefix itself is required. [#287](https://github.com/pingcap/dumpling/pull/287) + TiCDC - - Fix extra partition dispatching when adding new table partition [#2205](https://github.com/pingcap/ticdc/pull/2205) - - Add `capture-session-ttl` in CDC server config [#2169](https://github.com/pingcap/ticdc/pull/2169) - - Fix panic when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) - - Reduce unnecessary memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) - - Fix Unified Sorter memory consumption when tables are many [#1957](https://github.com/pingcap/ticdc/pull/1957) - - Fix a bug that some MySQL connection could leak after MySQL sink meets error and pauses [#1945](https://github.com/pingcap/ticdc/pull/1945) - - Fix the issue that TiCDC changefeed cannot be created when start ts is less than current ts - gcttl [#1871](https://github.com/pingcap/ticdc/pull/1871) - - Reduce memory malloc in sort heap to avoid too much CPU overhead [#1862](https://github.com/pingcap/ticdc/pull/1862) - - Fix a bug about resolved ts stopped when move a table [#1827](https://github.com/pingcap/ticdc/pull/1827) + - Fix the issue of extra partition dispatching after adding new table partitions [#2205](https://github.com/pingcap/ticdc/pull/2205) + + - Fix the panic issue that occurs when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) + - Reduce TiCDC's runtime memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) [#1957](https://github.com/pingcap/ticdc/pull/1957) + - Fix a bug that some MySQL connection might leak after MySQL sink meets the error and pauses [#1945](https://github.com/pingcap/ticdc/pull/1945) + - Fix the issue that TiCDC changefeed cannot be created when start TS is less than current TS minus GC TTL [#1871](https://github.com/pingcap/ticdc/pull/1871) + - Reduce memory `malloc` in sort heap to avoid too much CPU overhead [#1862](https://github.com/pingcap/ticdc/pull/1862) + - Fix a bug that the replication task might stop when moving a table [#1827](https://github.com/pingcap/ticdc/pull/1827) From e4261766fb8572643bb4f8bcefe24e8b6a292b75 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 23 Jul 2021 11:27:57 +0800 Subject: [PATCH 17/24] pr to issue --- releases/release-4.0.14.md | 50 +++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index a7b9e9ae63f1a..677ca96cec54d 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -30,7 +30,7 @@ TiDB version: 4.0.14 + TiDB Dashboard - Support OIDC SSO. By setting the OIDC-compatible SSO services (such as Okta and Auth0), users can log into TiDB Dashboard without entering the SQL password. [#960](https://github.com/pingcap/tidb-dashboard/pull/960) - - Add the **Debug API** UI, which is an alternative method to the command line to call several common TiDB and PD internal APIs for advanced debugging. [#927](https://github.com/pingcap/tidb-dashboard/pull/927) + - Add the **Debug API** UI, which is an alternative method to the command line to call several common TiDB and PD internal APIs for advanced debugging [#927](https://github.com/pingcap/tidb-dashboard/pull/927) ## Improvements @@ -82,47 +82,47 @@ TiDB version: 4.0.14 - Fix the issue that the `SELECT` result is incompatible with MySQL when joining a subquery with a `WHERE` clause evaluated to `false` [#24865](https://github.com/pingcap/tidb/issues/24865) - Fix the calculation error of the `ifnull` function that occurs when the argument is the `ENUM` or `SET` type [#24944](https://github.com/pingcap/tidb/issues/24944) - - Fix the wrong aggregate pruning in some cases [#26033](https://github.com/pingcap/tidb/pull/26033) - - Fix the incorrect result of the merge join operation that might occur when the column is the `SET` type [#26032](https://github.com/pingcap/tidb/pull/26032) - - Fix the issue that TiDB returns wrong results for cartesian join [#25665](https://github.com/pingcap/tidb/pull/25665) - - Fix the panic issue that occurs when `SELECT ... FOR UPDATE` works on a join operation and the join uses a partitioned table [#25501](https://github.com/pingcap/tidb/pull/25501) - - Fix the issue that the cached `prepared` plan is incorrectly used for `point get` [#24764](https://github.com/pingcap/tidb/pull/24764) - - Fix the issue that the `LOAD DATA` statement can abnormally import non-utf8 data [#26142](https://github.com/pingcap/tidb/pull/26142) + - Fix the wrong aggregate pruning in some cases [#25202](https://github.com/pingcap/tidb/issues/25202) + - Fix the incorrect result of the merge join operation that might occur when the column is the `SET` type [#25669](https://github.com/pingcap/tidb/issues/25669) + - Fix the issue that TiDB returns wrong results for cartesian join [#25591](https://github.com/pingcap/tidb/issues/25591) + - Fix the panic issue that occurs when `SELECT ... FOR UPDATE` works on a join operation and the join uses a partitioned table [#20028](https://github.com/pingcap/tidb/issues/20028) + - Fix the issue that the cached `prepared` plan is incorrectly used for `point get` [#24741](https://github.com/pingcap/tidb/issues/24741) + - Fix the issue that the `LOAD DATA` statement can abnormally import non-utf8 data [#25979](https://github.com/pingcap/tidb/issues/25979) - Fix a potential memory leak issue that occurs when accessing the statistics via an HTTP API [#24650](https://github.com/pingcap/tidb/pull/24650) - - Fix a security issue that occurs when executing the `ALTER USER` statement [#25347](https://github.com/pingcap/tidb/pull/25347) - - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status. [#24918](https://github.com/pingcap/tidb/pull/24918) - - Fix the issue that invalid strings are not truncated when parsing `DateTime` [#22260](https://github.com/pingcap/tidb/pull/22260) - - Fix the issue that the `select into outfile` statement might have no result when the column type is `YEAR` [#22185](https://github.com/pingcap/tidb/pull/22185) + - Fix a security issue that occurs when executing the `ALTER USER` statement [#25225](https://github.com/pingcap/tidb/issues/25225) + - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status [#24879](https://github.com/pingcap/tidb/issues/24879) + - Fix the issue that invalid strings are not truncated when parsing `DateTime` [#22231](https://github.com/pingcap/tidb/issues/22231) + - Fix the issue that the `select into outfile` statement might have no result when the column type is `YEAR` [#22159](https://github.com/pingcap/tidb/issues/22159) + TiKV - - Fix the issue that the duration calculation might panic on certain platforms [#10572](https://github.com/tikv/tikv/pull/10572) + - Fix the issue that the duration calculation might panic on certain platforms [#related-issue](https://github.com/rust-lang/rust/issues/86470#issuecomment-877557654) - Fix the wrong function that casts `DOUBLE` to `DOUBLE` [#25200](https://github.com/pingcap/tidb/issues/25200) - Fix the issue that the panic log might be lost when using the async logger [#8998](https://github.com/tikv/tikv/issues/8998) - Fix the panic issue that occurs when building a snapshot twice if encryption is enabled [#10462](https://github.com/tikv/tikv/pull/10462) - Fix the wrong arguments type of the `json_unquote()` function in the coprocessor [#10176](https://github.com/tikv/tikv/issues/10176) - - Fix the issues of suspicious warnings during shutdown and the non-deterministic response from Raftstore [#10395](https://github.com/tikv/tikv/pull/10395) - - Fix the issue of backup threads leak [#10360](https://github.com/tikv/tikv/pull/10360) - - Fix the issue that Region split might panic and corrupt the metadata if the split process is too slow and Region merge is on-going [#9584](https://github.com/tikv/tikv/pull/9584) - - Fix the issue that the Region heartbeats prevent TiKV from splitting large Regions in some situation [#10274](https://github.com/tikv/tikv/pull/10274) - - Fix the wrong statistics caused by the format inconsistency of CM Sketch between TiKV and TiDB [#10433](https://github.com/tikv/tikv/pull/10433) - - Fix the wrong statistics of the `apply wait duration` metric [#9966](https://github.com/tikv/tikv/pull/9966) + - Fix the issues of suspicious warnings during shutdown and the non-deterministic response from Raftstore [#10353](https://github.com/tikv/tikv/issues/10353) [#10307](https://github.com/tikv/tikv/issues/10307) + - Fix the issue of backup threads leak [#10287](https://github.com/tikv/tikv/issues/10287) + - Fix the issue that Region split might panic and corrupt the metadata if the split process is too slow and Region merge is on-going [#8456](https://github.com/tikv/tikv/issues/8456) [#8783](https://github.com/tikv/tikv/issues/8783) + - Fix the issue that the Region heartbeats prevent TiKV from splitting large Regions in some situation [#10111](https://github.com/tikv/tikv/issues/10111) + - Fix the wrong statistics caused by the format inconsistency of CM Sketch between TiKV and TiDB [#25638](https://github.com/pingcap/tidb/issues/25638) + - Fix the wrong statistics of the `apply wait duration` metric [#9893](https://github.com/tikv/tikv/issues/9893) - Fix the "Missing Blob" error after using `delete_files_in_range` in Titan [#10232](https://github.com/tikv/tikv/pull/10232) + PD - - Fix a bug that the scheduler might reappear after executing the delete operation [#3825](https://github.com/pingcap/pd/pull/3825) - - Fix the data race issue that might occur when the scheduler is started before the temporary configuration is loaded [#3773](https://github.com/pingcap/pd/pull/3773) + - Fix a bug that the scheduler might reappear after executing the delete operation [#2572](https://github.com/tikv/pd/issues/2572) + - Fix the data race issue that might occur when the scheduler is started before the temporary configuration is loaded [#3771](https://github.com/tikv/pd/issues/3771) - Fix a PD panic issue that might occur during the Region scattering operation [#3761](https://github.com/pingcap/pd/pull/3761) - Fix the issue that the priority of some operators is not set correctly [#3703](https://github.com/pingcap/pd/pull/3703) - - Fix a PD panic issue that might occur when deleting the `evict-leader` scheduler from a non-existent store [#3680](https://github.com/pingcap/pd/pull/3680) + - Fix a PD panic issue that might occur when deleting the `evict-leader` scheduler from a non-existent store [#3660](https://github.com/tikv/pd/issues/3660) - Fix the issue that the PD Leader re-election is slow when there are many stores [#3697](https://github.com/tikv/pd/issues/3697) + TiDB Dashboard - Fix the issue that the **Profiling** UI cannot profile all TiDB instances [#944](https://github.com/pingcap/tidb-dashboard/pull/944) - Fix the issue that the **Statements** UI does not display "Plan Count" [#939](https://github.com/pingcap/tidb-dashboard/pull/939) - - Fix the issue that the **Slow Query** UI might display the "unknown field" error after cluster upgrade [#930](https://github.com/pingcap/tidb-dashboard/pull/930) + - Fix the issue that the **Slow Query** UI might display the "unknown field" error after cluster upgrade [#902](https://github.com/pingcap/tidb-dashboard/issues/902) + TiFlash @@ -151,7 +151,7 @@ TiDB version: 4.0.14 + Dumpling - - When using Dumpling to export to S3, the `s3:ListBucket` permission is no longer required on the entire bucket. Only the data source prefix itself is required. [#287](https://github.com/pingcap/dumpling/pull/287) + - When using Dumpling to export to S3, the `s3:ListBucket` permission is no longer required on the entire bucket. Only the data source prefix itself is required [#898](https://github.com/pingcap/br/issues/898) + TiCDC @@ -160,6 +160,6 @@ TiDB version: 4.0.14 - Fix the panic issue that occurs when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) - Reduce TiCDC's runtime memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) [#1957](https://github.com/pingcap/ticdc/pull/1957) - Fix a bug that some MySQL connection might leak after MySQL sink meets the error and pauses [#1945](https://github.com/pingcap/ticdc/pull/1945) - - Fix the issue that TiCDC changefeed cannot be created when start TS is less than current TS minus GC TTL [#1871](https://github.com/pingcap/ticdc/pull/1871) - - Reduce memory `malloc` in sort heap to avoid too much CPU overhead [#1862](https://github.com/pingcap/ticdc/pull/1862) + - Fix the issue that TiCDC changefeed cannot be created when start TS is less than current TS minus GC TTL [#1839](https://github.com/pingcap/ticdc/issues/1839) + - Reduce memory `malloc` in sort heap to avoid too much CPU overhead [#1853](https://github.com/pingcap/ticdc/issues/1853) - Fix a bug that the replication task might stop when moving a table [#1827](https://github.com/pingcap/ticdc/pull/1827) From 597b4ebbfd126b79ebb44adc7d3abb14c38d909f Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 23 Jul 2021 11:44:01 +0800 Subject: [PATCH 18/24] minor update --- releases/release-4.0.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 677ca96cec54d..a0d082d042507 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -36,7 +36,7 @@ TiDB version: 4.0.14 + TiDB - ?- Change the lock records into put record for the index keys using `point get` or `batch point get` for `UPDATE` reads [#26223](https://github.com/pingcap/tidb/pull/26223) + - Change the `LOCK` records into the `PUT` record for the index keys using `point get` or `batch point get` for `UPDATE` reads [#26223](https://github.com/pingcap/tidb/pull/26223) - Support the MySQL system variable `init_connect` and its associated features [#26031](https://github.com/pingcap/tidb/pull/26031) - Support the stable result mode to make the query results more stable [#26003](https://github.com/pingcap/tidb/pull/26003) - Support pushing down the built-in function `json_unquote()` to TiKV [#25721](https://github.com/pingcap/tidb/pull/25721) From c8c61d5ab6687146defa228462989785b768684a Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 23 Jul 2021 11:58:32 +0800 Subject: [PATCH 19/24] Update release-4.0.14.md --- releases/release-4.0.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index a0d082d042507..dd604e8449b87 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -99,7 +99,7 @@ TiDB version: 4.0.14 - Fix the issue that the duration calculation might panic on certain platforms [#related-issue](https://github.com/rust-lang/rust/issues/86470#issuecomment-877557654) - Fix the wrong function that casts `DOUBLE` to `DOUBLE` [#25200](https://github.com/pingcap/tidb/issues/25200) - Fix the issue that the panic log might be lost when using the async logger [#8998](https://github.com/tikv/tikv/issues/8998) - - Fix the panic issue that occurs when building a snapshot twice if encryption is enabled [#10462](https://github.com/tikv/tikv/pull/10462) + - Fix the panic issue that occurs when building a snapshot twice if encryption is enabled [#9786](https://github.com/tikv/tikv/issues/9786) [#10407](https://github.com/tikv/tikv/issues/10407) - Fix the wrong arguments type of the `json_unquote()` function in the coprocessor [#10176](https://github.com/tikv/tikv/issues/10176) - Fix the issues of suspicious warnings during shutdown and the non-deterministic response from Raftstore [#10353](https://github.com/tikv/tikv/issues/10353) [#10307](https://github.com/tikv/tikv/issues/10307) - Fix the issue of backup threads leak [#10287](https://github.com/tikv/tikv/issues/10287) From fff84c084035b75462fb7e4251987bd9355426e8 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 23 Jul 2021 16:08:25 +0800 Subject: [PATCH 20/24] Update releases/release-4.0.14.md Co-authored-by: kennytm --- releases/release-4.0.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index dd604e8449b87..7f4b16dcd49f4 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -144,7 +144,7 @@ TiDB version: 4.0.14 + TiDB Lightning - - Fix the issue that TiDB fails to parse the `DECIMAL` type data in Parquet files [#1276](https://github.com/pingcap/br/pull/1276) + - Fix the issue that TiDB Lightning fails to parse the `DECIMAL` type data in Parquet files [#1276](https://github.com/pingcap/br/pull/1276) - Fix the EOF error reported when TiDB Lightning splits the imported large CSV files [#1133](https://github.com/pingcap/br/issues/1133) - Fix a bug that an excessively large base value is generated when TiDB Lightning imports tables with the `auto_increment` column of the `FLOAT` or `DOUBLE` type [#1185](https://github.com/pingcap/br/pull/1185) - Fix the issue of TiDB Lightning panic that occurs when generating KV data larger than 4 GB [#1128](https://github.com/pingcap/br/pull/1128) From 77efdb4df4b70a06be2f73c4d7c1a0d1cddf5c29 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 23 Jul 2021 16:08:31 +0800 Subject: [PATCH 21/24] Update releases/release-4.0.14.md Co-authored-by: kennytm --- releases/release-4.0.14.md | 1 - 1 file changed, 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 7f4b16dcd49f4..3aad243ae18a3 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -156,7 +156,6 @@ TiDB version: 4.0.14 + TiCDC - Fix the issue of extra partition dispatching after adding new table partitions [#2205](https://github.com/pingcap/ticdc/pull/2205) - - Fix the panic issue that occurs when TiCDC fails to read `/proc/meminfo` [#2023](https://github.com/pingcap/ticdc/pull/2023) - Reduce TiCDC's runtime memory consumption [#2011](https://github.com/pingcap/ticdc/pull/2011) [#1957](https://github.com/pingcap/ticdc/pull/1957) - Fix a bug that some MySQL connection might leak after MySQL sink meets the error and pauses [#1945](https://github.com/pingcap/ticdc/pull/1945) From 19db34e3f273af643886e4aeceaa632086bfb245 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 26 Jul 2021 15:03:02 +0800 Subject: [PATCH 22/24] change date --- releases/release-4.0.14.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 3aad243ae18a3..8ff8a83661dc9 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -4,11 +4,11 @@ title: TiDB 4.0.14 Release Notes # TiDB 4.0.14 Release Notes -Release date: July 22, 2021 +Release date: July 29, 2021 TiDB version: 4.0.14 -## Compatibility Changes +## Compatibility changes + TiDB @@ -20,11 +20,11 @@ TiDB version: 4.0.14 - Change the default value of `merge-check-tick-interval` from `10` to `2` to speed up the Region merge process [#9676](https://github.com/tikv/tikv/pull/9676) -## Feature Enhancements +## Feature enhancements + TiKV - - Add a metric to monitor the number of pending PD heartbeats, which helps locate the issue of slow PD threads [#10008](https://github.com/tikv/tikv/pull/10008) + - Add a metric `pending` to monitor the number of pending PD heartbeats, which helps locate the issue of slow PD threads [#10008](https://github.com/tikv/tikv/pull/10008) - Support using the virtual-host addressing mode to make BR support the S3-compatible storage [#10242](https://github.com/tikv/tikv/pull/10242) + TiDB Dashboard @@ -36,7 +36,7 @@ TiDB version: 4.0.14 + TiDB - - Change the `LOCK` records into the `PUT` record for the index keys using `point get` or `batch point get` for `UPDATE` reads [#26223](https://github.com/pingcap/tidb/pull/26223) + - Change the `LOCK` record into the `PUT` record for the index keys using `point get` or `batch point get` for `UPDATE` reads [#26223](https://github.com/pingcap/tidb/pull/26223) - Support the MySQL system variable `init_connect` and its associated features [#26031](https://github.com/pingcap/tidb/pull/26031) - Support the stable result mode to make the query results more stable [#26003](https://github.com/pingcap/tidb/pull/26003) - Support pushing down the built-in function `json_unquote()` to TiKV [#25721](https://github.com/pingcap/tidb/pull/25721) @@ -76,7 +76,7 @@ TiDB version: 4.0.14 - Add metrics for the table memory consumption [#1884](https://github.com/pingcap/ticdc/pull/1884) - Add `capture-session-ttl` to the TiCDC server configuration [#2169](https://github.com/pingcap/ticdc/pull/2169) -## Bug Fixes +## Bug fixes + TiDB @@ -90,7 +90,7 @@ TiDB version: 4.0.14 - Fix the issue that the `LOAD DATA` statement can abnormally import non-utf8 data [#25979](https://github.com/pingcap/tidb/issues/25979) - Fix a potential memory leak issue that occurs when accessing the statistics via an HTTP API [#24650](https://github.com/pingcap/tidb/pull/24650) - Fix a security issue that occurs when executing the `ALTER USER` statement [#25225](https://github.com/pingcap/tidb/issues/25225) - - Fix the bug that `TIKV_REGION_PEERS` table did not have the correct `DOWN` status [#24879](https://github.com/pingcap/tidb/issues/24879) + - Fix a bug that the `TIKV_REGION_PEERS` table cannot correctly handle the `DOWN` status [#24879](https://github.com/pingcap/tidb/issues/24879) - Fix the issue that invalid strings are not truncated when parsing `DateTime` [#22231](https://github.com/pingcap/tidb/issues/22231) - Fix the issue that the `select into outfile` statement might have no result when the column type is `YEAR` [#22159](https://github.com/pingcap/tidb/issues/22159) @@ -104,7 +104,7 @@ TiDB version: 4.0.14 - Fix the issues of suspicious warnings during shutdown and the non-deterministic response from Raftstore [#10353](https://github.com/tikv/tikv/issues/10353) [#10307](https://github.com/tikv/tikv/issues/10307) - Fix the issue of backup threads leak [#10287](https://github.com/tikv/tikv/issues/10287) - Fix the issue that Region split might panic and corrupt the metadata if the split process is too slow and Region merge is on-going [#8456](https://github.com/tikv/tikv/issues/8456) [#8783](https://github.com/tikv/tikv/issues/8783) - - Fix the issue that the Region heartbeats prevent TiKV from splitting large Regions in some situation [#10111](https://github.com/tikv/tikv/issues/10111) + - Fix the issue that the Region heartbeats prevent TiKV from splitting large Regions in some situations [#10111](https://github.com/tikv/tikv/issues/10111) - Fix the wrong statistics caused by the format inconsistency of CM Sketch between TiKV and TiDB [#25638](https://github.com/pingcap/tidb/issues/25638) - Fix the wrong statistics of the `apply wait duration` metric [#9893](https://github.com/tikv/tikv/issues/9893) - Fix the "Missing Blob" error after using `delete_files_in_range` in Titan [#10232](https://github.com/tikv/tikv/pull/10232) @@ -140,7 +140,7 @@ TiDB version: 4.0.14 + Backup & Restore (BR) - - Fix the issue that data restore from the `mysql` schema might fail [#1142](https://github.com/pingcap/br/pull/1142) + - Fix the issue that the data restore from the `mysql` schema might fail [#1142](https://github.com/pingcap/br/pull/1142) + TiDB Lightning @@ -151,7 +151,7 @@ TiDB version: 4.0.14 + Dumpling - - When using Dumpling to export to S3, the `s3:ListBucket` permission is no longer required on the entire bucket. Only the data source prefix itself is required [#898](https://github.com/pingcap/br/issues/898) + - When using Dumpling to export data to the S3 storage, the `s3:ListBucket` permission is no longer required on the entire bucket. The permission is required only on the data source prefix. [#898](https://github.com/pingcap/br/issues/898) + TiCDC From 1d68e4f6df53fd4ccf03f5b0779ea3cca255a7fe Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 27 Jul 2021 14:51:12 +0800 Subject: [PATCH 23/24] Update releases/release-4.0.14.md --- releases/release-4.0.14.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 8ff8a83661dc9..25661de314b89 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -93,6 +93,8 @@ TiDB version: 4.0.14 - Fix a bug that the `TIKV_REGION_PEERS` table cannot correctly handle the `DOWN` status [#24879](https://github.com/pingcap/tidb/issues/24879) - Fix the issue that invalid strings are not truncated when parsing `DateTime` [#22231](https://github.com/pingcap/tidb/issues/22231) - Fix the issue that the `select into outfile` statement might have no result when the column type is `YEAR` [#22159](https://github.com/pingcap/tidb/issues/22159) + - Fix the issue that the query result might be wrong when `NULL` is in the `UNION` subquery [#26532](https://github.com/pingcap/tidb/issues/26532) + - Fix the issue that the projection operator in execution might cause panic in some cases [#26534](https://github.com/pingcap/tidb/pull/26534) + TiKV From be1ab36028ba9cd2279ea79bd56574cf6916f607 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 27 Jul 2021 18:05:12 +0800 Subject: [PATCH 24/24] Update releases/release-4.0.14.md --- releases/release-4.0.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.14.md b/releases/release-4.0.14.md index 25661de314b89..00c5ab903a326 100644 --- a/releases/release-4.0.14.md +++ b/releases/release-4.0.14.md @@ -4,7 +4,7 @@ title: TiDB 4.0.14 Release Notes # TiDB 4.0.14 Release Notes -Release date: July 29, 2021 +Release date: July 27, 2021 TiDB version: 4.0.14