From 4c6ba0071052a41e61da3144b675268ffdb189b0 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 18 Nov 2021 10:39:12 +0800 Subject: [PATCH 01/28] releases: add TiDB 5.3.0 release notes --- TOC.md | 2 + _index.md | 2 +- releases/release-5.3.0.md | 388 +++++++++++++++++++++++++++++++++++ releases/release-notes.md | 4 + releases/release-timeline.md | 1 + 5 files changed, 396 insertions(+), 1 deletion(-) create mode 100644 releases/release-5.3.0.md diff --git a/TOC.md b/TOC.md index 68c97d3730960..e76ee3fcf5d36 100644 --- a/TOC.md +++ b/TOC.md @@ -574,6 +574,8 @@ + Release Notes + [All Releases](/releases/release-notes.md) + [Release Timeline](/releases/release-timeline.md) + + v5.3 + + [5.3.0](/releases/release-5.3.0.md) + v5.2 + [5.2.2](/releases/release-5.2.2.md) + [5.2.1](/releases/release-5.2.1.md) diff --git a/_index.md b/_index.md index 4123282d1a73f..896dc05ce32a9 100644 --- a/_index.md +++ b/_index.md @@ -16,7 +16,7 @@ Designed for the cloud, TiDB provides flexible scalability, reliability and secu - [TiDB Introduction](/overview.md) - [Basic Features](/basic-features.md) -- [TiDB 5.2 Release Notes](/releases/release-5.2.0.md) +- [TiDB 5.3 Release Notes](/releases/release-5.3.0.md) - [TiDB Release Timeline](/releases/release-timeline.md) - [Compatibility with MySQL](/mysql-compatibility.md) - [Usage Limitations](/tidb-limitations.md) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md new file mode 100644 index 0000000000000..180b4a399a8bb --- /dev/null +++ b/releases/release-5.3.0.md @@ -0,0 +1,388 @@ +--- +title: TiDB 5.3 Release Notes +--- + +# TiDB 5.3 Release Notes + +Release date: November 23, 2021 + +TiDB version: 5.3.0 + +In v5.3, the key new features or improvements are as follows: + ++ Introduce temporary tables to simplify your application logic and improve performance ++ Support setting attributes for tables and partitions ++ Support creating users with minimum privilege on TiDB Dashboard to enhance system security ++ Optimize the process of handling timestamps in TiDB to improve the overall performance ++ Enhance the performance of TiDB Data Migration (DM) so that data is migrated from MySQL to TiDB with lower latency ++ Support parallel import in TiDB Lightning to improve the efficiency of full data migration ++ Support saving and restoring the on-site information of a cluster with a single SQL statement, which helps improve the efficiency of troubleshooting execution plan issues ++ Support the continuous profiling experimental feature to improve the observability of database performance ++ Continue optimizing the storage and computing engines to improve the system performance and stability + +## Compatibility changes + +> **Note:** +> +> When upgrading from an earlier TiDB version to v5.3.0, if you want to know the compatibility change notes of all intermediate versions, you can check the [Release Notes](/releases/release-notes.md) of the corresponding version. + +### System variables + +| Variable name | Change type | Description | +| :---------- | :----------- | :----------- | +| [tidb_enable_noop_functions](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | No longer controls the `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` behaviors. | +| [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Newly added | Controls the behavior of the optimizer when the statistics on a table expire. The default value is `ON`. When the number of modified rows in the table is greater than 80% of the total rows (This ratio can be adjusted by the configuration [`pseudo-estimate-ratio`](/tidb-configuration-file.md#pseudo-estimate-ratio)), the optimizer considers that the statistics other than the total number of rows are no longer reliable and use pseudo statistics instead. When you set the value as `OFF`, even if the statistics expire, the optimizer still uses them. | +|[`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53) | Newly added | Determines whether to enable or disable the TSO Follower Proxy feature. The default value is `OFF`, which means the TSO Follower Proxy feature is disabled. At this time, TiDB only gets TSO from PD leader. When this feature is enabled, TiDB evenly sends the requests to all PD nodes when acquiring TSO. The PD follower then forwards the TSO requests to reduce the CPU pressure of PD leader. | +|[`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) | Newly added | Sets the maximum waiting time for a batch saving operation when TiDB requests TSO from PD. The default value is `0`, which means no additional waiting. | +| [tidb_tmp_table_max_size](/system-variables.md#tidb_tmp_table_max_size-new-in-v53) | Newly added | Limits the maximum size of a single [temporary table](/temporary-tables.md). If the temporary table exceeds this size, an error will occur. | + +### Configuration file parameters + +| Configuration file | Configuration item | Change type | Description | +| :---------- | :----------- | :----------- | :----------- | +| TiDB | [`prepared-plan-cache.capacity`](/tidb-configuration-file.md#capacity) | Modified | Controls the number of cached statements. The default value is changed from `100` to `1000`.| +| TiKV | [`storage.reserve-space`](/tikv-configuration-file.md#reserve-space) | Modified | Controls space reserved for disk protection when TiKV is started. Starting from v5.3.0, 80% of the reserved space is spared for operations and maintenance when disk space is insufficient, and the remaining 20% space stores temporary disk files. | +| TiKV | memory-usage-limit | Modified | This configuration item is new in TiDB v5.3.0 and its value is calculated based on storage.block-cache.capacity. | +| PD | [`log.file.max-days`](/pd-configuration-file.md#max-days) | Modified | Controls the maximum number of days that logs are retained. The default value is changed from `1` to `0`. | +| PD | [`log.file.max-backups`](/pd-configuration-file.md#max-backups) | Modified | Controls the maximum number of logs that are retained. The default value is changed from `7` to `0`. | +| PD | [`patrol-region-interval`](/pd-configuration-file.md#patrol-region-interval) | Modified | Controls the running frequency at which replicaChecker checks the health state of a Region. The smaller this value is, the faster replicaChecker runs. Normally, you do not need to adjust this parameter. The default value is changed from `100ms` to `10ms`. | +| PD | [`max-snapshot-count`](/pd-configuration-file.md#max-snapshot-count) | Modified | Controls the maximum number of snapshots that a single store receives or sends at the same time. PD schedulers depend on this configuration to prevent the resources used for normal traffic from being preempted. The default value is changed from `3` to `64`. | +| PD | [`max-pending-peer-count`](/pd-configuration-file.md#max-pending-peer-count) | Modified | Controls the maximum number of pending peers in a single store. PD schedulers depend on this configuration to prevent too many Regions with outdated logs from being generated on some nodes. The default value is changed from `16` to `64`. | + +### Others + +- Temporary tables: + + - Regarding local temporary tables, if you have created local temporary tables on a cluster earlier than TiDB v5.3.0, these tables are actually ordinary tables, and handled as ordinary tables after the cluster is upgraded to TiDB v5.3.0 or later versions. Regarding global temporary tables, if you have created global temporary tables on a cluster in TiDB v5.3.0 or later versions, when the cluster is downgraded to a version earlier than v5.3.0, these tables are handled as ordinary tables and a data error occures. + - TiCDC and BR support [global temporary tables](/temporary-tables.md#global-temporary-tables) since v5.3.0. If you use TiCDC and BR in a version earlier than v5.3.0 to replicate global temporary tables to the downstream, a table definition error occurs. + - The following clusters should be in TiDB v5.3.0 or later versions: the cluster to be exported using TiDB ecosystem tools, the cluster after data is restored using the tools, and the downstream cluster that has replications using the tools. Otherwise, an error is reported when creating a global temporary table. + - For the compatibility of temporary tables, refer to [Compatibility with MySQL temporary tables](/temporary-tables.md#compatibility-with-mysql-temporary-tables) and [Compatibility restrictions with other TiDB features](/temporary-tables.md#compatibility-restrictions-with-other-tidb-features) + +- Fix an issue that `SHOW CREATE VIEW` does not require the `SHOW VIEW` permission. Now you must have `SHOW VIEW` permission to execute the `SHOW CREATE VIEW` statement. +- The system variable `sql_auto_is_null` is added to the Noop Function. When `tidb_enable_noop_functions = 0/OFF`, an error is reported after modified this variable. +- TiDB no longer allows to execute the `GRANT ALL ON performance_schema.*` syntax. When you execute the statement in TiDB, an error is reported. +- Fix an issue that auto-analyze is triggered outside the specified time period when new indexes are added in earlier versions of v5.3.0. In v5.3.0, after you set the time period with `tidb_auto_analyze_start_time` and `tidb_auto_analyze_end_time`, auto-analyze is only triggered during this time period. +- The default storage directory for plugins is changed from "" to /data/deploy/plugin. +- The DM code is migrated to [the folder "dm" in TiCDC code repository](https://github.com/pingcap/ticdc/tree/master/dm). The version numbers of DM are changed from v2.0.x to v5.3.0, and you can upgrade from v2.0.x to v5.3.0 without any risk. + +## New features + +### SQL + +- **Use SQL interface to set placement rules for data (experimental)** + + Support the `[CREATE | ALTER] PLACEMENT POLICY` syntax that provides a SQL interface to set placement rules for data. Using this feature, you can specify tables and partitions to be scheduled to specific regions, data centers, racks, or hosts. This meets your application demands for lower cost and higher flexibility. The typical user scenarios are as follows: + + - Merge multiple databases of different applications to reduce the cost on database maintenance + - Increase replica count for important data to improve the application availability and data reliability + - Store new data into SSDs and store old data into HHDs to lower the cost on data archiving and storage + - Schedule the leaders of hotspot data to high-performance TiKV instances + - Separate cold data to lower-cost storage mediums to improve cost efficiency + + [User document](/information-schema/information-schema-placement-rules.md), [#18030](https://github.com/pingcap/tidb/issues/18030) + +- **Temporary tables** + + Support the `CREATE [GLOBAL] TEMPORARY TABLE` syntax to create temporary tables. Using this feature, you can easily manage the temporary data generated in the calculation process of an application. Temporary data is stored in memory and you can use the `tidb_tmp_table_max_size` variable to limit the size of a temporary table. TiDB supports the following types of temporary tables: + + - Global temporary tables + - Visible to all sessions in the cluster, and table schemas are persistent. + - Provides transaction-level data isolation. The temporary data is effective only in the transaction. After the transaction finishes, the data is automatically dropped. + - Local temporary tables + - Visible only to the current session, and tables schemas are not persistent. + - Supports duplicated table names. You do not need to design complicated naming rules for your application. + - Provides session-level data isolation, which enables you to design a simpler application logic. After the transaction finishes, the temporary tables are dropped. + + [User document](/temporary-tables.md), [#24169](https://github.com/pingcap/tidb/issues/24169) + +- **Support the `FOR UPDATE OF TABLES` syntax** + + For a SQL statement that joins multiple tables, TiDB supports acquiring pessimistic locks on the rows correlated to the tables that are included in `OF TABLES`. + + [User document](/sql-statements/sql-statement-select.md), [#28689](https://github.com/pingcap/tidb/issues/28689) + +- **Set table attributes** + + Support the `ALTER TABLE [PARTITION] ATTRIBUTES` statement that allows you to set attributes to a table or partition. Currently, TiDB only supports setting the `merge_option` attribute. By adding this attribute, you can directly control the Region merge behavior. + + User scenarios: When you perform the `SPLIT TABLE` operation, if no data is inserted after a certain period of time, the empty Region is automatically merged by default. In this case, you can set the table attribute to `merge_option=deny` to avoid the automatic merging of the Region. + + [User document](/table-attributes.md),[#3839](https://github.com/tikv/pd/issues/3839) + +### Security + +- **Support creating a least-privileged user for TiDB Dashboard** + +The account system of TiDB Dashboard is consistent with that of TiDB SQL. Users accessing TiDB Dashboard are authenticated and authorized based on TiDB SQL user's privileges. Therefore, TiDB Dashboard requires limited privileges, or merely the read-only privilege. You can configure users to access TiDB Dashboard based on the principle of least privilege, thus avoiding access of high-privileged users. + +It is recommended that you create a least-privileged SQL user to access and sign in with TiDB Dashboard. This avoids access of high-privileged users and improves security. + +[User document](/dashboard/dashboard-user.md) + +### Performance + +- **Optimize the PD timestamp processing flow** + + TiDB optimizes its timestamp processing flow and reduces the timestamp processing load of PD by enabling PD Follower Proxy and modifying the batch waiting time when the PD client requests TSO in batches. This helps improve the overall scalability of the system. + + - Support enabling or disabling PD Follower Proxy through the system variable [`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53). Suppose that the TSO requests load of PD is too high. In this case, enabling PD follower proxy can batch forward the TSO requests collected during the request cycle on followers to leader nodes. This solution can effectively reduce the number of direct interactions between clients and leaders, reduce the pressure of the leaders' load, and improve the overall performance of TiDB. + + > **Note:** + > + > When the number of clients is small and the PD leader CPU is not full, it is NOT recommended to enable PD Follower Proxy. + + - Support setting the maximum waiting time through the system variable [`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53). This time is the waiting time for batching TSO requests when PD client requests TSO in batches. The unit of this time is milliseconds. In case that PD has a high TSO requests load, you can reduce the load and improve the throughput by increasing the waiting time to get a larger batch size. + + > **Note:** + > + > When the TSO request load is not high, it is NOT recommended to modify this variable. + + [User document](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53),[#3149](https://github.com/tikv/pd/issues/3149) + +### Stability + +- **Support Online Unsafe Recovery after stores permanently damaged (experimental feature)** + + Support the `unsafe remove-failed-stores` command that allows data unsafe recovery. Suppose that multiple replicas of data encounter problems like permanent damage (such as disk damage), and this issue causes the data in an application to be not readable or writable. In this case, Online Unsafe Recovery using PD can recover the data to be readable or writable again. + +> **Warning:** +> +> - This feature is a lossy recovery, so TiKV cannot guarantee data integrity and data indexes integrity after using the feature. +> - Online unsafe recovery is an experimental feature, and it is **not** recommended to use it in the production environment. The interface, strategy, and internal implementation of the feature might be changed when it is finally released. Also, although the feature is tested in some scenarios, it is not thoroughly validated, so the feature might cause system unavailability. +> - It is recommended to perform the feature-related operations with the support of the TiDB team. If some incorrect operations are performed, it might be hard to recover the cluster. + +[User document](/online-unsafe-recovery.md), [#10483](https://github.com/tikv/tikv/issues/10483) + +### Data migration + +- **DM replication performance enhanced** + + Supports the following features to ensure lower-latency data replication from MySQL to TiDB: + + - Compact multiple updates on a single row into one statement + - Merge batch updates of multiple rows into one statement + +- **Add DM OpenAPI to better maintain DM clusters (experimental feature)** + + DM provides the OpenAPI feature for querying and operating the DM cluster. It is similar to the feature of [dmctl tools](./dmctl-introduction.md). + + Currently, DM OpenAPI is an experimental feature and disabled by default. It is not recommended to use it in a production environment. + + [User Document](https://docs.pingcap.com/zh/tidb-data-migration/stable/open-api) + +- **TiDB Lightning Parallel Import** + + TiDB Lightning provides parallel import capability to extend the original feature. It allows you to deploy multiple Lightning instances at the same time, and import single tables or multiple tables to downstream TiDB in parallel. Without changing the way customers use it, it greatly improves the data migration ability, allowing you to obtain data in a more real-time way to further process, integrate and analyze them. It improves the efficiency of enterprise data management. + + In our test, using 10 TiDB Lightning instances, a total of 20 TiB MySQL data can be imported to TiDB within 8 hours. The performance of multiple table import is also improved. A single TiDB Lightning instance can support importing at 250 GB/s, 8 times the original performance. + + [User Document](/tidb-lightning/tidb-lightning-distributed-import.md) + +- **TiDB Lightning Prechecks** + + TiDB Lightning provides the ability to check the configuration before running a migration task. It is enabled by default. This feature automatically performs some routine checks for disk space and execution configuration. The main purpose is to ensure that the whole subsequent import process goes smoothly. + + [User Document](tidb-lightning/tidb-lightning-prechecks.md) + +- **TiDB Lightning supports importing files of GBK character set** + + You can specify the character set of the source data file. TiDB Lightning will convert the source file from the specified character set to UTF-8 encoding during the import process. + + [User Document](/tidb-lightning/tidb-lightning-configuration.md) + +- **Sync-diff-inspector improvement** + + - Improve the comparison speed from 375 MB/s to 700 MB/s + - Reduce the memory consumption of TiDB nodes by nearly half during comparison + - Optimize the user interface and display the progress bar during comparison + + [User Document](/sync-diff-inspector/sync-diff-inspector-overview.md) + +### Diagnostic efficiency + +- **Save and restore the on-site information of a cluster** + + When you locate and troubleshoot the issues of a TiDB cluster, you often need to provide information on the system and the execution plan. To help you get the information and troubleshoot cluster issues in a more convenient and efficient way, the `PLAN REPLAY` command is introduced in TiDB v5.3.0. This command enables you to easily save and restore the on-site information of a cluster, improves the efficiency of troubleshooting, and helps you more easily archive the issue for management. + + The features of `PLAN REPLAYER` are as follows: + + - Exports the information of a TiDB cluster at an on-site troubleshooting to a ZIP-formatted file for storage. + - Imports into a cluster the ZIP-formatted file exported from another TiDB cluster. This file contains the information of the latter TiDB cluster at an on-site troubleshooting. + + [User document](/sql-plan-replayer.md), [#26325](https://github.com/pingcap/tidb/issues/26325) + +### TiDB data share subscription + +- **TiCDC Eventually Consistent Replication** + + TiCDC provides the eventually consistent replication capability in disaster scenarios. When a disaster occurs in the primary TiDB cluster and the service cannot be resumed in a short period of time, TiCDC needs to provide the ability to ensure the consistency of data in the secondary cluster. Meanwhile, TiCDC needs to allow the business to quickly switch the traffic to the secondary cluster to avoid the database being unavailable for a long time and affecting the business. + + This feature supports TiCDC to replicate incremental data from a TiDB cluster to the secondary relational database TiDB/Aurora/MySQL/MariaDB. In case the primary cluster crashes, TiCDC can recover the secondary cluster to a certain snapshot in the primary cluster within 30 minutes, given the condition that before disaster the replication status of TiCDC is normal and replication lag is small. It allows data loss of less than 5 minutes, that is, RPO <= 30min, and RTO <= 5min. + + [User Document](/ticdc/manage-ticdc.md) + +### Deployment and O&M + +- **Continuous Profiling (experimental feature)** + + Introduced in TiDB Dashboard, Continuous Profiling allows storing instance performance analysis results automatically in real time without interrupting TiDB clusters. The performance analysis result is displayed using a flame graph, which is observable and shortens troubleshooting time. + + This feature is disabled by default and needs to be enabled on the **Continuous Profile** page of TiDB Dashboard. + + This feature is only available for clusters upgraded or installed using TiUP v1.7.0 or above. + + [User document](/dashboard/continuous-profiling.md) + +## Removed feature + +Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters (an experimental feature in v5.0.0) has been removed. If you have already used this feature to replicate data before upgrading TiCDC, the related data is not affected after the upgrade. + +## Improvements + ++ TiDB + + - Show the affected SQL statements in the debug log when the coprocessor encounters a lock, which is helpful in diagnosing problems [#27718](https://github.com/pingcap/tidb/issues/27718) + - Support showing the size of the backup and restore data when backing up and restoring data in the SQL logical layer [#27247](https://github.com/pingcap/tidb/issues/27247) + - Improve the default collection logic of ANALYZE when `tidb_analyze_version=2`, which improves the collection speed and reduces resource overhead + - Introduce the `ANALYZE TABLE table_name COLUMNS col_1, col_2, ... , col_n` syntax. The syntax provides a solution for wide tables to collect statistics only on a portion of the columns, which improves the speed of collecting statistics for wide tables. + ++ TiKV + + - Enhance disk space protection to improve storage stability + + To solve the issue that TiKV might panic when getting the disk fully written error, TiKV introduces a two-level threshold defense mechanism for the disk remaining space to prevent excess traffic from exhausting disk space. Additionally, the mechanism provides the ability to reclaim space when the threshold is triggered. When the remaining space threshold is triggered, some write operations will fail and return a disk full error and a list of disk full nodes. In this case, to recover the space and restore the service, you can execute `Drop/Truncate Table` or scale out the nodes. + + - Simplify the algorithm of L0 flow control [#10879](https://github.com/tikv/tikv/issues/10879) + - Improve the error log report in the raft client module [#10944](https://github.com/tikv/tikv/pull/10944) + - Improve logging threads to avoid them becoming a performance bottleneck [#10841](https://github.com/tikv/tikv/issues/10841) + - Add more statistics types of write queries [#10507](https://github.com/tikv/tikv/issues/10507) + ++ PD + + - Add more types of write queries to QPS dimensions in the hotspot scheduler [#3869](https://github.com/tikv/pd/issues/3869) + - Support dynamically adjusting the retry limit of the balance region scheduler to improve the performance of the scheduler [#3744](https://github.com/tikv/pd/issues/3744) + - Update TiDB Dashboard to v2021.10.08.1 [#4070](https://github.com/tikv/pd/pull/4070) + - Support that the evict leader scheduler can schedule regions with unhealthy peers [#4093](https://github.com/tikv/pd/issues/4093) + - Speed up the exit process of schedulers [#4146](https://github.com/tikv/pd/issues/4146) + ++ TiFlash + + - Improve the execution efficiency of the TableScan operator greatly + - Improve the execution efficiency of the Exchange operator + - Reduce write amplification and memory usage during GC of the storage engine (experimental feature) + - Improve the stability and availability of TiFlash when TiFlash restarts, which reduces the possibility of query failures during a short period of time after the restart + - Support pushing down multiple new String and Time functions to the MPP engine + + - String functions: LIKE pattern, FORMAT(), LOWER(), LTRIM(), RTRIM(), SUBSTRING_INDEX(), TRIM(), UCASE(), UPPER() + - Mathematical functions: ROUND (decimal, int) + - Date and time functions: HOUR (), MICROSECOND (), MINUTE (), SECOND (), SYSDATE () + - Type conversion function: CAST(time, real) + - Aggregation functions: GROUP_CONCAT(), SUM(enum) + + - Support 512-bit SIMD + - Enhance the cleanup algorithm for outdated data versions to reduce disk usage and improve read file performance + - Fix the issue that users cannot get memory or CPU information when viewing the dashboard on some non-Linux systems + - Unify the naming style of TiFlash log files (keep the naming style consistent with that of TiKV) and support dynamic modification of logger.count and logger.size + - Improve the data validation capability of column-based files (checksums, experimental feature) + ++ Tools + + + TiCDC + + - Reduce the default value of the Kafka sink configuration item `MaxMessageBytes` from 64 MB to 1 MB to fix the issue that large messages are rejected by the Kafka Broker [#3104](https://github.com/pingcap/ticdc/pull/3104) + - Reduce memory usage in the relpication pipeline [#2553](https://github.com/pingcap/ticdc/issues/2553)[#3037](https://github.com/pingcap/ticdc/pull/3037) [#2726](https://github.com/pingcap/ticdc/pull/2726) + - Optimize monitoring items and alert rules to improve observability of synchronous links, memory GC, and stock data scanning processes [#2735](https://github.com/pingcap/ticdc/pull/2735) [#1606](https://github.com/pingcap/ticdc/issues/1606) [#3000](https://github.com/pingcap/ticdc/pull/3000) [#2985](https://github.com/pingcap/ticdc/issues/2985) [#2156](https://github.com/pingcap/ticdc/issues/2156) + - When the sync task status is normal, no more historical error messages are displayed to avoid misleading users [#2242](https://github.com/pingcap/ticdc/issues/2242) + +## Bug Fixes + ++ TiDB + + - Fix an error that occurs during execution caused by the wrong execution plan. The wrong execution plan is caused by the shallow copy of schema columns when pushing down the aggregation operators on partitioned tables. [#27797](https://github.com/pingcap/tidb/issues/27797) [#26554](https://github.com/pingcap/tidb/issues/26554) + - Fix the issue that plan-cache cannot detect changes of unsigned flags [#28254](https://github.com/pingcap/tidb/issues/28254) + - Fix the wrong partition pruning when the partition function is out of range [#28233](https://github.com/pingcap/tidb/issues/28233) + - Fix the issue that planner might cache invalid plans for `join` in some cases [#28087](https://github.com/pingcap/tidb/issues/28087) + - Fix wrong index hash join when hash column type is enum [#27893](https://github.com/pingcap/tidb/issues/27893) + - Fix a batch client bug that recycling idle connection might block sending requests in some rare cases [#27688](https://github.com/pingcap/tidb/pull/27688) + - Fix the TiDB Lightning panic issue when it fails to perform checksum on a target cluster [#27686](https://github.com/pingcap/tidb/pull/27686) + - Fix wrong results of the `date_add` and `date_sub` functions in some cases [#27232](https://github.com/pingcap/tidb/issues/27232) + - Fix wrong results of the `hour` function in vectorized expression [#28643](https://github.com/pingcap/tidb/issues/28643) + - Fix the authenticating issue when connecting to MySQL 5.1 or an older client version [#27855](https://github.com/pingcap/tidb/issues/27855) + - Fix the issue that auto analyze might be triggered out of the specified time when a new index is added [#28698](https://github.com/pingcap/tidb/issues/28698) + - Fix a bug that setting any session variable invalidates `tidb_snapshot` [#28683](https://github.com/pingcap/tidb/pull/28683) + - Fix a bug that BR is not working for clusters with many missing-peer regions [#27534](https://github.com/pingcap/tidb/issues/27534) + - Fix the unexpected error like `tidb_cast to Int32 is not supported` when the unsupported `cast` is pushed down to TiFlash [#23907](https://github.com/pingcap/tidb/issues/23907) + - Fix the issue that `DECIMAL overflow` is missing in the `%s value is out of range in '%s'`error message [#27964](https://github.com/pingcap/tidb/issues/27964) + - Fix a bug that the availability detection of MPP node does not work in some corner cases [#3118](https://github.com/pingcap/tics/issues/3118) + - Fix the `DATA RACE` issue when assigning `MPP task ID` [#27952](https://github.com/pingcap/tidb/issues/27952) + - Fix the `INDEX OUT OF RANGE` error for a MPP query after deleting an empty `dual table`. [#28250](https://github.com/pingcap/tidb/issues/28250) + - Fix the issue of false positive error log `invalid cop task execution summaries length` for MPP queries [#1791](https://github.com/pingcap/tics/issues/1791) + - Fix the issue of error log `cannot found column in Schema column` for MPP queries [#28149](https://github.com/pingcap/tidb/pull/28149) + - Fix the issue that TiDB might panic when TiFlash is shuting down [#28096](https://github.com/pingcap/tidb/issues/28096) + - Remove the support for insecure 3DES (Triple Data Encryption Algorithm) based TLS cipher suites [#27859](https://github.com/pingcap/tidb/pull/27859) + - Fix the issue that Lightning connects to offline TiKV nodes during pre-check and causes import failures [#27826](https://github.com/pingcap/tidb/pull/27826) + - Fix the issue that pre-check cost too much time when importing many files to tables [#27605](https://github.com/pingcap/tidb/issues/27605) + - Fix the issue that rewriting expressions makes `between` infer wrong collation [#27146](https://github.com/pingcap/tidb/issues/27146) + - Fix the issue that `group_concat` function did not consider the collation [#27429](https://github.com/pingcap/tidb/issues/27429) + - Fix the result wrong that occurs when the argument of the `extract` function is a negative duration [#27236](https://github.com/pingcap/tidb/issues/27236) + - Fix the issue that creating partition fails if `NO_UNSIGNED_SUBTRACTION` is set [#26765](https://github.com/pingcap/tidb/issues/26765) + - Avoid expressions with side effects in column pruning and aggregation pushdown [#27106](https://github.com/pingcap/tidb/issues/27106) + - Remove useless gRPC logs [#24190](https://github.com/pingcap/tidb/issues/24190) + - Limit the valid decimal length to fix precision-related issues [#3091](https://github.com/pingcap/tics/issues/3091) + - Fix the issue of a wrong way to check for overflow in `plus` expression [#26977](https://github.com/pingcap/tidb/issues/26977) + - Fix the issue of `data too long` error when dumping statistics from the table with `new collation` data [#27024](https://github.com/pingcap/tidb/issues/27024) + - Fix the issue that the retried transactions' statements are not included in `TIDB_TRX` [#28670](https://github.com/pingcap/tidb/pull/28670) + - Fix the wrong default value of the `plugin_dir` configuration [28084](https://github.com/pingcap/tidb/issues/28084) + ++ TiKV + + - Fix the issue of unavailable TiKV caused by Raftstore deadlock when migrating Regions. The workaround is to disable the scheduling and restart the unavailable TiKV. [#10909](https://github.com/tikv/tikv/issues/10909) + - Fix the issue that CDC adds scan retries frequently due to the Congest error [#11082](https://github.com/tikv/tikv/issues/11082) + - Fix the issue that the raft connection is broken when the channel is full [#11047](https://github.com/tikv/tikv/issues/11047) + - Fix the issue that batch messages are too large in Raft client implementation [#9714](https://github.com/tikv/tikv/issues/9714) + - Fix the issue that some coroutines leak in `resolved_ts` [#10965](https://github.com/tikv/tikv/issues/10965) + - Fix a panic issue that occurs to the coprocessor when the size of response exceeds 4 GiB [#9012](https://github.com/tikv/tikv/issues/9012) + - Fix the issue that snapshot Garbage Collection (GC) misses GC snapshot files when snapshot files cannot be garbage collected [#10813](https://github.com/tikv/tikv/issues/10813) + - Fix a panic issue caused by timeout when processing Coprocessor requests [#10852](https://github.com/tikv/tikv/issues/10852) + - Fix a memory leak caused by monitoring data of statistics threads [#11195](https://github.com/tikv/tikv/issues/11195) + - Fix a panic issue caused by getting the cgroup information from some platforms [#10980](https://github.com/tikv/tikv/pull/10980) + ++ PD + + - Fix the issue that PD incorrectly delete the peers with data and in pending status because the number of peers exceeds the number of configured peers [#4045](https://github.com/tikv/pd/issues/4045) + - Fix the issue that PD does not fix down peers in time [#4077](https://github.com/tikv/pd/issues/4077) + - Fix the issue that the scatter range scheduler cannot schedule empty regions [#4118](https://github.com/tikv/pd/pull/4118) + - Fix the issue that the key manager cost too much CPU [#4071](https://github.com/tikv/pd/issues/4071) + - Fix the data race issue that might occur when setting configurations of hot region scheduler [#4159](https://github.com/tikv/pd/issues/4159) + - Fix slow leader election caused by stucked region syncer[#3936](https://github.com/tikv/pd/issues/3936) + ++ TiFlash + + - Fix the inaccurate issue of the TiFlash Store Size statistics + - Fix the issue that TiFlash fails to start up on some platforms due to the absence of library `nsl` + - Block the infinite wait of `wait index` when writing pressure is heavy (a default timeout of 5 minutes is added), which prevents TiFlash from waiting too long for data synchronization to provide services + - Fix the slow and no result issues of the log search when the log volume is large + - Fix the issue that only the most recent logs can be searched when searching old historical logs + - Fix the wrong result that might occur when a new collation is enabled + - Fix the possible parsing errors when an SQL statement contains extremely long nested expressions + - Fix the `Block schema mismatch` error of the Exchange operator + - Fix the `Can't compare` error when comparing Decimal types + - Fix the `3rd arguments of function substringUTF8 must be constants` error of the `left/substring` function + ++ Tools + + + TiCDC + + - Fix the issue that TiCDC replication task might terminate when the upstream TiDB instance unexpectedly exits [#3061](https://github.com/pingcap/ticdc/issues/3061) + - Fix the issue that TiCDC process might panic when TiKV sends duplicate requests to the same Region [#2386](https://github.com/pingcap/ticdc/issues/2386) + - Fix unnecessary CPU consumption when verifying downstream TiDB/MySQL availability [#3073](https://github.com/pingcap/ticdc/issues/3073) + - Fix the issue that the volume of Kafka messages generated by TiCDC is not constrained by `max-message-size` [#2962](https://github.com/pingcap/ticdc/issues/2962) + - Fix the issue that TiCDC sync task might pause when an error occurs during writing a Kafka message [#2978](https://github.com/pingcap/ticdc/issues/2978) + - Fix the issue that some partitioned tables without valid indexes might be ignored when `force-replicate` is enabled [#2834](https://github.com/pingcap/ticdc/issues/2834) + - Fix the issue that scanning stock data might fail due to TiKV performing GC when scanning stock data takes too long [#2470](https://github.com/pingcap/ticdc/issues/2470) + - Fix a possible panic issue when encoding some types of columns into Open Protocol format [#2758](https://github.com/pingcap/ticdc/issues/2758) + - Fix a possible panic issue when encoding some types of columns into Avro format [#2648](https://github.com/pingcap/ticdc/issues/2648) + + + TiDB Binlog + + - Fix the issue that when most tables are filtered out, checkpoint can not be updated under some special load [#1075](https://github.com/pingcap/tidb-binlog/pull/1075) diff --git a/releases/release-notes.md b/releases/release-notes.md index 2bba58d85278f..1ed0703c75a79 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -5,6 +5,10 @@ aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] # TiDB Release Notes +## 5.3 + +- [5.3.0](/releases/release-5.3.0.md) + ## 5.2 - [5.2.2](/releases/release-5.2.2.md) diff --git a/releases/release-timeline.md b/releases/release-timeline.md index f4549a75124a5..b48697551e898 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -9,6 +9,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | +| [5.3.0](/releases/release-5.3.0.md) | 2021-11-23 | | [5.2.2](/releases/release-5.2.2.md) | 2021-10-29 | | [5.1.2](/releases/release-5.1.2.md) | 2021-09-27 | | [5.0.4](/releases/release-5.0.4.md) | 2021-09-27 | From b4abe603a1333d81d093d5643df84880fef2703b Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 18 Nov 2021 10:44:16 +0800 Subject: [PATCH 02/28] Update release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 180b4a399a8bb..fdc5ae5ce5e8a 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -182,7 +182,7 @@ It is recommended that you create a least-privileged SQL user to access and sign TiDB Lightning provides the ability to check the configuration before running a migration task. It is enabled by default. This feature automatically performs some routine checks for disk space and execution configuration. The main purpose is to ensure that the whole subsequent import process goes smoothly. - [User Document](tidb-lightning/tidb-lightning-prechecks.md) + [User Document](/tidb-lightning/tidb-lightning-prechecks.md) - **TiDB Lightning supports importing files of GBK character set** From dbac243f0fcd669a6caafb0e70e4a7eec1cd7400 Mon Sep 17 00:00:00 2001 From: Enwei Date: Thu, 18 Nov 2021 06:51:27 +0100 Subject: [PATCH 03/28] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- releases/release-5.3.0.md | 46 +++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 21 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index fdc5ae5ce5e8a..630a8a0e96f2b 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -53,17 +53,21 @@ In v5.3, the key new features or improvements are as follows: - Temporary tables: - - Regarding local temporary tables, if you have created local temporary tables on a cluster earlier than TiDB v5.3.0, these tables are actually ordinary tables, and handled as ordinary tables after the cluster is upgraded to TiDB v5.3.0 or later versions. Regarding global temporary tables, if you have created global temporary tables on a cluster in TiDB v5.3.0 or later versions, when the cluster is downgraded to a version earlier than v5.3.0, these tables are handled as ordinary tables and a data error occures. - - TiCDC and BR support [global temporary tables](/temporary-tables.md#global-temporary-tables) since v5.3.0. If you use TiCDC and BR in a version earlier than v5.3.0 to replicate global temporary tables to the downstream, a table definition error occurs. - - The following clusters should be in TiDB v5.3.0 or later versions: the cluster to be exported using TiDB ecosystem tools, the cluster after data is restored using the tools, and the downstream cluster that has replications using the tools. Otherwise, an error is reported when creating a global temporary table. - - For the compatibility of temporary tables, refer to [Compatibility with MySQL temporary tables](/temporary-tables.md#compatibility-with-mysql-temporary-tables) and [Compatibility restrictions with other TiDB features](/temporary-tables.md#compatibility-restrictions-with-other-tidb-features) - -- Fix an issue that `SHOW CREATE VIEW` does not require the `SHOW VIEW` permission. Now you must have `SHOW VIEW` permission to execute the `SHOW CREATE VIEW` statement. -- The system variable `sql_auto_is_null` is added to the Noop Function. When `tidb_enable_noop_functions = 0/OFF`, an error is reported after modified this variable. -- TiDB no longer allows to execute the `GRANT ALL ON performance_schema.*` syntax. When you execute the statement in TiDB, an error is reported. -- Fix an issue that auto-analyze is triggered outside the specified time period when new indexes are added in earlier versions of v5.3.0. In v5.3.0, after you set the time period with `tidb_auto_analyze_start_time` and `tidb_auto_analyze_end_time`, auto-analyze is only triggered during this time period. + - If you have created local temporary tables in a TiDB cluster earlier than v5.3.0, these tables are actually ordinary tables, and handled as ordinary tables after the cluster is upgraded to v5.3.0 or a later version. If you have created global temporary tables in a TiDB cluster of v5.3.0 or a later version, when the cluster is downgraded to a version earlier than v5.3.0, these tables are handled as ordinary tables and cause a data error. + - Since v5.3.0, TiCDC and BR support [global temporary tables](/temporary-tables.md#global-temporary-tables). If you use TiCDC and BR of a version earlier than v5.3.0 to replicate global temporary tables to the downstream, a table definition error occurs. + - The following clusters are expected to be v5.3.0 or later; otherwise, data error is reported when you create a global temporary table: + + - the cluster to be imported using TiDB ecosystem tools + - the cluster restored using TiDB ecosystem tools + - the downstream cluster in a replication task using TiDB ecosystem tools + - For the compatibility information of temporary tables, refer to [Compatibility with MySQL temporary tables](/temporary-tables.md#compatibility-with-mysql-temporary-tables) and [Compatibility restrictions with other TiDB features](/temporary-tables.md#compatibility-restrictions-with-other-tidb-features). + +- Fix the issue that the `SHOW VIEW` permission is not required to execute `SHOW CREATE VIEW`. Now you are expected to have the `SHOW VIEW` permission to execute the `SHOW CREATE VIEW` statement. +- The system variable `sql_auto_is_null` is added to the noop functions. When `tidb_enable_noop_functions = 0/OFF`, modifying this variable value causes an error. +- The `GRANT ALL ON performance_schema.*` syntax is no longer permitted. If you execute this statement in TiDB, an error occurs. +- Fix the issue that auto-analyze is unexpectedly triggered outside the specified time period when new indexes are added before v5.3.0. In v5.3.0, after you set the time period through the `tidb_auto_analyze_start_time` and `tidb_auto_analyze_end_time` variables, auto-analyze is triggered only during this time period. - The default storage directory for plugins is changed from "" to /data/deploy/plugin. -- The DM code is migrated to [the folder "dm" in TiCDC code repository](https://github.com/pingcap/ticdc/tree/master/dm). The version numbers of DM are changed from v2.0.x to v5.3.0, and you can upgrade from v2.0.x to v5.3.0 without any risk. +- The DM code is migrated to [the folder "dm" in TiCDC code repository](https://github.com/pingcap/ticdc/tree/master/dm). Now DM follows TiDB in version numbers. Next to v2.0.x, the new DM version is v5.3.0, and you can upgrade from v2.0.x to v5.3.0 without any risk. ## New features @@ -101,13 +105,13 @@ In v5.3, the key new features or improvements are as follows: [User document](/sql-statements/sql-statement-select.md), [#28689](https://github.com/pingcap/tidb/issues/28689) -- **Set table attributes** +- **Table attributes** - Support the `ALTER TABLE [PARTITION] ATTRIBUTES` statement that allows you to set attributes to a table or partition. Currently, TiDB only supports setting the `merge_option` attribute. By adding this attribute, you can directly control the Region merge behavior. + Support the `ALTER TABLE [PARTITION] ATTRIBUTES` statement that allows you to set attributes for a table or partition. Currently, TiDB only supports setting the `merge_option` attribute. By adding this attribute, you can explicitly control the Region merge behavior. - User scenarios: When you perform the `SPLIT TABLE` operation, if no data is inserted after a certain period of time, the empty Region is automatically merged by default. In this case, you can set the table attribute to `merge_option=deny` to avoid the automatic merging of the Region. + User scenarios: When you perform the `SPLIT TABLE` operation, if no data is inserted after a certain period of time, the empty Regions are automatically merged by default. In this case, you can set the table attribute to `merge_option=deny` to avoid the automatic merging of Regions. - [User document](/table-attributes.md),[#3839](https://github.com/tikv/pd/issues/3839) + [User document](/table-attributes.md), [#3839](https://github.com/tikv/pd/issues/3839) ### Security @@ -121,27 +125,27 @@ It is recommended that you create a least-privileged SQL user to access and sign ### Performance -- **Optimize the PD timestamp processing flow** +- **Optimize the timestamp processing flow of PD** - TiDB optimizes its timestamp processing flow and reduces the timestamp processing load of PD by enabling PD Follower Proxy and modifying the batch waiting time when the PD client requests TSO in batches. This helps improve the overall scalability of the system. + TiDB optimizes its timestamp processing flow and reduces the timestamp processing load of PD by enabling PD Follower Proxy and modifying the batch waiting time required when the PD client requests TSO in batches. This helps improve the overall scalability of the system. - - Support enabling or disabling PD Follower Proxy through the system variable [`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53). Suppose that the TSO requests load of PD is too high. In this case, enabling PD follower proxy can batch forward the TSO requests collected during the request cycle on followers to leader nodes. This solution can effectively reduce the number of direct interactions between clients and leaders, reduce the pressure of the leaders' load, and improve the overall performance of TiDB. + - Support enabling or disabling PD Follower Proxy through the system variable [`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53). Suppose that the TSO requests load of PD is too high. In this case, enabling PD follower proxy can batch forward the TSO requests collected during the request cycle on followers to the leader nodes. This solution can effectively reduce the number of direct interactions between clients and leaders, reduce the pressure of the load on leaders, and improve the overall performance of TiDB. > **Note:** > > When the number of clients is small and the PD leader CPU is not full, it is NOT recommended to enable PD Follower Proxy. - - Support setting the maximum waiting time through the system variable [`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53). This time is the waiting time for batching TSO requests when PD client requests TSO in batches. The unit of this time is milliseconds. In case that PD has a high TSO requests load, you can reduce the load and improve the throughput by increasing the waiting time to get a larger batch size. + - Support using the system variable [`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) to set the maximum waiting time needed for the PD client to batch request TSO. The unit of this time is milliseconds. In case that PD has a high TSO requests load, you can reduce the load and improve the throughput by increasing the waiting time to get a larger batch size. > **Note:** > - > When the TSO request load is not high, it is NOT recommended to modify this variable. + > When the TSO request load is not high, it is NOT recommended to modify this variable value. - [User document](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53),[#3149](https://github.com/tikv/pd/issues/3149) + [User document](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53), [#3149](https://github.com/tikv/pd/issues/3149) ### Stability -- **Support Online Unsafe Recovery after stores permanently damaged (experimental feature)** +- **Support Online Unsafe Recovery after some stores are permanently damaged (experimental feature)** Support the `unsafe remove-failed-stores` command that allows data unsafe recovery. Suppose that multiple replicas of data encounter problems like permanent damage (such as disk damage), and this issue causes the data in an application to be not readable or writable. In this case, Online Unsafe Recovery using PD can recover the data to be readable or writable again. From 01f22fc4440cfc2d39087d62cb2625a9288a7590 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Thu, 18 Nov 2021 14:56:12 +0800 Subject: [PATCH 04/28] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-5.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 630a8a0e96f2b..0e6f31cf04144 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -41,9 +41,9 @@ In v5.3, the key new features or improvements are as follows: | Configuration file | Configuration item | Change type | Description | | :---------- | :----------- | :----------- | :----------- | | TiDB | [`prepared-plan-cache.capacity`](/tidb-configuration-file.md#capacity) | Modified | Controls the number of cached statements. The default value is changed from `100` to `1000`.| -| TiKV | [`storage.reserve-space`](/tikv-configuration-file.md#reserve-space) | Modified | Controls space reserved for disk protection when TiKV is started. Starting from v5.3.0, 80% of the reserved space is spared for operations and maintenance when disk space is insufficient, and the remaining 20% space stores temporary disk files. | +| TiKV | [`storage.reserve-space`](/tikv-configuration-file.md#reserve-space) | Modified | Controls space reserved for disk protection when TiKV is started. Starting from v5.3.0, 80% of the reserved space is used as the extra disk space required for operations and maintenance when the disk space is insufficient, and the other 20% is used to store temporary files. | | TiKV | memory-usage-limit | Modified | This configuration item is new in TiDB v5.3.0 and its value is calculated based on storage.block-cache.capacity. | -| PD | [`log.file.max-days`](/pd-configuration-file.md#max-days) | Modified | Controls the maximum number of days that logs are retained. The default value is changed from `1` to `0`. | +| PD | [`log.file.max-days`](/pd-configuration-file.md#max-days) | Modified | Controls the maximum number of days that logs are retained for. The default value is changed from `1` to `0`. | | PD | [`log.file.max-backups`](/pd-configuration-file.md#max-backups) | Modified | Controls the maximum number of logs that are retained. The default value is changed from `7` to `0`. | | PD | [`patrol-region-interval`](/pd-configuration-file.md#patrol-region-interval) | Modified | Controls the running frequency at which replicaChecker checks the health state of a Region. The smaller this value is, the faster replicaChecker runs. Normally, you do not need to adjust this parameter. The default value is changed from `100ms` to `10ms`. | | PD | [`max-snapshot-count`](/pd-configuration-file.md#max-snapshot-count) | Modified | Controls the maximum number of snapshots that a single store receives or sends at the same time. PD schedulers depend on this configuration to prevent the resources used for normal traffic from being preempted. The default value is changed from `3` to `64`. | @@ -115,9 +115,9 @@ In v5.3, the key new features or improvements are as follows: ### Security -- **Support creating a least-privileged user for TiDB Dashboard** +- **Support creating users with the least privileges on TiDB Dashboard** -The account system of TiDB Dashboard is consistent with that of TiDB SQL. Users accessing TiDB Dashboard are authenticated and authorized based on TiDB SQL user's privileges. Therefore, TiDB Dashboard requires limited privileges, or merely the read-only privilege. You can configure users to access TiDB Dashboard based on the principle of least privilege, thus avoiding access of high-privileged users. +The account system of TiDB Dashboard is consistent with that of TiDB SQL. Users accessing TiDB Dashboard are authenticated and authorized based on TiDB SQL users' privileges. Therefore, TiDB Dashboard requires limited privileges, or merely the read-only privilege. You can configure users to access TiDB Dashboard based on the principle of least privilege, thus avoiding access of high-privileged users. It is recommended that you create a least-privileged SQL user to access and sign in with TiDB Dashboard. This avoids access of high-privileged users and improves security. @@ -225,11 +225,11 @@ It is recommended that you create a least-privileged SQL user to access and sign [User Document](/ticdc/manage-ticdc.md) -### Deployment and O&M +### Deployment and maintenance - **Continuous Profiling (experimental feature)** - Introduced in TiDB Dashboard, Continuous Profiling allows storing instance performance analysis results automatically in real time without interrupting TiDB clusters. The performance analysis result is displayed using a flame graph, which is observable and shortens troubleshooting time. + TiDB Dashboard supports the Continuous Profiling feature, which stores instance performance analysis results automatically in real time when TiDB clusters are running. You can check the performance analysis result in a flame graph, which is more observable and shortens troubleshooting time. This feature is disabled by default and needs to be enabled on the **Continuous Profile** page of TiDB Dashboard. From 5e04b2d870aaabb9a515691e98b69557080a1ea2 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 18 Nov 2021 14:58:42 +0800 Subject: [PATCH 05/28] Apply suggestions from code review Co-authored-by: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> --- releases/release-5.3.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 0e6f31cf04144..538fde2076c9e 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -274,18 +274,18 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters - Improve the execution efficiency of the TableScan operator greatly - Improve the execution efficiency of the Exchange operator - Reduce write amplification and memory usage during GC of the storage engine (experimental feature) - - Improve the stability and availability of TiFlash when TiFlash restarts, which reduces the possibility of query failures during a short period of time after the restart + - Improve the stability and availability of TiFlash when TiFlash restarts, which reduces possible query failures following the restart - Support pushing down multiple new String and Time functions to the MPP engine - String functions: LIKE pattern, FORMAT(), LOWER(), LTRIM(), RTRIM(), SUBSTRING_INDEX(), TRIM(), UCASE(), UPPER() - Mathematical functions: ROUND (decimal, int) - - Date and time functions: HOUR (), MICROSECOND (), MINUTE (), SECOND (), SYSDATE () + - Date and time functions: HOUR(), MICROSECOND(), MINUTE(), SECOND(), SYSDATE() - Type conversion function: CAST(time, real) - Aggregation functions: GROUP_CONCAT(), SUM(enum) - Support 512-bit SIMD - Enhance the cleanup algorithm for outdated data versions to reduce disk usage and improve read file performance - - Fix the issue that users cannot get memory or CPU information when viewing the dashboard on some non-Linux systems + - Fix the issue that dashboard does not display memory or CPU information in some non-Linux systems - Unify the naming style of TiFlash log files (keep the naming style consistent with that of TiKV) and support dynamic modification of logger.count and logger.size - Improve the data validation capability of column-based files (checksums, experimental feature) @@ -362,15 +362,15 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters + TiFlash - - Fix the inaccurate issue of the TiFlash Store Size statistics + - Fix the issue of inaccurate TiFlash Store Size statistics - Fix the issue that TiFlash fails to start up on some platforms due to the absence of library `nsl` - Block the infinite wait of `wait index` when writing pressure is heavy (a default timeout of 5 minutes is added), which prevents TiFlash from waiting too long for data synchronization to provide services - Fix the slow and no result issues of the log search when the log volume is large - Fix the issue that only the most recent logs can be searched when searching old historical logs - - Fix the wrong result that might occur when a new collation is enabled + - Fix the possible wrong result when a new collation is enabled - Fix the possible parsing errors when an SQL statement contains extremely long nested expressions - - Fix the `Block schema mismatch` error of the Exchange operator - - Fix the `Can't compare` error when comparing Decimal types + - Fix the possible `Block schema mismatch` error of the Exchange operator + - Fix the possible `Can't compare` error when comparing Decimal types - Fix the `3rd arguments of function substringUTF8 must be constants` error of the `left/substring` function + Tools From a078161589ed8b8fc1a87f0e9002a67d0f88a830 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 18 Nov 2021 14:59:25 +0800 Subject: [PATCH 06/28] Apply suggestions from code review Co-authored-by: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 538fde2076c9e..a5608de152b51 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -254,7 +254,7 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters - Enhance disk space protection to improve storage stability - To solve the issue that TiKV might panic when getting the disk fully written error, TiKV introduces a two-level threshold defense mechanism for the disk remaining space to prevent excess traffic from exhausting disk space. Additionally, the mechanism provides the ability to reclaim space when the threshold is triggered. When the remaining space threshold is triggered, some write operations will fail and return a disk full error and a list of disk full nodes. In this case, to recover the space and restore the service, you can execute `Drop/Truncate Table` or scale out the nodes. + To solve the issue that TiKV might panic in case of a disk fully-written error, TiKV introduces a two-level threshold defense mechanism to protect the disk remaining space from being exhausted by excess traffic. Additionally, the mechanism provides the ability to reclaim space when the threshold is triggered. When the remaining space threshold is triggered, some write operations will fail and TiKV will return a disk full error as well as a list of disk full nodes. In this case, to recover the space and restore the service, you can execute `Drop/Truncate Table` or scale out the nodes. - Simplify the algorithm of L0 flow control [#10879](https://github.com/tikv/tikv/issues/10879) - Improve the error log report in the raft client module [#10944](https://github.com/tikv/tikv/pull/10944) From a9af6127c739eadb97b688e771b4d84b41e92136 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 18 Nov 2021 15:00:00 +0800 Subject: [PATCH 07/28] Update releases/release-5.3.0.md Co-authored-by: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> --- releases/release-5.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index a5608de152b51..d2f9c09da573f 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -247,8 +247,8 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters - Show the affected SQL statements in the debug log when the coprocessor encounters a lock, which is helpful in diagnosing problems [#27718](https://github.com/pingcap/tidb/issues/27718) - Support showing the size of the backup and restore data when backing up and restoring data in the SQL logical layer [#27247](https://github.com/pingcap/tidb/issues/27247) - - Improve the default collection logic of ANALYZE when `tidb_analyze_version=2`, which improves the collection speed and reduces resource overhead - - Introduce the `ANALYZE TABLE table_name COLUMNS col_1, col_2, ... , col_n` syntax. The syntax provides a solution for wide tables to collect statistics only on a portion of the columns, which improves the speed of collecting statistics for wide tables. + - Improve the default collection logic of ANALYZE when `tidb_analyze_version` is `2`, which accelerates collection and reduces resource overhead + - Introduce the `ANALYZE TABLE table_name COLUMNS col_1, col_2, ... , col_n` syntax. The syntax allows collecting statistics only on a portion of the columns in wide tables, which improves the speed of statistics collection + TiKV From e3c055bda37e63f06c74b759503f60c036ffa4f2 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 18 Nov 2021 15:04:51 +0800 Subject: [PATCH 08/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index d2f9c09da573f..c3b7dae3f834b 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -284,7 +284,7 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters - Aggregation functions: GROUP_CONCAT(), SUM(enum) - Support 512-bit SIMD - - Enhance the cleanup algorithm for outdated data versions to reduce disk usage and improve read file performance + - Enhance the cleanup algorithm for outdated data to reduce disk usage and read files more efficiently - Fix the issue that dashboard does not display memory or CPU information in some non-Linux systems - Unify the naming style of TiFlash log files (keep the naming style consistent with that of TiKV) and support dynamic modification of logger.count and logger.size - Improve the data validation capability of column-based files (checksums, experimental feature) From 2eb84a3fa31b2331a000d88f58ebb254e6fb8687 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Thu, 18 Nov 2021 15:06:23 +0800 Subject: [PATCH 09/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index c3b7dae3f834b..6a4b0eb9d7aba 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -44,7 +44,7 @@ In v5.3, the key new features or improvements are as follows: | TiKV | [`storage.reserve-space`](/tikv-configuration-file.md#reserve-space) | Modified | Controls space reserved for disk protection when TiKV is started. Starting from v5.3.0, 80% of the reserved space is used as the extra disk space required for operations and maintenance when the disk space is insufficient, and the other 20% is used to store temporary files. | | TiKV | memory-usage-limit | Modified | This configuration item is new in TiDB v5.3.0 and its value is calculated based on storage.block-cache.capacity. | | PD | [`log.file.max-days`](/pd-configuration-file.md#max-days) | Modified | Controls the maximum number of days that logs are retained for. The default value is changed from `1` to `0`. | -| PD | [`log.file.max-backups`](/pd-configuration-file.md#max-backups) | Modified | Controls the maximum number of logs that are retained. The default value is changed from `7` to `0`. | +| PD | [`log.file.max-backups`](/pd-configuration-file.md#max-backups) | Modified | Controls the maximum number of logs that are retained for. The default value is changed from `7` to `0`. | | PD | [`patrol-region-interval`](/pd-configuration-file.md#patrol-region-interval) | Modified | Controls the running frequency at which replicaChecker checks the health state of a Region. The smaller this value is, the faster replicaChecker runs. Normally, you do not need to adjust this parameter. The default value is changed from `100ms` to `10ms`. | | PD | [`max-snapshot-count`](/pd-configuration-file.md#max-snapshot-count) | Modified | Controls the maximum number of snapshots that a single store receives or sends at the same time. PD schedulers depend on this configuration to prevent the resources used for normal traffic from being preempted. The default value is changed from `3` to `64`. | | PD | [`max-pending-peer-count`](/pd-configuration-file.md#max-pending-peer-count) | Modified | Controls the maximum number of pending peers in a single store. PD schedulers depend on this configuration to prevent too many Regions with outdated logs from being generated on some nodes. The default value is changed from `16` to `64`. | From f71fba5fc1a0cb2ff4c1bd4bc994da7bd09f9b2e Mon Sep 17 00:00:00 2001 From: Enwei Date: Thu, 18 Nov 2021 19:13:39 +0100 Subject: [PATCH 10/28] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- releases/release-5.3.0.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 6a4b0eb9d7aba..b4d8056299b74 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -147,13 +147,9 @@ It is recommended that you create a least-privileged SQL user to access and sign - **Support Online Unsafe Recovery after some stores are permanently damaged (experimental feature)** - Support the `unsafe remove-failed-stores` command that allows data unsafe recovery. Suppose that multiple replicas of data encounter problems like permanent damage (such as disk damage), and this issue causes the data in an application to be not readable or writable. In this case, Online Unsafe Recovery using PD can recover the data to be readable or writable again. + Support the `unsafe remove-failed-stores` command that performs online data unsafe recovery. Suppose that the majority of data replicas encounter issues like permanent damage (such as disk damage), and these issues cause the data ranges in an application to be unreadable or unwritable. In this case, you can use the Online Unsafe Recovery feature implemented in PD to recover the data, so that the data is readable or writable again. -> **Warning:** -> -> - This feature is a lossy recovery, so TiKV cannot guarantee data integrity and data indexes integrity after using the feature. -> - Online unsafe recovery is an experimental feature, and it is **not** recommended to use it in the production environment. The interface, strategy, and internal implementation of the feature might be changed when it is finally released. Also, although the feature is tested in some scenarios, it is not thoroughly validated, so the feature might cause system unavailability. -> - It is recommended to perform the feature-related operations with the support of the TiDB team. If some incorrect operations are performed, it might be hard to recover the cluster. + It is recommended to perform the feature-related operations with the support of the TiDB team. [User document](/online-unsafe-recovery.md), [#10483](https://github.com/tikv/tikv/issues/10483) From 7ce8f28de8263e2d8bd32e996872566fdf437416 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 19 Nov 2021 12:50:19 +0800 Subject: [PATCH 11/28] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Grace Cai Co-authored-by: Enwei Co-authored-by: Morgan Tocker Co-authored-by: Daniël van Eeden --- releases/release-5.3.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index b4d8056299b74..80b3e1928d02c 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -12,11 +12,11 @@ In v5.3, the key new features or improvements are as follows: + Introduce temporary tables to simplify your application logic and improve performance + Support setting attributes for tables and partitions -+ Support creating users with minimum privilege on TiDB Dashboard to enhance system security -+ Optimize the process of handling timestamps in TiDB to improve the overall performance ++ Support creating users with the least privileges on TiDB Dashboard to enhance system security ++ Optimize the timestamp processing flow in TiDB to improve the overall performance + Enhance the performance of TiDB Data Migration (DM) so that data is migrated from MySQL to TiDB with lower latency + Support parallel import in TiDB Lightning to improve the efficiency of full data migration -+ Support saving and restoring the on-site information of a cluster with a single SQL statement, which helps improve the efficiency of troubleshooting execution plan issues ++ Support saving and restoring the on-site information of a cluster with a single SQL statement, which helps improve the efficiency of troubleshooting issues relating to execution plans + Support the continuous profiling experimental feature to improve the observability of database performance + Continue optimizing the storage and computing engines to improve the system performance and stability @@ -83,11 +83,11 @@ In v5.3, the key new features or improvements are as follows: - Schedule the leaders of hotspot data to high-performance TiKV instances - Separate cold data to lower-cost storage mediums to improve cost efficiency - [User document](/information-schema/information-schema-placement-rules.md), [#18030](https://github.com/pingcap/tidb/issues/18030) + [User document](/placement-rules-in-sql.md), [#18030](https://github.com/pingcap/tidb/issues/18030) - **Temporary tables** - Support the `CREATE [GLOBAL] TEMPORARY TABLE` syntax to create temporary tables. Using this feature, you can easily manage the temporary data generated in the calculation process of an application. Temporary data is stored in memory and you can use the `tidb_tmp_table_max_size` variable to limit the size of a temporary table. TiDB supports the following types of temporary tables: + Support the `CREATE [GLOBAL] TEMPORARY TABLE` statement to create temporary tables. Using this feature, you can easily manage the temporary data generated in the calculation process of an application. Temporary data is stored in memory and you can use the `tidb_tmp_table_max_size` variable to limit the size of a temporary table. TiDB supports the following types of temporary tables: - Global temporary tables - Visible to all sessions in the cluster, and table schemas are persistent. @@ -202,7 +202,7 @@ It is recommended that you create a least-privileged SQL user to access and sign - **Save and restore the on-site information of a cluster** - When you locate and troubleshoot the issues of a TiDB cluster, you often need to provide information on the system and the execution plan. To help you get the information and troubleshoot cluster issues in a more convenient and efficient way, the `PLAN REPLAY` command is introduced in TiDB v5.3.0. This command enables you to easily save and restore the on-site information of a cluster, improves the efficiency of troubleshooting, and helps you more easily archive the issue for management. + When you locate and troubleshoot the issues of a TiDB cluster, you often need to provide information on the system and the query plan. To help you get the information and troubleshoot cluster issues in a more convenient and efficient way, the `PLAN REPLAY` command is introduced in TiDB v5.3.0. This command enables you to easily save and restore the on-site information of a cluster, improves the efficiency of troubleshooting, and helps you more easily archive the issues for management. The features of `PLAN REPLAYER` are as follows: @@ -290,7 +290,7 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters + TiCDC - Reduce the default value of the Kafka sink configuration item `MaxMessageBytes` from 64 MB to 1 MB to fix the issue that large messages are rejected by the Kafka Broker [#3104](https://github.com/pingcap/ticdc/pull/3104) - - Reduce memory usage in the relpication pipeline [#2553](https://github.com/pingcap/ticdc/issues/2553)[#3037](https://github.com/pingcap/ticdc/pull/3037) [#2726](https://github.com/pingcap/ticdc/pull/2726) + - Reduce memory usage in the replication pipeline [#2553](https://github.com/pingcap/ticdc/issues/2553)[#3037](https://github.com/pingcap/ticdc/pull/3037) [#2726](https://github.com/pingcap/ticdc/pull/2726) - Optimize monitoring items and alert rules to improve observability of synchronous links, memory GC, and stock data scanning processes [#2735](https://github.com/pingcap/ticdc/pull/2735) [#1606](https://github.com/pingcap/ticdc/issues/1606) [#3000](https://github.com/pingcap/ticdc/pull/3000) [#2985](https://github.com/pingcap/ticdc/issues/2985) [#2156](https://github.com/pingcap/ticdc/issues/2156) - When the sync task status is normal, no more historical error messages are displayed to avoid misleading users [#2242](https://github.com/pingcap/ticdc/issues/2242) From ef5cfcd91519c92fea845810bab7e20a8dc8291c Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 22 Nov 2021 09:57:38 +0800 Subject: [PATCH 12/28] Update releases/release-5.3.0.md Co-authored-by: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 80b3e1928d02c..d432a41176290 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -360,7 +360,7 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters - Fix the issue of inaccurate TiFlash Store Size statistics - Fix the issue that TiFlash fails to start up on some platforms due to the absence of library `nsl` - - Block the infinite wait of `wait index` when writing pressure is heavy (a default timeout of 5 minutes is added), which prevents TiFlash from waiting too long for data synchronization to provide services + - Block the infinite wait of `wait index` when writing pressure is heavy (a default timeout of 5 minutes is added), which prevents TiFlash from waiting too long for data replication to provide services - Fix the slow and no result issues of the log search when the log volume is large - Fix the issue that only the most recent logs can be searched when searching old historical logs - Fix the possible wrong result when a new collation is enabled From 2bcad45312bb8d441b4a4071e3168924ddd84139 Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 <82579298+Liuxiaozhen12@users.noreply.github.com> Date: Mon, 22 Nov 2021 11:17:17 +0800 Subject: [PATCH 13/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index d432a41176290..fc7fec950712d 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -164,7 +164,7 @@ It is recommended that you create a least-privileged SQL user to access and sign - **Add DM OpenAPI to better maintain DM clusters (experimental feature)** - DM provides the OpenAPI feature for querying and operating the DM cluster. It is similar to the feature of [dmctl tools](./dmctl-introduction.md). + DM provides the OpenAPI feature for querying and operating the DM cluster. It is similar to the feature of [dmctl tools](https://docs.pingcap.com/zh/tidb-data-migration/stable/dmctl-introduction). Currently, DM OpenAPI is an experimental feature and disabled by default. It is not recommended to use it in a production environment. From d8b7b7f2457a24579e1b7ccd0ca78066c832ac3a Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 <82579298+Liuxiaozhen12@users.noreply.github.com> Date: Mon, 22 Nov 2021 13:25:30 +0800 Subject: [PATCH 14/28] Apply suggestions from code review Co-authored-by: Morgan Tocker --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index fc7fec950712d..ddecc7661b5ca 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -30,7 +30,7 @@ In v5.3, the key new features or improvements are as follows: | Variable name | Change type | Description | | :---------- | :----------- | :----------- | -| [tidb_enable_noop_functions](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | No longer controls the `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` behaviors. | +| [tidb_enable_noop_functions](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | Temporary tables are now supported by TiDB so `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` no longer requires enabling `tidb_enable_noop_functions`. | | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Newly added | Controls the behavior of the optimizer when the statistics on a table expire. The default value is `ON`. When the number of modified rows in the table is greater than 80% of the total rows (This ratio can be adjusted by the configuration [`pseudo-estimate-ratio`](/tidb-configuration-file.md#pseudo-estimate-ratio)), the optimizer considers that the statistics other than the total number of rows are no longer reliable and use pseudo statistics instead. When you set the value as `OFF`, even if the statistics expire, the optimizer still uses them. | |[`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53) | Newly added | Determines whether to enable or disable the TSO Follower Proxy feature. The default value is `OFF`, which means the TSO Follower Proxy feature is disabled. At this time, TiDB only gets TSO from PD leader. When this feature is enabled, TiDB evenly sends the requests to all PD nodes when acquiring TSO. The PD follower then forwards the TSO requests to reduce the CPU pressure of PD leader. | |[`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) | Newly added | Sets the maximum waiting time for a batch saving operation when TiDB requests TSO from PD. The default value is `0`, which means no additional waiting. | From fca2e30e82475722de70d4c6b94c8e453a2f5112 Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 <82579298+Liuxiaozhen12@users.noreply.github.com> Date: Mon, 22 Nov 2021 13:26:14 +0800 Subject: [PATCH 15/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index ddecc7661b5ca..fbabe9a28789c 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -30,7 +30,7 @@ In v5.3, the key new features or improvements are as follows: | Variable name | Change type | Description | | :---------- | :----------- | :----------- | -| [tidb_enable_noop_functions](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | Temporary tables are now supported by TiDB so `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` no longer requires enabling `tidb_enable_noop_functions`. | +| [tidb_enable_noop_functions](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | Temporary tables are now supported by TiDB so `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` no longer require enabling `tidb_enable_noop_functions`. | | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Newly added | Controls the behavior of the optimizer when the statistics on a table expire. The default value is `ON`. When the number of modified rows in the table is greater than 80% of the total rows (This ratio can be adjusted by the configuration [`pseudo-estimate-ratio`](/tidb-configuration-file.md#pseudo-estimate-ratio)), the optimizer considers that the statistics other than the total number of rows are no longer reliable and use pseudo statistics instead. When you set the value as `OFF`, even if the statistics expire, the optimizer still uses them. | |[`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53) | Newly added | Determines whether to enable or disable the TSO Follower Proxy feature. The default value is `OFF`, which means the TSO Follower Proxy feature is disabled. At this time, TiDB only gets TSO from PD leader. When this feature is enabled, TiDB evenly sends the requests to all PD nodes when acquiring TSO. The PD follower then forwards the TSO requests to reduce the CPU pressure of PD leader. | |[`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) | Newly added | Sets the maximum waiting time for a batch saving operation when TiDB requests TSO from PD. The default value is `0`, which means no additional waiting. | From d7b5248d174fffed01791ed862052e7b546667cd Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 22 Nov 2021 14:22:18 +0800 Subject: [PATCH 16/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index fbabe9a28789c..b91720031ced0 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -43,6 +43,7 @@ In v5.3, the key new features or improvements are as follows: | TiDB | [`prepared-plan-cache.capacity`](/tidb-configuration-file.md#capacity) | Modified | Controls the number of cached statements. The default value is changed from `100` to `1000`.| | TiKV | [`storage.reserve-space`](/tikv-configuration-file.md#reserve-space) | Modified | Controls space reserved for disk protection when TiKV is started. Starting from v5.3.0, 80% of the reserved space is used as the extra disk space required for operations and maintenance when the disk space is insufficient, and the other 20% is used to store temporary files. | | TiKV | memory-usage-limit | Modified | This configuration item is new in TiDB v5.3.0 and its value is calculated based on storage.block-cache.capacity. | +| TiKV | `raftstore.raft-reject-transfer-leader-duration` | Deleted | Determines the smallest duration that a Leader is transferred to a newly added node. | | PD | [`log.file.max-days`](/pd-configuration-file.md#max-days) | Modified | Controls the maximum number of days that logs are retained for. The default value is changed from `1` to `0`. | | PD | [`log.file.max-backups`](/pd-configuration-file.md#max-backups) | Modified | Controls the maximum number of logs that are retained for. The default value is changed from `7` to `0`. | | PD | [`patrol-region-interval`](/pd-configuration-file.md#patrol-region-interval) | Modified | Controls the running frequency at which replicaChecker checks the health state of a Region. The smaller this value is, the faster replicaChecker runs. Normally, you do not need to adjust this parameter. The default value is changed from `100ms` to `10ms`. | From 50d577c33a7561bd418c96f9f45bd083ea7f232e Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 22 Nov 2021 14:25:53 +0800 Subject: [PATCH 17/28] Update releases/release-5.3.0.md Co-authored-by: Enwei --- releases/release-5.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index b91720031ced0..14d83a35905d8 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -74,11 +74,11 @@ In v5.3, the key new features or improvements are as follows: ### SQL -- **Use SQL interface to set placement rules for data (experimental)** +- **Use SQL interface to set placement rules for data (experimental feature)** - Support the `[CREATE | ALTER] PLACEMENT POLICY` syntax that provides a SQL interface to set placement rules for data. Using this feature, you can specify tables and partitions to be scheduled to specific regions, data centers, racks, or hosts. This meets your application demands for lower cost and higher flexibility. The typical user scenarios are as follows: + Support the `[CREATE | ALTER] PLACEMENT POLICY` syntax that provides a SQL interface to set placement rules for data. Using this feature, you can specify tables and partitions to be scheduled to specific regions, data centers, racks, hosts, or replica count rules. This meets your application demands for lower cost and higher flexibility. The typical user scenarios are as follows: - - Merge multiple databases of different applications to reduce the cost on database maintenance + - Merge multiple databases of different applications to reduce the cost on database maintenance, and achieve application resource isolation through the rule configuration - Increase replica count for important data to improve the application availability and data reliability - Store new data into SSDs and store old data into HHDs to lower the cost on data archiving and storage - Schedule the leaders of hotspot data to high-performance TiKV instances From 30f0f6c48a2b66756ddc759041645327d0f814c3 Mon Sep 17 00:00:00 2001 From: qiancai Date: Mon, 22 Nov 2021 14:46:57 +0800 Subject: [PATCH 18/28] update_tidb_download_link --- pd-control.md | 2 +- pd-recover.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pd-control.md b/pd-control.md index 1bcdc64c1c243..4875d321d0d93 100644 --- a/pd-control.md +++ b/pd-control.md @@ -28,7 +28,7 @@ If you want to download the latest version of `pd-ctl`, directly download the Ti > **Note:** > -> `{version}` indicates the version number of TiDB. For example, if `{version}` is `v5.2.2`, the package download link is `https://download.pingcap.org/tidb-v5.2.2-linux-amd64.tar.gz`. +> `{version}` indicates the version number of TiDB. For example, if `{version}` is `v5.3.0`, the package download link is `https://download.pingcap.org/tidb-v5.3.0-linux-amd64.tar.gz`. ### Compile from source code diff --git a/pd-recover.md b/pd-recover.md index 878fdcac6a26b..e49b4c557045b 100644 --- a/pd-recover.md +++ b/pd-recover.md @@ -27,7 +27,7 @@ To download the latest version of PD Recover, directly download the TiDB package > **Note:** > -> `{version}` indicates the version number of TiDB. For example, if `{version}` is `v5.2.2`, the package download link is `https://download.pingcap.org/tidb-v5.2.2-linux-amd64.tar.gz`. +> `{version}` indicates the version number of TiDB. For example, if `{version}` is `v5.3.0`, the package download link is `https://download.pingcap.org/tidb-v5.3.0-linux-amd64.tar.gz`. ## Quick Start From a85c7570921d2f0008a49ca5acd141c5903ba59d Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 24 Nov 2021 10:06:24 +0800 Subject: [PATCH 19/28] Update release-5.3.0.md --- releases/release-5.3.0.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 14d83a35905d8..36ef490f9be1e 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -238,6 +238,12 @@ It is recommended that you create a least-privileged SQL user to access and sign Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters (an experimental feature in v5.0.0) has been removed. If you have already used this feature to replicate data before upgrading TiCDC, the related data is not affected after the upgrade. +### Telemetry + +TiDB adds the information about whether the TEMPORARY TABLE is enabled in telemetry. + +To learn more about telemetry and how to disable this behavior, refer to [Telemetry](/telemetry.md). + ## Improvements + TiDB From 8fa35ae33c39a40fe84e17e5a2a198096d837968 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 25 Nov 2021 14:24:31 +0800 Subject: [PATCH 20/28] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> Co-authored-by: Ian --- releases/release-5.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 36ef490f9be1e..4d1d311633471 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -4,7 +4,7 @@ title: TiDB 5.3 Release Notes # TiDB 5.3 Release Notes -Release date: November 23, 2021 +Release date: November 30, 2021 TiDB version: 5.3.0 @@ -15,7 +15,7 @@ In v5.3, the key new features or improvements are as follows: + Support creating users with the least privileges on TiDB Dashboard to enhance system security + Optimize the timestamp processing flow in TiDB to improve the overall performance + Enhance the performance of TiDB Data Migration (DM) so that data is migrated from MySQL to TiDB with lower latency -+ Support parallel import in TiDB Lightning to improve the efficiency of full data migration ++ Support parallel import using multiple TiDB Lightning instances to improve the efficiency of full data migration + Support saving and restoring the on-site information of a cluster with a single SQL statement, which helps improve the efficiency of troubleshooting issues relating to execution plans + Support the continuous profiling experimental feature to improve the observability of database performance + Continue optimizing the storage and computing engines to improve the system performance and stability @@ -173,7 +173,7 @@ It is recommended that you create a least-privileged SQL user to access and sign - **TiDB Lightning Parallel Import** - TiDB Lightning provides parallel import capability to extend the original feature. It allows you to deploy multiple Lightning instances at the same time, and import single tables or multiple tables to downstream TiDB in parallel. Without changing the way customers use it, it greatly improves the data migration ability, allowing you to obtain data in a more real-time way to further process, integrate and analyze them. It improves the efficiency of enterprise data management. + TiDB Lightning provides parallel import capability to extend the original feature. It allows you to deploy multiple Lightning instances at the same time to import single tables or multiple tables to downstream TiDB in parallel. Without changing the way customers use it, it greatly improves the data migration ability, allowing you to migrate data in a more real-time way to further process, integrate and analyze them. It improves the efficiency of enterprise data management. In our test, using 10 TiDB Lightning instances, a total of 20 TiB MySQL data can be imported to TiDB within 8 hours. The performance of multiple table import is also improved. A single TiDB Lightning instance can support importing at 250 GB/s, 8 times the original performance. From 4a5fe2ce8db33fdb838e9b4d245a1c325c042720 Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 <82579298+Liuxiaozhen12@users.noreply.github.com> Date: Thu, 25 Nov 2021 15:09:16 +0800 Subject: [PATCH 21/28] Apply suggestions from code review Co-authored-by: Ian --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 4d1d311633471..afe82f15c8068 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -175,7 +175,7 @@ It is recommended that you create a least-privileged SQL user to access and sign TiDB Lightning provides parallel import capability to extend the original feature. It allows you to deploy multiple Lightning instances at the same time to import single tables or multiple tables to downstream TiDB in parallel. Without changing the way customers use it, it greatly improves the data migration ability, allowing you to migrate data in a more real-time way to further process, integrate and analyze them. It improves the efficiency of enterprise data management. - In our test, using 10 TiDB Lightning instances, a total of 20 TiB MySQL data can be imported to TiDB within 8 hours. The performance of multiple table import is also improved. A single TiDB Lightning instance can support importing at 250 GB/s, 8 times the original performance. + In our test, using 10 TiDB Lightning instances, a total of 20 TiB MySQL data can be imported to TiDB within 8 hours. The performance of multiple table import is also improved. A single TiDB Lightning instance can support importing at 250 GB/s, and the overall migration is 8 times faster than the original performance. [User Document](/tidb-lightning/tidb-lightning-distributed-import.md) From f1cab7ce5930f8114220d0579a39a1ec1ed68905 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 25 Nov 2021 15:47:09 +0800 Subject: [PATCH 22/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index afe82f15c8068..7f8827e999a7a 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -134,7 +134,7 @@ It is recommended that you create a least-privileged SQL user to access and sign > **Note:** > - > When the number of clients is small and the PD leader CPU is not full, it is NOT recommended to enable PD Follower Proxy. + > When the number of clients is small and the PD leader CPU load is not full, it is NOT recommended to enable PD Follower Proxy. - Support using the system variable [`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) to set the maximum waiting time needed for the PD client to batch request TSO. The unit of this time is milliseconds. In case that PD has a high TSO requests load, you can reduce the load and improve the throughput by increasing the waiting time to get a larger batch size. From 1889f9b7dc9bc7f4abadb0ff17cfae40c31d88c9 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 29 Nov 2021 10:38:39 +0800 Subject: [PATCH 23/28] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: xixirangrang Co-authored-by: Daniël van Eeden Co-authored-by: Enwei --- releases/release-5.3.0.md | 11 +++++++---- releases/release-timeline.md | 2 +- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 7f8827e999a7a..fb5e010fe047c 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -30,11 +30,11 @@ In v5.3, the key new features or improvements are as follows: | Variable name | Change type | Description | | :---------- | :----------- | :----------- | -| [tidb_enable_noop_functions](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | Temporary tables are now supported by TiDB so `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` no longer require enabling `tidb_enable_noop_functions`. | +| [`tidb_enable_noop_functions`](/system-variables.md#tidb_enable_noop_functions-new-in-v40) | Modified | Temporary tables are now supported by TiDB so `CREATE TEMPORARY TABLE` and `DROP TEMPORARY TABLE` no longer require enabling `tidb_enable_noop_functions`. | | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Newly added | Controls the behavior of the optimizer when the statistics on a table expire. The default value is `ON`. When the number of modified rows in the table is greater than 80% of the total rows (This ratio can be adjusted by the configuration [`pseudo-estimate-ratio`](/tidb-configuration-file.md#pseudo-estimate-ratio)), the optimizer considers that the statistics other than the total number of rows are no longer reliable and use pseudo statistics instead. When you set the value as `OFF`, even if the statistics expire, the optimizer still uses them. | |[`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53) | Newly added | Determines whether to enable or disable the TSO Follower Proxy feature. The default value is `OFF`, which means the TSO Follower Proxy feature is disabled. At this time, TiDB only gets TSO from PD leader. When this feature is enabled, TiDB evenly sends the requests to all PD nodes when acquiring TSO. The PD follower then forwards the TSO requests to reduce the CPU pressure of PD leader. | |[`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) | Newly added | Sets the maximum waiting time for a batch saving operation when TiDB requests TSO from PD. The default value is `0`, which means no additional waiting. | -| [tidb_tmp_table_max_size](/system-variables.md#tidb_tmp_table_max_size-new-in-v53) | Newly added | Limits the maximum size of a single [temporary table](/temporary-tables.md). If the temporary table exceeds this size, an error will occur. | +| [`tidb_tmp_table_max_size`](/system-variables.md#tidb_tmp_table_max_size-new-in-v53) | Newly added | Limits the maximum size of a single [temporary table](/temporary-tables.md). If the temporary table exceeds this size, an error will occur. | ### Configuration file parameters @@ -222,6 +222,9 @@ It is recommended that you create a least-privileged SQL user to access and sign [User Document](/ticdc/manage-ticdc.md) +- **TiCDC supports the HTTP protocol OpenAPI for managing TiCDC tasks** + Since TiDB v5.3.0, TiCDC OpenAPI becomes an General Availability (GA) feature. You can query and operate TiCDC clusters using OpenAPI in the production environment. + ### Deployment and maintenance - **Continuous Profiling (experimental feature)** @@ -240,7 +243,7 @@ Starting from TiCDC v5.3.0, the cyclic replication feature between TiDB clusters ### Telemetry -TiDB adds the information about whether the TEMPORARY TABLE is enabled in telemetry. +TiDB adds the information to the telemetry report about whether or not the TEMPORARY TABLE feature is used. This does not include table names or table data. To learn more about telemetry and how to disable this behavior, refer to [Telemetry](/telemetry.md). @@ -345,7 +348,7 @@ To learn more about telemetry and how to disable this behavior, refer to [Teleme - Fix the issue of unavailable TiKV caused by Raftstore deadlock when migrating Regions. The workaround is to disable the scheduling and restart the unavailable TiKV. [#10909](https://github.com/tikv/tikv/issues/10909) - Fix the issue that CDC adds scan retries frequently due to the Congest error [#11082](https://github.com/tikv/tikv/issues/11082) - - Fix the issue that the raft connection is broken when the channel is full [#11047](https://github.com/tikv/tikv/issues/11047) + - Fix the issue that the Raft connection is broken when the channel is full [#11047](https://github.com/tikv/tikv/issues/11047) - Fix the issue that batch messages are too large in Raft client implementation [#9714](https://github.com/tikv/tikv/issues/9714) - Fix the issue that some coroutines leak in `resolved_ts` [#10965](https://github.com/tikv/tikv/issues/10965) - Fix a panic issue that occurs to the coprocessor when the size of response exceeds 4 GiB [#9012](https://github.com/tikv/tikv/issues/9012) diff --git a/releases/release-timeline.md b/releases/release-timeline.md index b48697551e898..cc0ab54ec63ff 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -9,7 +9,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | -| [5.3.0](/releases/release-5.3.0.md) | 2021-11-23 | +| [5.3.0](/releases/release-5.3.0.md) | 2021-11-30 | | [5.2.2](/releases/release-5.2.2.md) | 2021-10-29 | | [5.1.2](/releases/release-5.1.2.md) | 2021-09-27 | | [5.0.4](/releases/release-5.0.4.md) | 2021-09-27 | From 7483bfd477280d7da467931654523dee58a5a0dc Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 30 Nov 2021 09:48:31 +0800 Subject: [PATCH 24/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index fb5e010fe047c..485335b09fc12 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -63,6 +63,7 @@ In v5.3, the key new features or improvements are as follows: - the downstream cluster in a replication task using TiDB ecosystem tools - For the compatibility information of temporary tables, refer to [Compatibility with MySQL temporary tables](/temporary-tables.md#compatibility-with-mysql-temporary-tables) and [Compatibility restrictions with other TiDB features](/temporary-tables.md#compatibility-restrictions-with-other-tidb-features). +For releases earlier than v5.3.0, TiDB reports an error when a system variable is set to an illegal value. For v5.3.0 and later releases, TiDB returns success with a warning such as "|Warning | 1292 | Truncated incorrect xxx: 'xx'" when a system variable is set to an illegal value. - Fix the issue that the `SHOW VIEW` permission is not required to execute `SHOW CREATE VIEW`. Now you are expected to have the `SHOW VIEW` permission to execute the `SHOW CREATE VIEW` statement. - The system variable `sql_auto_is_null` is added to the noop functions. When `tidb_enable_noop_functions = 0/OFF`, modifying this variable value causes an error. - The `GRANT ALL ON performance_schema.*` syntax is no longer permitted. If you execute this statement in TiDB, an error occurs. From 7d105e1333f5c25b5c6ea55c9c31ac623be0682f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 30 Nov 2021 10:00:42 +0800 Subject: [PATCH 25/28] Update releases/release-5.3.0.md Co-authored-by: Connor --- releases/release-5.3.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 485335b09fc12..f2332b2869558 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -357,6 +357,7 @@ To learn more about telemetry and how to disable this behavior, refer to [Teleme - Fix a panic issue caused by timeout when processing Coprocessor requests [#10852](https://github.com/tikv/tikv/issues/10852) - Fix a memory leak caused by monitoring data of statistics threads [#11195](https://github.com/tikv/tikv/issues/11195) - Fix a panic issue caused by getting the cgroup information from some platforms [#10980](https://github.com/tikv/tikv/pull/10980) + - Fix the issue of poor scan performance because MVCC Deletion versions are not dropped by compaction filter GC [#11248](https://github.com/tikv/tikv/pull/11248) + PD From c95f149e0d0e0dff47532581db447f465e213326 Mon Sep 17 00:00:00 2001 From: qiancai Date: Tue, 30 Nov 2021 10:09:50 +0800 Subject: [PATCH 26/28] Update release-5.3.0.md --- releases/release-5.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index f2332b2869558..0af41d59aa69a 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -56,14 +56,14 @@ In v5.3, the key new features or improvements are as follows: - If you have created local temporary tables in a TiDB cluster earlier than v5.3.0, these tables are actually ordinary tables, and handled as ordinary tables after the cluster is upgraded to v5.3.0 or a later version. If you have created global temporary tables in a TiDB cluster of v5.3.0 or a later version, when the cluster is downgraded to a version earlier than v5.3.0, these tables are handled as ordinary tables and cause a data error. - Since v5.3.0, TiCDC and BR support [global temporary tables](/temporary-tables.md#global-temporary-tables). If you use TiCDC and BR of a version earlier than v5.3.0 to replicate global temporary tables to the downstream, a table definition error occurs. - - The following clusters are expected to be v5.3.0 or later; otherwise, data error is reported when you create a global temporary table: - + - The following clusters are expected to be v5.3.0 or later; otherwise, data error is reported when you create a global temporary table: + - the cluster to be imported using TiDB ecosystem tools - the cluster restored using TiDB ecosystem tools - the downstream cluster in a replication task using TiDB ecosystem tools - For the compatibility information of temporary tables, refer to [Compatibility with MySQL temporary tables](/temporary-tables.md#compatibility-with-mysql-temporary-tables) and [Compatibility restrictions with other TiDB features](/temporary-tables.md#compatibility-restrictions-with-other-tidb-features). -For releases earlier than v5.3.0, TiDB reports an error when a system variable is set to an illegal value. For v5.3.0 and later releases, TiDB returns success with a warning such as "|Warning | 1292 | Truncated incorrect xxx: 'xx'" when a system variable is set to an illegal value. +- For releases earlier than v5.3.0, TiDB reports an error when a system variable is set to an illegal value. For v5.3.0 and later releases, TiDB returns success with a warning such as "|Warning | 1292 | Truncated incorrect xxx: 'xx'" when a system variable is set to an illegal value. - Fix the issue that the `SHOW VIEW` permission is not required to execute `SHOW CREATE VIEW`. Now you are expected to have the `SHOW VIEW` permission to execute the `SHOW CREATE VIEW` statement. - The system variable `sql_auto_is_null` is added to the noop functions. When `tidb_enable_noop_functions = 0/OFF`, modifying this variable value causes an error. - The `GRANT ALL ON performance_schema.*` syntax is no longer permitted. If you execute this statement in TiDB, an error occurs. @@ -225,7 +225,7 @@ It is recommended that you create a least-privileged SQL user to access and sign - **TiCDC supports the HTTP protocol OpenAPI for managing TiCDC tasks** Since TiDB v5.3.0, TiCDC OpenAPI becomes an General Availability (GA) feature. You can query and operate TiCDC clusters using OpenAPI in the production environment. - + ### Deployment and maintenance - **Continuous Profiling (experimental feature)** From 750a6d474c599eead3487aafb0b16b2a3964a07f Mon Sep 17 00:00:00 2001 From: qiancai Date: Tue, 30 Nov 2021 10:17:26 +0800 Subject: [PATCH 27/28] Update release-5.3.0.md --- releases/release-5.3.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index 0af41d59aa69a..da217d0d831ca 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -224,6 +224,7 @@ It is recommended that you create a least-privileged SQL user to access and sign [User Document](/ticdc/manage-ticdc.md) - **TiCDC supports the HTTP protocol OpenAPI for managing TiCDC tasks** + Since TiDB v5.3.0, TiCDC OpenAPI becomes an General Availability (GA) feature. You can query and operate TiCDC clusters using OpenAPI in the production environment. ### Deployment and maintenance From 06485f58620275c484beabd060940f28eaec984b Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 30 Nov 2021 10:32:26 +0800 Subject: [PATCH 28/28] Update releases/release-5.3.0.md --- releases/release-5.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-5.3.0.md b/releases/release-5.3.0.md index da217d0d831ca..cf953cd22e279 100644 --- a/releases/release-5.3.0.md +++ b/releases/release-5.3.0.md @@ -34,7 +34,7 @@ In v5.3, the key new features or improvements are as follows: | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Newly added | Controls the behavior of the optimizer when the statistics on a table expire. The default value is `ON`. When the number of modified rows in the table is greater than 80% of the total rows (This ratio can be adjusted by the configuration [`pseudo-estimate-ratio`](/tidb-configuration-file.md#pseudo-estimate-ratio)), the optimizer considers that the statistics other than the total number of rows are no longer reliable and use pseudo statistics instead. When you set the value as `OFF`, even if the statistics expire, the optimizer still uses them. | |[`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v53) | Newly added | Determines whether to enable or disable the TSO Follower Proxy feature. The default value is `OFF`, which means the TSO Follower Proxy feature is disabled. At this time, TiDB only gets TSO from PD leader. When this feature is enabled, TiDB evenly sends the requests to all PD nodes when acquiring TSO. The PD follower then forwards the TSO requests to reduce the CPU pressure of PD leader. | |[`tidb_tso_client_batch_max_wait_time`](/system-variables.md#tidb_tso_client_batch_max_wait_time-new-in-v53) | Newly added | Sets the maximum waiting time for a batch saving operation when TiDB requests TSO from PD. The default value is `0`, which means no additional waiting. | -| [`tidb_tmp_table_max_size`](/system-variables.md#tidb_tmp_table_max_size-new-in-v53) | Newly added | Limits the maximum size of a single [temporary table](/temporary-tables.md). If the temporary table exceeds this size, an error will occur. | +| [`tidb_tmp_table_max_size`](/system-variables.md#tidb_tmp_table_max_size-new-in-v530) | Newly added | Limits the maximum size of a single [temporary table](/temporary-tables.md). If the temporary table exceeds this size, an error will occur. | ### Configuration file parameters