MySQL 8.0 Compatibility

[morgo]

This issue is to track what is required to move from MySQL 5.7 to MySQL 8.0 compatibility.

See the complete list.

Essential

• Common Table Expressions (regular and recursive) support common table expression (CTE) #17472
• Window Functions Support window functions for TiDB #4807 docs/design: add proposal for window functions #8117
• utf8mb4 as default (*: change default charset and collation from 'utf8 utf8_bin' to 'utf8mb4 utf8mb4_bin' #7965)
• Role Based Authentication
• Support utf8mb4_0900_ai_ci when new collation is enabled Support utf8mb4_0900_ai_ci when new collation is enabled #37566

Nice to Have

8.0.11 and before

• NOWAIT
• Invisible Indexes UCP: Support Invisible Indexes #9246
• SET_VAR optimizer hint Support Variable-Setting Hint SET_VAR #18748
• information_schema.keywords table. Example here. Add information_schema.keywords #48801
• SHUTDOWN command Support SHUTDOWN SQL command (and protocol cmd) #5046
• Improved Regular expression support support regular expression related functions #23881
• Maintain information about password history *: support password reuse policy #39162
• Resource Management Global Resource Quota Control  #38825
• MAX_EXECUTION_TIME hint Feature Request: Support MAX_EXECUTION_TIME #7008
• Non-binary utf8mb4 collations, such as utf8mb4_0900_ai_ci https://github.com/pingcap/tidb/pull/33212/files
• helper functions uuid_to_bin and bin_to_uuid Support bin-to-uuid and uuid-to-bin #20119
• support is_uuid function #30295
• caching_sha2_password Authentication Plugin Support for caching_sha2_password #9411
• JSON_TABLE function + add remaining JSON functions that were backported to 5.7
• SKIP LOCKED Support SKIP LOCKED Syntax #18207
• RESTART command Support for 'RESTART' statement #26016
• Cleanup SHOW VARIABLES output (remove query cache etc.)
• Remove removed functions (insecure crypto etc)
• GIS Support Support for SPATIAL functions, data types and indexes #6347
• Support ascending or descending index  #2519
  • Descending Indexes + GROUP BY no longer implying ORDER BY
• Functional Indexes, aka expression indexes in TiDB Make Expression Index generally available #18008
• Index Skip Scan
• support more clear combinations of UNION and INTO
• SQL Mode TIME_TRUNCATE_FRACTIONAL https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html#sqlmode_time_truncate_fractional Support sql_mode TIME_TRUNCATE_FRACTIONAL #29406
• Support MySQL 8.0 bitwise behavior bitwise operations with binary args behavior changes by mysql 5.7 and 8.0 both #30637
• Support Partitioning management DDL ALTER TABLE t REMOVE PARTITIONING Support ALTER TABLE t REMOVE PARTITIONING #42616
• Support Partitioning management DDL ALTER TABLE t PARTITION BY ... Support ALTER TABLE t REMOVE PARTITIONING #42616
• New multi table DROP TABLE behavior When DROP TABLE with multiple tables listed, and one of them does not exist, TiDB still drops all the remaining tables #64218

8.0.13

• Option to require tables to have a primary key (8.0.13) Add support for sql_require_primary_key #28544
• Default values for BLOB/TEXT + default value as function (part 1: RAND()) Incompatible with MySQL 8.0 about create table with default expression #10377
• Default values for BLOB/TEXT + default value as function (part 2: UUID()) support specifying expr uuid() as column default value #33870
• Default values for BLOB/TEXT + default value as function (part 3) Default value for TEXT, BLOB or JSON columns #45506
• Require current password to be specified to change password (8.0.13)

8.0.14

• Support Lateral derived tables #40328
  • Lateral derived tables Details
• Admin-only TCP/IP port
  • Details
• Support RETAIN CURRENT PASSWORD aka dual passwords #60587
  • Two passwords per account Details

8.0.15

• Check Constraints (from 8.0.15) https://github.com/pingcap/tidb/projects/46 Support CHECK constraint #41711

8.0.17

• utf8mb4_0900_bin TiDB should support collation utf8mb4_0900_bin #46268
• multi-valued indexes on json arrays / json_overlaps() and member of() support multi-valued index  #39592
• JSON schema validation with JSON_SCHEMA_VALID() #52779

8.0.18

• Create user/alter user/set password can generate random passwords for you.

8.0.19

• Table constructors. i.e. TABLE t ORDER BY c LIMIT 10 OFFSET 3 - 8.0.19 Support VALUES and TABLE statements syntax #19934
• Make integer display with optional util: rewrite parser warings for integer display width #18775
• Lock accounts after too many failed logins Support Failed-Login Tracking and Temporary Account Locking #38938
• New syntax for INSERT ON DUPLICATE KEY UPDATE https://dev.mysql.com/worklog/task/?id=6312 Support MySQL 8.0.19 feature - aliases in the values and the set clause of insert into ... on duplicate key update #51650
• Row-value constructors. i.e. VALUES ROW(1, "foo", 3.2) -8.0.19 VALUES statement is not supported #21486
• Support TIMESTAMP with explicit time zone Support TIMESTAMP with explicit time zone #51742

8.0.20

• New Optimizer hints: join_index, no_join_index, group_index, no_group_index, order_index, no_order_index, index, no_index.

8.0.21

• User attributes Support commenting in CREATE USER and ALTER USER statements #38172
• JSON_VALUE function (8.0.21)
• CREATE TABLE ... START TRANSACTION support mysql8.0 create table with start transaction feature #52477

8.0.22

• Parenthesized Query Expressions

8.0.23

• Invisible Columns #59233
• Query Attributes

8.0.28

• Deprecated TLSv1.0 and TLSv1.1 The minimum TLS version should comply with IETF RFC 8996 #36036
• Support wider range of UNIX_TIMESTAMP Support wider range of UNIX_TIMESTAMP, as MySQL 8.0.28 #30133

8.0.30

• Generated Invisible Primary Key #59253
• KDF, salt and iterations support for AES_ENCRYPT() and AES_DECRYPT().

8.0.31

• INTERSECT Support EXCEPT/INTERSECT Operators #18031
• EXCEPT Support EXCEPT/INTERSECT Operators #18031
• Support for EXCEPT ALL #66392

8.0.34

• CURRENT_USER() as default value for VARCHAR and TEXT columns in CREATE TABLE statements Default value for TEXT, BLOB or JSON columns #45506

8.0.35

8.0.36

8.0.37

Cleanup

• Remove variables that have been removed.
• Remove features that have been removed (this will require several sub-tasks)
• Add deprecation warnings for deprecated variables
• Add deprecation warnings for deprecated features

Not Applicable

• New Data Dictionary
• Performance Schema enhancements
• Replication enhancements
• InnoDB enhancements
• SET PERSIST (settings automatically save cluster-wide)
• Log services (log_error_services etc)

[shenli]

@morgo Nice work! We will add it to our roadmap.
/cc @winkyao @zz-jason @jackysp

[elvizlai]

is there any progress or plan about JSON_TABLE ?

[morgo]

I moved skip-locked to the essential list. At the time this list was created, the default was optimistic locking. But since it is now pessmistic, it is highly desirable.

[dveeden]

User attributes might also be something to add

mysql 8.0.22 > CREATE USER 'foo'@'%' IDENTIFIED BY 'affalfajljfjlk' COMMENT 'Support ticket #123';
Query OK, 0 rows affected (0.01 sec)

mysql 8.0.22 > CREATE USER 'bar'@'%' IDENTIFIED BY 'affalfajljfjlk' ATTRIBUTE '{"ticket": 123, "full_name": "John Doe"}';
Query OK, 0 rows affected (0.03 sec)

mysql 8.0.22 > select User_attributes from mysql.user where user in ('foo','bar');
+--------------------------------------------------------+
| User_attributes                                        |
+--------------------------------------------------------+
| {"metadata": {"ticket": 123, "full_name": "John Doe"}} |
| {"metadata": {"comment": "Support ticket #123"}}       |
+--------------------------------------------------------+
2 rows in set (0.01 sec)