Skip to content

[fix](function) resubmit-fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug #17907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yiguolei merged 8 commits into from
Mar 29, 2023
Merged

[fix](function) resubmit-fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug #17907

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
52415a0
Revert "[fix](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm in…
xinyiZzz Mar 17, 2023
d60083e
[fix-resubmit](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm i…
xinyiZzz Mar 9, 2023
1b435f6
1
xinyiZzz Mar 17, 2023
fd9aa85
3
xinyiZzz Mar 27, 2023
15a3def
4
xinyiZzz Mar 28, 2023
fc1b7ea
4
xinyiZzz Mar 28, 2023
80fe65c
4
xinyiZzz Mar 29, 2023
4b3b5c2
5
xinyiZzz Mar 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion be/src/vec/functions/function_encryption.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,8 @@ void exectue_result(std::vector<const ColumnString::Offsets*>& offsets_list,
template <typename Impl, EncryptionMode mode, bool is_encrypt>
struct EncryptionAndDecryptTwoImpl {
static DataTypes get_variadic_argument_types_impl() {
return {std::make_shared<DataTypeString>(), std::make_shared<DataTypeString>()};
return {std::make_shared<DataTypeString>(), std::make_shared<DataTypeString>(),
std::make_shared<DataTypeString>()};
}

static Status vector_vector(std::vector<const ColumnString::Offsets*>& offsets_list,
Expand All @@ -167,6 +168,17 @@ struct EncryptionAndDecryptTwoImpl {
continue;
}
EncryptionMode encryption_mode = mode;
int mode_size = (*offsets_list[2])[i] - (*offsets_list[2])[i - 1];
const auto mode_raw =
reinterpret_cast<const char*>(&(*chars_list[2])[(*offsets_list[2])[i - 1]]);
if (mode_size != 0) {
std::string mode_str(mode_raw, mode_size);
if (aes_mode_map.count(mode_str) == 0) {
StringOP::push_null_string(i, result_data, result_offset, null_map);
continue;
}
encryption_mode = aes_mode_map.at(mode_str);
}
exectue_result<Impl, is_encrypt>(offsets_list, chars_list, i, encryption_mode, nullptr,
0, result_data, result_offset, null_map);
}
Expand Down
97 changes: 59 additions & 38 deletions be/test/vec/function/function_string_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -676,8 +676,9 @@ TEST(function_string_test, function_sm3sum_test) {
TEST(function_string_test, function_aes_encrypt_test) {
std::string func_name = "aes_encrypt";
{
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String};
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String, TypeIndex::String};

const char* mode = "AES_128_ECB";
const char* key = "doris";
const char* src[6] = {"aaaaaa", "bbbbbb", "cccccc", "dddddd", "eeeeee", ""};
std::string r[5];
Expand All @@ -692,13 +693,13 @@ TEST(function_string_test, function_aes_encrypt_test) {
r[i] = std::string(p, outlen);
}

DataSet data_set = {{{std::string(src[0]), std::string(key)}, r[0]},
{{std::string(src[1]), std::string(key)}, r[1]},
{{std::string(src[2]), std::string(key)}, r[2]},
{{std::string(src[3]), std::string(key)}, r[3]},
{{std::string(src[4]), std::string(key)}, r[4]},
{{std::string(src[5]), std::string(key)}, Null()},
{{Null(), std::string(key)}, Null()}};
DataSet data_set = {{{std::string(src[0]), std::string(key), std::string(mode)}, r[0]},
{{std::string(src[1]), std::string(key), std::string(mode)}, r[1]},
{{std::string(src[2]), std::string(key), std::string(mode)}, r[2]},
{{std::string(src[3]), std::string(key), std::string(mode)}, r[3]},
{{std::string(src[4]), std::string(key), std::string(mode)}, r[4]},
{{std::string(src[5]), std::string(key), std::string(mode)}, Null()},
{{Null(), std::string(key), std::string(mode)}, Null()}};

check_function<DataTypeString, true>(func_name, input_types, data_set);
}
Expand Down Expand Up @@ -743,8 +744,9 @@ TEST(function_string_test, function_aes_encrypt_test) {
TEST(function_string_test, function_aes_decrypt_test) {
std::string func_name = "aes_decrypt";
{
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String};
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String, TypeIndex::String};

const char* mode = "AES_128_ECB";
const char* key = "doris";
const char* src[5] = {"aaaaaa", "bbbbbb", "cccccc", "dddddd", "eeeeee"};
std::string r[5];
Expand All @@ -759,12 +761,12 @@ TEST(function_string_test, function_aes_decrypt_test) {
r[i] = std::string(p, outlen);
}

DataSet data_set = {{{r[0], std::string(key)}, std::string(src[0])},
{{r[1], std::string(key)}, std::string(src[1])},
{{r[2], std::string(key)}, std::string(src[2])},
{{r[3], std::string(key)}, std::string(src[3])},
{{r[4], std::string(key)}, std::string(src[4])},
{{Null(), std::string(key)}, Null()}};
DataSet data_set = {{{r[0], std::string(key), std::string(mode)}, std::string(src[0])},
{{r[1], std::string(key), std::string(mode)}, std::string(src[1])},
{{r[2], std::string(key), std::string(mode)}, std::string(src[2])},
{{r[3], std::string(key), std::string(mode)}, std::string(src[3])},
{{r[4], std::string(key), std::string(mode)}, std::string(src[4])},
{{Null(), std::string(key), std::string(mode)}, Null()}};

check_function<DataTypeString, true>(func_name, input_types, data_set);
}
Expand Down Expand Up @@ -806,29 +808,39 @@ TEST(function_string_test, function_aes_decrypt_test) {
TEST(function_string_test, function_sm4_encrypt_test) {
std::string func_name = "sm4_encrypt";
{
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String};
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String, TypeIndex::String,
TypeIndex::String};

const char* key = "doris";
const char* iv = "0123456789abcdef";
const char* mode = "SM4_128_ECB";
const char* src[6] = {"aaaaaa", "bbbbbb", "cccccc", "dddddd", "eeeeee", ""};
std::string r[5];

for (int i = 0; i < 5; i++) {
int cipher_len = strlen(src[i]) + 16;
char p[cipher_len];

int outlen = EncryptionUtil::encrypt(
EncryptionMode::SM4_128_ECB, (unsigned char*)src[i], strlen(src[i]),
(unsigned char*)key, strlen(key), nullptr, 0, true, (unsigned char*)p);
int iv_len = 32;
std::unique_ptr<char[]> init_vec;
init_vec.reset(new char[iv_len]);
std::memset(init_vec.get(), 0, strlen(iv) + 1);
memcpy(init_vec.get(), iv, strlen(iv));
int outlen =
EncryptionUtil::encrypt(EncryptionMode::SM4_128_ECB, (unsigned char*)src[i],
strlen(src[i]), (unsigned char*)key, strlen(key),
init_vec.get(), strlen(iv), true, (unsigned char*)p);
r[i] = std::string(p, outlen);
}

DataSet data_set = {{{std::string(src[0]), std::string(key)}, r[0]},
{{std::string(src[1]), std::string(key)}, r[1]},
{{std::string(src[2]), std::string(key)}, r[2]},
{{std::string(src[3]), std::string(key)}, r[3]},
{{std::string(src[4]), std::string(key)}, r[4]},
{{std::string(src[5]), std::string(key)}, Null()},
{{Null(), std::string(key)}, Null()}};
DataSet data_set = {
{{std::string(src[0]), std::string(key), std::string(iv), std::string(mode)}, r[0]},
{{std::string(src[1]), std::string(key), std::string(iv), std::string(mode)}, r[1]},
{{std::string(src[2]), std::string(key), std::string(iv), std::string(mode)}, r[2]},
{{std::string(src[3]), std::string(key), std::string(iv), std::string(mode)}, r[3]},
{{std::string(src[4]), std::string(key), std::string(iv), std::string(mode)}, r[4]},
{{std::string(src[5]), std::string(key), std::string(iv), std::string(mode)},
Null()},
{{Null(), std::string(key), std::string(iv), std::string(mode)}, Null()}};

check_function<DataTypeString, true>(func_name, input_types, data_set);
}
Expand Down Expand Up @@ -875,28 +887,37 @@ TEST(function_string_test, function_sm4_encrypt_test) {
TEST(function_string_test, function_sm4_decrypt_test) {
std::string func_name = "sm4_decrypt";
{
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String};
InputTypeSet input_types = {TypeIndex::String, TypeIndex::String, TypeIndex::String,
TypeIndex::String};

const char* key = "doris";
const char* iv = "0123456789abcdef";
const char* mode = "SM4_128_ECB";
const char* src[5] = {"aaaaaa", "bbbbbb", "cccccc", "dddddd", "eeeeee"};
std::string r[5];

for (int i = 0; i < 5; i++) {
int cipher_len = strlen(src[i]) + 16;
char p[cipher_len];

int outlen = EncryptionUtil::encrypt(
EncryptionMode::SM4_128_ECB, (unsigned char*)src[i], strlen(src[i]),
(unsigned char*)key, strlen(key), nullptr, 0, true, (unsigned char*)p);
int iv_len = 32;
std::unique_ptr<char[]> init_vec;
init_vec.reset(new char[iv_len]);
std::memset(init_vec.get(), 0, strlen(iv) + 1);
memcpy(init_vec.get(), iv, strlen(iv));
int outlen =
EncryptionUtil::encrypt(EncryptionMode::SM4_128_ECB, (unsigned char*)src[i],
strlen(src[i]), (unsigned char*)key, strlen(key),
init_vec.get(), strlen(iv), true, (unsigned char*)p);
r[i] = std::string(p, outlen);
}

DataSet data_set = {{{r[0], std::string(key)}, std::string(src[0])},
{{r[1], std::string(key)}, std::string(src[1])},
{{r[2], std::string(key)}, std::string(src[2])},
{{r[3], std::string(key)}, std::string(src[3])},
{{r[4], std::string(key)}, std::string(src[4])},
{{Null(), std::string(key)}, Null()}};
DataSet data_set = {
{{r[0], std::string(key), std::string(iv), std::string(mode)}, std::string(src[0])},
{{r[1], std::string(key), std::string(iv), std::string(mode)}, std::string(src[1])},
{{r[2], std::string(key), std::string(iv), std::string(mode)}, std::string(src[2])},
{{r[3], std::string(key), std::string(iv), std::string(mode)}, std::string(src[3])},
{{r[4], std::string(key), std::string(iv), std::string(mode)}, std::string(src[4])},
{{Null(), std::string(key), std::string(iv), std::string(mode)}, Null()}};

check_function<DataTypeString, true>(func_name, input_types, data_set);
}
Expand Down
8 changes: 7 additions & 1 deletion docs/en/docs/sql-manual/sql-functions/encrypt-digest-functions/aes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ AES_ENCRYPT
### description

Encryption of data using the OpenSSL. This function is consistent with the `AES_ENCRYPT` function in MySQL. Using AES_128_ECB algorithm by default, and the padding mode is PKCS7.
Reference: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_aes-decrypt

### Compatibility

1. aes_decrypt/aes_encrypt/sm4_decrypt/sm4_encrypt When the initial vector is not provided, block_encryption_mode will not take effect, and AES_128_ECB will be used for encryption and decryption in the end, which is inconsistent with the behavior of MySQL.
2. Add aes_decrypt_v2/aes_encrypt_v2/sm4_decrypt_v2/sm4_encrypt_v2 functions to support correct behavior. When no initial vector is provided, block_encryption_mode can take effect, aes-192-ecb and aes-256-ecb will be correctly encrypted and decrypted, and other block encryption modes will report an error. If there is no need to be compatible with old data, the v2 function can be used directly.

#### Syntax

Expand All @@ -42,7 +48,7 @@ AES_ENCRYPT(str,key_str[,init_vector])

- `str`: Content to be encrypted
- `key_str`: Secret key
- `init_vector`: Initialization Vector
- `init_vector`: Initialization Vector. The default value for the block_encryption_mode system variable is aes ecb mode, which does not require an initialization vector. The alternative permitted block encryption modes CBC, CFB1, CFB8, CFB128, and OFB all require an initialization vector.

#### Return Type

Expand Down
8 changes: 7 additions & 1 deletion docs/zh-CN/docs/sql-manual/sql-functions/encrypt-digest-functions/aes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ AES_ENCRYPT
### Description

Aes 加密函数。该函数与 MySQL 中的 `AES_ENCRYPT` 函数行为一致。默认采用 AES_128_ECB 算法，padding 模式为 PKCS7。底层使用 OpenSSL 库进行加密。
Reference: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_aes-decrypt

### Compatibility

1. aes_decrypt/aes_encrypt/sm4_decrypt/sm4_encrypt 当没有提供初始向量时，block_encryption_mode 不生效，最终都会使用 AES_128_ECB 加解密，这和 MySQL 的行为不一致。
2. 增加 aes_decrypt_v2/aes_encrypt_v2/sm4_decrypt_v2/sm4_encrypt_v2 函数支持正确的行为，没有提供初始向量时，block_encryption_mode 可以生效，aes-192-ecb 和 aes-256-ecb 将正确加解密，其他块加密模式将报错。如果无需兼容旧数据，可直接使用v2函数。

#### Syntax

Expand All @@ -42,7 +48,7 @@ AES_ENCRYPT(str,key_str[,init_vector])

- `str`: 待加密的内容
- `key_str`: 密钥
- `init_vector`: 初始向量
- `init_vector`: 初始向量。block_encryption_mode 默认值为 aes-128-ecb，它不需要初始向量，可选的块加密模式 CBC、CFB1、CFB8、CFB128 和 OFB 都需要一个初始向量。

#### Return Type

Expand Down
Loading