KMS Encryption Plugin #1543
KMS Encryption Plugin #1543
Conversation
| - AWS KMS key with appropriate permissions | ||
| - Database table to store encryption metadata | ||
| - AWS credentials configured (via IAM roles, profiles, or environment variables) | ||
| - **JSqlParser 4.5.x dependency** - Required for SQL parsing and analysis |
There was a problem hiding this comment.
we're in the SQL parsing business now!
There was a problem hiding this comment.
No choice... that is the only way to tell if a column requires encryption (at the moment)
|
|
||
| ```sql | ||
| -- Key storage table (must be created first due to foreign key) | ||
| CREATE TABLE key_storage ( |
There was a problem hiding this comment.
Do we generally rely on user connecting to a database which has this table? What if I have a user with multiple schemas available to me, where this table needs to be? Also of course, what is multiple tables with the same name exist in different schemas? Could there be a any confusion as to what metadata table is relevant to them?
There was a problem hiding this comment.
should we have tables prefixed with like "aws_" or configurable via properties?
There was a problem hiding this comment.
sure, implementation detail... could be in an aws schema
| // Configure encryption for sensitive columns | ||
| keyManagementUtility.initializeEncryptionForColumn("customers", "ssn", masterKeyArn); | ||
| keyManagementUtility.initializeEncryptionForColumn("customers", "credit_card", masterKeyArn); | ||
| keyManagementUtility.initializeEncryptionForColumn("customers", "phone", masterKeyArn); | ||
| keyManagementUtility.initializeEncryptionForColumn("customers", "address", masterKeyArn); |
There was a problem hiding this comment.
It feels like to make sure there is no sneaky insert of unencrypted data by mistake the table should have a trigger which looks for some KMS-related prefixes in the values inserted and reject if it doesn't find them. Maybe we should recommend customers setting something up. I believe KMS-encrypted strings have specific look to them
There was a problem hiding this comment.
This will cover all possible paths how the data might end up inserted into the table, such as via PL/SQL function or copied from another table...
There was a problem hiding this comment.
This is certainly a valid concern. Since this will work with MySQL as well we may have to figure something out.
|
|
||
| ## Limitations | ||
|
|
||
| - Currently supports string data types for encryption |
There was a problem hiding this comment.
I assume ORDER BY or LIKE are not much useful since server side cannot figure it out.
However, if I do group by encrypted column, will it work?
There was a problem hiding this comment.
nope, won't work. You can't do any sorting, indexing etc on an encrypted column. You would need another column with say the last 4 digits if the ssn to do equaliity
There was a problem hiding this comment.
If the same value is re-written to the encrypted column, is it going to have a different cyphertext?
There was a problem hiding this comment.
It will have a different cypher text. The encryption is non-determistic.
| optionalImplementation("io.opentelemetry:opentelemetry-api:1.52.0") | ||
| optionalImplementation("io.opentelemetry:opentelemetry-sdk:1.52.0") | ||
| optionalImplementation("io.opentelemetry:opentelemetry-sdk-metrics:1.52.0") | ||
| optionalImplementation("com.github.jsqlparser:jsqlparser:4.5") // JSqlParser SQL parser (Java 8 compatible) |
There was a problem hiding this comment.
have we done a security review / audit for this dependency?
There was a problem hiding this comment.
no, but I may replace it with my own.
working code updated packages and added license files moved to correct package and now compiles recovered the parser code fix javadoc warnings removed reflection and added test for SqlAnalysisService add AwsWrapperProperty to EncryptionConfig cleaned up the configuration properties and added docs working code update SQLAnalyzer to handle more complex joins refactored to use jooq parser instead of our own PostgreSQL parser refactored to use jooq parser instead of our own PostgreSQL parser removed jooq and used JSQLParser removed unused methods fixed documentation
…ionIntegration test to use the key management utility instead of manually updating tables fix useless test
remember where we are finished switching to JUL removed all Jooq, updated SqlAnalyzer to handle columns better fixed tests Parser now handles more complex queries make sure the parser can handle schema qualified names and add a schema to the key metadata tables added a property ENCRYPTION_METADATA_SCHEMA to specify the schema that the key metadata is store in run the KmsEncryptionIntegrationTest
…so that we can verify that the data has been HMAC encrypted
working now that we use binary transfer and an EncryptedData PGObject add in the extension code removed the extension code that created the encrypted_data type, use a domain over bytea now. Fixed tests, now the trigger works to ensure the data is hmac encrypted before changing it
davecramer
force-pushed
the
kms_encryption1
branch
from
74fba5b to
e3db7a7
Compare
2 participants
Summary
Plugin to enable encrypting columns on the client
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.