release-22.2: pkg/keys: implement SafeFormatter for roachpb.Key#87647
Merged
abarganier merged 1 commit intorelease-22.2from Sep 20, 2022
Merged
release-22.2: pkg/keys: implement SafeFormatter for roachpb.Key#87647abarganier merged 1 commit intorelease-22.2from
abarganier merged 1 commit intorelease-22.2from
Conversation
Currently, when a key is logged, the entirety of the pretty-printed key is redacted, hindering observability when parsing through redacted logs (something that will become more common with upcoming compliance requirements). For example, prior to this patch, a pretty-printed key would appear in the following way for the unredacted/redacted cases, respectively: - unredacted: ‹/Table/42/1222/"index key"› - redacted: ‹x› This patch addresses this by implementing the SafeFormatter interface for `roachpb.Key` and `roachpb.RKey`, yielding the following result when looking at the same example above: - unredacted: /Table/42/1222/‹"index key"› - redacted: /Table/42/1222/‹x› While the index key itself remains redacted, the ability to see the specific table, index, and in the case of tenant tables, the tenant itself, provides much better observability into which table & index a log line is referring to than before. Note that this implementation is only partial. It currently only supports keys that fall in the `/Table` keyspace for application tenants and system tenants. Keyspaces such as Meta1, Meta2, Local, etc. are not yet supported, but can be added with much more ease in the future now that the example has been set. Finally, we remove the `maxLen` and related truncation logic from `StringWithDirs`, as this is no longer used. Furthermore, the truncation was invalid as it could have truncated a utf-8 sequence in the wrong place, making the result invalid utf-8. Release note (security update): redacted logs will now reveal pretty-printed keys, except for the index key values themselves. For example `/Table/42/1222/‹x›` will be shown instead of `‹x›` (which was shown previously). This improved redaction is available for the `/Table` keyspace for both system and application tenants. Other keyspaces such as `/Meta1`, `/Meta2`, `/Local`, etc. are not yet supported. Release justification: low risk, high benefit observability changes
ff5ad33 to
6f93145
Compare
Author
|
Thanks for opening a backport. Please check the backport criteria before merging:
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
Add a brief release justification to the body of your PR to justify this backport. Some other things to consider:
|
Member
dhartunian
approved these changes
Sep 19, 2022
andreimatei
approved these changes
Sep 20, 2022
Contributor
|
TFTR! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #86813 on behalf of @abarganier.
/cc @cockroachdb/release
Currently, when a key is logged, the entirety of the pretty-printed
key is redacted, hindering observability when parsing through redacted
logs (something that will become more common with upcoming compliance
requirements).
For example, prior to this patch, a pretty-printed key would appear
in the following way for the unredacted/redacted cases, respectively:
This patch addresses this by implementing the SafeFormatter interface
for
roachpb.Keyandroachpb.RKey, yielding the following resultwhen looking at the same example above:
While the index key itself remains redacted, the ability to see the
specific table, index, and in the case of tenant tables, the tenant
itself, provides much better observability into which table & index
a log line is referring to than before.
Note that this implementation is only partial. It currently only
supports keys that fall in the
/Tablekeyspace for applicationtenants and system tenants. Keyspaces such as Meta1, Meta2, Local,
etc. are not yet supported, but can be added with much more ease
in the future now that the example has been set.
Finally, we remove the
maxLenand related truncation logic fromStringWithDirs, as this is no longer used. Furthermore, thetruncation was invalid as it could have truncated a utf-8
sequence in the wrong place, making the result invalid utf-8.
This PR is a continuation of the work originally done by @kzh in
#67065. See the original PR for some initial discussions.
Release note (security update): redacted logs will now reveal
pretty-printed keys, except for the index key values themselves.
For example
/Table/42/1222/‹x›will be shown instead of‹x›(which was shown previously). This improved redaction is available
for the
/Tablekeyspace for both system and application tenants.Other keyspaces such as
/Meta1,/Meta2,/Local, etc. are notyet supported.
Release justification: low risk, high benefit observability changes
Addresses #86316
Release justification: