fix(utils): Fix UnicodeDecodeError on Python 2
#2657
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There were at least two things wrong with our
strip_stringimplementation:UnicodeDecodeErrors on Python 2 if a non-unicode string (i.e.,bytesin Python 2) was provided. When encoding such a (byte) string, Python 2 will first try to coerce it to a unicode string, usingasciito decode, leading to aUnicodeDecodeErrorif the byte string contains anything beyond ascii.strip_stringtrims by number of chars instead of bytes #2634: We found out how many bytes should be stripped from the string, but were then actually stripping that many characters/code points, not bytes.This PR addresses both issues.
Closes #2612
Closes #2634
General Notes
Thank you for contributing to
sentry-python!Please add tests to validate your changes, and lint your code using
tox -e linters.Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.
For maintainers
Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.
Before running sensitive test suites, please carefully check the PR. Then, apply the
Trigger: tests using secretslabel. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.