UnicodeEncodeError on source files containing non-ascii characters#4074
Closed
spg wants to merge 2 commits intopytest-dev:masterfrom
Closed
UnicodeEncodeError on source files containing non-ascii characters#4074spg wants to merge 2 commits intopytest-dev:masterfrom
spg wants to merge 2 commits intopytest-dev:masterfrom
Conversation
… non-ascii characters
nicoddemus
reviewed
Oct 3, 2018
| if astnode is None: | ||
| content = str(source) | ||
| if six.PY2: | ||
| content = unicode(source) |
Member
There was a problem hiding this comment.
Unfortunately this still might fail, because unicode will try to use the ascii codec to decode the bytestream (see py27 failure). Not sure if by this point getstatementrange_ast shouldn't already be a unicode stream actually. 🤔
Member
|
Your original reproduction does not reproduce for me: Can you list all of your currently installed dependencies? I suspect something suspicious is afoot |
Author
|
@asottile Yes you are right, I created a fresh virtualenv (Python 2.7.14), and like you said, the problem is not happening. |
Member
|
@spg could you list the deps in your (broken) venv so we can triage what package is doing nefarious things? |
Author
|
Here is the output from |
Author
|
I have been able to reproduce the issue with a fresh venv and by installing the following packages: pytest (latest), |
Author
|
I have narrowed down the venv even further. I can reproduce the issue by installing only: |
Member
|
I swear, it's going to be the number of week-hours of developer time I've lost to that thing. |
Member
|
ok yep, it's a bug in pyfakefs -- I'll look on making a PR there |
Member
|
Here's that PR, with that applied the example works again: pytest-dev/pyfakefs#441 Thanks again for the issue and reproduction steps, this was paramount to finding the root cause! 🎉 |
Author
|
Thanks for the fix! I applied your patch locally, and it all works now! 🎉 |
mrbean-bremen
pushed a commit
to pytest-dev/pyfakefs
that referenced
this pull request
Oct 11, 2018
`python-future` is notorious for breaking modules which use `try:` / `except:` to import modules based on version. In this case, `pyfakefs` imported the backported `builtins` module which changes the semantics of the `open()` function. `pyfakefs` then monkeypatches `linecache` which breaks any module which attempts to use `linecache` (in this case `pytest`). The downstream issue is pytest-dev/pytest#4074
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes an issue I had when running tests that:
Here's an example of such a test (
test_bug.py):Running this test outputs the following:
Aside from the obvious stacktrace, you can see that pytest reported that 0 tests ran.
With my fix:
I'm not sure if or how I should write a test for this particular case. Any help is appreciated.