"ParseError: bad input" (lib2to3) when target file has Python3 print using keyword args

[alwilson]

I'm running into issues with applying generated annotations to my Python3 snippet. I narrowed down my issue to keyword args(sep='' or end='') passed to print statements. After some debugging I learned that pyannotate is using 2to3 to refactor the code. It looks like 2to3 scans for certain python3 features and then fails to parse the entire file if it finds an instance.

Also, great tool! I'm excited to make use of this. It's worked very well on files without keyword args in print.

example.py:

#!/usr/bin/env python3

def hello(a, b):
    print('hello\n', end='')
    return a + b

from pyannotate_runtime import collect_types
collect_types.init_types_collection()
with collect_types.collect():
    ret = hello(4, 7)
collect_types.dump_stats("./type_info.json")

Applying annotations to example.py:

$ pyannotate --py3 ./example.py
Can't parse ./example.py: ParseError: bad input: type=22, value='=', context=('', (4, 24))
No files need to be modified.
There was 1 error:
Can't parse ./example.py: ParseError: bad input: type=22, value='=', context=('', (4, 24))
NOTE: this was a dry run; use -w to write files

Version info:

$ python --version
Python 3.7.4
$ pip search pyannotate
pyannotate (1.2.0)                    - PyAnnotate: Auto-generate PEP-484 annotations
  INSTALLED: 1.2.0 (latest)

Exception stack trace (from PyCharm):

refactor_string, refactor.py:360
refactor_file, refactor.py:335
refactor_file, refactor.py:728
refactor_file, __main__.py:54
refactor, refactor.py:282
refactor, refactor.py:687
main, __main__.py:139
<module>, pyannotate:10

[alwilson]

As a workaround, it appears that importing print_function from future helps 2to3. I'm not sure what the right fix is here though. Perhaps pyannotate can prepend python3 files with this import to bypass issues with lib2to3?

from __future__ import print_function

[alwilson]

Haha, well, disregard all of this. The -p or --print-function appears to do exactly what I wanted. Maybe there could be a warning or suggestion on using the print function? It is right in the help though, so...