[Fix] decode error(#1806)

[mingchuno]

Short Description:

[Fix] decode error(#1806)

Fixes:

• fix exception like "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 31: ordinal not in range(128)"

[gregtampa]

someone test and comment please

[vieira]

Not working for me.

$ python pokecli.py -cf config.json 
[10:20:29] PokemonGO Bot v1.0
[10:20:29] Configuration initialized
[10:20:29] 
[10:20:29] Not logged in!
Traceback (most recent call last):
  File "pokecli.py", line 464, in <module>
    main()
  File "pokecli.py", line 79, in main
    report_summary(bot)
  File "pokecli.py", line 87, in report_summary
    metrics.capture_stats()
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/metrics.py", line 75, in capture_stats
    response_dict = self.bot.api.call()
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 57, in call
    if not self._can_call():
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 29, in _can_call
    raise NotLoggedInException()
pgoapi.exceptions.NotLoggedInException

As soon as I make decoded_str = string instead of decoded_string = string.decode('utf-8') it starts again but the original problem is still there, of course.

[mingchuno]

I am running it myself and it works
@vieira for your case, It looks like totally different issue from your stacktrace

[vieira]

The fix that works for me is:

--- a/pokemongo_bot/cell_workers/move_to_fort_worker.py
+++ b/pokemongo_bot/cell_workers/move_to_fort_worker.py
@@ -40,7 +40,8 @@ class MoveToFortWorker(object):
         )

         if dist > Constants.MAX_DISTANCE_FORT_IS_REACHABLE:
-            logger.log('Moving towards fort {}, {} left'.format(fort_name, format_dist(dist, unit))
+            logger.log(u'Moving towards fort {}, {} left'
+                    .format(fort_name.decode('utf-8'), format_dist(dist, unit)))

             step_walker = StepWalker(
                 self.bot,

@mingchuno Fell free the use the fix above in your PR if you see fit.

[vieira]

@mingchuno It's the same problem, the Unicode exception is probably getting caught by a generic except that just says unable to login. It's probably because my first name as a special character 😉

[mingchuno]

@vieira We should solve it in lower layer since the logger is shared by the whole app.
Adding .decode('utf-8') inside the logger impl is better than adding .decode('utf-8') every where in the code

[vieira]

@mingchuno Yes, I agree but in that case you have the fix every call to the logger that are already passing utf-8 encoded strings. Forget about my first name, it's probably the initial location.

[vieira]

@mingchuno Have a look at https://github.com/PokemonGoF/PokemonGo-Bot/blob/dev/pokemongo_bot/__init__.py#L456

[mingchuno]

@vieira Well, I am not actually know much on Python.
Isn't that u'test'.decode('utf-8') still give a valid unicode string?

[Di9]

it works for me fine, thx! But Instead of Cyrillic title i got in terminal pokestot name with wrong encoding anyway (before it also was with not-cyrilic encoding, just another one) :)

[vieira]

@mingchuno You encode a sequence of unicode characters into a str, e.g. print(u'ç'.encode('utf-8')). u'ç'.decode('utf-8') won't work. You use decode, for instance, to bring a some encoded str back to unicode, you have to know how the str was encoded and decode it with the same codec, e.g. 'utf-8'. I am no expert in this, either.

[vieira]

@mingchuno Can you reproduce my problem if you set your initial location to something with special chars, e.g. "ç" or "ü"?

[vieira]

@mingchuno Without changing anything else, I think this works:

def log(string, color='white'):
    try:
        string = string.decode('utf8')
    except UnicodeEncodeError:
        pass

<rest of the original code>

Sometimes log is called with an already decoded str, other with an unicode sequence. If it is already a string decode will fail but it is safe to continue.

[sgitkene]

@vieira your fix seems to do it for me: (I used to have an error due to the "ü" in Sihlbrücke)

[12:31:10] PokeStop on cooldown. Time left: 4 minutes, 59 seconds
[12:31:13] Moving towards fort 1866 Sihlbrücke, 0.11km left
[12:31:19] -- Recycled 3x Pokeball (keeps only 50 maximum)
[12:31:20] -- Recycled 1x Super Potion (keeps only 1 maximum)
[12:31:20] Moving towards fort 1866 Sihlbrücke, 0.10km left
[12:31:25] Moving towards fort 1866 Sihlbrücke, 0.08km left
[12:31:31] Something rustles nearby!

Edit: Apparently somethng's still wrong: (Why do we use ascii???)

Traceback (most recent call last):
File "pokecli.py", line 464, in
main()
File "pokecli.py", line 68, in main
bot.tick()
File "/home/max/PokemonGo-Bot/pokemongo_bot/init.py", line 84, in tick
if worker.work() == WorkerResult.RUNNING:
File "/home/max/PokemonGo-Bot/pokemongo_bot/cell_workers/seen_fort_worker.py", line 32, in work
logger.log('Now at Pokestop: {0}'.format(fort_name), 'cyan')
File "/home/max/PokemonGo-Bot/pokemongo_bot/logger.py", line 28, in log
print('[' + time.strftime("%H:%M:%S") + '] ' + u'\033[' + color_hex[color] + string.decode('utf-8') + '\033[0m')
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 28: ordinal not in range(128)

Edit2:

I put @CapCap 's fix in too and now it works. I believe nowhere should we still use ascii.
reload(sys)
sys.setdefaultencoding('utf8')

[vieira]

@sgitkene Ooops, missed an u'' in line 28. I will open a PR with both fixes.

Should be working now (Now at Pokestop: Quiosque Cais do Sodré):

[11:45:01] Moving towards fort Quiosque Cais do Sodré , 0.05km left
[11:45:06] Moving towards fort Quiosque Cais do Sodré , 0.04km left
[11:45:11] Now at Pokestop: Quiosque Cais do Sodré 
[11:45:11] Spinning ...

[vieira]

@sgitkene Could you help testing #1824? Thanks!

[WebberTakkn]

Please approve

[DayBr3ak]

190640f fixed this

[sgitkene]

sorry @vieira I was away. I assume it is no longer needed?

[vieira]

@sgitkene No problem at all! I guess #1839 should have fixed this but I can't test right now. If it works with:

1. initial location with special characters
2. moving to fort with special characters
3. arrived at pokestop with special characters

then we can close #1824.

[marksweb]

@vieira I've come up against a couple more UnicodeDecode errors with some French place names. It seems that setting the encoding when creating the string passed to logger.log() proves a better solution, e.g. 'Now at Pokestop: {0}'.format(unicode(fort_name, 'utf-8'))

I've done this with spin_fort.py and move_to_fort.py & had it running for an hour or so with no issues. Maybe it's worth another PR?

[elicwhite]

The logger has been deprecated and replaced with websockets. Hopefully this isn't an issue anymore. If it is, please re-open against the new approach.