MemoryError using custom warning formatter in python 2.7

[Sup3rGeo]

• pytest master
• python 2.7
• Windows 10

Running the following test makes pytest hang. Python uses the whole PC memory and eventually it exists with MemoryError:

# -*- coding: utf8 -*-
from __future__ import unicode_literals
import warnings
import pytest


def _custom_showwarning(message, *a, **b):
    return "WARNING: {}".format(message)

warnings.formatwarning = _custom_showwarning

@pytest.mark.filterwarnings("default")
def test_custom_warning_formatter():
    warnings.warn("¥")

Maybe I am doing something wrong here. This problems does not happen with python 3.6.

[pytestbot]

GitMate.io thinks possibly related issues are #2917 (Asserting repeated warning does not work in Python 2.7), #2642 (Introduce six to the codebase to deal with Python 2/Python 3 differences), #3339 (deprecation warnings when imported on python 3.7 beta), #2327 (capsys not capturing on Python 2.7), and #3 (not executable on Python 2.7).

[asottile]

monkeypatching the warnings module probably isn't the best idea. In this case it's leading to the following output (I ^Cd after a second)

============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /tmp, inifile:
collected 1 item

test.py .                                                                [100%]

=============================== warnings summary ===============================
test.py::test_custom_warning_formatter
  WARNING: \xa5
  WARNING: Warning is using unicode non convertible to ascii, converting to a sa
fe representation:\n  WARNING: \\xa5
  WARNING: Warning is using unicode non convertible to ascii, converting to a sa
fe representation:\n  WARNING: Warning is using unicode non convertible to ascii
, converting to a safe representation:\\n  WARNING: \\\\xa5
  WARNING: Warning is using unicode non convertible to ascii, converting to a sa
fe representation:\n  WARNING: Warning is using unicode non convertible to ascii
, converting to a safe representation:\\n  WARNING: Warning is using unicode non
 convertible to ascii, converting to a safe representation:\\\\n  WARNING: \\\\\
\\\xa5
  WARNING: Warning is using unicode non convertible to ascii, converting to a sa
fe representation:\n  WARNING: Warning is using unicode non convertible to ascii
, converting to a safe representation:\\n  WARNING: Warning is using unicode non
 convertible to ascii, converting to a safe representation:\\\\n  WARNING: Warni
ng is using unicode non convertible to ascii, converting to a safe representatio
n:\\\\\\\\n  WARNING: \\\\\\\\\\\\\\\\xa5
  WARNING: Warning is using unicode non convertible to ascii, converting to a sa
fe representation:\n  WARNING: Warning is using unicode non convertible to ascii
, converting to a safe representation:\\n  WARNING: Warning is using unicode non
 convertible to ascii, converting to a safe representation:\\\\n  WARNING: Warni
ng is using unicode non convertible to ascii, converting to a safe representatio
n:\\\\\\\\n  WARNING: Warning is using unicode non convertible to ascii, convert
ing to a safe representation:\\\\\\\\\\\\\\\\n  WARNING: \\\\\\\\\\\\\\\\\\\\\\\
\\\\\\\\\xa5

This patch """fixes""" the crash, but I suspect you shouldn't be monkeypatching in the first place -- was this code found in the wild somewhere?

diff --git a/src/_pytest/warnings.py b/src/_pytest/warnings.py
index abd04801..67548e03 100644
--- a/src/_pytest/warnings.py
+++ b/src/_pytest/warnings.py
@@ -100,7 +100,7 @@ def catch_warnings_for_item(item):
             if unicode_warning:
                 warnings.warn(
                     "Warning is using unicode non convertible to ascii, "
-                    "converting to a safe representation:\n  %s" % msg,
+                    "converting to a safe representation:\n  %r" % msg,
                     UnicodeWarning,
                 )
 

[Sup3rGeo]

Actually if you search for "format warnings python" you get into this stackoverflow post from which one of the answers is to monkey patch that function (easier than patching warning.showwarning which is unnecessarily more complex).

It also seems that warnings somehow expects warnings.formatwarning to be replaced, although not mentioned in the documentation:
https://github.com/python/cpython/blob/5d4cb54800966947db2e86f65fb109c5067076be/Lib/warnings.py#L114-L116

Is there a better way of doing this?

I would assume that many other people might be using this as well to format their warnings. If there is no downside in the "fix" and it avoids the problem in this case, I would say it is a good idea to have it merged?

[RonnyPfannschmidt]

@Sup3rGeo missuses warnings by using unicode literals - they need native strings for correct operation

we should use safe_str to ensure the message is a native string - ironically the code that fails is precisely in place because people mess it up all the time ^^

[Sup3rGeo]

thanks @RonnyPfannschmidt for the information! Question is, if the "fix" from @asottile makes everything work fine regardless of unicode in warnings formatting, then isn't that a good thing to merge?

[RonnyPfannschmidt]

@Sup3rGeo on its own it fixes the wrong problem, we need to fix the right problem ^^