supplying filterwarnings overrides default filters

[jaraco]

I'm really happy about pytest 3.8 and the work in #2908.

And now that I'm seeing warnings again, I realize how blissful ignorance is.

So I've gone through one project and fixed all of the deprecation warnings for the project under test (there are still many coming from other projects). In order to assert that the project no longer has any warnings, I added this to pytest.ini:

filterwarnings = 
    error:::mylib[.*]

Now tests pass, but all of the deprecation warnings (from the dependencies) are suppressed. That wasn't what I'd intended by setting the project to error on warnings.

I don't think that's the right behavior. Instead, pytest should configure the filters for DeprecationWarnings and PendingDeprecationWarnings and then allow the user to override those filters but not suppress the defaults with any value.

If the project must suppress the defaults, the docs should at least describe what the defaults are so they can be copied into the configuration wishing to define other filters. I had to traverse to the source to determine what the defaults are:

filterwarnings =
	always::DeprecationWarning
	always::PendingDeprecationWarning

Is there any reason not to always configure the defaults and allow overrides?

[nicoddemus]

Hi @jaraco,

Now tests pass, but all of the deprecation warnings (from the dependencies) are suppressed. That wasn't what I'd intended by setting the project to error on warnings.

Indeed. When we introduced the warnings plugin a few versions back we had some serious regressions, so this time around I decided to be really conservative to minimize disruption.

The unittest module only changes the warnings filters to diplay deprecations if sys.warnoptions is set (usually by PYTHONWARNINGS). I decided to go with the same route with our warnings plugin:

pytest/src/_pytest/warnings.py

Lines 72 to 90 in 58aa4f9

	with warnings.catch_warnings(record=True) as log:
	filters_configured = args or inifilters or sys.warnoptions
	for arg in args:
	warnings._setoption(arg)
	for arg in inifilters:
	_setoption(warnings, arg)
	if item is not None:
	for mark in item.iter_markers(name="filterwarnings"):
	for arg in mark.args:
	_setoption(warnings, arg)
	filters_configured = True
	if not filters_configured:
	# if user is not explicitly configuring warning filters, show deprecation warnings by default (#2908)
	warnings.filterwarnings("always", category=DeprecationWarning)
	warnings.filterwarnings("always", category=PendingDeprecationWarning)

So basically we only install the filters if the user did not specify any filter configuration themselves, either using pytest's mechanisms or python's own PYTHONWARNINGS.

But I agree with you that this is not ideal, because as you point out any customization effectively silences deprecation warnings again.

I believe that for 3.9 we can install the deprecation filters before all other filters, right after catch_warnings() in the previous snippet. This shows deprecations by default, but any other filter that comes after it will take precedence.

[jaraco]

Wonderful. Yes, I completely understand the desire to be cautious... and I concur with your assessment and plan. Most importantly, thank you for all the time and effort you put into this project.

[nicoddemus]

Thanks @jaraco! 😊

PR opened at #4019 if you want to provide feedback. 👍