Skip to content

FIX: suppress invalid data warnings for handling NaNs in DataRange classes#835

Merged
nicolasap-dm merged 5 commits into
mainfrom
fix/834-nan-warnings
Sep 15, 2022
Merged

FIX: suppress invalid data warnings for handling NaNs in DataRange classes#835
nicolasap-dm merged 5 commits into
mainfrom
fix/834-nan-warnings

Conversation

@nicolasap-dm
Copy link
Copy Markdown
Contributor

@nicolasap-dm nicolasap-dm commented Sep 15, 2022

Fixes #834

  • suppresses "invalid value" warnings in numpy when computing range masks in the BaseDataRange API implementations
    • in DataRange1D
    • in DataRange2D ¹
  • doesn't change behavior, apart from warning suppression

The default behavior of float comparison with nans is consistent with the mask_data abstract specification (being "inside the range" implies being finite), so it should be fine to confidently use it without warnings.

By the way, all uses of mask_data in the Chaco codebase outside of the classes themselves are already paired with a & nan_mask. I'm not sure if that's redundant, but it makes me confident that exluding nans from masks is expected everywhere.


¹ a curious finding: apparently transposing an array defuses the warning in the first place

Details
>>> import numpy as np
>>> nan = float("nan")

>>> np.array([1.0, nan]) > 0.5
__main__:1: RuntimeWarning: invalid value encountered in greater
array([ True, False])

>>> np.array([[1.0], [nan]]).transpose()[0] > 0.5
array([ True, False]) 

Edit: I was wrong, there is an issue of state. Whichever of the above comparisons gets run first will emit the warning.

@nicolasap-dm
Copy link
Copy Markdown
Contributor Author

nicolasap-dm commented Sep 15, 2022

The tests check that NaNs are False in masks.

They try to check also that warnings are not emitted, however it is not guaranteed that they would spot a regression, because if a matching warning has already been emitted anywhere else in the test suite, the "once" rule on warnings will already have prevented it to be emitted again.

From the docs, it seems like it should be overcomable (emphasis mine):

One thing to be aware of is that if a warning has already been raised because of a once/default rule, then no matter what filters are set the warning will not be seen again unless the warnings registry related to the warning has been cleared.

https://docs.python.org/3/library/warnings.html#testing-warnings

however, clearing the registry doesn't seem to be easy or supported: https://bugs.python.org/issue21724.

CC @mdickinson since this came up in offline discussion.


Mark, you're welcome to review if you have time. The core of the PR is based on @rkern's suggestion on Slack.

@nicolasap-dm nicolasap-dm changed the title FIX: suppress invalid data warnings for handling NaNs in DataRange1D FIX: suppress invalid data warnings for handling NaNs in DataRange classes Sep 15, 2022
Copy link
Copy Markdown
Contributor

@corranwebster corranwebster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Thanks for the contribution.

@nicolasap-dm nicolasap-dm merged commit c55f8a6 into main Sep 15, 2022
@corranwebster corranwebster added the needs backport Needs to be backported to current maint/* label Oct 13, 2022
@corranwebster corranwebster deleted the fix/834-nan-warnings branch January 24, 2023 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs backport Needs to be backported to current maint/*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plotting data with nans causes numpy warnings

2 participants