-
-
Notifications
You must be signed in to change notification settings - Fork 19.4k
CLN/TST: delegate StringArray.fillna() to parent class + add tests #37987
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CLN/TST: delegate StringArray.fillna() to parent class + add tests #37987
Conversation
pandas/core/arrays/string_.py
Outdated
|
|
||
| def fillna(self, value=None, method=None, limit=None): | ||
| # TODO: validate dtype | ||
| if not isinstance(value, str): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
np.str_?
its a weird usage, but there's no reason why a user couldn't pass a n NA fill value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added both, including the missing value
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
np.str_ is already be covered by str (it subclasses str):
In [13]: arr = np.array(["a"])
In [14]: type(arr[0])
Out[14]: numpy.str_
In [15]: isinstance(arr[0], str)
Out[15]: True
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right. Got rid of the np.str_ check
0a85704 to
34e3d82
Compare
c5da53a to
616a6c7
Compare
…ring-validate-fillna-args
|
Does this need a whatsnew? |
I don't think it is necessarily needed, it's only a change in the error message I think? |
jreback
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pandas/core/arrays/string_.py
Outdated
|
|
||
| def fillna(self, value=None, method=None, limit=None): | ||
| # TODO: validate dtype | ||
| if value is not None and not ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should just be able to call _validate_setitem_value(value), maybe we can it it in the baseclass.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i would like this since it would allow us to implement a correct ExtensionBlock._can_hold_element, xref #36226
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should just be able to call
_validate_setitem_value(value), maybe we can it it in the baseclass.
Gave that a go - is the last commit what you meant? @jreback @jbrockmendel
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am -1 adding it in this PR (or at least adding it on the base class). If we add it to the base class, it needs to be part of the EA interface, documented that way, have a base extension test, have a fall back implementation, .. (and first discuss if we actually want it).
Also, we actually don't need to explicitly call the setitem validation here, since fillna is already raising that exception because it uses the setitem implementation under the hood.
So we could also simply add the test to ensure fillna raises the proper error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am -1 adding it in this PR (or at least adding it on the base class). If we add it to the base class, it needs to be part of the EA interface, documented that way, have a base extension test, have a fall back implementation, .. (and first discuss if we actually want it).
Ok! I'll open an issue
Also, we actually don't need to explicitly call the setitem validation here, since
fillnais already raising that exception because it uses the setitem implementation under the hood.So we could also simply add the test to ensure
fillnaraises the proper error.
There was an existing test and I added more testcases here
…ring-validate-fillna-args
…ring-validate-fillna-args
pandas/core/arrays/string_.py
Outdated
| def fillna(self, value=None, method=None, limit=None): | ||
| # TODO: validate dtype | ||
| if value is not None: | ||
| value = self._validate_setitem_value(value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is actually not needed, as the setitem under the hood of fillna already calls it. So calling it here as well means we validate the value twice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right, reverted all the changes here
…ring-validate-fillna-args
|
I pared down this PR to just the fillna test for strings. Discussion on |
pandas/core/arrays/string_.py
Outdated
|
|
||
| def fillna(self, value=None, method=None, limit=None): | ||
| # TODO: validate dtype | ||
| return super().fillna(value, method, limit) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this method need to be overriden here at all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it doesn't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pushed that now
|
thanks @arw2019 |
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff