Skip to content

BUG: Copy categorical codes if empty (fixes #18051) #18279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghasemnaddaf wants to merge 2 commits into from
Closed

BUG: Copy categorical codes if empty (fixes #18051) #18279

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6c751c0
BUG: Copy categorical codes if empty (fixes #18051)
Nov 15, 2017
21734aa
Unit tests
Nov 20, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.21.1.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ Categorical
- Error messages in the testing module have been improved when items have
different ``CategoricalDtype`` (:issue:`18069`)
- ``CategoricalIndex`` can now correctly take a ``pd.api.types.CategoricalDtype`` as its dtype (:issue:`18116`)
- Bug in ``Categorical.unique()`` returning read-only ``codes`` array when all categories were ``NaN`` (:issue:`18051`)

Other
^^^^^
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/categorical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2250,7 +2250,7 @@ def _recode_for_categories(codes, old_categories, new_categories):

if len(old_categories) == 0:
# All null anyway, so just retain the nulls
return codes
return codes.copy()
Copy link
Contributor Author

@ghasemnaddaf ghasemnaddaf Nov 14, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return codes causes writable flag to be False hence we get the error reported in #18051

indexer = coerce_indexer_dtype(new_categories.get_indexer(old_categories),
new_categories)
new_codes = take_1d(indexer, codes.copy(), fill_value=-1)
Expand Down
10 changes: 10 additions & 0 deletions pandas/tests/test_categorical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1673,6 +1673,12 @@ def test_unique(self):
exp_cat = Categorical(["b", np.nan, "a"], categories=["b", "a"])
tm.assert_categorical_equal(res, exp_cat)

# GH 18051
cat = Categorical([np.nan])
res = cat.unique()
exp_cat = Categorical([np.nan], categories=[])
tm.assert_categorical_equal(res, exp_cat)
Copy link
Contributor Author

@ghasemnaddaf ghasemnaddaf Nov 20, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback , do you mean like this? (This is not failing on 0.21.0 though)

Copy link
Contributor

@topper-123 topper-123 Nov 21, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a test for

assert pd.Series(pd.Categorical([np.nan])).nunique() == 0

Note that nunique is a method on Series, not Categorical.

Copy link
Contributor Author

@ghasemnaddaf ghasemnaddaf Nov 21, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@topper-123 I have added that test in https://github.com/pandas-dev/pandas/pull/18279/files#diff-ed4f442894a2f521dfac3193a3a8d8a0R2185 (L2185) below.

Copy link
Contributor

@topper-123 topper-123 Nov 21, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks great IMO.

Also, you mention above that Categorical([np.nan]).unique() doesn't fail in 0.21.0, but pd.Series(pd.Categorical([np.nan])).unique() does fail. So if you could add a test for that as well in the test for series, then IMO the PR is good (but let @jreback make the final call on that).


def test_unique_ordered(self):
# keep categories order when ordered=True
cat = Categorical(['b', 'a', 'b'], categories=['a', 'b'], ordered=True)
Expand Down Expand Up @@ -2174,6 +2180,10 @@ def test_basic(self):
result = x.person_name.loc[0]
assert result == expected

# GH 18051
s = pd.Series(pd.Categorical([np.nan]))
assert s.nunique() == 0

def test_creation_astype(self):
l = ["a", "b", "c", "a"]
s = pd.Series(l)
Expand Down