BUG: GroupBy.size created by TimeGrouper raises AttributeError

[sinhrks]

Related to #7453 2nd issue:

• GroupBy.size created by TimeGrouper raises AttributeError

[jreback]

why are you not using value_counts? this should be very similar (if not call irectly), BaseGrouper.size

[sinhrks]

There are 2 reasons.

• value_counts cannot fulfill intermediate timestamps which is not included in the group key. For example, if the TimeGrouper categorize [2011-01-01, 2011-01-02, 2011-01-04] with daily frequency, the resulted group keys must be [2011-01-01, 2011-01-02, 2011-01-03, 2011-01-04]
• BinGrouper doesn't retain original Index to be directly passed to value_counts. To use value_counts, it must be created using similar logic.

indices = self.indices
labels = []
for k, v in compat.iteritems(indices):
            labels.extend([k] * len(v))
value_counts(labels)

[jreback]

ok can u add a bench for size in that case?
for both a regular groupby and TimeGrouper?

[sinhrks]

OK, added benchmarks.

[jreback]

can I post benchmarks for those 2? thxs

[sinhrks]

Here it is. Cannot measure base performance of groupby_dt_timegrouper_size because of the bug, but it looks not slow comparing to groupby_dt_size

-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
groupby_dt_size                              |  42.2956 |  55.4504 |   0.7628 |
groupby_dt_timegrouper_size                  |  34.2657 |  ------- |   ------ |

[jreback]

run then with -Hto test current perf (with your fix) with comparable samples (or do in ipython).

[sinhrks]

Yep, current perf of groupby_dt_timegrouper_size is attached above.