Skip to content

Conversation

@jbrockmendel
Copy link
Member

Preliminary pushing towards #34714

@jreback jreback added Apply Apply, Aggregate, Transform, Map Internals Related to non-user accessible pandas implementation labels Jul 20, 2020
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ultimately you want to put this _apply_blockwise in a BlockwiseMixin, similar to SelectionMixin, so you can add it to groupby and so on, but suppose could be done later

return VariableWindowIndexer(index_array=self._on.asi8, window_size=window)
return FixedWindowIndexer(window_size=window)

def _apply_blockwise(self, homogeneous_func: Callable):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you type the output

# of homogenenous DataFrames.
blocks, obj = self._create_blocks(self._selected_obj)

skipped = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you type these

@jbrockmendel
Copy link
Member Author

jbrockmendel commented Aug 5, 2020

I think ultimately you want to put this _apply_blockwise in a BlockwiseMixin, similar to SelectionMixin, so you can add it to groupby and so on, but suppose could be done later

I'm planning a BlockManager method that subsumes all of the apply-blockwise usages.

For that to work, we need to get all of the relevant looping-over-blocks functions into a form that can be passed to such a method. Thats the intermediate goal of this and #35356.

@jbrockmendel
Copy link
Member Author

@simonjayhawkins is this new/expected?

pandas/core/window/rolling.py:500: error: Type variable "pandas._typing.ArrayLike" is unbound
pandas/core/window/rolling.py:500: note: (Hint: Use "Generic[ArrayLike]" or "Protocol[ArrayLike]" base class to bind "ArrayLike" inside a class)
pandas/core/window/rolling.py:500: note: (Hint: Use "ArrayLike" in function signature to bind "ArrayLike" inside a function)

Co-authored-by: Simon Hawkins <simonjayhawkins@gmail.com>
@jbrockmendel
Copy link
Member Author

that worked, thanks @simonjayhawkins

@jreback jreback added this to the 1.2 milestone Aug 6, 2020
@jreback jreback merged commit 3e10b7c into pandas-dev:master Aug 6, 2020
@jreback
Copy link
Contributor

jreback commented Aug 6, 2020

thanks

@jbrockmendel jbrockmendel deleted the ref-mgr-transform-2 branch August 6, 2020 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Apply Apply, Aggregate, Transform, Map Internals Related to non-user accessible pandas implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants