REF/BUG: cast back to datetimelike in pad/backfill #40052
Conversation
pandas/core/missing.py
Outdated
|
|
||
| return values | ||
|
|
||
| def _datetimelike_compat(func): |
There was a problem hiding this comment.
I've just started (and I mean just) experimenting with having a numba version of the libs.
numpy datetime and timedeltas can be handled by numba. This means that _cast_values_for_fillna as removed here (but replaced with the decorator) and creating the view of the mask are unnecessary with numba. d449ca0
i suspect that many of the i8 conversions in the main codebase could become redundant (presumably numba deals with this)
with this in mind, would it be worth considering moving any i8 conversions into _libs? This may produce a cleaner _libs api and simplify the python code.
There was a problem hiding this comment.
oh. and back to the PR. can you type the decorator to preserve the signature of the decorated function (ignoring that values to the decorator will be superset of values in the function, since we don't yet have type parameters for numpy arrays)
There was a problem hiding this comment.
I've just started (and I mean just) experimenting with having a numba version of the libs.
neat!
with this in mind, would it be worth considering moving any i8 conversions into _libs? This may produce a cleaner _libs api and simplify the python code.
ill have to give that some thought. one wrinkle that comes to mind is that cdef functions dont tend to play nicely with decorators
There was a problem hiding this comment.
may need a python interface layer in _libs, but early days yet since don't know how will handle pandas types that are defined as ctypes, without a performance hit.
| # This needs to occur before datetime/timedeltas are cast to int64 | ||
| mask = isna(values) | ||
| result = func(values.view("i8"), limit=limit, mask=mask) | ||
| return result.view(values.dtype) |
There was a problem hiding this comment.
since the cython funcs operate inplace, is it possible to just return values. or is a view of a view nbd.
There was a problem hiding this comment.
yah lets save this for a future step
jreback
added
the
Missing-data
3 participants
Goal: simplify/perf for simple_new by requiring correct-type (no i8) values
Problem: a bunch of places pass i8 values
Solution: track then down and pass/return the right thing in the first place.