Polars/Pyarrow Support

[mkuelkerRapidSpan]

I like this library and may end up using it for my company.

We use Polars in our data Pipelines. This is because it is faster and has a smaller size than pandas. Works great for event driven serverless functions.

I'm marking this as an issue as I may want to open a PR later to add support for polars data frames.

[MarcusRisanger]

Hello there 👋

I think extending with polars support sounds like a great idea.

Perhaps this can be solved without too much hassle by abstracting the dataframe "engine" parts, have pandas/polars as dev dependencies only, and throw runtime errors if the environment is missing both packages.

I will be off on vacation for the next 3 weeks but will have time to look at this after!

[MarcusRisanger]

Hi again @mkuelkerRapidSpan - I've looked at this now and it seems fairly straightforward with narwhals. Take a look at the PR and let me know what you think!

I added tests to cover both pandas and polars dataframes in create/upsert, but feel free to fork the branch and test locally.

[mkuelkerRapidSpan]

The PR looks good @MarcusRisanger. I'm impressed by how easy this was to implement.

Thanks for adding this!

[MarcusRisanger]

Deployed to pypi now, thanks again for the suggestion!