Support polars, etc. through narwhals#957
Merged
Martin Stancsics (stanmart) merged 36 commits intomainfrom Mar 9, 2026
Merged
Conversation
Martin Stancsics (stanmart)
changed the title
Martin Stancsics (stanmart)
changed the title
This was
linked to
issues
Jan 8, 2026
Collaborator
Author
|
Needs tabmat #502 and a new tabmat release. |
Copilot started reviewing on behalf of
Martin Stancsics (stanmart)
January 12, 2026 16:30
View session
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds support for Polars and other dataframe libraries through the narwhals compatibility layer, enabling glum to work with multiple dataframe backends beyond pandas. It also implements backwards compatibility for pickled models from earlier glum versions.
Changes:
- Integrated narwhals library to support pandas, polars, and other dataframe backends
- Migrated from
feature_dtypes_to_categorical_levels_for categorical column tracking - Added backwards compatibility handling in
__setstate__for unpickling models from glum 3.0+ - Updated minimum Python version to 3.10 and bumped minimum dependency versions
Reviewed changes
Copilot reviewed 15 out of 18 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| setup.py | Updated Python version requirement to 3.10+, added narwhals dependency and version constraints for core dependencies |
| pyproject.toml | Updated mypy Python version target to 3.10 |
| pixi.toml | Updated Python version constraints and dependency versions, removed py39 environment |
| conda.recipe/recipe.yaml | Updated dependency versions in conda recipe |
| src/glum/_glm.py | Added narwhals support, implemented pickle compatibility via setstate, migrated to categorical_levels_, added deprecated feature_dtypes_ property |
| src/glum/_glm_cv.py | Changed copy_X parameter type to Optional[bool] |
| src/glum/_utils.py | Converted utility functions to work with narwhals DataFrames instead of pandas-only |
| src/glum/_validation.py | Updated array checking to use narwhals for dataframe detection |
| src/glum/_typing.py | Added IntoDataFrame and ShapedArrayLikeConverted type aliases |
| tests/glm/test_utils.py | Updated tests to use narwhals DataFrames |
| tests/glm/test_pickle_compatibility.py | New test file for pickle backwards compatibility |
| tests/glm/test_golden_master.py | Added polars parametrization for existing tests |
| tests/glm/test_glm_regressor.py | Added polars test cases alongside pandas tests |
| tests/glm/test_glm_base.py | Parametrized tests for both pandas and polars |
| tests/glm/test_formula.py | Added polars support to formula tests (with fixture issue) |
| tests/glm/pickles/*.pkl | Pickle files from glum v3.0 for compatibility testing |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
1 task
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Martin Stancsics (stanmart)
requested review from
Marc-Antoine Schmidt (MarcAntoineSchmidtQC) and
Jan Tilly (jtilly)
as code owners
Collaborator
Author
|
This is now ready for review. CI is failing because of an unrelated issue and should be fixed by #965 |
Collaborator
Author
Marc-Antoine Schmidt (MarcAntoineSchmidtQC)
approved these changes
Jan 30, 2026
Member
Marc-Antoine Schmidt (MarcAntoineSchmidtQC)
left a comment
Marc-Antoine Schmidt (MarcAntoineSchmidtQC)
left a comment
There was a problem hiding this comment.
Looks good to me!
Merged
The GeneralizedLinearRegressorCV fitting behavior changed intentionally on main (alpha path computation + test weight normalization), so the v3.0 pickle predictions no longer match a fresh fit. Regenerate the non-CV pickle with actual glum 3.0.0 and drop the CV test case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous pickle was generated with pandas 3.0.1, causing a StringDtype constructor mismatch when loaded in the oldies environment (pandas 1.4.4). Regenerate using matching versions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Belated follow up for tabmat #388. Solves #896.
Also implements Luca Bittarello (@lbittarello)'s idea for backwards compatibility wrt. pickles (no guarantees though, let's do it on a best effort basis for the time being) and addresses #932.
Checklist
CHANGELOG.rstentry