Update concat for multi-variable indexes.#10371
Merged
dcherian merged 11 commits intopydata:mainfrom Jun 5, 2025
Merged
Conversation
2 tasks
When concatenating along a dimension associated with a multi-variable index, avoid merging the other dimensions. Instead we rely on a strict equality check along those dimensions (7551a7a)
dcherian
force-pushed
the
concat-multi-var-index
branch
from
988503e to
a996655
Compare
benbovy
reviewed
May 30, 2025
benbovy
reviewed
May 30, 2025
Co-authored-by: Benoit Bovy <benbovy@gmail.com>
* main: Fix performance regression in interp from pydata#9881 (pydata#10370) html repr: improve style for dropdown sections (pydata#10354) Grouper tweaks. (pydata#10362) Docs: Add links to getting help mermaid diagram (pydata#10324) Enforce ruff/flynt rules (FLY) (pydata#10375) Add missing AbstractWritableDataStore base methods and arguments (pydata#10343) Improve html repr in dark mode (Jupyterlab + Xarray docs) (pydata#10353) Pin Mypy to 1.15 (pydata#10378) use numpy dtype exposed by zarr array instead of metadata.data_type (pydata#10348) Fix doc typo for caption "Interoperability" (pydata#10374) Implement cftime vectorization as discussed in PR pydata#8322 (pydata#8324) Enforce ruff/flake8-pyi rules (PYI) (pydata#10359) Apply assorted ruff/Pylint rules (PL) / Enforce PLE rules (pydata#10366) (fix): pandas extension array repr for int64[pyarrow] (pydata#10317) Enforce ruff/flake8-implicit-str-concat rules (ISC) (pydata#10368) Enforce ruff/refurb rules (FURB) (pydata#10367) Ignore ruff/Pyflakes rule F401 more precisely (pydata#10369) Apply assorted ruff/flake8-simplify rules (SIM) (pydata#10364) Apply assorted ruff/flake8-pytest-style rules (PT) (pydata#10363) Fix "a array" misspelling (pydata#10365)
dcherian
commented
May 30, 2025
| @@ -0,0 +1,79 @@ | |||
| from collections.abc import Hashable, Iterable, Sequence | |||
Contributor
Author
There was a problem hiding this comment.
refactored these out from test_dataset.py so they can be reused
dcherian
commented
May 30, 2025
| y=PandasIndex.from_variables({"y": variables["y"]}, options=options), | ||
| ) | ||
|
|
||
| def create_variables(self, variables): |
dcherian
commented
May 30, 2025
| return x_eq and y_eq | ||
|
|
||
| @classmethod | ||
| def concat( |
benbovy
reviewed
Jun 3, 2025
| # skip merging these variables. | ||
| # if concatenating over a dimension 'x' that is associated with an index over 2 variables, | ||
| # 'x' and 'y', then we assert join="equals" on `y` and don't need to merge it. | ||
| # that assertion happens in the align step prior to this function being called |
Member
There was a problem hiding this comment.
I was wondering how this would behave for the corner case where the "y" coordinate (and/or the "x" coordinate) does not have any index in one or more of the objects to concatenate. We could leave it for now, though. It is quite unlikely that it will occur in practice I'd say.
Contributor
Author
There was a problem hiding this comment.
Not sure what you mean, we defer to the Index, so it can do what's sensible?
benbovy
reviewed
Jun 3, 2025
benbovy
approved these changes
Jun 3, 2025
Co-authored-by: Benoit Bovy <benbovy@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When concatenating along a dimension associated with a multi-variable index, avoid merging the other dimensions. Instead we rely on a strict equality check along those dimensions (7551a7a)
Still needs tests, but seems to work on https://github.com/dcherian/rasterix/pull/18