Skip to content

Conversation

@rajeeja
Copy link
Contributor

@rajeeja rajeeja commented Jun 24, 2025

UxArray crashes with an AttributeError: 'NoneType' object has no attribute '_ds' when attempting to display the HTML representation of a UxDataset that has lost its grid reference during certain operations, particularly groupby operations.

added null checking in _obj_repr_with_grid() and overloaded groupby and resample methods to preserve uxgrid after operations.

test cases added to test these chages.

cftime, I thought would be handle by xarray, but was causing issues:
We now detecti cftime objects, extract year/month information directly from them and perform manual binning without requiring conversion to pandas datetime

@rajeeja rajeeja requested a review from philipc2 June 24, 2025 17:43
@rajeeja rajeeja self-assigned this Jun 24, 2025
@rajeeja
Copy link
Contributor Author

rajeeja commented Jun 24, 2025

d=ds0.groupby("time.month").mean('time')
d.uxgrid

== NO GRID OBJECT

this works:

c=ds0['TSA'].groupby("time.month").mean('time')
c.uxgrid

==

uxarray.Grid

Dimensions:
n_node: 48602, n_face.....

@philipc2
Copy link
Member

The issue in #1220 is due to the Grid being lost after certain Xarray operations. The changes here only impact the printing, which is not handling the underlying issue.

@rajeeja
Copy link
Contributor Author

rajeeja commented Jun 25, 2025

The issue in #1220 is due to the Grid being lost after certain Xarray operations. The changes here only impact the printing, which is not handling the underlying issue.

Right, I put what you mentioned in the PR description. The checks here should still be fine as a safety mechanism.

@philipc2
Copy link
Member

The issue in #1220 is due to the Grid being lost after certain Xarray operations. The changes here only impact the printing, which is not handling the underlying issue.

Right, I put what you mentioned in the PR description. The checks here should still be fine as a safety mechanism.

Oops I must have missed that!

@rajeeja rajeeja requested a review from philipc2 June 30, 2025 19:44
@github-project-automation github-project-automation bot moved this to 👀 In review in UXarray Development Jul 1, 2025
@philipc2 philipc2 requested a review from erogluorhan July 1, 2025 18:18
@philipc2 philipc2 linked an issue Jul 1, 2025 that may be closed by this pull request
Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I'm going to take a deeper look on Monday when I'm back from PTO this week.

Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just ran the example provided in #1220, and the code executes, however the final UxDataset is no longer linked to a Grid, with .uxgrid being None

@philipc2
Copy link
Member

philipc2 commented Jul 7, 2025

Actually, looks like the groupby works as expected, it's only the resample now.

Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May you update the title of this PR to reflect the changes? It helps when creating releases since they are automatically generated from the PR titles.

rajeeja added 3 commits July 8, 2025 15:35
…ling, The implementation works with objects used in climate data, Time dimensions are correctly reduced by resampling operations
@rajeeja rajeeja requested a review from philipc2 July 10, 2025 16:06
@rajeeja
Copy link
Contributor Author

rajeeja commented Aug 20, 2025

While this does seem to work, I'm skeptical if this is the best solution in terms of maintainability moving forward. Pinging @erogluorhan so he can take a look.

Good observation and I have been thinking about an alternate approach that might be better - How about wrapping the xarray accessor objects (GroupBy/Bin, Resample, Rolling, Coarsen,Weighted(found these are in need also:)) instead of using proxies? When methods like groupby(), resample(), or rolling() are called, we return lightweight wrappers that delegate to xarray but preserve uxgrid in results.

rajeeja and others added 2 commits August 19, 2025 23:51
…entation with accessor classes that wrap xarray operations. Removed inline proxy classes and the _wrap_callable_attr helper in favor of a centralized accessor module.

Preserve Key Attribute: This ensures the uxgrid attribute is preserved through all groupby, resample, rolling, coarsen, weighted, and cumulative operations.

Improve Code Quality: The new approach provides better maintainability, cleaner separation of concerns, and easier extensibility for future accessor methods.
@rajeeja rajeeja requested a review from philipc2 August 20, 2025 15:50
@rajeeja
Copy link
Contributor Author

rajeeja commented Aug 20, 2025

While this does seem to work, I'm skeptical if this is the best solution in terms of maintainability moving forward. Pinging @erogluorhan so he can take a look.

Good observation and I have been thinking about an alternate approach that might be better - How about wrapping the xarray accessor objects (GroupBy/Bin, Resample, Rolling, Coarsen,Weighted(found these are in need also:)) instead of using proxies? When methods like groupby(), resample(), or rolling() are called, we return lightweight wrappers that delegate to xarray but preserve uxgrid in results.

Please review this now, I made changes throughout and this implementation look way better.

Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! A few suggestions:

Rajeev Jain added 4 commits August 20, 2025 17:52
- Created new test/test_accessors.py for all groupby/resample/rolling/etc tests
- Removed accessor tests from test_dataset.py to keep it focused on core Dataset functionality
- Fixed import sorting in accessors.py (pre-commit)
@erogluorhan
Copy link
Member

While this does seem to work, I'm skeptical if this is the best solution in terms of maintainability moving forward. Pinging @erogluorhan so he can take a look.

Sorry for the delayed response! Seems like @rajeeja made some changes since this ping. Should I still have a look, @philipc2 ?

@philipc2
Copy link
Member

While this does seem to work, I'm skeptical if this is the best solution in terms of maintainability moving forward. Pinging @erogluorhan so he can take a look.

Sorry for the delayed response! Seems like @rajeeja made some changes since this ping. Should I still have a look, @philipc2 ?

Yes please! I think two reviews are needed for the scope of this PR.

Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed.

Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments.

Co-authored-by: Philip Chmielowiec <67855069+philipc2@users.noreply.github.com>
@rajeeja rajeeja force-pushed the rajeeja/issue_1220 branch 2 times, most recently from 8f0eb38 to 806a539 Compare August 21, 2025 17:18
@erogluorhan
Copy link
Member

While this does seem to work, I'm skeptical if this is the best solution in terms of maintainability moving forward. Pinging @erogluorhan so he can take a look.

Sorry for the delayed response! Seems like @rajeeja made some changes since this ping. Should I still have a look, @philipc2 ?

Yes please! I think two reviews are needed for the scope of this PR.

Sure! Give me some time, maybe until tomorrow, I will give this a review before the weekend.

Copy link
Member

@erogluorhan erogluorhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please find my thoughts about the Grid preservation and cftime handling below:

  1. I might really be lost in the code, but shouldn't we just override the xarray functions, wherever there is a loss of the Grid object, and simply add back the uxgrid to the data array/set object to be returned? I think that might be a much simpler and adhoc solution, and that way I don't think we should be concerned about _repr_.

I'd imagine this'd look something like (for UxDataset):

def resample(self, ...):
  xrds = self.resample(...)
  uxds = uxarray.core.dataset.UxDataset(xrds, uxgrid=self.uxgrid)
  
  return uxds
  1. Not sure about the cftime handling since even xarray does not work on the dataset pointed out in the #1220 . Didn't look into those aprts of this PR yet either. Nevertheless, if we can get it to work with uxarray easily, maybe it is worth handling here.

@rajeeja
Copy link
Contributor Author

rajeeja commented Aug 22, 2025

Please find my thoughts about the Grid preservation and cftime handling below:

  1. I might really be lost in the code, but shouldn't we just override the xarray functions, wherever there is a loss of the Grid object, and simply add back the uxgrid to the data array/set object to be returned? I think that might be a much simpler and adhoc solution, and that way I don't think we should be concerned about _repr_.

I'd imagine this'd look something like (for UxDataset):

def resample(self, ...):
  xrds = self.resample(...)
  uxds = uxarray.core.dataset.UxDataset(xrds, uxgrid=self.uxgrid)
  
  return uxds

The accessor pattern is necessary because:
Without accessors, this breaks:

ds.resample('D').mean() # resample returns xr.Resample, mean() loses grid

With accessors:
   ds.resample('D')  # returns UxDatasetResampleAccessor
      .mean()         # accessor wraps mean(), preserves grid

Direct override only handles the first call, not the chained operations. The accessor wraps the entire
operation chain.

  1. cftime handling:

You're right that xarray itself has issues with cftime in certain cases. The current implementation passes
cftime objects through to xarray's resample, which means we inherit both its capabilities and limitations.
We've added tests to ensure grid preservation works with cftime when xarray supports it, but we don't
attempt to fix xarray's underlying cftime issues. If xarray improves cftime support in the future, uxarray
will automatically benefit.

@erogluorhan
Copy link
Member

Direct override only handles the first call, not the chained operations. The accessor wraps the entire operation chain.

Never mind, I made an assumption, too early, as if xarray.Dataset/DataArray.resample() returns a new xarray.Dataset/DataArray object, but it does not.

Now understanding this, I will give your code a second review.

  1. cftime handling:

You're right that xarray itself has issues with cftime in certain cases. The current implementation passes cftime objects through to xarray's resample, which means we inherit both its capabilities and limitations. We've added tests to ensure grid preservation works with cftime when xarray supports it, but we don't attempt to fix xarray's underlying cftime issues. If xarray improves cftime support in the future, uxarray will automatically benefit.

Got it; thanks for clarifying!

Copy link
Member

@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work!

@philipc2 philipc2 merged commit c9a4ead into main Sep 5, 2025
13 of 15 checks passed
@erogluorhan erogluorhan deleted the rajeeja/issue_1220 branch September 26, 2025 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xarray features not working on datasets

4 participants