Remove breaking check about `auto_mkdir` for FSSpecStore by maxrjones · Pull Request #3193 · zarr-developers/zarr-python

maxrjones · 2025-07-02T12:51:52Z

This change is causing downstream breakages since auto_mkdir is only relevant for stores with read_only = False and stores that are in the root of the provided directory.

E.g., https://github.com/developmentseed/titiler/actions/runs/16025154840/job/45211163911#step:8:2093

This fixes a regression introduced in #2774. I apologize for the mistake in that pull request.

TODO:

Add unit tests and/or doctests in docstrings
Add docstrings and API docs for any new/modified user-facing classes and functions
New/modified features documented in docs/user-guide/*.rst
Changes documented as a new file in changes/
GitHub Actions have all passed
Test coverage is 100% (Codecov passes)

…ith auto_mkdir=False

d-v-b

I pushed some changes in the interest of speed. thanks for this quick fix max!

d-v-b · 2025-07-03T17:00:46Z

@meeseeksdev backport to 3.0.10

lumberbot-app · 2025-07-03T17:00:53Z

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

Checkout backport branch and update it.

git checkout 3.0.10
git pull

Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 5a24487f09d499c91ce25f24af910c2ede055b5a

You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #3193: Remove breaking check about `auto_mkdir` for FSSpecStore'

Push to a named branch:

git push YOURFORK 3.0.10:auto-backport-of-pr-3193-on-3.0.10

Create a PR against branch 3.0.10, I would have named this PR:

"Backport PR #3193 on branch 3.0.10 (Remove breaking check about auto_mkdir for FSSpecStore)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

…pers#3193) * Remove breaking check from _make_async * Update expected error * Change import structure to protect against AttributeError * changelog * add test to ensure that we can create a read-only copy of the store with auto_mkdir=False * only test if the async wrapper is available --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> (cherry picked from commit 5a24487)

* Remove breaking check from _make_async * Update expected error * Change import structure to protect against AttributeError * changelog * add test to ensure that we can create a read-only copy of the store with auto_mkdir=False * only test if the async wrapper is available --------- (cherry picked from commit 5a24487) Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>

* Remove breaking check about `auto_mkdir` for FSSpecStore (#3193) * Remove breaking check from _make_async * Update expected error * Change import structure to protect against AttributeError * changelog * add test to ensure that we can create a read-only copy of the store with auto_mkdir=False * only test if the async wrapper is available --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> (cherry picked from commit 5a24487) * Add missing import for AsyncFileSystemWrapper for `_make_async` in `_fsspec.py` (#3195) * Add missing import for AsyncFileSystemWrapper in `_fsspec.py` * Add missing changelog entry for AsyncFileSystemWrapper import fix * Move AsyncFileSystemWrapper import past the version check in `_fsspec.py` * Add newline after AsyncFileSystemWrapper import in `_fsspec.py` * Simplify import statement for AsyncFileSystemWrapper in `_fsspec.py` --------- Co-authored-by: Altay Sansal <altay.sansal@tgs.com> (cherry picked from commit 97aa42f) --------- Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> Co-authored-by: Altay Sansal <tasansal@users.noreply.github.com> Co-authored-by: Altay Sansal <altay.sansal@tgs.com>

* Enable tests on 3.0.x branch (#3135) * Changelog for 3.0.9 * Backport PR #3149: Add GroupNotFound error to API docs (#3179) Co-authored-by: David Stansby <dstansby@gmail.com> * Backport PR #3140: Suppress FileNotFoundError when deleting keys in the obstore adapter (#3180) Co-authored-by: Lukas Bindreiter <lukas.bindreiter@tilebox.com> * Backport PR #3138: Add with_read_only() convenience method to store (#3181) Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> * Create read only copy if needed when opening a store path (#3156) * Create read only copy if needed when opening a store path * Add ValueError to Raises section * Update expected warning * Update src/zarr/storage/_common.py Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> * Use ANY_ACCESS_MODE * Update src/zarr/storage/_common.py Co-authored-by: David Stansby <dstansby@gmail.com> * Update src/zarr/storage/_common.py Co-authored-by: David Stansby <dstansby@gmail.com> * Update changes * Try using get_args on definition * Revert "Try using get_args on definition" This reverts commit 7ad760f. * Add test * Remove warning * Apply suggestion for try; except shortening Co-authored-by: Tom Nicholas <tom@earthmover.io> * Improve code coverage --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> Co-authored-by: David Stansby <dstansby@gmail.com> Co-authored-by: Tom Nicholas <tom@earthmover.io> (cherry picked from commit 5731c6c) * Create read only copy if needed when opening a store path (#3156) (#3182) * Create read only copy if needed when opening a store path * Add ValueError to Raises section * Update expected warning * Update src/zarr/storage/_common.py * Use ANY_ACCESS_MODE * Update src/zarr/storage/_common.py * Update src/zarr/storage/_common.py * Update changes * Try using get_args on definition * Revert "Try using get_args on definition" This reverts commit 7ad760f. * Add test * Remove warning * Apply suggestion for try; except shortening * Improve code coverage --------- (cherry picked from commit 5731c6c) Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> Co-authored-by: David Stansby <dstansby@gmail.com> Co-authored-by: Tom Nicholas <tom@earthmover.io> * Remove breaking check about `auto_mkdir` for FSSpecStore (#3193) * Remove breaking check from _make_async * Update expected error * Change import structure to protect against AttributeError * changelog * add test to ensure that we can create a read-only copy of the store with auto_mkdir=False * only test if the async wrapper is available --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> (cherry picked from commit 5a24487) * Remove breaking check about `auto_mkdir` for FSSpecStore (#3193) (#3203) * Remove breaking check from _make_async * Update expected error * Change import structure to protect against AttributeError * changelog * add test to ensure that we can create a read-only copy of the store with auto_mkdir=False * only test if the async wrapper is available --------- (cherry picked from commit 5a24487) Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> * Add missing import for AsyncFileSystemWrapper for `_make_async` in `_fsspec.py` (#3195) * Add missing import for AsyncFileSystemWrapper in `_fsspec.py` * Add missing changelog entry for AsyncFileSystemWrapper import fix * Move AsyncFileSystemWrapper import past the version check in `_fsspec.py` * Add newline after AsyncFileSystemWrapper import in `_fsspec.py` * Simplify import statement for AsyncFileSystemWrapper in `_fsspec.py` --------- Co-authored-by: Altay Sansal <altay.sansal@tgs.com> (cherry picked from commit 97aa42f) * Auto backport of pr 3195 on 3.0.10 (#3204) * Remove breaking check about `auto_mkdir` for FSSpecStore (#3193) * Remove breaking check from _make_async * Update expected error * Change import structure to protect against AttributeError * changelog * add test to ensure that we can create a read-only copy of the store with auto_mkdir=False * only test if the async wrapper is available --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> (cherry picked from commit 5a24487) * Add missing import for AsyncFileSystemWrapper for `_make_async` in `_fsspec.py` (#3195) * Add missing import for AsyncFileSystemWrapper in `_fsspec.py` * Add missing changelog entry for AsyncFileSystemWrapper import fix * Move AsyncFileSystemWrapper import past the version check in `_fsspec.py` * Add newline after AsyncFileSystemWrapper import in `_fsspec.py` * Simplify import statement for AsyncFileSystemWrapper in `_fsspec.py` --------- Co-authored-by: Altay Sansal <altay.sansal@tgs.com> (cherry picked from commit 97aa42f) --------- Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> Co-authored-by: Altay Sansal <tasansal@users.noreply.github.com> Co-authored-by: Altay Sansal <altay.sansal@tgs.com> * 3.0.9 release notes (#3183) * Create read only copy if needed when opening a store path (#3156) * Create read only copy if needed when opening a store path * Add ValueError to Raises section * Update expected warning * Update src/zarr/storage/_common.py Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> * Use ANY_ACCESS_MODE * Update src/zarr/storage/_common.py Co-authored-by: David Stansby <dstansby@gmail.com> * Update src/zarr/storage/_common.py Co-authored-by: David Stansby <dstansby@gmail.com> * Update changes * Try using get_args on definition * Revert "Try using get_args on definition" This reverts commit 7ad760f. * Add test * Remove warning * Apply suggestion for try; except shortening Co-authored-by: Tom Nicholas <tom@earthmover.io> * Improve code coverage --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> Co-authored-by: David Stansby <dstansby@gmail.com> Co-authored-by: Tom Nicholas <tom@earthmover.io> (cherry picked from commit 5731c6c) * release notes --------- Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> Co-authored-by: David Stansby <dstansby@gmail.com> Co-authored-by: Tom Nicholas <tom@earthmover.io> * release notes --------- Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com> Co-authored-by: Lukas Bindreiter <lukas.bindreiter@tilebox.com> Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> Co-authored-by: Tom Nicholas <tom@earthmover.io> Co-authored-by: Altay Sansal <tasansal@users.noreply.github.com> Co-authored-by: Altay Sansal <altay.sansal@tgs.com>

Remove breaking check from _make_async

49c3955

github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jul 2, 2025

maxrjones changed the title ~~Remove breaking check from _make_async~~ Remove breaking check about auto_mkdir for FSSpecStore Jul 2, 2025

maxrjones added 2 commits July 2, 2025 09:03

Update expected error

1853231

Change import structure to protect against AttributeError

1772c89

maxrjones mentioned this pull request Jul 2, 2025

Add missing import for AsyncFileSystemWrapper for _make_async in _fsspec.py #3195

Merged

6 tasks

d-v-b added this to the 3.0.10 milestone Jul 3, 2025

d-v-b added 2 commits July 3, 2025 11:08

Merge branch 'main' into remove-check

3a9e307

changelog

cec89ff

github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Jul 3, 2025

d-v-b added 2 commits July 3, 2025 11:42

add test to ensure that we can create a read-only copy of the store w…

13238ff

…ith auto_mkdir=False

only test if the async wrapper is available

46e4148

d-v-b approved these changes Jul 3, 2025

View reviewed changes

d-v-b merged commit 5a24487 into zarr-developers:main Jul 3, 2025
30 checks passed

lumberbot-app bot added the Still Needs Manual Backport label Jul 3, 2025

d-v-b removed the Still Needs Manual Backport label Jul 3, 2025

maxrjones mentioned this pull request Jul 6, 2025

Use a release checklist and narrative announcement for breaking changes #3209

Closed

maxrjones mentioned this pull request Jan 30, 2026

ODD PI 25.4 Objective 4: 🛠️ Zarr Development NASA-IMPACT/veda-odd#204

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove breaking check about `auto_mkdir` for FSSpecStore#3193

Remove breaking check about `auto_mkdir` for FSSpecStore#3193
d-v-b merged 7 commits intozarr-developers:mainfrom
maxrjones:remove-check

maxrjones commented Jul 2, 2025 •

edited

Loading

Uh oh!

d-v-b left a comment

Uh oh!

Uh oh!

d-v-b commented Jul 3, 2025

Uh oh!

lumberbot-app bot commented Jul 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

maxrjones commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

d-v-b left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

d-v-b commented Jul 3, 2025

Uh oh!

lumberbot-app bot commented Jul 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maxrjones commented Jul 2, 2025 •

edited

Loading