Skip to content

Conversation

@d33bs
Copy link
Member

@d33bs d33bs commented Oct 10, 2025

Description

This PR adds a new function cloud_glob for improved large data access on cloud providers. We add AWS S3-focused capability here as a way of addressing the needs mentioned in #377 . Other cloud providers fall back to existing capabilities within cloudpathlib. I've also added a quick local test of cpg0043-segmentation to ensure nested data are concatenated together properly when using nested data capabilities with CytoTable.

The use case involves referencing data from Cell Painting Gallery dataset cpg0043-segmentation which includes many nested objects (which causes cloudpathlib by default to pause for a long time while loading data).

References #377

What is the nature of your change?

  • Bug fix (fixes an issue).
  • Enhancement (adds functionality).
  • Breaking change (fix or feature that would cause existing functionality to not work as expected).
  • This change requires a documentation update.

Checklist

Please ensure that all boxes are checked before indicating that a pull request is ready for review.

  • I have read the CONTRIBUTING.md guidelines.
  • My code follows the style guidelines of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • New and existing unit tests pass locally with my changes.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have deleted all non-relevant text in this pull request template.

@d33bs d33bs requested a review from gwaybio October 10, 2025 22:08
@d33bs d33bs marked this pull request as ready for review October 10, 2025 22:08
@d33bs
Copy link
Member Author

d33bs commented Oct 16, 2025

Thanks @gwaybio !

@d33bs d33bs merged commit b973a13 into cytomining:main Oct 16, 2025
12 checks passed
@d33bs d33bs deleted the gaia-troubleshooting branch October 16, 2025 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants