Fail to summarize across plate

[GaiaGerbaka]

I'm trying to summarize the CellProfiler outputs across a whole plate (for multiple well and sites), I tried using s3 and local paths but it seems like the convert function is unable to grab the Cells.csv, Nuclei.csv and Cytoplasm.csv from each Well-Site folder in order to collate them. Here's a google colab version of my code, the code is able to compile but not complete a successful run, it looks like it's timing out because it can't find the csv. I tried multiple versions of the path in case that was the issue: "s3://cellpainting-gallery/cpg0043-segmentation/broad/workspace/analysis/2025_08_21_Batch1/BR00116991/analysis/" and "s3://cellpainting-gallery/cpg0043-segmentation/broad/workspace/analysis/2025_08_21_Batch1/BR00116991/. From my understanding this is a feature of cytotable so let me know if I'm doing anything incorrectly.

[d33bs]

Thanks @GaiaGerbaka for raising this issue! We built CytoTable with capability in mind but haven't yet added a test to ensure the functionality works as expected. I'll dig in here to find out what might be going on and follow up.

[d33bs]

I found that the challenge here likely relates to how many files are stored under analysis and how cloudpathlib (a dependency which handles cloud-based paths like pathlib) handles globbing (see here for more drivendataorg/cloudpathlib#513). I'll focus on implementing a workaround for S3 to increase the speed of object search based on how CytoTable operates.

[d33bs]

Hi @GaiaGerbaka - we've just updated CytoTable to resolve a bug in performance which this issue helped to understand. When you have a chance, could you retry with the latest version of CytoTable? Please don't hesitate to let me know if you still run into any issues.