`sandbox` diverged from `biocOSN` ?

[Artur-man]

We have two sources now. CC @LucaMarconato

SpatialData.data/R/merfish_demo.R

Lines 9 to 15 in 74c519b

	build_osn_xdemo_path <- function(zipname) {
	sprintf("https://mghp.osn.xsede.org/bir190004-bucket01/BiocXenDemo/%s", zipname)
	}
	build_sandbox_path <- function(zipname) {
	sprintf("https://s3.embl.de/spatialdata/spatialdata-sandbox/%s", zipname)
	}

which seem to have different versions of the same datasets since sandbox versions are throwing an error when an old version of spatialdata-io is being used.

SpatialData.data/tests/testthat/test-caching.R

Lines 59 to 64 in 74c519b

	dem <- SpatialData.data:::.unzip_spd_demo(
	zipname="mibitof.zip",
	cache=BiocFileCache::BiocFileCache(),
	destination=tf,
	source="sandbox")
	grab <- readSpatialData(dem)

> dem <- SpatialData.data:::.unzip_spd_demo(
+     zipname="mibitof.zip",
+     cache=BiocFileCache::BiocFileCache(),
+     destination=tf,
+     source="sandbox")
retrieving from sandbox, caching, and returning path

> grab <- readSpatialData(dem)
Error in py_call_impl(callable, call_args$unnamed, call_args$named) : 
  KeyError: '0.2'
Run `reticulate::py_last_error()` for details.

However BiocOSN works fine.

> dem <- SpatialData.data:::.unzip_spd_demo(
...
+     source="biocOSN")
retrieving from biocOSN, caching, and returning path
> readSpatialData(dem)
class: SpatialData
- images(3):
  - point16_image (3,1024,1024)
  - point23_image (3,1024,1024)
  - point8_image (3,1024,1024)
- labels(3):
  - point16_labels (1024,1024)
  - point23_labels (1024,1024)
  - point8_labels (1024,1024)
...

[LucaMarconato]

Two comments here:

• the latest datasets available from the docs https://spatialdata.scverse.org/en/stable/tutorials/notebooks/datasets/README.html will be updated at the next release. They have been produced with an old spatilaldata version that IIRC doesn't use the latest zarr v3 format
• KeyError: '0.2' suggests that the reader is not able to parse one of the latest formats. 0.2 can mean many things, all the meanings are listed here: https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_io/format.py

[Artur-man]

Here is the full error:

> reticulate::py_last_error()

── Python Exception Message ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Traceback (most recent call last):
  File "/Users/amanuky/Library/Caches/org.R-project.R/R/basilisk/1.22.0/SpatialData/0.99.23/anndata_env/lib/python3.12/site-packages/spatialdata/_io/io_zarr.py", line 78, in read_zarr
    element = _read_multiscale(f_elem_store, raster_type="image")
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amanuky/Library/Caches/org.R-project.R/R/basilisk/1.22.0/SpatialData/0.99.23/anndata_env/lib/python3.12/site-packages/spatialdata/_io/io_raster.py", line 47, in _read_multiscale
    format = RasterFormatV01() if version is None else RasterFormats[version]
                                                       ~~~~~~~~~~~~~^^^^^^^^^
KeyError: '0.2'

── R Traceback ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
    ▆
 1. └─SpatialData::readSpatialData(dem)
 2.   └─SpatialData:::.readTables_basilisk(x) at SpatialData/R/read.R:212:5
 3.     └─basilisk::basiliskRun(...) at SpatialData/R/read.R:133:5
 4.       └─SpatialData (local) fun(...)
 5.         └─sd$read_zarr(x) at SpatialData/R/read.R:136:9
 6.           └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
See `reticulate::py_last_error()$r_trace$full_call` for more details.

[LucaMarconato]

Looks like the spatialdata (Python package) version being used it too old.

[Artur-man]

We can set up a new basilisk env with new spatialdata/anndata versions ... but then these should be able to read the old data @vjcitn stashed in his s3 bucket.