Integrity check fails on files with basins

[B-Hartmann]

DCKit 0.17.2
dclab 0.62.6
h5py 3.11

I have a processed RTDC file containing additional texture features and basins. I wanted to run integrity checks on the files and got the following error message:

Unhandled exception in DCKit version 0.17.2:
Traceback (most recent call last):
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dckit\main.py", line 337, in on_integrity_check
    dlg = IntegrityCheckDialog(self, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dckit\dlg_icheck.py", line 46, in __init__
    self.populate_ui()
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dckit\dlg_icheck.py", line 85, in populate_ui
    cues = self.check(use_metadata=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dckit\dlg_icheck.py", line 197, in check
    cues = check_dataset(self.path, metadata_dump, expand_section)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dckit\dlg_icheck.py", line 315, in check_dataset
    cues = ic.check(expand_section=expand_section)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dclab\rtdc_dataset\check.py", line 247, in check
    cues += funcs[ff](self, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\dclab\rtdc_dataset\check.py", line 264, in check_basin_features_internal
    if feat not in self.ds.h5file[bpaths[0]]:
                   ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "h5py\\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "C:\Users\bhartma\.environments\dckit\Lib\site-packages\h5py\_hl\group.py", line 357, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py\\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\\h5o.pyx", line 241, in h5py.h5o.open
KeyError: "Unable to synchronously open object (object 'basin_events' doesn't exist)"

[paulmueller]

I understand where the error comes from, but this should not have happened.

Could you please give me access to this dataset? To me it looks like something there might be broken (no path to an internal basin given which must be defined). I don't need access to any of the basins, just the dataset you are opening in DCKit.

Thanks!

[paulmueller]

OK thanks, I understand now. The pipeline computed the background images image_bg as an internal basin. Subsequent pipeline steps copied over the basin (which should not happen). The resulting basin definition refers to data that is not present in the given file. So there are two issues:

• fix opening such files: Handle situations where an internal basin is defined but the corresponding data are missing dclab#268
• prevent creating such files: When exporting data, do not write the internal basin if the internal basin data is not supposed to be exported dclab#269