Numpy byte order conversion error with Dataset and dictionaries

[chrisfoulon]

Hi MONAI team and thanks for your awesome work!

I was previously using the NiftiDataset class and the 'array' version of the transformations and everything was running fine.
For the need of my project, I had to switch to dictionaries (and the corresponding transformations) and thus instead of the NiftiDataset I used the regular Dataset class and added LoadImaged to the transformations.
However, I now face the following error when I try to pull a batch from the loader:

Traceback (most recent call last):
File "/home/tolhsadum/anaconda3/bin/lesseg_unet", line 8, in
sys.exit(main())
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/lesseg_unet/main.py", line 71, in main
dataloader_workers=args.num_workers)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/lesseg_unet/training.py", line 60, in training_loop
train_ds, val_ds = init_training_data(img_path_list, seg_path_list, img_pref)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/lesseg_unet/training.py", line 41, in init_training_data
data_loading.data_loader_checker_first(train_ds, 'training')
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/lesseg_unet/data_loading.py", line 67, in data_loader_checker_first
img_batch, seg_batch = monai.utils.misc.first(check_loader)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/monai/utils/misc.py", line 46, in first
for i in iterable:
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in next
data = self._next_data()
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
return self._process_data(data)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
data.reraise()
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
data = fetcher.fetch(index)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
return self.collate_fn(data)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/monai/data/utils.py", line 217, in list_data_collate
return default_collate(data)
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 73, in default_collate
return {key: default_collate([d[key] for d in batch]) for key in elem}
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 73, in
return {key: default_collate([d[key] for d in batch]) for key in elem}
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 73, in default_collate
return {key: default_collate([d[key] for d in batch]) for key in elem}
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 73, in
return {key: default_collate([d[key] for d in batch]) for key in elem}
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 63, in default_collate
return default_collate([torch.as_tensor(b) for b in batch])
File "/home/tolhsadum/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 63, in
return default_collate([torch.as_tensor(b) for b in batch])
ValueError: given numpy array has byte order different from the native byte order. Conversion between byte orders is currently not supported.

So, I tried many things for ages, I thought it was because of transformations so I stripped the pipeline from pretty much every transformation but I still have this error.

Eventually, I started questioning my data so I tried a minimal:
I took the example from https://github.com/Project-MONAI/tutorials/blob/0.4.0/3d_segmentation/torch/unet_training_dict.py
and just replaced the crop by resize

data = [{'image': 'path/to/image, 'label': 'path/to/label'}]
train_transforms = Compose(
             [
                 LoadImaged(keys=keys),
                 AsChannelFirstd(keys=keys, channel_dim=-1),
                 ScaleIntensityd(keys="image"),
                 Resized(keys=keys, spatial_size=[96,96,96]),
                 RandRotate90d(keys=keys, prob=0.5, spatial_axes=[0, 2]),
                 ToTensord(keys=keys),
             ]
     )
check_ds = Dataset(data=data, transform=Compose(train_transforms))
check_loader = DataLoader(check_ds, batch_size=1, pin_memory=torch.cuda.is_available())
check_data = monai.utils.misc.first(check_loader)

And with that, it does not work on my initial data but it DOES work on another dataset I have.
So, yes there's probably an issue with my data but this did not seem to bother the NiftiDataset classes when I was using arrays instead of dict.

So, is it possible to use something like the NiftiDataset but with dictionaries? (I need the dictionaries because I need to use external transformations and I need to apply them to both the image and label in the exact same way)
Or maybe there is an option I don't know about that would fix this problem?

Environment

MONAI version: 0.4.0
Numpy version: 1.19.4
Pytorch version: 1.7.0
MONAI flags: HAS_EXT = False, USE_COMPILED = False
MONAI rev id: 0563a44

Optional dependencies:
Pytorch Ignite version: 0.4.2
Nibabel version: 3.2.1
scikit-image version: 0.17.2
Pillow version: 8.0.1
Tensorboard version: 2.4.0
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.8.1
ITK version: 5.1.2
tqdm version: 4.54.0
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.7.3

Thank you in advance!
Chris.

[rijobro]

Hi, I've been looking into this this morning, and have put together the following test code:

from monai.data import Dataset, DataLoader
from monai.transforms import LoadImaged, LoadImage
from monai.data import create_test_image_2d
import nibabel as nib
import numpy as np
import os

im, _ = create_test_image_2d(100, 100)

for endianness in ["<", ">"]:

    hdr = nib.Nifti1Header(endianness=endianness)
    nii = nib.Nifti1Image(im, np.eye(4), header=hdr)
    fname = os.getcwd() + "im.nii"
    nib.save(nii, fname)

    data_arr = [fname]
    data_dct = [{"image": fname}]

    tr_arr = LoadImage(image_only=True)
    tr_dct = LoadImaged("image")

    for name, data, transform in zip(["arr", "dct"], [data_arr, data_dct], [tr_arr, tr_dct]):
        print(f"checking {name}, endianness: {endianness}")
        check_ds = Dataset(data, transform)
        check_loader = DataLoader(check_ds, batch_size=1)
        try:
            check_data = next(iter(check_loader))
            print("OK!")
        except ValueError:
            print("not OK")

Which gives the output:

checking arr, endianness: <
OK!
checking dct, endianness: <
OK!
checking arr, endianness: >
OK!
checking dct, endianness: >
not OK

I'm guessing that you see the error for big endian nifti images, but not little endian ones.

I found it strange that the array version was fine with big endian data, whereas the dictionary version wasn't.

After a bit of debugging, I found the error to be in the loaded metadata. If the dictionary version is used to load only the image without the metadata (required changing source code, but I'll add this as PR as it's useful), then the problem went away.

I'll look into this a bit more as to what exactly in the metadata has the wrong byte order.

[chrisfoulon]

Hi! Thank you for your answer!

Yes it seems to be linked with endianness. I also have the problem with images showing '=' when loaded with nibabel and using dtype.byteorder. A colleague found a quick fix for that (but we need to modify the data before running monai for that so it can only be a temporary fix): We just load the nifti with nibabel, create a new Nifti1Image with the data and affine of the original and save it to replace the original.
Maybe it can help finding a fix for it