LoadImageD and MetaTensor

[myron]

It's more of question to clarify, not a bug. can you help clarify plz

• LoadImage() and LoadImaged() always return MetaTensor. Is it by design? I'm okay either way, just want to clarify. So we can't return the original data format ?

• line 281 return img, img.meta if isinstance(img, MetaTensor) else meta_data. The img seems to be always a MetaTensor, is that check necessary?

MONAI/monai/transforms/io/array.py

Lines 273 to 281 in c27ab36

	meta_data[Key.FILENAME_OR_OBJ] = f"{ensure_tuple(filename)[0]}" # Path obj should be strings for data loader
	img = MetaTensor.ensure_torch_and_prune_meta(
	img_array, meta_data, self.simple_keys, pattern=self.pattern, sep=self.sep
	)
	if self.ensure_channel_first:
	img = EnsureChannelFirst()(img)
	if self.image_only:
	return img
	return img, img.meta if isinstance(img, MetaTensor) else meta_data

• inside of MetaTensor.ensure_torch_and_prune_meta(), we seems to do conversion to Tensor twice on lines 487 and 504. on 487 we img = convert_to_tensor(im) and return MetaTensor(img, meta=meta) will internally call torch.as_tensor on img.

MONAI/monai/data/meta_tensor.py

Lines 487 to 504 in c27ab36

	img = convert_to_tensor(im) # potentially ascontiguousarray
	# if not tracking metadata, return `torch.Tensor`
	if not get_track_meta() or meta is None:
	return img
	# remove any superfluous metadata.
	if simple_keys:
	# ensure affine is of type `torch.Tensor`
	if MetaKeys.AFFINE in meta:
	meta[MetaKeys.AFFINE] = convert_to_tensor(meta[MetaKeys.AFFINE]) # bc-breaking
	remove_extra_metadata(meta) # bc-breaking
	if pattern is not None:
	meta = monai.transforms.DeleteItemsd(keys=pattern, sep=sep, use_re=True)(meta)
	# return the `MetaTensor`
	return MetaTensor(img, meta=meta)

Thank you

[wyli]

it's possible to globally turn off the metatensor...

>>> from monai.data import set_track_meta
>>> set_track_meta(False)
>>> from monai.transforms import LoadImage
>>> x = LoadImage(image_only=True)("IBSR_01_ana.nii.gz")
>>> type(x)
<class 'torch.Tensor'>

but these are not systematically tested for end-to-end workflows, for example, some transform inverting pipelines might not work properly...

[myron]

it's possible to globally turn off the metatensor...

>>> from monai.data import set_track_meta
>>> set_track_meta(False)
>>> from monai.transforms import LoadImage
>>> x = LoadImage(image_only=True)("IBSR_01_ana.nii.gz")
>>> type(x)
<class 'torch.Tensor'>

but these are not systematically tested for end-to-end workflows, for example, some transform inverting pipelines might not work properly...

I see, thank you. what about the last question: lines 487 and 504 seems both convert to tensor internally, or can we remove line 487 img = convert_to_tensor(im) ?

[wyli]

I think it is about returning a fresh copy of the input, @rijobro perhaps can confirm

[rijobro]

Looking at the code, I'm not 100% sure if both are needed. I'm also not 100% which of them would create a deepcopy of the data. Feel free to mess around with it for a variety of inputs (torch.Tensor, MetaTensor and np.ndarray) and see how many copies are created (when debugging, you could do print(img is im) etc.). Also check for contiguous vs non-contiguous input, as the comment mentions this: img = convert_to_tensor(im) # potentially ascontiguousarray.

If they can be combined into one, I'm happy for us to do that.

[wyli]

thanks, now the repeated convert_to_tensor is changed to a simple meta assignment:

MONAI/monai/data/meta_tensor.py

Lines 545 to 552 in b592164

	# return the `MetaTensor`
	if meta is None:
	meta = {}
	img.meta = meta
	if MetaKeys.AFFINE in meta:
	img.affine = meta[MetaKeys.AFFINE] # this uses the affine property setter
	else:
	img.affine = MetaTensor.get_default_affine()