Is this MPO file decoded correctly?

[bigcat88]

What did you do?

from PIL import Image

im = Image.open("328384445-d4972d5b-3409-4d5f-a107-ccb8c5dc0177.jpg")  # MPO file
im.show()

What did you expect to happen?

Correctly decode second image

What actually happened?

What are your OS, Python and Pillow versions?

• OS: macOS/Linux
• Python: 3.10/3.11
• Pillow: 10.2.0 (edited) (was 10.3.0, my mistake)

Test image can be found here: Comfy-Org/ComfyUI#3416 (comment)

[bigcat88]

Sorry my bad, I tested it on Pillow 10.2

on Pillow 10.3 there is no error when decoding, just second image looks wrong.

Updated issue to reflect this. Sorry for mess.

[liusida]

Hi, @bigcat88 , 10.3.0 correctly extract the second frame with a different resolution. However, the second frame is not actually a frame, it's a gain map according to this UltraHDR format.
A JPG file should only have one frame. And people naturally expect the second frame would in the same resolution as the first one. That's the problem we have in Comfy-Org/ComfyUI#3416 (comment)

So, supporting UltraHDR is a good way to go, I guess. #8036

[bigcat88]

10.3.0 correctly extract the second frame with a different resolution

No it does not, imho.

Pillow should either reject a gain map and show only primary image, or it should decode second frame correctly, even with lower resolution.

[liusida]

btw, the tomato photo I provided eariler is not very representitive, since the gain map is all 1.0s (completely white). I didn't understand this was related to the UltraHDR format earlier.

Here is another photo I took that has something in the gain map:

we can see the gain map is smaller comparing to the original photo.

[bigcat88]

btw, the tomato photo I provided eariler is not very representitive, since the gain map is all 1.0s (completely white).

please look at my first message.
It is not white.

[shawnington]

perhaps gain maps need their own type returned from .get_bands() or some way to explicitly exclude them from ImageSequence.Iterator

[radarhere]

Are there any images with a small file size that could be added to our test suite and distributed under the Pillow license?

[liusida]

Are there any images with a small file size that could be added to our test suite and distributed under the Pillow license?

I, as the author of the tomato photo and the trees photo, agree to add these to the Pillow test suite and to adhere to its licensing terms.

I don't have smaller ones, since this is already the 'medium resolution'; the other option is 'full resolution.'

[radarhere]

Pillow should either reject a gain map and show only primary image, or it should decode second frame correctly, even with lower resolution.

I've created #8056 to treat Ultra HDR images as standard JPEGs, ignoring the gain map. I would consider decoding the second frame correctly to be a matter for #8036

[radarhere]

Thanks very much for the test image. I've included it in the PR.

[shawnington]

Pillow should either reject a gain map and show only primary image, or it should decode second frame correctly, even with lower resolution.

I've created #8056 to treat Ultra HDR images as standard JPEGs, ignoring the gain map. I would consider decoding the second frame correctly to be a matter for #8036

Im not familiar with the specs for MPO or Ultra HDR, but I am assuming the smaller gain map is embedded in the exif data of the main photo and is being extracted as a seperate file when opening images? Is there any way to reattach the gain map to the exif of the main image so the data is still there later but it is only presented as a single image? I imagine there are future uses where preserving the data is preferred behavior, but presenting them as seperate image is not.