OpenJPEG failing tests on big endian

[nijel]

Looks like there are quite some issues within the code when executed on big endian machine, see testsuite failures:

======================================================================
ERROR: TestFileIco.test_save_to_bytes
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_ico.py", line 27, in test_save_to_bytes
    reloaded = Image.open(output)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8992cb90>

======================================================================
ERROR: TestFileJpeg2k.test_16bit_j2k_roundtrips
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 157, in test_16bit_j2k_roundtrips
    im = self.roundtrip(j2k)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_16bit_jp2_roundtrips
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 163, in test_16bit_jp2_roundtrips
    im = self.roundtrip(jp2)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_irreversible_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 83, in test_irreversible_rt
    im = self.roundtrip(test_card, irreversible=True, quality_layers=[20])
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_layers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 107, in test_layers
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_lossless_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 65, in test_lossless_rt
    im = self.roundtrip(test_card)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_lossy_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 69, in test_lossy_rt
    im = self.roundtrip(test_card, quality_layers=[20])
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_prog_qual_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 88, in test_prog_qual_rt
    test_card, quality_layers=[60, 40, 20], progression='LRCP')
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_prog_res_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 92, in test_prog_res_rt
    im = self.roundtrip(test_card, num_resolutions=8, progression='RLCP')
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_tiled_offset_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 79, in test_tiled_offset_rt
    tile_offset=(0, 0), offset=(32, 32))
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8992cb90>

======================================================================
ERROR: TestFileJpeg2k.test_tiled_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 73, in test_tiled_rt
    im = self.roundtrip(test_card, tile_size=(128, 128))
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8992cb90>

======================================================================
FAIL: TestFileJpeg2k.test_16bit_monchrome_j2k_like_tiff
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 152, in test_16bit_monchrome_j2k_like_tiff
    self.assert_image_similar(j2k, tiff_16bit, 1e-3)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/helper.py", line 105, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 250.8379 > epsilon 0.0010

======================================================================
FAIL: TestFileJpeg2k.test_16bit_monchrome_jp2_like_tiff
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 146, in test_16bit_monchrome_jp2_like_tiff
    self.assert_image_similar(jp2, tiff_16bit, 1e-3)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/helper.py", line 105, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 250.8379 > epsilon 0.0010

Full build log can be found here:

https://build.opensuse.org/package/live_build_log/devel:languages:python/python-Pillow/openSUSE_Tumbleweed/ppc64

[wiredfool]

I've been testing on big-endian (ubuntu 12.04 PPC), and Jpeg2k has always had issues. I think there's something with the underlying library on that version. What version is is suse testing against?

[nijel]

It was openjpeg2-devel-2.1.0

[wiredfool]

Just got the PPC fired up here, and I'm seeing similar issues. (32bit, g4 mini on ubuntu 12.04). It's got openjpeg 2.1.0, and most of the jpeg2k bits are failing.

I'm seeing a slightly different error in TestFileIco which at least points to something:

  File "/home/erics/Pillow/Tests/test_file_ico.py", line 27, in test_save_to_bytes
    reloaded = Image.open(output)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/Image.py", line 2266, in open
    im = factory(fp, filename)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/ImageFile.py", line 97, in __init__
    self._open()
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 264, in _open
    self.load()
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 267, in load
    im = self.ico.getimage(self.size)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 150, in getimage
    return self.frame(i)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 160, in frame
    self.buf.seek(header['offset'])
OverflowError: cannot fit 'long' into an index-sized integer

[hugovk]

(Partial) Fix #1232 is merged.

[hugovk]

Is this still an issue after the fix from #1232?

[nijel]

I don't see how ICO fix would relate to OpenJPEG bug...