9.2.0 breaks compatibility to pass PIL.Image to tensorflow (tf.reshape)

[sla-te]

On passing it tf.reshape throws:

Attempt to convert a value (<PIL.Image.Image image mode=RGB size=100x100 at 0x153B2CF8EE0>) with an unsupported type (<class 'PIL.Image.Image'>) to a Tensor.

[radarhere]

Could we get a self-contained example? A block of code that we can run, by itself, without any inputs (or with those inputs attached), to demonstrate the error?

And you're saying that the error doesn't occur with Pillow 9.1.1?

[sla-te]

Could we get a self-contained example? A block of code that we can run, by itself, without any inputs (or with those inputs attached), to demonstrate the error?

Thats a little difficult, as you would need a model, that takes images as input

And you're saying that the error doesn't occur with Pillow 9.1.1?

Correct, just tested again to make sure - 9.2.0 throws and 9.1.1 works

My guess, that some kind of modifications were made to PIL.Image class and now tensorflow does not recognize it anymore and throws the ValueError.

Might want to add, that I am not an expert for tensorflow but I've been running the same code for many months and since 9.2.0 it broke. During testing I found, that if we call PIL.Image.getvalue() (if i recall correctly, that was the methods name, that returns the raw binary of the image) fixes it.

[radarhere]

If you aren't able to post a self-contained example, could we just get a copy of relevant lines of code where you call Pillow and tensorflow?

[radarhere]

Testing with

from PIL import Image
import tensorflow as tf

im = Image.new("L", (1, 1))
tf.reshape(im, (1, 1))

I find that it #6394 is what caused the change. This situation also didn't work for Pillow < 8.3.0, before #5379, so TensorFlow works with __array__, but not __array_interface__.

If you would like an immediate workaround, you can convert the Pillow image to a NumPy array first.

from PIL import Image
import tensorflow as tf
import numpy as np

im = Image.new("L", (1, 1))
arr = np.array(im)
tf.reshape(arr, (1, 1))

[radarhere]

I've created tensorflow/tensorflow#56723 to try and see if TensorFlow is interested in supporting __array_interface__.

[radarhere]

@sla-te to explain a bit, we never intentionally supported the conversion to TensorFlow that you're performing.

We have a mechanism to allow conversion to NumPy. For a while there, we altered it to workaround a NumPy bug, and during that period of time, conversion to TensorFlow started working. The NumPy bug has been fixed, so we switched back, and now conversion to TensorFlow has stopped working.

So while we could add an __array__ method specifically for the sake of TensorFlow, it could also be solved from their side by adding support for __array_interface__. Or, you could use tf.reshape(np.array(im), (1, 1)) as a workaround.

[radarhere]

@sla-te are you happy for this to be closed? You could instead monitor the TensorFlow issue.

[radarhere]

The TensorFlow issue has been resolved, addressed in tensorflow/tensorflow@8b3369b. This should be fixed in a future release of TensorFlow.