test_8bit_with_16bit_depth fails on s390x (64 bit/big endian machine)

[olebole]

When running the tests on a s390x Debian machine, I get the following failure:

__________________________ test_8bit_with_16bit_depth __________________________

im = <PIL.Image.Image image mode=I;16B size=128x128 at 0x3FF8C149350>
fp = <_io.BytesIO object at 0x3ff8c22fc90>, filename = ''
chunk = <function putchunk at 0x3ff8d1e8040>, save_all = False

    def _save(im, fp, filename, chunk=putchunk, save_all=False):
        […]
        # get the corresponding PNG mode
        try:
>           rawmode, mode = _OUTMODES[mode]
E           KeyError: 'I;16B'

/usr/lib/python3/dist-packages/PIL/PngImagePlugin.py:1286: KeyError

The above exception was the direct cause of the following exception:

    def test_8bit_with_16bit_depth():
        rng = np.random.default_rng()
    
        img16 = rng.integers(2**0, 2**8, (128, 128), dtype=np.uint16)
>       img16_bytes = iio.imwrite("<bytes>", img16, extension=".png", plugin="pillow")

test_pillow.py:634: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/imageio/v3.py:147: in imwrite
    encoded = img_file.write(image, **kwargs)
/usr/lib/python3/dist-packages/imageio/plugins/pillow.py:441: in write
    self._flush_writer()
/usr/lib/python3/dist-packages/imageio/plugins/pillow.py:457: in _flush_writer
    primary_image.save(self._request.get_file(), **self.save_args)
/usr/lib/python3/dist-packages/PIL/Image.py:2432: in save
    save_handler(self, fp, filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

im = <PIL.Image.Image image mode=I;16B size=128x128 at 0x3FF8C149350>
fp = <_io.BytesIO object at 0x3ff8c22fc90>, filename = ''
chunk = <function putchunk at 0x3ff8d1e8040>, save_all = False

    def _save(im, fp, filename, chunk=putchunk, save_all=False):
        […]
        # get the corresponding PNG mode
        try:
            rawmode, mode = _OUTMODES[mode]
        except KeyError as e:
            msg = f"cannot write mode {mode} as PNG"
>           raise OSError(msg) from e
E           OSError: cannot write mode I;16B as PNG

/usr/lib/python3/dist-packages/PIL/PngImagePlugin.py:1289: OSError

This is imageio 2.31.1 with PIL 9.5.0.

[FirefoxMetzger]

I can reproduce the exception on an amd64 architecture using the following snippet:

>>> import imageio.v3 as iio
>>> import numpy as np
>>> img = iio.imread("imageio:chelsea.png", mode="L")
>>> img.dtype
dtype('uint8')
>>> iio.imwrite("foo.png", img.astype("<u2"))
>>> iio.imwrite("foo.png", img.astype(">u2"))
Traceback (most recent call last):
  File "/Users/<user>/projects/imageio/.venv/lib/python3.10/site-packages/PIL/PngImagePlugin.py", line 1277, in _save
    rawmode, mode = _OUTMODES[mode]
KeyError: 'I;16B'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/<user>/projects/imageio/imageio/v3.py", line 139, in imwrite
    with imopen(
  File "/Users/<user>/projects/imageio/imageio/core/v3_plugin_api.py", line 367, in __exit__
    self.close()
  File "/Users/<user>/projects/imageio/imageio/plugins/pillow.py", line 123, in close
    self._flush_writer()
  File "/Users/<user>/projects/imageio/imageio/plugins/pillow.py", line 457, in _flush_writer
    primary_image.save(self._request.get_file(), **self.save_args)
  File "/Users/<user>/projects/imageio/.venv/lib/python3.10/site-packages/PIL/Image.py", line 2413, in save
    save_handler(self, fp, filename)
  File "/Users/<user>/projects/imageio/.venv/lib/python3.10/site-packages/PIL/PngImagePlugin.py", line 1280, in _save
    raise OSError(msg) from e
OSError: cannot write mode I;16B as PNG

This originates from Pillow. Internally their PNG writer maintains a list of valid input buffer dtypes and I;16B (big-endian 16-bit) is not listed there. However, I;16 (little-endian 16-bit) is. If I add I;16B in their list of allowed dtypes everything runs as it should. --- I will create an issue over at pillow and ask if they can add it, since this looks like a bug/oversight to me :)

---

The reason this error shows up on s390x is because it is a big-endian architecture. For PNGs (though this will likely extend in the future) ImageIO will unpack pixels into native endianness. As such you get a big-endian buffer on a big-endian machine, which is more performant but, for now, not allowed as input when writing with pillow.

[FirefoxMetzger]

xref: python-pillow/Pillow#7301