New behavior of imageio.imsave

[aarpon]

Hi,

in an old code base, we used imsave() from imageio version 2.15.0 to save 3-channel, 16-bit arrays to RGB JPEGs. imageio.imsave() would take care of the conversion for us. I am now porting the code to use more recent versions of several libraries, and among those we now use imageio version 2.31.1. Our code now fails with TypeError: Cannot handle this data type: (1, 1, 3), <u2 when passing the 3-channel, 16-bit array to imsave().

I tentatively added a simple conversion to 8-bit before saving:

rgb = (rgb >> 8).astype(np.uint8)

and the result is very similar to what imageio 2.15.0 would create, but not exactly the same.

Would it be possible to know (a pointer to the code would be enough, I guess) how the 2.15.0 version of imsave() was converting the 3-channel, 16-bit image to 8-bit RGB?

I tested with both Pillow version 8.4 and 10.0, and the result is the same.

Thanks!

[FirefoxMetzger]

to save 3-channel, 16-bit arrays to RGB JPEGs

Interesting. Afaik JPEG only supports 8-bit depth, so saving 16-bit JPEG sounds a bit surprising; did you verify that you actually save in 16-bit and don't internally convert to 8-bit before saving?

I get the same exception when I try to save a 16-bit array (uint16) as JPEG:

>>> import imageio.v3 as iio
>>> import numpy as np
>>> img = iio.imread("imageio:chelsea.png")
>>> img = img.astype(np.uint16)
>>> iio.imwrite("foo.jpeg", img)
Traceback (most recent call last):
  File "/Users/sebastian.wallkotter@schibsted.com/projects/imageio/.venv/lib/python3.10/site-packages/PIL/Image.py", line 3070, in fromarray
    mode, rawmode = _fromarray_typemap[typekey]
KeyError: ((1, 1, 3), '<u2')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/sebastian.wallkotter@schibsted.com/projects/imageio/imageio/v3.py", line 147, in imwrite
    encoded = img_file.write(image, **kwargs)
  File "/Users/sebastian.wallkotter@schibsted.com/projects/imageio/imageio/plugins/pillow.py", line 416, in write
    pil_frame = Image.fromarray(frame, mode=mode)
  File "/Users/sebastian.wallkotter@schibsted.com/projects/imageio/.venv/lib/python3.10/site-packages/PIL/Image.py", line 3073, in fromarray
    raise TypeError(msg) from e
TypeError: Cannot handle this data type: (1, 1, 3), <u2

However, I think this behavior is a desirable default because the alternative (a silent downcast) loses quality in a hard to detect way. Granted, for JPEG specifically this isn't to big an issue since compression affects quality much more. I think there is value in having consistent defaults across commonly used formats on this behavior, though we could view it as a regression and bring it back for JPEG specifically if you feel strongly about it.

Would it be possible to know (a pointer to the code would be enough, I guess) how the 2.15.0 version of imsave() was converting the 3-channel, 16-bit image to 8-bit RGB?

The implicit conversion was done via im = np.right_shift(im, 8) and the function that does it lives here:

imageio/imageio/core/util.py

Lines 45 to 108 in 3dd6574

	def image_as_uint(im, bitdepth=None):
	"""Convert the given image to uint (default: uint8)
	If the dtype already matches the desired format, it is returned
	as-is. If the image is float, and all values are between 0 and 1,
	the values are multiplied by np.power(2.0, bitdepth). In all other
	situations, the values are scaled such that the minimum value
	becomes 0 and the maximum value becomes np.power(2.0, bitdepth)-1
	(255 for 8-bit and 65535 for 16-bit).
	"""
	if not bitdepth:
	bitdepth = 8
	if not isinstance(im, np.ndarray):
	raise ValueError("Image must be a numpy array")
	if bitdepth == 8:
	out_type = np.uint8
	elif bitdepth == 16:
	out_type = np.uint16
	else:
	raise ValueError("Bitdepth must be either 8 or 16")
	dtype_str1 = str(im.dtype)
	dtype_str2 = out_type.__name__
	if (im.dtype == np.uint8 and bitdepth == 8) or (
	im.dtype == np.uint16 and bitdepth == 16
	):
	# Already the correct format? Return as-is
	return im
	if dtype_str1.startswith("float") and np.nanmin(im) >= 0 and np.nanmax(im) <= 1:
	_precision_warn(dtype_str1, dtype_str2, "Range [0, 1].")
	im = im.astype(np.float64) * (np.power(2.0, bitdepth) - 1) + 0.499999999
	elif im.dtype == np.uint16 and bitdepth == 8:
	_precision_warn(dtype_str1, dtype_str2, "Losing 8 bits of resolution.")
	im = np.right_shift(im, 8)
	elif im.dtype == np.uint32:
	_precision_warn(
	dtype_str1,
	dtype_str2,
	"Losing {} bits of resolution.".format(32 - bitdepth),
	)
	im = np.right_shift(im, 32 - bitdepth)
	elif im.dtype == np.uint64:
	_precision_warn(
	dtype_str1,
	dtype_str2,
	"Losing {} bits of resolution.".format(64 - bitdepth),
	)
	im = np.right_shift(im, 64 - bitdepth)
	else:
	mi = np.nanmin(im)
	ma = np.nanmax(im)
	if not np.isfinite(mi):
	raise ValueError("Minimum image value is not finite")
	if not np.isfinite(ma):
	raise ValueError("Maximum image value is not finite")
	if ma == mi:
	return im.astype(out_type)
	_precision_warn(dtype_str1, dtype_str2, "Range [{}, {}].".format(mi, ma))
	# Now make float copy before we scale
	im = im.astype("float64")
	# Scale the values between 0 and 1 then multiply by the max value
	im = (im - mi) / (ma - mi) * (np.power(2.0, bitdepth) - 1) + 0.499999999
	assert np.nanmin(im) >= 0
	assert np.nanmax(im) < np.power(2.0, bitdepth)
	return im.astype(out_type)

How did you verify that the result "is similar but not exactly the same"?

[aarpon]

Hi,

thanks for your feedback.

This is my code:

with rawpy.imread(in_filename) as raw:
    rgb = raw.postprocess(gamma=(1, 1), no_auto_bright=False, output_bps=16)
    image.imsave(out_filename, rgb)

I checked that in both my environments (with different versions of the various dependencies), the rgb array I try to save has the same dtype = np.uint16. In imageio 2.15, imsave() would save an 8-bit RGB JPEG without error or warnings, while in 2.31.1 I get TypeError: Cannot handle this data type: (1, 1, 3), <u2.

My code is now:

with rawpy.imread(in_filename) as raw:
    rgb = raw.postprocess(gamma=(1, 1), no_auto_bright=False, output_bps=16)
    rgb = (rgb >> 8).astype(np.uint8)
    image.imsave(out_filename, rgb)

I checked whether the output RGB images in 2.15 vs. 2.31.1 would be the same by simply (and crudely) comparing the mean pixel intensities in a series of small patches of the images generated by imageio 2.15 vs. version 2.31 with my additional bit-shift operation. It didn't need to be highly scientific: I just needed to see whether it was a simple normalisation or something a bit more sophisticated. I will look into image_as_uint.

Thanks!

[FirefoxMetzger]

Odd that your tests fail. When I write a JPEG with either version I get identical results:

>>> import imageio
>>> import numpy as np
>>> imageio.__version__
'2.15.0'
>>> img = 4*imageio.imread("imageio:chelsea.png").astype(np.uint16)
>>> imageio.imwrite("imageio-2-15-0.jpeg", img)
Lossy conversion from uint16 to uint8. Losing 8 bits of resolution. Convert image to uint8 prior to saving to suppress this warning.

>>> import imageio
>>> import numpy as np
>>> import imageio.v3 as iio
>>> imageio.__version__
'2.31.1'
>>> img = 4*iio.imread("imageio:chelsea.png").astype(np.uint16)
>>> iio.imwrite("imageio-2-31-1.jpeg", (img >> 8).astype(np.uint8))
>>> img1 = iio.imread("imageio-2-31-1.jpeg")
>>> img2 = iio.imread("imageio-2-15-0.jpeg")
>>> np.all(img1==img2)
True

[aarpon]

Maybe rawpy changed what raw.postprocess()returns, somehow. I am out of town now and can't check what versions I was and am now using (what I can say is that the version of the underlying Python interpreter was 3.6 and now is 3.10).

However, I also noticed in your code that in version 2.15.0, imageio.imwrite outputs Lossy conversion from uint16 to uint8. Losing 8 bits of resolution. Convert image to uint8 prior to saving to suppress this warning., whereas version 2.31.1 does not. Did you remove that output in your example, or is there really a difference in behaviour?

Anyhow, I'll investigate further when I am back. Thanks!

[FirefoxMetzger]

Did you remove that output in your example, or is there really a difference in behaviour?

No, I didn't remove any warning. The code for 2.31 does the conversion from 16bit to 8bit explicitly (it would raise an exception otherwise). The code for 2.15 relies on the implicit conversion in order to compare it to (img >> 8) which is the conversion you reported leading to different results when used with 2.31.