io.imread() not opening renamed NDTiffStacks

[l-sala]

Description:

Hello,

I am acquiring NDTiffStack.tiff files (XYt) with a scientific camera and MicroManager (MM) and using scikit-image to analyze the files. My script works with all the files except those that have been manually renamed.

For example, I acquire a file MM and the file is saved on the SSD as "CellLine1_1Hz_Field_1.NDTiffStack.tif" and placed within a folder with the same prefix "CellLine1_1Hz_Field_1".
If I open this file with io.imread(CellLine1_1Hz_Field_1.NDTiffStack.tif), it works.

If I manually rename both the file and its directory, i.e. to "CellLine1_1Hz_Field_2.NDTiffStack.tif", the command io.imread(CellLine1_1Hz_Field_2.NDTiffStack.tif) provides the following error:

[Errno 2] No such file or directory: CellLine1_1Hz_Field_2/CellLine1_1Hz_Field_1.NDTiffStack.tif

If I add in the same folder a file with the original name, both files are correctly opened.
If I move the renamed file into another directory that contains a non-renamed file, both files are correctly opened.

I tried to reboot the machine, restart the kernel, update the Python version, change computer, change the loading package (io.imread or tifffile) but no luck.

Way to reproduce:

No response

Version information:

Python version: 3.11.8
Skimage version: 0.22.0
Numpy version: 1.26.4
Platform: Windows + Anaconda

[lagru]

Thanks for the report!

If I manually rename both the file and its directory, i.e. to "CellLine1_1Hz_Field_2.NDTiffStack.tif", the command io.imread(CellLine1_1Hz_Field_2.NDTiffStack.tif) provides the following error:

[Errno 2] No such file or directory: CellLine1_1Hz_Field_2/CellLine1_1Hz_Field_1.NDTiffStack.tif

This seems a little confusing to me. I'm not sure if this is an oversight while writing the report or if this points to the actual cause of the issue. But these paths don't line up. You rename to "CellLine1_1Hz_Field_2.NDTiffStack.tif" including the directory but the error message indicates that you used "CellLine1_1Hz_Field_1.NDTiffStack.tif". Can you check if that's the case and maybe try to describe the issue in clearer more explicit terms? E.g.

# path before renaming
directory_1/file_name_1.tif
# path after renaming
directory_2/file_name_2.tif

It would be even better if you could provide a snippet to reproduce this. 🙏

Note that we are planning on deprecating the IO plugin infrastructure altogether (#7353). You could also try to use imageio directly.

[l-sala]

Thank you, Lars, for the quick reply. Apologies for the confusion.

That point is the actual cause of the issue. I follow up below with some specifications.

# 1. path before renaming
## correctly read by io.imread()
file_name_1_directory/file_name_1.NDTiffStack.tif

# 2. path after renaming
## not read by io.imread() 
file_name_2_directory/file_name_2.NDTiffStack.tif

### Here the error that is outputed will be:
[Errno 2] No such file or directory: 'file_name_2_directory/file_name_1.NDTiffStack.tif' (yes, it's looking for the original file name into the renamed directory)

# 3. solution that works
## if I duplicate the file at point no. 2 and I rename one of the two as the file in point no. 1, both are correctly read.
file_name_2_directory/file_name_1.NDTiffStack.tif
file_name_2_directory/file_name_2.NDTiffStack.tif

To me, it seems that io.imread() somehow have access to the pre-renamed filename in the renamed folder, and this of course causes the error as the file in the new folder has indeed another name, since it has been indeed renamed. I have no clue on how/where io.imread() gets the information on the old file name. I already emptied Win10 cache.

The snippet is very minimal in my case, as the error is recreated already with:

from skimage import io
import os
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

main_dir = input("Enter the path to the main directory: ") #this would be, e.g., file_name_1_directory or file_name_2_directory

tif_files = []
for root, dirs, files in os.walk(main_dir):
    for file in files:
        if file.endswith('.tif'):
            tif_path = os.path.join(root, file)
            tif_files.append(tif_path)

for tif_path in tqdm(tif_files, desc="Processing .tif files"):
    try:
        # Check if the path has enough directories
        tif_path_parts = tif_path.split(os.sep)
        # Extract the second and third from the last directory names
        if len(tif_path_parts) >= 4:
            dir2 = tif_path_parts[-3]  # Second from the last
            last = tif_path_parts[-1]  # This is the file
            # Combine them as a new file name
            file_name = f"{dir2}_{last}"

            stack = io.imread(tif_path)

The files to be analyzed are 2-3 Gb in size. Let me know if you think one sample is required.

[cgohlke]

The metadata for NDTiff stacks, including the file names where image data are stored, are found in a separate NDTiff.index file. Renaming a NDTiff file invalidates the metadata. Try to remove the NDTiff.index file or tell tifffile not to use NDTiff metadata: imread(..., is_ndtiff=False).

[l-sala]

The metadata for NDTiff stacks, including the file names where image data are stored, are found in a separate NDTiff.index file. Renaming a NDTiff file invalidates the metadata. Try to remove the NDTiff.index file or tell tifffile not to use NDTiff metadata: imread(..., is_ndtiff=False).

Thanks a lot; the imread(..., is_ndtiff=False) solved the issue completely.