Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix to #3465. Prevent, resaving of duplicate images if overwrite not specified #3472

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

shawnington
Copy link
Contributor

@shawnington shawnington commented May 13, 2024

This is a fix to #3465 for LoadImage node. Behavior also generalizes to preventing duplication of existing files.

Adds function compare_image_hash to do a sha256 hash comparison between an uploaded image and existing images with matching file names.

This changes the behavior so that only images having the same filename that are actually different images are saved to input with increment, existing images are instead now opened instead.

Currently, exact duplicates with the same filename are resave saved with an incremented filename in the format:

filename (i).ext

with the code:

while os.path.exists(filepath): 
    filename = f"{split[0]} ({i}){split[1]}"
    filepath = os.path.join(full_output_folder, filename)
    i += 1

This commit changes this to include a sh256 hash comparison:

while os.path.exists(filepath): 
    if compare_image_hash(filepath, image):
        image_is_duplicate = True
        break
    filename = f"{split[0]} ({i}){split[1]}"
    filepath = os.path.join(full_output_folder, filename)
    i += 1

a check for if image_is_duplicate = False is done before saving the file.

Currently, if you load the same image of a cat named cat.jpg into the LoadImage node 3 times, you will get 3 new files in your input folder with incremented file names.

With this change, you will now only have the single copy of cat.jpg, that will be re-opened instead of re-saved with increment.

However if you load 3 different images of cats all named cat.jpg, you will get the expected behavior of having:
cat.jpg
cat (1).jpg
cat (2).jpg

This saves space and clutter. After checking my own input folder, I have 800+ images that are duplicates that were resaved with incremented file names amounting to more than 5GB of duplicated data.

… if overwrite not specified

This is a fix to comfyanonymous#3465 

Adds function compare_image_hash to do a sha256 hash comparison between an uploaded image and existing images with matching file names. 

This changes the behavior so that only images having the same filename that are actually different are saved to input, existing images are instead now opened instead of resaved with increment. 

Currently, exact duplicates with the same filename are resave saved with an incremented filename in the format:

<filename> (n).ext 

with the code: 

```
while os.path.exists(filepath): 
                        filename = f"{split[0]} ({i}){split[1]}"
                        filepath = os.path.join(full_output_folder, filename)
                        i += 1
```

This commit changes this to: 

```
while os.path.exists(filepath): 
                        if compare_image_hash(filepath, image):
                            image_is_duplicate = True
                            break
                        filename = f"{split[0]} ({i}){split[1]}"
                        filepath = os.path.join(full_output_folder, filename)
                        i += 1
```

a check for if image_is_duplicate = False is done before saving the file. 

Currently, if you load the same image of a cat named cat.jpg into the LoadImage node 3 times, you will get 3 new files in your input folder with incremented file names.

With this change, you will now only have the single copy of cat.jpg, that will be re-opened instead of re-saved. 

However if you load 3 different images of cats named cat.jpg, you will get the expected behavior of having:
cat.jpg
cat (1).jpg
cat (2).jpg

This saves space and clutter. After checking my own input folder, I have 800+ images that are duplicates that were resaved with incremented file names amounting to more than 5GB of duplicated data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant