Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRITICAL FEATURE REQUEST - Visual indication that folders are being excluded even when duplicati shows them as included #4751

Closed
oucil opened this issue Jun 16, 2022 · 5 comments · Fixed by #4989

Comments

@oucil
Copy link

oucil commented Jun 16, 2022

Since #4487 was closed and I can't comment on it, which deals with the fact that Nextcloud is considered a "system" folder, but unchecking the "Exclude -> System" box doesn't actually fix it, with the expected result of having those files included... Instead, there is a note in the last comment indicating that you have to add a custom "Exclude Filter Group -> System" to get this to work.

There is NO visual indicator until you actually go to restore a file that happens to be in the Nextcloud file structure, and then only that the folder tree isn't listed, despite it being listed when you go back to check your configuration and the list of included folders.

I just lost weeks worth of work because of this due to a simple accidentally overwritten file, and on top of that have found that 2 YEARS worth of incremental backups were not taking place on my mission critical work files.... it's dumb luck that I didn't have a complete failure and need those files! My business would have folded.

The "not my problem" response to that scenario in the other bug was even more concerning. It didn't have to be a Nextcloud specific fix... the issue is that Duplicati shows that the file tree is included in one place, but then simply doesn't include it, and doesn't produce even a warning that files you've selected are not actually included. This would happen with any folder with those attributes.... silently.

When selecting folders in the Source list, it should be exceedingly clear that when I select a folder that is considered a System folder, and I've unchecked the exclude System filter, that those files are STILL NOT INCLUDED.

PLEASE ADDRESS THIS, you can't imagine how many people out there are "backing up" Nextcloud without realizing that they're not actually backing anything up.

Kevin.

@Xavron
Copy link

Xavron commented Jun 17, 2022

If its showing up in the Duplicati UI then it should be backed up or removed from showing for sure. That shouldn't be up to the user to deal with two minds.

But, I'm not sure what's going on here entirely. I would have to think that its failing to gain required access during the file crawl and moves along skipping it.

Also side note that I will not discuss, and you might not like it, but you shouldn't let 2 years or even 1 go by without checking and/or starting fresh your backups. There's no backup application that you shouldn't continually check. You should also use at least two different applications. Your business is at stake and you put all your eggs in one basket.

@drwtsn32x
Copy link
Contributor

The UI green check/red X doesn't show more complex inclusions/exclusions especially regex ones. This is a deficiency for sure, but might be difficult to solve. Some filters must be evaluated at time of backup.

@oucil
Copy link
Author

oucil commented Jun 18, 2022

@Xavron Thanks for seeing the issue for what it is and recognizing it shouldn't be up to the end user. The other thread I referenced hints at the root cause and the solution (requiring a custom exception) is in the last post which helped me get around the issue for now.

As for your advice regarding backups, you shouldn't make assumptions and then state you wont discuss them, I have three copies of all files, the live versions, the nextcloud versions and the duplicati versions, and I also take a yearly offline backup to an 8TB drive of all critical stuff. I also use the Duplicati restore on a regular basis (at least every month or two) so I knew my backups were running and retaining as they should.... the one and only issue is that the software shows it's backing something up, that it actually isn't. I shouldn't have to restore every file I'm backing up on a regular basis just to check that it's backed up properly. Please don't try to blame this on a users lack of understanding of how backups should be run and tested. Enough said.

@Xavron
Copy link

Xavron commented Jun 18, 2022

"As for your advice regarding backups, you shouldn't make assumptions and then state you wont discuss them,"

Not an assumption. "it's dumb luck that I didn't have a complete failure and need those files! My business would have folded." You would then only have one backup. Its not an issue if you have two or three other backups (each using a different application) unless you somehow managed to destroy or lose all of them at once. (edit: in which case it wouldn't matter anyway lol) So no, I will not discuss it further :-)

Also, this issue is really the same as to work with hidden files, etc kind of issue. It shouldn't be treated any differently. Either it should in whole or it shouldn't in whole. It is the way I develop stuff, but I do disagree at times with devs, so if they choose to see it the same way or not that's up to them. As in no problem. I see it the same way as you.

@devs. At least I would go by seeing if the files could be backed up. If just not possible, then put in a check during the listing of the files & folders in the Duplicati file browser to eliminate them from being listed or something like that. If neither could be done then there's a red text warning situation with the file browser so the user knows its funky. Temporary measures could be done too. But, you might see some other way =)

Also have to deal with the catch all User folder or whatever. If its not backing up stuff under it then it definitely needs user warning about that.

I do assume here that there's no message after a backup runs. I didn't look if that was something happening there. Duplicati loves having messages after so I'd be surprised if there wasn't already something happening here for the user in knowing. And if there is a message then there's already a nice check in place. Though needing a listing fix at the very least.

@oucil
Copy link
Author

oucil commented Jun 18, 2022

@Xavron I agree it's a bit like hidden files, even down to the dedicated checkbox to not exclude System files, but then it silently excludes this specific type of system file still (which is what the other thread still called them), that's where it becomes a little more serious. The solution is to manually add "Exclude Filter Group" -> "System Files".

@drwtsn32x A warning when selecting a folder tree that includes files/folders that will be excluded without manual intervention would be great, but from a programming perspective might be more work than is necessary. If it didn't happen at the folder tree selection stage but instead at Submit / "Next" with a warning message indicating that there are resources that will be excluded... or even a quick confirmation step which lists the selected files/folders that are going to be excluded and a prompt "I agree, these should be excluded" with an alternate option with an instructions on how to include them, or an simple option to say, "Please include these files anyway" that adds the Exclude Filter Group rule in the background.

This would make the whole process more friendly and transparent, and more capable of dealing with potential edge situations without adding a lot of work.

Jojo-1000 added a commit to Jojo-1000/duplicati that referenced this issue Jun 24, 2023
Add more metadata to the file list returned by server, so that the source file picker can display filters based on file attributes.
Correctly display 'X' in file list if a file is excluded by attributes and not included via an include filter (duplicati#4751).
Fix directory separator issue when loading the page and prevent opening folders in source file list.
Jojo-1000 added a commit to Jojo-1000/duplicati that referenced this issue Jul 9, 2023
Add more metadata to the file list returned by server, so that the source file picker can display filters based on file attributes.
Correctly display 'X' in file list if a file is excluded by attributes and not included via an include filter (duplicati#4751).
Fix directory separator issue when loading the page and prevent opening folders in source file list.
Jojo-1000 added a commit to Jojo-1000/duplicati that referenced this issue Jul 9, 2023
Add more metadata to the file list returned by server, so that the source file picker can display filters based on file attributes.
Correctly display 'X' in file list if a file is excluded by attributes and not included as a source (duplicati#4751).
Fix directory separator issue when loading the page and prevent opening folders in source file list.
Jojo-1000 added a commit to Jojo-1000/duplicati that referenced this issue Jan 16, 2024
Add more metadata to the file list returned by server, so that the source file picker can display filters based on file attributes.
Correctly display 'X' in file list if a file is excluded by attributes and not included as a source (duplicati#4751).
Fix directory separator issue when loading the page and prevent opening folders in source file list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants