Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backup Quota Warning Error #4970

Closed
durcom2 opened this issue Jun 19, 2023 · 10 comments · Fixed by #4993
Closed

Backup Quota Warning Error #4970

durcom2 opened this issue Jun 19, 2023 · 10 comments · Fixed by #4993
Labels
pending user feedback needs information from the original poster stale

Comments

@durcom2
Copy link

durcom2 commented Jun 19, 2023

  • [x ] I have searched open and closed issues for duplicates.
  • [x ] I have searched the forum for related topics.

Environment info

  • Duplicati version: 2.0.7.1_beta_2023-05-25
  • Operating system: ubuntu 22.04
  • Backend: local filesystem

Description

Backup throws storage quota warning incorrectly:

2023-06-17 13:00:06 -04 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-BackendQuotaNear]: Backend quota is close to being exceeded: Using 156.72 GB of 14.94 GB (4.11 GB available)

Backup is to local storage with 10TB free...

Looks like backup is looking at root (/) of the filesystem for quota calcs instead of mountpoint (/mnt/backups) where backups are stored.

Steps to reproduce

  1. Run backup
  2. Run Backup
  3. Run Backup
  • Actual result:
    warning
  • Expected result:
    no warning

Screenshots

Debug log

@Jojo-1000
Copy link
Contributor

There is some code in place to manage this:

if (Platform.IsClientPosix)
{
string path = Util.AppendDirSeparator(systemIO.PathGetFullPath(m_path));
root = "/";
//Find longest common prefix from mounted devices
//TODO: Can trick this with symlinks, where the symlink is on one mounted volume,
// and the actual storage is on another
foreach (System.IO.DriveInfo di in System.IO.DriveInfo.GetDrives())
if (path.StartsWith(Util.AppendDirSeparator(di.Name), StringComparison.Ordinal) && di.Name.Length > root.Length)
root = di.Name;
}

Maybe this code is bugged for your situation, or you use symlinks like the todo suggests? I don't have a linux setup to test this.

@ts678
Copy link
Collaborator

ts678 commented Jun 19, 2023

I don't have a linux setup to test this.

Unfortunate. I only have old VMs. Maybe @durcom2 can test whether this old theory about what's a drive can explain the issue:

Backend quota has been exceeded after apparently successful backup

@gpatel-fr gpatel-fr added the pending user feedback needs information from the original poster label Jun 22, 2023
@durcom2
Copy link
Author

durcom2 commented Jun 28, 2023

I have no resources to test code changes. This is a definite issue/bug though that developers need to look at.

@Jojo-1000
Copy link
Contributor

I have no resources to test code changes. This is a definite issue/bug though that developers need to look at.

For anyone to look at this, you should probably give a bit more information about your specific setup, so that others can reproduce the issue:

  • what kind of drives / mounts are you using and are there other mounts in your system that might confuse this function?
  • does this happen with all mounted drives or only specific ones?

Assuming that GetDrives() returns your mountpoint as a drive, it should make a correct free space calculation. So there must be something wrong with that function on your system. In that case, it is not a Duplicati issue but rather a mono issue.

In the linked forum thread above there are more troubleshooting steps that might give more insight. Without more info probably nobody can fix this.

@durcom2
Copy link
Author

durcom2 commented Jun 28, 2023

nothing special about my setup...minimal ubuntu 22 install
only "unusual" thing is the mountpoint where the duplicati files are stored is on a zfs volume
The warning on the disk quota mentioned above lists Exactly the space used and free space available for the root volume (/) (a standard ext4 linux volume) so it is returning the space for the root volume and not the zfs volume. It does not appear that duplicati can handle a zfs mountpoint properly.

@ts678
Copy link
Collaborator

ts678 commented Jun 29, 2023

I have no resources to test code changes.

That wasn't the request. It was to see how the mono GetDrives example program works on your system if you can compile it.

Assuming that GetDrives() returns your mountpoint as a drive

is the question that we can't answer on your system. We probably can't get mono changed, but possibly we can work around it.

@ts678
Copy link
Collaborator

ts678 commented Jul 4, 2023

I have no resources

If you installed the mono-devel package per the manual, you should have gotten mono-mcs with man mcs and /usr/bin/mcs.
DriveInfo.GetDrives Method at learn.microsoft.com has an example with a Copy button. Put it in a file and run it, for example:

mcs -out:GetDrives.exe GetDrives.cs
mono GetDrives.exe

If your drive isn't listed, by mono, there's your problem. Duplicati doesn't look at or know your filesystem type, but mono does.

If you're totally uncomfortable with running Microsoft example code, then could you grep /etc/mtab for ZFS filesystem to post?
Testing with a loop versus tmpfs mount (I don't have ZFS on this VM) make me think a line starting with a device will get in, e.g.

/dev/loop3 /mnt/loop ext2 rw,relatime 0 0

but a line that doesn't start like that takes its chances on the mono lib linked from above link providing special code to get it in. Here is a direct link, and reading it lines up well with test results. Also adding credibility is that .NET 6 doesn't show the problem.
That's no help until Duplicati can migrate to .NET. Sadly, DriveInfo(String) Constructor has the same issue (maybe same cause).

I'm not sure what other ways one can get quota information. Maybe someone more familiar with .NET can think of something...
If there's none, then maybe a workaround would be to provide an option to just turn off quota check until the value is available.

So asking again for information from system.

@duplicatibot
Copy link

This issue has been mentioned on Duplicati. There might be relevant details there:

https://forum.duplicati.com/t/backend-quota-exceeded-i-know-its-been-covered-but/16488/16

@github-actions
Copy link

This issue is stale because it has been open for 15 days with no activity.

@github-actions github-actions bot added the stale label Aug 29, 2023
@github-actions
Copy link

This issue was closed because it has been inactive for 15 days since being marked as stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pending user feedback needs information from the original poster stale
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants