As confirmed in this link, DejaDup wants a target size at least twice as great as the backup source.
My computer is looking for >10x the size of the backup source. I’m trying to backup to a cloud drive that has 100GB available. My source data is 13GB (just /home/<me>) so currently the target has more than 5x the amount of space needed. Yet, DejaDup error message said the backup target needs to be >140GB. I admit I’m trying to encrypt the backup, but that shouldn’t introduce that much overhead.
How is this being calculated? How do I get this working? Eventually I’ll add an external drive and TimeShift, but I’d still like to have this working.
Hello! We grab the expected size from duplicity itself (the underlying backup tool). So for some reason it thinks we need 70GB. We do a dry-run backup first, and duplicity scans all the files it will back up and then gives us a summary. We do the check for space, then complain if it’s not enough.
One thought is to run deja-dup with full debugging on, which will be A LOT. But it will print the same messages that deja-dup sees, so if you can look at the INFO 2 messages it prints, those are the “progress reports” as it scans files.
My INFO 2 messages just contained a bunch of numbers. Do they represent the summation of the file sizes?
It does appear that a backup completed this morning, but I’m still getting the error message and subsequent backups fail. I’m totally unclear why it succeeded this morning.
Hey @mterry, any further help with this? I’m still getting the error about the location being too small and I’m not sure how to interpret the numbers with INFO 2 messages.
If that number does reflect the cumulative sizes of the files being scanned, I believe it is way off.
For example:
DUPLICITY: INFO 5 'home/tim/.bash_history'
DUPLICITY: . M home/tim/.bash_history
DUPLICITY: INFO 2 9068
and
-rw------- 1 tim tim 396 Oct 21 09:36 .bash_history
For the second file:
DUPLICITY: INFO 4 'home/tim/.config/Code/Cache/Cache_Data/6932e96d6fd70c24_0'
DUPLICITY: . A home/tim/.config/Code/Cache/Cache_Data/6932e96d6fd70c24_0
DUPLICITY: INFO 2 15253740
and
-rw------- 1 tim tim 18K Oct 21 19:04 /home/tim/.config/Code/Cache/Cache_Data/6932e96d6fd70c24_0
Is that how I should be interpreting INFO 2 messages?
Heyo! Yes you are right it’s the cumulative file size as it goes.
It looks like it pulls that size from stat()'s st_size field. So it’s surprising that it’d be wrong, but there’s a million ways for software to mess up.
Is there anything “interesting” about your home dir? Like… is it encrypted or doing anything unusual? (I don’t think encryption should affect this… I’m just clutching at straws)
No encryption. Although Pop!_OS has whole disk encryption on my default, I turned it off.
Maybe I’ll try creating a dummy account and seeing if that works. I thought it might be something in the cache, but even the calculation of the .bash_history is way off.
I created a new user account, set it up with the Pop!_OS defaults, and did not add any software other than DejaDup.
Here are the log entries for the first 3 files:
DUPLICITY: INFO 4 'home'
DUPLICITY: INFO 4 'home/test'
DUPLICITY: INFO 4 'home/test/.bash_logout'
DUPLICITY: INFO 2 8892
DUPLICITY: INFO 4 'home/test/.bashrc'
DUPLICITY: INFO 2 12663
DUPLICITY: INFO 4 'home/test/.config'
DUPLICITY: INFO 4 'home/test/.config/.gsd-keyboard.settings-ported'
DUPLICITY: INFO 2 16759
Using ls here are the file entries:
-rw-r--r-- 1 test test 220 Oct 28 10:21 .bash_logout
-rw-r--r-- 1 test test 3.7K Oct 28 10:21 .bashrc
-rw-rw-r-- 1 test test 0 Oct 28 10:22 .config/.gsd-keyboard.settings-ported
While all three are off (as I understand what I’m reading), I found the .gsd-keyboard.settings-ported most interesting as it is an empty file.
What other steps can I take to help troubleshoot this issue?
The jump you see in some of your logs (like for .gsd-keyboard.settings-ported) is because of intervening folders which each count as 4096 bytes – which can be debated whether that is how we should count them, but that’s what stat() returns at least.
One difference I found when I was playing with this is that duplicity takes each hardlink separately but depending on how you’re counting local files, it may or may not separately count those bytes. I was using du so passing du -l made sure it was correctly not trying to skip hardlinked files.
I appreciate you investigating this. I’ll work to translate your command to use flatpak which is how Pop!_OS installs apps.
Tbh, I don’t know what is considered a lot of hard links. For the test user I created, I installed zero software except DejaDup. Here are the contents of the /home/test/.config folder:
test@bench:~$ ls -lah .config
total 64K
drwxr-xr-x 13 test test 4.0K Oct 28 10:38 .
drwxr-x--- 16 test test 4.0K Oct 28 10:37 ..
-rw-rw-r-- 1 test test 0 Oct 28 10:22 .gsd-keyboard.settings-ported
drwx------ 2 test test 4.0K Oct 28 10:25 dconf
drwx------ 3 test test 4.0K Oct 28 10:22 evolution
-rw-rw-r-- 1 test test 5 Oct 28 10:25 gnome-initial-setup-done
drwx------ 3 test test 4.0K Oct 28 10:38 gnome-session
drwxr-xr-x 2 test test 4.0K Oct 28 10:24 goa-1.0
drwx------ 2 test test 4.0K Oct 28 10:22 gtk-3.0
drwx------ 3 test test 4.0K Oct 28 10:22 ibus
drwxr-xr-x 2 test test 4.0K Oct 28 10:22 nautilus
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 pop-shell
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 pop-system-updater
drwx------ 2 test test 4.0K Oct 28 10:22 pulse
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 touchegg
-rw------- 1 test test 633 Oct 28 10:22 user-dirs.dirs
-rw-rw-r-- 1 test test 1 Oct 28 10:22 user-dirs.locale
Inside /home/test/.cache:
test@bench:~$ ls -lah .cache
total 80K
drwx------ 18 test test 4.0K Oct 28 10:30 .
drwxr-x--- 16 test test 4.0K Oct 28 10:37 ..
drwxr-xr-x 3 test test 4.0K Oct 28 10:24 appstream
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 com.system76.FirmwareManager
drwx------ 8 test test 4.0K Oct 28 10:22 evolution
drwxr-xr-x 3 test test 4.0K Oct 28 10:22 flatpak
drwxr-xr-x 2 test test 4.0K Oct 28 10:30 fontconfig
drwxrwxr-x 3 test test 4.0K Oct 28 10:22 fwupd-client
drwxrwxr-x 4 test test 4.0K Oct 28 10:23 gnome-initial-setup
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 gstreamer-1.0
drwxrwxr-x 3 test test 4.0K Oct 28 10:22 ibus
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 ibus-table
drwxrwxr-x 4 test test 4.0K Oct 28 10:23 io.elementary.appcenter
drwxr-xr-x 132 test test 4.0K Oct 28 10:38 mesa_shader_cache
drwx------ 3 test test 4.0K Oct 28 10:30 mozilla
-rw-rw-r-- 1 test test 2 Oct 28 10:22 pop-app-folders
-rw-rw-r-- 1 test test 2 Oct 28 10:22 pop-flatpak-repos
drwxrwxr-x 2 test test 4.0K Oct 28 10:25 pop-launcher
drwxrwxr-x 2 test test 4.0K Oct 28 10:22 pop-system-updater
drwx------ 3 test test 4.0K Oct 28 10:22 tracker3
You should be able to duplicate my test with a system-installed duplicity if you install it on the command line with sudo apt install duplicity
It might be different than the flatpak version! Which would be interesting. Here’s how to do my test using the duplicity shipped inside the Deja Dup flatpak:
Some difference, but both are showing over 99G as the source size which doesn’t make a lot of sense to me.
I then ran the duplicity test under the “virgin” test account I created:
INFO 2 6433935973
6433935973 /home/test
which is still about 2.5x what du -sh . reports.
I guess I’m not really sure what to make of all of this. I suppose I can poke around/ask the duplicity maintainers if they know of any issues with flatpak-based systems, but I also welcome other suggestions from you.
