Déjà Dup - restore stuck on "scanning backup files"

Hi,

My computer went bust and I’m trying to restore files from a Gnome Backup backup on a new machine. I’ve installed Manjaro Gnome and am running the Backup restore, however it is forever stuck on “Scanning backup files…” Any advice?

More detail:
The backup was created with Gnome Backup on Manjaro Gnome, saving files on Google Drive, encrypted with a password. The backup was a regular (repeated) differential (I think), and is quite large (172 GB). Google Drive containes a number of files names s/t like duplicity-full.dateTtimeZ.volX.difftar.gpg

On the new computer, I have started Gnome Backup, selected “Restore files” and selected Google Drive and the correct folder. I was not asked for a password to restore files, so far (6 hours) the backup programme is stuck as outlined above. The backup dates have appeared in the bottom right of the Restore window, but nothing else has happened so far.

Gnome Backup version (restore): 42.7
Gnome Backup version (backup): not sure, latest on Manjaro, but won’t be far off current version.

Any advice would be appreciated.

Thanks,
nico

1 Like

Hello! I’m sorry to hear about your computer and that Deja Dup is giving you problems.

So the restore screen does two things in order - it scans for a list of dates, then scans for all files in a given date (by default, the newest one). Sounds like we’re getting stuck for some reason on that second step, since you say the dates appeared in the bottom right.

Let’s go through some options. I would assume your priority is restoring your data, so let’s start with a possible silver bullet.

Direct Restore

If you know the path of the file/directory you want to restore, you can skip the browse interface by using the command line. Run something like:

deja-dup --restore path/to/restore

This file or directory does not need to currently exist. Deja Dup should pop up its restore interface at that point.

Debug Deja Dup

Can you get Deja Dup stuck again, and then paste the output of the Debug Information screen in the About dialog? That contains a log of the last operation we tried to do.

Avoid Google Drive

If we think the issue is how we’re talking to Google Drive, you could try downloading the duplicity files locally and restore from that location. That sounds obnoxious though, since it’s 172GB. Let’s not worry about this yet.

Avoid Deja Dup

If Deja Dup is just not being helpful, you can try the underlying backup tool duplicity directly. It’s command line can be a little confusing, especially if you’re going to be talking to Google Drive. But you can read its man page / documentation and try that. If you think this is the way to go, I can help you navigate how to connect to Google Drive.

1 Like

Hi Michael,

Thanks for your quick and detailed response. I will try out your suggestions over the next few hours and respond as and when (I might not have time to try out everything today, I’ll come back to it when I have the chance).

Direct Restore
I’m not 100% sure of the paths, the once I tried (which should work) did not:
deja-dup --restore /home/rgij/Pictures
should give something, but didn’t. I also tried
deja-dup --restore Pictures
deja-dup --restore /rgij-e540
(the root backup folder on Google Drive)
deja-dup --restore /rgij-e540/Pictures
deja-dup --restore *
deja-dup --restore /home/rgij/Pictures/*

The first four commands popped up a Deja-Dup window, but this said “Restore Failed: No backups to restore”.

One potential issue is that my new computer has a different device name, so that Deja-Dup has assumed a different Google Drive root folder. When I try to restore through the normal process, I can amend this to the correct folder, but using the command line I cannot (or I don’t know how).

Debug Deja Dup
The log is here: https://pastebin.com/VC40TDWM
It seems to ‘understand’ the backup well enough. Is it supposed to ask me for the backup encryption password and not doing so?

Avoid Google Drive
I’ll try duplicity first. :slight_smile:

Avoid Deja Dup
Happy to give that a try, though I’d like to persist with Deja Dup a bit more, if for no reason than to find out where the problem is. I think I have a fairly common ‘restore’ scenario…

Again, thanks for your help, any further support is much appreciated.

1 Like

The first four commands popped up a Deja-Dup window, but this said “Restore Failed: No backups to restore”.

Huh. That’s an odd error in this case, because that means it didn’t find any backup snapshots (any dates). But we know the browse & restore interface did when it scanned, because the dropdown in the bottom right gets filled in and the log you gave shows some.

When I try to restore through the normal process, I can amend this to the correct folder, but using the command line I cannot (or I don’t know how).

Ah right! Because the command line interface was designed for a different use case - of restoring not from a clean sweep but for restoring a file you accidentally deleted or something, while you still have an active system. So it doesn’t prompt for the usual “where is your backup?” questions and instead assumes your current settings are where it is.

(that should probably change, but we can work around it for now)

If you can change your current backup settings to match where your old backups are, the command line might work. (Also, sorry for all this hassle!)

Can you open the preferences dialog and change your settings to point to your Google Drive backup? Then try from the command line again?

The log is here: https://pastebin.com/VC40TDWM

That log makes it look like it’s correctly finishing the first scan (“give me all the backup dates”) but didn’t even start the second scan (“give me all the files in this one backup date”).

It seems to ‘understand’ the backup well enough. Is it supposed to ask me for the backup encryption password and not doing so?

That would be a reasonable interpretation. It knows the backup is encrypted from the dates scan, and would need the password for the files scan.

1 Like

Thanks again for your response. So, I’m half a step further:

If you can change your current backup settings to match where your old backups are, the command line might work.

I found the settings and amended accordingly. The result was a little better but no cigar. The restore process went as far as asking me to choose the date, then thought for a while (like five minutes), then crashed without error. The log from Deja Dup is the wrong date, it seems using the command line doesn’t result in a log, or it’s stored elsewhere.

One question: when I use the command line, how should I specify the backed-up directory? Do I give the full path from the original system, i.e. --restore /home/user/Pictures, or do I assume the root of the backup folder (i.e. --restore Pictures)? Also, is there a way to restore all files through the command line? I honestly am not sure I know all the folders I backed up, as I included some settings folders.

Thanks.

1 Like

crashed without error

Oh geeze!! Crashes are pretty rare and I’d love to fix this. Though if this is a real problem that is still getting in our way, we may want to move into “use duplicity directly, and then later worry about deja-dup’s hangups” territory, just so we can get to the point where you have your data.

But let’s try one more time with deja-dup to possibly get some useful data:

The log from Deja Dup is the wrong date, it seems using the command line doesn’t result in a log, or it’s stored elsewhere.

The crash might stop it from logging the results? You can get on-the-fly logging if you run the command line like so:

DEJA_DUP_DEBUG=1 deja-dup --restore ...

It’ll be a firehose, but that’s probably what we want right now.

when I use the command line, how should I specify the backed-up directory?

Both Pictures and /home/user/Pictures should be fine and equivalent (the former works as long as you’re running the command from /home/user). You’re specifying a path to the location of where the file was originally, either as a relative or absolute path.

Also, is there a way to restore all files through the command line?

Try --restore /home/user, but you’ll probably want to restore to a new location (not to original locations) because you don’t want to overwrite your current home files immediately, but rather move files from the restored set into place.

Duplicity

OK, let’s talk a little bit about what next steps are, if we want to pause debugging deja-dup for a second.

Let’s first set up rclone, which will expose your google drive account as a normal folder on your machine. This will let us interact with it more easily (and honestly, is probably easier than driving duplicity’s drive backend which gets a little technical).

rclone

You can probably install rclone from your distro. Then follow the following setup instructions (setting up a new remote called, let’s say gdrive and just using the default for most of the questions, to use the built-in default developer keys that rclone has):

https://rclone.org/drive/

Back to Duplicity

OK, now with that set up, we can do something like (note the lack of a slash in front of home/user):

duplicity restore --file-to-restore=home/user/Pictures rclone://gdrive:/path/to/backup/files /tmp/Pictures

1 Like

Hi Michael,

I seem to be getting more stuck in more avenues…

Deja-Dup

DEJA_DUP_DEBUG=1 deja-dup --restore ...

I’ve got a 2.7MB log file, but not that much seems to be relevant. The main bits are here: https://pastebin.com/aZgZF7F9

I’ve stuck to --restore Picture

Duplicity
So I managed to set up rclone as described, and run duplicity.

Restoring didn’t work because of a GPG error:

duplicity restore --file-to-restore=home/nico/Pictures rclone://gdrive:/nico-e540 /home/nico/restore/Pictures
Synchronising remote metadata to local cache...
Copying duplicity-full-signatures.20210121T095528Z.sigtar.gpg to local cache.
GPGError: GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: AES256.CFB encrypted data
gpg: encrypted with 1 passphrase
gpg: decryption failed: Bad session key
===== End GnuPG log =====

I tried the suggested stackoverflow suggestion, but it didn’t make a difference.

Cheers.

1 Like

Yeah that looks like it just abruptly ends before it gets going. It’s ending at the expected part - just as it is about to start listing files in the backup. I guess that’s consistent with a crash…

That is the unhelpful error message you get when you enter the wrong encryption passphrase. Is it possible you had a typo?

(There is also an old bug that might cause that error message, but that would only hit you if you were on a fairly old version of deja-dup – less than 39.1 – when making the backups - how do I see Manjaro’s version history for deja-dup?)

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.