How to deal with off-site backup of the local backup

[NobbZ]

I am currently migrating my restic scripts to rustic, as the --as-path feature makes some of the scripts much easier for me. Also restic takes 5 to 10 minutes to open my local repository taking a full core, and relatively huge amounts of memory for all tasks.

The only thing that I have trouble translating in the process is my current use of restic copy which once day copies my local repository over to B2.

How would I reimplement this workflow using rustic?

I am aware that I can use restic for the copy and rustic for anything else, though I would prefer to only require a single tool, especially as restic has that huge delay in operation and usually also makes my fans spin at full speed…

Thanks in advance,
NobbZ

[aawsome]

Thanks for your question @NobbZ!

Yes, there is no copy command yet in rustic, see below about this topic.
But can you explain your use case a bit more in detail, why do you want to use the copy command`? Is there a reason why you want to copy once a day?

FWIW, I do backups with rustic to a local storage and to a remote (hot/cold) storage. I simply have two config files to define two repository destinations and run rustic backup twice to backup to both destinations.

About the copy command

Currently, there is no copy command implemented for rustic. I can of course put this a bit higher on my prio list. The main struggling point about copy is however a usage/syntax thing, as we have two repositories involved here. This is already a bit unpleasant with restic (e.g. trying to copy from S3 to S3 but with different S3 credentials).

For rustic it is even more to think about as we support 1) configuration through a config file and 2) hot/cold repositories which itself are basically two backend locations.

So, I don't really know what is best:

• Not support copy at all with config files, only via command-line parameters (this becomes however a bit ugly with hot/cold repos)
• Support to copy using two config files, e.g. rustic -P local -Pdest xyz copy <Snapshots>
• Support to specify one or more destinations in the config file like we do with backup sources

Moreover, there is the option to extend the hot/cold approach into a multi-repo approach which would be something like specifying multiple backend locations and define if it is a hot part or a additional full backend.This could include syncing options, e.g. if one of the repositories is not available or temporarily excluded for some reasiong.
However, I didn't think about this in detail so far...

[NobbZ]

I need the copy mostly as one of the hosts I back up from is a computer that does not have full internet access and can not access the b2 storage directly, so I used the copy as a workaround.

I can totally understand the reasoning behind the wonky UX, and I am aware of the S3 to S3 (or similar) problems.

Though indeed it is currently the only way for me to get some of my backups to the off-site.

[aawsome]

Ok, I see the use case.

If you want to copy all snapshots of the repository, syncing all files in the local storage to b2 (e.g. using rclone directly) is a possible workaround. This however doesn't help if you only want to selectively copy snapshots or if you start with an already-existing repo produced by restic copy.

I'll try to work on a copy implementation during the Christmas holidays! Actually I think I'll start with only allowing to define "targets" in the config file, like

[[copy.targets]]
repository = "rclone:b2:backup"
password-file =  "/root/key-b2"

[[copy.targets]]
repository = "rclone:s3:backup-cold"
repo-hot = "rclone:s3:backup-hot"
password = "mySecret"

Then rustic copy would copy from the "main" repository defined under [global] to all specified targets.

[aawsome]

I started #358 which should be already working. Feel free to test it out, but read the description and be aware that it is WIP.
However, for today I'm done, will maybe finish this tomorrow or beginning of next week.

[aawsome]

PR #358 is now finished and can be tried. Please ask if you need binaries.

Example output:

alex-dev@latitude:~/rust/rustic$ ./target/x86_64-unknown-linux-musl/release/rustic -P copy_example copy 
using config /home/alex-dev/.config/rustic/copy_example.toml
[INFO] repository /tmp/repo: password is correct.
[00:00:00] reading index...               ████████████████████████████████████████          1/1                                                                             
[INFO] repository /tmp/repo2: password is correct.
[INFO] using no cache
[INFO] copying to target /tmp/repo2...
| ID       | Time                | Host     | Label | Tags | Paths | Status   |
|----------|---------------------|----------|-------|------|-------|----------|
| 1e87670d | 2022-12-24 22:57:50 | latitude |       |      | src   | existing |
[INFO] no snapshot to copy.
[INFO] repository /tmp/repo3: password is correct.
[INFO] using no cache
[INFO] copying to target /tmp/repo3...
| ID       | Time                | Host     | Label | Tags | Paths | Status  |
|----------|---------------------|----------|-------|------|-------|---------|
| 1e87670d | 2022-12-24 22:57:50 | latitude |       |      | src   | to copy |
[00:00:00] reading index...               ████████████████████████████████████████          0/0                                                                             
[00:00:00] copying snapshot blobs...      ████████████████████████████████████████          1/1                                                                             
[00:00:00] saving snapshots...            ████████████████████████████████████████          1/1  

[aawsome]

The copy command is now available in main and in the latest beta builds.

[aawsome]

The copy command is now available in rustic 0.4.2