Regularely disable append-only mode automatically (optionally)

[rugk]

Append-only mode has been introduced in #160; doc: https://borgwarehouse.com/docs/user-manual/repositories/#append-only-mode

However, the issue obviously is this can fill up your disk space.

Can you somehow add an option to enable or disable it automatically, say… every 30days? (And re-enables it after pruning or after some interval again like maybe 10h or whatever – where the backup client is scheduled to do it's backup.) So you have at least 30 days of backups?

Threat model assumes a compromised client, but one, where you would at least in some interval notice this. Like a ransomware will hopefully after… say… 30 days at most show you a notice to pay.
Of course, there is a risk that "real pruning" happens at some time, when the data has been deleted, but AFAIK, there is not a lot you can do here. The risk is always there… because… even if you don't see it ransomware could have encrypted/deleted everything already. If the malware is as sophisticated to check for Borgbackup, it would also be silent for a long enough time for your backups to be "too old" and be pruned, potentially, likely…

Alternatives

The alternative is kinda the same thing: You might manually check this whether backups have been made/the system is okay and then disable append-only, Here you also cannot be sure whether your client is really not compromised already – how could you? If you would know that already, you would not need the backup for later, you would just restore the current one…
So depending on your threat model and how sophisticated you guess your attacker is, the "automatic append-only disable" mode may be sufficient IMHO and be better than not having append-only mode at all. Because realistically, this is the more relativistic alternative…

More ideas

Thinking about it, you could have a simple "sanity check" (or some would make market it and solve it with AI haha…) that just checks before the mode is disabled, whether:

• much da has been pruned recently
• any other criteria that could be a tell for a compromise of the client?

In such a case (even when this option is not enabled), there could be some warning (/triangle) or so being displayed like: "In the last 7 [30|60] days, more than 6GB of data has been deleted. This could be generally a nice feature.
AFAIK this is how Nextcloud's ransomware protection app kinda works. (though it may have more data to analyze, like file names etc.)

[Ravinou]

Thanks for your comment. As I was developing the feature, I was thinking of the same thing as you, but in reverse 😄

In other words, a temporary deactivation of append-only mode. That way you can do your pruning and not forget to reactivate the option afterwards, for example.

But, However, an option to disable append-only mode doesn't seem very convincing to me. In addition to greatly complicating the UI, I'm not convinced of its value, I can't see the point of such a feature. Automatically disabling append-only mode would mean that your client would have to do the prune in the same time window, and automatically too. It all sounds very flawed to me 🙄

[rugk]

Yeah, I meant exactly such option.

[Forceu]

How about having a button in the interface instead for compacting? This can be done server-side without a passphrase.

borg compact does not need a key, so it is possible to invoke it from the client or also from the server.

from https://borgbackup.readthedocs.io/en/stable/usage/compact.html

[rugk]

Hmm but borg compact is not prune, but does this work in a fully automated fashion maybe (so not client-server coordination needs to be done)?

At regular intervals invoke the following "algorithm":

0. Edit: Temporarily disallow remote access (see edit below)
1. Disable append-only mode.
2. Invoke compact.
3. Enable append-only mode afterwards.
4. Enable remote access again.

Clients can then run prune as usual and the server just "once upon a month" or so, cleans up what needs to be cleaned up while keeping the repo in append-only mode otherwise?

Edit: Ah read the docs and this is obviously also important:

Ensure that no remote access is possible if the repository is temporarily set to normal mode for e.g. regular pruning.

[hozza]

Borg Base handles this by allowing the client to to specify prune but in append only mode it will not actually delete those repos marked as "to be pruned".

Then periodically (e.g. every 6 weeks) the server side runs compact which actually deletes the previously specified prunes. Essentially converting the append only option into a safe "delayed deletion" determined by the server side compact command.

IMO this is the best of both worlds. Safer than "trusted client" deletion but also keeping storage space used managable.

[rugk]

I agree! IIRC when I wrote my comments before borg did not had the compact command or I did not have it in mind yet…