Trying to reproduce/understand --append-only

[kmille]

Hey,
I'm trying to reproduce/understand the behaviour of --append-only (once again)

In the authorized_keys of backup01, I have two keys (one in append-only mode and one with full access)

command="borg serve --append-only --restrict-to-path /opt/borg",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG0H3T9YtJEzGnK+y6ol18ss6Bzjyh2a+wFYDwvrJ0ou append
command="borg serve --restrict-to-path /opt/borg",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGd456vtmLqQ2doHV3feKnZ4TKRA0Zj1hmSDrpRaGAWk full

1. Initialize repo with --append-only

unset SSH_AUTH_SOCK
export BORG_RSH="ssh -i borg-test-full"
export BORG_REPO="root@backup01:/opt/borg"
export BORG_PASSPHRASE="test"
borg init --encryption repokey --append-only

2. Check it

root@backup01:/opt$ cat /opt/borg/config | grep append
append_only = 1

3. Create two backups in --append-only mode

unset SSH_AUTH_SOCK
export BORG_RSH="ssh -i borg-test-append"
export BORG_REPO="root@backup01:/opt/borg"
export BORG_PASSPHRASE="test"

function create() {
    borg --verbose create "::$(date +%T)" ~/Downloads/CTS1_quickstartguide_EN.pdf
}

create
create
borg list

Results in:

Creating archive at "root@backup01:/opt/borg::11:09:20"
+ create
++ date +%T
+ borg --verbose create ::11:09:25 /home/kmille/Downloads/CTS1_quickstartguide_EN.pdf
Creating archive at "root@backup01:/opt/borg::11:09:25"
+ borg list
11:09:20                             Sun, 2024-01-28 11:09:24 [f72f8fd0f4d5a4a32b53b4cce0643e2faf58a998d8cbe9857b0ac1de7f3ad144]
11:09:25                             Sun, 2024-01-28 11:09:28 [4e0e44b3f95627573e3d68e764470dc1c77c5ed59ceaa61e3d924f533a3eb379]

4. Prune backups with --append-only results in (as expected)

+ borg list
11:09:20                             Sun, 2024-01-28 11:09:24 [f72f8fd0f4d5a4a32b53b4cce0643e2faf58a998d8cbe9857b0ac1de7f3ad144]
11:09:25                             Sun, 2024-01-28 11:09:28 [4e0e44b3f95627573e3d68e764470dc1c77c5ed59ceaa61e3d924f533a3eb379]
+ borg prune --keep-last 1
+ borg list
11:09:25                             Sun, 2024-01-28 11:09:28 [4e0e44b3f95627573e3d68e764470dc1c77c5ed59ceaa61e3d924f533a3eb379]

4. Running borg check --verbose --progress --verify-data with full permission should show an error, if I understand the docs correctly, but it doesn't.

+ borg check --verbose --progress --verify-data
Remote: Starting repository check
Remote: finished segment check at segment 13
Remote: Starting repository index check
Remote: Index object count match.
Remote: Finished full repository check, no problems found.
Starting archive consistency check...
Starting cryptographic data integrity verification...
Finished cryptographic data integrity verification, verified 4 chunks with 0 integrity errors.                         
Analyzing archive 11:09:25 (1/1)                                                                                       
Archive consistency check complete, no problems found. 

Hm. Any ideas what's wrong here?

[ThomasWaldmann]

Looks mostly ok, but there is one maybe confusing issue:

You set append-only using 2 mechanisms:

• via the command line option in authorized_keys
• via the repo config (guess that will also make your "full access" repo append-only)

[kmille]

Hmm. As far as I understood, I need to globally enable append-only for the repo and then I can specify it per key. I just tried it again (but with borg init without --append-only and in the config it was set to 0). But then the behaviour was the same. In the end, I just get

+ borg list
13:59:49                             Sun, 2024-01-28 13:59:53 [54123f7b1814cc5d33688966f11b87f4945416ee84e147ddcd11290e1bc4855d]
+ borg --verbose --progress check --verify-data
Remote: Starting repository check
Remote: finished segment check at segment 13
Remote: Starting repository index check
Remote: Index object count match.
Remote: Finished full repository check, no problems found.
Starting archive consistency check...
Starting cryptographic data integrity verification...
Finished cryptographic data integrity verification, verified 5 chunks with 0 integrity errors.                                                                             
Analyzing archive 13:59:49 (1/1)                                                                                                                                           
Archive consistency check complete, no problems found.  

with full access ssh/borg. (one backup left, no message about a prune in append-mode)

Same if I run it on the host itself: root@backup01:/opt$ borg check --verbose --progress --verify-data borg

[ThomasWaldmann]

From the docs:

https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode-forbid-compaction

Running borg init --append-only creates an append-only repository
regardless of server settings.

("server settings" refers to borg serve cmdline options)

[ThomasWaldmann]

"Running borg check --verbose --progress --verify-data with full permission should show an error"

I don't think so.

What you need to do is to remove append_only=1 from your repo config and just use borg serve with / without --append-only.

[kmille]

That's what I did... I have a key that is not append-only-restricted:

command="borg serve --restrict-to-path /opt/borg",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGd456vtmLqQ2doHV3feKnZ4TKRA0Zj1hmSDrpRaGAWk full

1. with full access: borg init --encryption repokey
2. in append-only mode: create archive, create archive, prune one archive (UPDATE: to clarify: with a ssh key that has --append-only in the autorized_keys)
3. with full access: borg check --verbose --progress check --verify-data

[ThomasWaldmann]

OK, so what's the problem?

[kmille]

I expect a message with "someone pruned your backups without permission" after a borg check with full permission. Otherwise I will do a borg compact (somewhere in the future) and also delete the backups which should not be deleted.

With the current behaviour, I always have to run borg list, check it manually, and then run borg compact to not accidentally delete my backups.

[ThomasWaldmann]

It doesn't work like you think it does.

On a logical level, all works as usual. You can delete and prune archives.

It just does not compact the segments (if you run borg compact [since borg 1.2] or implicitly, if you run older borg versions), so you could rollback the repo to an earlier state (see the docs how to do this). Also, deleting the whole repo is disallowed.

[kmille]

Yes, that's what I thought. But expected borg to tell me with borg check that something strange is going on. Two small other questions:

1. by deleting the whole repo, what do you mean? I'm not aware of a delete-repo command (only borg prune --keep-last 1 and borg delete <last archive>)
2. Is there a way to restict ssh access/borg serve to only borg compact (so disallow listing/mounting/extracting existing backups, just compacting)?

I'm quiet unhappy how things are, but now I think I understood everything. Thanks for the explanation. I think I will not use the append-only-feature in the way it currently works and more rely on offiste-backups.

[ThomasWaldmann]

From borg's perspective, nothing strange happened.

A1: I meant borg delete REPO (without giving ::ARCHIVE). Now you are aware. :-)

A2: No.