How does Borg interact with ZFS? Does Borg mostly use sync writes over a network?

[Zerophase]

I'm building a storage server, and trying to make it extremely fast and have large amounts of data. In order to do this I'm creating the server with two zpools. One smaller SSD write cache zpool, (technically not a cache in ZFS terminology but functionally the same in practice) and a larger HDD zpool. I'm trying to see if I can saturate my network with the write cache. So, what happens is I write to the cache, and then use ZFS send / receive in scripts to move my data from the cache to storage. But, I don't know how Borg interacts with ZFS.

I don't believe my cache will have a borg database necessarily, and doing the sends from the cache wouldn't be calling borg. So, I think I'll have to run borg differently from how I've been running it on my desktop. More accurately, would I have to call borg from my file server to move the files on my desktop that changed to the write cache in ZFS? Would Borg then be capable of handling the ZFS send / receives to update the database stored on my HDDs?

I won't have the cache initially since I need to design the HDDs with all of their special disks first. Borg should just work until I add the write cache functionality to speed up network writes. I'm new to ZFS and I don't know how using multiple pools of disks would work exactly. Maybe, Borg and ZFS are already smart enough to just write new data to the write cache, and ZFS can handle moving it all down to the HDDs. How does Borg interact with Hierchachal Storage Management?

Also are Borg writes over a network all sync writes? If they are that would change how I design my disk layout.

[ThomasWaldmann]

Borg does not interact with ZFS at all, it just uses the normal OS file api (open, read/write, close, fsync, fadvise).

For the source files it also uses the OS calls to get attributes, xattrs and ACLs.

[Zerophase]

I mean a backup that involves moving the changes from disk 1 to disk 2, and then merging that with the borg repo on disk 3.

It's possible to send data from a zfs disk to another zfs disk without the transfer having errors. The intermediate disk can just be written to faster than the final storage disks. I'm trying to build a server that could back up 100 computers with borg, and the write cache might be necessary to prevent network congestion.

So, borg would be writing the changes on disk 1 to disk 2. Then disk 2 sends it to disk 3 to merge into the already existing Borg repo.

[RonnyPfannschmidt]

Borg only works with the Filesystem

So if your scheme is transparent on that level it works

If any awarenes is needed then it won't

[Zerophase]

Yeah, borgmatic does what I need. might make sense to add zfs send/receive support natively to borg eventually.

I'm guessing borgmatic can recognize filesystem changes between my source and borg repo. It then transfers the files to my cache, and sends it to where the borg repo is before running borg create to add those changes.

[RonnyPfannschmidt]

borg is unaware of zfs internals and ought to stay that way for practical reasons

[Zerophase]

I agree for the most part. I think borg 2.0 does what I need with the transfer command. It might make sense to add support for the zfs/btrfs send/receive commands in there after borg 2.0 is finalized. Might make borg even more data secure for complex backups on those filesystems.

Once my backup server is fully built I'll be testing out the workflow. I'll gladly give feedback if it seems like hierarchical storage systems could simplify running / make borg more efficient on platforms like that. What I'm worried about currently is I might have to run borg multiple times depending on how transfer ends up working. Once to get the data to the fast disk cache, and again to merge into the repo. I'll realistically have this built out six months from now. I have to do a lot of benchmarking for the HDD zpool, before even adding an SSD cache zpool.