Replies: 3 comments 1 reply
-
|
I don't use onedrive and I also don't know what it does internally. Borg has 2 places where it checks for unchanged / changed files:
Note that borg usually uses ctime and not mtime, because ctime is only controlled by the kernel (and thus more reliable) while mtime is controllable also by userspace applications and some apps are known to "roll back" mtime after they modified files. While the warning you have seen comes from the 2nd place, I guess you should try to make the 1st place work better for your case. If the files cache works as it should, borg will not read the file's contents in a 2nd+ backup if the file did not change. See that FAQ entry (and also the following 2) about how to debug/improve this: https://borgbackup.readthedocs.io/en/stable/faq.html#why-is-backup-slow-for-me |
Beta Was this translation helpful? Give feedback.
-
|
You are absolutely correct: the warning comes from the before and after stat. Based on your hint I could manually reproduce it: ctime changed (see below). But please consider: OneDrive (like others) is a local-cloud sync tool and you can't change the inner working, of course. Given that a lot of built-in disks (spinning, SSD, NVMe, ...) are smaller than some cloud storage spaces, you simply can't fill a local cache with all content from the cloud at the same time. You are bound to the cycle request file >> download file >> cache file locally >> borg process file >> free local cache (delete file; this is automatically and unavoidable done by OneDrive) >> move on to next file Freeing the local cache /deleting the locally cached file triggers a crime modification by OneDrive (again, no chance to prevent this). Of course, this is a simplification, because the file gets deleted sometime after downloading other files, but before the local disks fills up. But it's the base process. Cloud storage spaces are ubiquitous. But they are not a backup! There are many reasons that could lead to a temporarily or permanent loss of your data or access to your data. I really would appreciate to use borg as the missing and freely available backup tool. So I hope that maybe a simple configuration, maybe a command line switch, can solve the describes situation without backing up every file each time borg runs. As you might already have guessed: I am a total newbie to borg :-) Here is a stat comparison. Steps done:
Here's the outcome:
The only thing that changed is the line starting with "Change". |
Beta Was this translation helpful? Give feedback.
-
|
You can add The first backup will be slow, but the 2nd then might work faster. Guess that is a bug in onedrive, I don't see a reason why it changes the ctime if the file did not change. Hmm, maybe it attaches some "onedrive metadata" to the file and that is what changes the ctime? |
Beta Was this translation helpful? Give feedback.
-
|
I would assume OneDrive is working correct in it's own definition and changes some metadata. I will try to test your options on a small folder. |
Beta Was this translation helpful? Give feedback.
-
First and foremost I hope that I have not overlooked obvious posts related to this question. I tried various keywords here and on Google, but couldn't find an answer.
Using borg to backup files from my OneDrive folder to another cloud storage shows "file changed while we backed it up" for each and every file and files will be fully processed again the next time. So one ends with a full (as in: full gigabytes) backup every time.
Background:
I have OneDrive configured on my macOS. It is syncing my OneDrive folder and doing files-on-demand, as the disk in my Mac is smaller than my OneDrive space.
OneDrive, being a sync not a backup solution, needs proper backup. So I tried borg to backup the Onedrive folder.
First, I was interested, whether borg would backup files-on-demand at all, for almost all of the files are links to OneDrive, rather than physical on my computer. OneDrive has to download them on access to make them available to borg backup. borg triggers the download correctly and then backs up the file.
Second, OneDrive should download the file, currently processed by borg and "delete" it (leave only the OneDrive link to it) soon afterwards to prevent my local disk from filling. OneDrive handles this, so my disk never got full.
Everything got properly backup from OneDrive to the other cloud destination. But the borg log file showed "file changed while we backed it up" for each and every file. Starting a second backup sees each file as changed and will backup it again.
Any idea or recommendation how to solve this?
Beta Was this translation helpful? Give feedback.
All reactions