Integrate open-android-backup with a deduplicating backup tool

[dev590t]

Advanced backup tool like borg, restic support Deduplicating archiver. This feature is very usefull for incremental backup with low storage usage.
Do it is possible to integrate open-android-backup with a deduplicating backup tool.
For instance, user can disable the encryption feature. So after run open-android-backup, user launch borg on backup-tmp dir to create the backup.
Or better, open-android-backup store data directly with borg.

[mrrfv]

Great idea! This is absolutely possible to integrate with open-android-backup if any of these tools support backing up directly from stdin (I think borg does?). Once I have some free time I'll try to add this in (PRs are always welcome, and if I forget about this, feel free to ping me, preferably on a weekend).

[dev590t]

It is what I think:

Existing advanced backup tool(restic,borg, duplicati,..), open-android-backup use some basic backup logic. Why do users want to still use open-android-backup? Because tools aren't out-of-the-box for Android usage, if a user uses this backup tool for Android backup, he should manage many tasks: extract apk, export data, indicate files to include in Android internal storage to back up, etc...
I think the main purpose of open-android-backup should be to export all elements necessary for restoring into the file system(apk, config of the app, data, ...), then let the user choose the backup solution he prefers. Because:

• Each backup solution has its advantages; locking the user in a specific solution prevents the user from profiting from another one: If open-android-backup uses borg backend, the user can't use duplicati 's encrypted backups on cloud storage services.
• Users already have a backup tool for computers with which they are familiar; if the user is familiar with restic, and open-android-backup uses the borg backend, user should learn borg additionally. He will prefer to use restic.

So open-android-backup should expose the backup-tmp directory, it is the bridge to allow users extends their computer backup solution to Android.

How open-android-backup could integrate with other advanced backup tool?
I think it is possible to remove the step of exporting all data into backup-tmp. This operation has a few issues:

• if the phone has 100GB, it is time expensive for the HDD disk, and bad for the life of SSD disk.
• if the process is interrupted (because of a non caught exception, or computer power down), the data will be let on backup-tmp without removing. It is a security issue.

I think the existing two solutions are:

• after that open-android-backup export apk into file system. It produces a configuration file for the integrated backup tool. The configuration file lists all dir in Android that the backup tool should back up.
• Or open-android-backup export apk in backup-tmp, backup-tmp contains symlink of all dir that should be backed up. The user just need to run the backup tool with the option follow symlink, and all elements can be backup.

The second solution is more universal because each backup tool configuration file uses a different format. In the first solution, open-android-backup should support the integrated backup's configuration file's format.

[mrrfv]

So open-android-backup should expose the backup-tmp directory, it is the bridge to allow users extends their computer backup solution to Android.

An option to simply expose the contents of backup-tmp without doing any sort of compression or encryption is quite easy to add. I'll do that later.

I think it is possible to remove the step of exporting all data into backup-tmp.
...
after that open-android-backup export apk into file system. It produces a configuration file for the integrated backup tool. The configuration file lists all dir in Android that the backup tool should back up.
Or open-android-backup export apk in backup-tmp, backup-tmp contains symlink of all dir that should be backed up. The user just need to run the backup tool with the option follow symlink, and all elements can be backup.

I feel like you might be misunderstanding how the script works (unless I'm in the wrong). open-android-backup works by running standard adb commands (with some scripting and other magic sprinkled on top) to copy files from the device to backup-tmp. It isn't possible to directly mount the phone's internal storage and lower-level databases as a filesystem and create symlinks for a backup tool.
It could be potentially possible to make adb e.g. output the contents of a specific file to stdin and directly send that to 7Zip or a backup tool bypassing backup-tmp, but that would require an inelegant solution that might not work on all devices.

On the other side, you are absolutely correct that backup-tmp might cause issues in some scenarios, like when the computer isn't encrypted or is low on space, so an open-android-backup version that omits the temporary directory could be made in the distant future (I don't have enough free time for that at the moment, and there's little demand for such change).

About the deduplicating backup tool integration, I was thinking to implement that by adding an option to use such tool - and if it's enabled, the script would use tar instead of 7Zip to pack all files from backup-tmp into a single file, and tar's output would be saved directly to the repository of a selected backup tool. Any of them can be supported as long as they support adding files from stdin.