Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporary backup name #189

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Temporary backup name #189

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8758506
running backup has temporary name
SimonHeimberg Dec 2, 2019
a5259ce
create backup.inprogress before pruning data, fixes #188
SimonHeimberg Dec 2, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 11 additions & 13 deletions rsync_tmbackup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ fn_parse_date() {
}

fn_find_backups() {
fn_run_cmd "find "$DEST_FOLDER/" -maxdepth 1 -type d -name \"????-??-??-??????\" -prune | sort -r"
fn_run_cmd "find "$DEST_FOLDER/" -maxdepth 1 -type d -name \"$1????-??-??-??????\" -prune | sort -r"
}

fn_expire_backup() {
Expand Down Expand Up @@ -429,7 +429,8 @@ KEEP_ALL_DATE=$((EPOCH - 86400)) # 1 day ago
KEEP_DAILIES_DATE=$((EPOCH - 2678400)) # 31 days ago

export IFS=$'\n' # Better for handling spaces in filenames.
DEST="$DEST_FOLDER/$NOW"
DEST="$DEST_FOLDER/_$NOW"
DEST_DONE="$DEST_FOLDER/$NOW"
PREVIOUS_DEST="$(fn_find_backups | head -n 1)"
INPROGRESS_FILE="$DEST_FOLDER/backup.inprogress"
MYPID="$$"
Expand Down Expand Up @@ -478,20 +479,16 @@ if [ -n "$(fn_find "$INPROGRESS_FILE")" ]; then
fi
fi

if [ -n "$PREVIOUS_DEST" ]; then

PREVIOUS_DEST_RUNNING="$(fn_find_backups _ | head -n 1)"
if [ -n "$PREVIOUS_DEST_RUNNING" ]; then
# - Last backup is moved to current backup folder so that it can be resumed.
# - 2nd to last backup becomes last backup.
fn_log_info "$SSH_DEST_FOLDER_PREFIX$INPROGRESS_FILE already exists - the previous backup failed or was interrupted. Backup will resume from there."
fn_run_cmd "mv -- $PREVIOUS_DEST $DEST"
if [ "$(fn_find_backups | wc -l)" -gt 1 ]; then
PREVIOUS_DEST="$(fn_find_backups | sed -n '2p')"
else
PREVIOUS_DEST=""
fi
# update PID to current process to avoid multiple concurrent resumes
fn_run_cmd "echo $MYPID > $INPROGRESS_FILE"
fn_run_cmd "mv -- $PREVIOUS_DEST_RUNNING $DEST"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mv moves the incomplete backup to the new name, to resume. It is the same as before except of the temporary name.

fi
fi
# update PID to current process to avoid multiple concurrent runs
fn_run_cmd "echo $MYPID > $INPROGRESS_FILE"

# Run in a loop to handle the "No space left on device" logic.
while : ; do
Expand Down Expand Up @@ -613,8 +610,9 @@ while : ; do
# Add symlink to last backup
# -----------------------------------------------------------------------------

fn_run_cmd "mv -- '$DEST' '$DEST_DONE'"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this use of mv can be dangerous on some systems (like on tape backups; yes these are really still used in the wild as they are very cheap and long lasting)
it might be safer (as these are backups after all) to use a conflict check or -f (force) flag to ensure backups make it when they have to overwrite partial or corrupt copies from previous runs.

also have you tested what happened with symlink conflicts?

Copy link
Contributor Author

@SimonHeimberg SimonHeimberg Mar 21, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just used the same arguments for mv it was used before (when continuing an aborted backup) (L485).
I see that mv -T (or with long option mv --no-target-directory) is safer than plain mv. With -f, it only overwrites empty directories, but does never move the directory to somewhere unexpected. Or what symlink conflict do you mean?

The porpose of this mv is to mark the backup as completed. It should not overwrite any data (except in very strange or constructed cases).

I do not know enough about tape backups. Can they be used with current version? How does rsync decide if a hardlink should be created without rewinding? By caching data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reply @reactive-firewall ?

fn_rm_file "$DEST_FOLDER/latest"
fn_ln "$(basename -- "$DEST")" "$DEST_FOLDER/latest"
fn_ln "$(basename -- "$DEST_DONE")" "$DEST_FOLDER/latest"

fn_rm_file "$INPROGRESS_FILE"

Expand Down