Skip to content

Commit

Permalink
Stop keeping old copies of eventbus files
Browse files Browse the repository at this point in the history
We've never restored from these "backups", and as long as we ensure that
the current file points to a fully written file, it's fine to only keep
at most 2 files around: the actual current file and a temporary "new"
file.

To ensure that we're not pointing at a half-written file, we continue
using the age-old pattern for atomic file updates: write to another file
and swap a symlink once the write is complete
  • Loading branch information
nemacysts committed Feb 14, 2024
1 parent c0d3a0a commit 9e715ca
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions tron/eventbus.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,10 @@ def sync_load_log(self):
duration = time.time() - started
log.info(f"log read from disk, took {duration:.4}s")

def sync_save_log(self, reason):
def sync_save_log(self, reason: str) -> bool:
started = time.time()
new_file = os.path.join(self.log_dir, f"{int(started)}.pickle")
previous_file = os.path.realpath(os.path.join(self.log_dir, "current.pickle"))
try:
with open(new_file, "xb") as f:
pickle.dump(self.event_log, f)
Expand All @@ -165,7 +166,15 @@ def sync_save_log(self, reason):
except FileNotFoundError:
pass
os.symlink(new_file, tmplink)
os.replace(tmplink, self.log_current)
os.replace(src=tmplink, dst=self.log_current)
# once we get here, `self.log_current` is now pointing to `new_file`
# so we can safely delete the previous `self.log_current` target without
# feat of losing data
try:
os.remove(previous_file)
except Exception:
# this shouldn't happen - but we also shouldn't crash if the impossible happens
log.exception(f"unable to delete {previous_file} - continuing anyway.")

duration = time.time() - started
log.info(f"log dumped to disk because {reason}, took {duration:.4}s")
Expand Down

0 comments on commit 9e715ca

Please sign in to comment.