-
Notifications
You must be signed in to change notification settings - Fork 2.5k
refactor(pkg/utils): add unified atomic file write utility #706
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mosir
wants to merge
12
commits into
sipeed:main
Choose a base branch
from
mosir:fix/atomic-file-writes
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+197
−71
Open
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
c56fced
refactor(pkg/utils): add unified atomic file write utility
mosir 4aed359
refactor(pkg/utils): improve WriteFileAtomic with stronger durability…
mosir 11996f1
refactor(pkg): move atomic file write to dedicated fileutil package
mosir 7a2d353
Merge branch 'main' of github.com:mosir/picoclaw into fix/atomic-file…
mosir f86de3c
Merge remote-tracking branch 'sipeed/main' into fix/atomic-file-writes
mosir 6e754a8
merge: resolve conflicts with main
mosir 87e674b
Merge branch 'sipeed:main' into fix/atomic-file-writes
mosir be4b8fa
Merge branch 'sipeed:main' into fix/atomic-file-writes
mosir 16a1c96
Merge branch 'sipeed:main' into fix/atomic-file-writes
mosir d887009
merge: resolve conflicts with main
mosir 433af43
style: fix gci import grouping in config, cron, and skills installer
mosir b8c0d13
Merge branch 'sipeed:main' into fix/atomic-file-writes
mosir File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| // PicoClaw - Ultra-lightweight personal AI agent | ||
| // Inspired by and based on nanobot: https://github.com/HKUDS/nanobot | ||
| // License: MIT | ||
| // | ||
| // Copyright (c) 2026 PicoClaw contributors | ||
|
|
||
| // Package fileutil provides file manipulation utilities. | ||
| package fileutil | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "os" | ||
| "path/filepath" | ||
| "time" | ||
| ) | ||
|
|
||
| // WriteFileAtomic atomically writes data to a file using a temp file + rename pattern. | ||
| // | ||
| // This guarantees that the target file is either: | ||
| // - Completely written with the new data | ||
| // - Unchanged (if any step fails before rename) | ||
| // | ||
| // The function: | ||
| // 1. Creates a temp file in the same directory (original untouched) | ||
| // 2. Writes data to temp file | ||
| // 3. Syncs data to disk (critical for SD cards/flash storage) | ||
| // 4. Sets file permissions | ||
| // 5. Syncs directory metadata (ensures rename is durable) | ||
| // 6. Atomically renames temp file to target path | ||
| // | ||
| // Safety guarantees: | ||
| // - Original file is NEVER modified until successful rename | ||
| // - Temp file is always cleaned up on error | ||
| // - Data is flushed to physical storage before rename | ||
| // - Directory entry is synced to prevent orphaned inodes | ||
| // | ||
| // Parameters: | ||
| // - path: Target file path | ||
| // - data: Data to write | ||
| // - perm: File permission mode (e.g., 0o600 for secure, 0o644 for readable) | ||
| // | ||
| // Returns: | ||
| // - Error if any step fails, nil on success | ||
| // | ||
| // Example: | ||
| // | ||
| // // Secure config file (owner read/write only) | ||
| // err := utils.WriteFileAtomic("config.json", data, 0o600) | ||
| // | ||
| // // Public readable file | ||
| // err := utils.WriteFileAtomic("public.txt", data, 0o644) | ||
| func WriteFileAtomic(path string, data []byte, perm os.FileMode) error { | ||
| dir := filepath.Dir(path) | ||
| if err := os.MkdirAll(dir, 0o755); err != nil { | ||
| return fmt.Errorf("failed to create directory: %w", err) | ||
| } | ||
|
|
||
| // Create temp file in the same directory (ensures atomic rename works) | ||
| // Using a hidden prefix (.tmp-) to avoid issues with some tools | ||
| tmpFile, err := os.OpenFile( | ||
| filepath.Join(dir, fmt.Sprintf(".tmp-%d-%d", os.Getpid(), time.Now().UnixNano())), | ||
| os.O_WRONLY|os.O_CREATE|os.O_EXCL, | ||
| perm, | ||
| ) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to create temp file: %w", err) | ||
| } | ||
|
|
||
| tmpPath := tmpFile.Name() | ||
| cleanup := true | ||
|
|
||
| defer func() { | ||
| if cleanup { | ||
| tmpFile.Close() | ||
| _ = os.Remove(tmpPath) | ||
| } | ||
| }() | ||
|
|
||
| // Write data to temp file | ||
| // Note: Original file is untouched at this point | ||
| if _, err := tmpFile.Write(data); err != nil { | ||
| return fmt.Errorf("failed to write temp file: %w", err) | ||
| } | ||
|
|
||
| // CRITICAL: Force sync to storage medium before any other operations. | ||
| // This ensures data is physically written to disk, not just cached. | ||
| // Essential for SD cards, eMMC, and other flash storage on edge devices. | ||
| if err := tmpFile.Sync(); err != nil { | ||
| return fmt.Errorf("failed to sync temp file: %w", err) | ||
| } | ||
|
|
||
| // Set file permissions before closing | ||
| if err := tmpFile.Chmod(perm); err != nil { | ||
| return fmt.Errorf("failed to set permissions: %w", err) | ||
| } | ||
|
|
||
| // Close file before rename (required on Windows) | ||
| if err := tmpFile.Close(); err != nil { | ||
| return fmt.Errorf("failed to close temp file: %w", err) | ||
| } | ||
|
|
||
| // Atomic rename: temp file becomes the target | ||
| // On POSIX: rename() is atomic | ||
| // On Windows: Rename() is atomic for files | ||
| if err := os.Rename(tmpPath, path); err != nil { | ||
| return fmt.Errorf("failed to rename temp file: %w", err) | ||
| } | ||
|
|
||
| // Sync directory to ensure rename is durable | ||
| // This prevents the renamed file from disappearing after a crash | ||
| if dirFile, err := os.Open(dir); err == nil { | ||
| _ = dirFile.Sync() | ||
| dirFile.Close() | ||
| } | ||
|
|
||
| // Success: skip cleanup (file was renamed, no temp to remove) | ||
| cleanup = false | ||
| return nil | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The way you implement "atomicwrite" just using tmpfile for the job, it will corrupt the file if write to tmp file failed too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're correct. I've fixed the issue。