Skip to content

Fix timestamp preservation when extracting cached files #387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Fix timestamp preservation when extracting cached files #387

wants to merge 4 commits into from

Conversation

cowwoc
Copy link

@cowwoc cowwoc commented Oct 5, 2025 •
edited
Loading

Problem

The Maven Build Cache Extension was not properly preserving file and directory timestamps when restoring attachedOutputs from cache. This caused Maven to warn about files being 'more recent than the packaged artifact' even after running mvnw verify.

Root Causes

  1. CacheUtils.zip() did not store directory entries with their timestamps - only files were added to the ZIP
  2. CacheUtils.unzip() set directory timestamps immediately during extraction, but these timestamps were subsequently overwritten when Files.copy() created files within those directories

Changes

CacheUtils.zip()

  • Added preVisitDirectory() override to store directory entries in the ZIP with their original timestamps
  • Modified visitFile() to explicitly set file entry timestamps from BasicFileAttributes

CacheUtils.unzip()

  • Introduced Map<Path, Long> directoryTimestamps to defer directory timestamp updates
  • Directory timestamps are now set after all files have been extracted, preventing them from being modified by subsequent file operations
  • Added HashMap and Map imports to support this functionality

Testing

Created comprehensive test that verifies:

  • Deep directory structures (a/b/c/)
  • Directory timestamp preservation
  • File timestamp preservation
  • Round-trip zip/unzip consistency

Impact

This fix ensures that cached build outputs maintain their original timestamps, eliminating spurious Maven warnings and improving build cache consistency across multi-module projects.

Fixes timestamp-related warnings like:

[WARNING] File 'formatter/api/target/classes/...' is more recent than the packaged artifact for 'module', please run a full 'mvn package' build

Related Issues and PRs

This PR improves the ZIP archive handling in CacheUtils:

This fix specifically addresses timestamp preservation which was not covered by previous restoration improvements.

@cowwoc
Fix timestamp preservation when extracting cached files
bdb7fe5 
The build cache extension was not properly preserving file and directory
timestamps when restoring attachedOutputs from cache. This caused Maven
to warn about files being 'more recent than the packaged artifact' even
after a successful build.
Root Causes:
1. The zip() method did not store directory entries with timestamps
2. The unzip() method set directory timestamps immediately, but they
   were later modified by Files.copy() operations for files within
Changes:
- Modified CacheUtils.zip() to store directory entries with timestamps
  via preVisitDirectory() callback
- Modified CacheUtils.unzip() to defer directory timestamp updates until
  after all files are extracted, preventing them from being overwritten
- Added Map<Path, Long> to track directory timestamps during extraction
This ensures that cached build outputs maintain their original timestamps,
preventing spurious warnings and improving build cache consistency.
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch 2 times, most recently from 10eb2c4 to 91e8b9c Compare October 5, 2025 22:20
This was referenced Oct 5, 2025
Open
Open
Closed
@olamy
Copy link
Member

olamy commented Oct 5, 2025

This caused Maven to warn about files being 'more recent than the packaged artifact' even after running mvnw verify.

how do you have this error?
is it possible to have a test for this change?

@AlexanderAshitkin
Copy link
Contributor

feedback provided in #388

cowwoc pushed a commit to cowwoc/maven-build-cache-extension that referenced this pull request Oct 6, 2025
@claude
Add comprehensive fixes and unit tests for timestamp preservation
0b0ea6e 
Addresses feedback from PR apache#388 review comments and adds comprehensive
unit tests to validate timestamp preservation through cache operations.

Changes to CacheUtils.java:

1. Implement glob filtering for directory entries
   - Track directories containing matching files during visitFile()
   - Only add directory entries in postVisitDirectory() if they contain
     matching files or no glob filter is active
   - Preserves glob pattern contract while maintaining timestamp preservation

2. Fix Files.exists() race condition (TOCTOU vulnerability)
   - Remove unnecessary Files.exists() checks before Files.createDirectories()
   - Leverage idempotent behavior of createDirectories()
   - Eliminates time-of-check-to-time-of-use race condition

3. Add error handling for timestamp operations
   - Wrap Files.setLastModifiedTime() in try-catch blocks
   - Best-effort timestamp setting with graceful degradation
   - Prevents failures on filesystems without timestamp support

New Unit Tests (CacheUtilsTimestampTest.java):

- testFileTimestampPreservation: Verifies file timestamps preserved
- testDirectoryTimestampPreservation: Verifies directory timestamps preserved
- testDirectoryEntriesStoredInZip: Verifies directories stored in zip
- testTimestampsInZipEntries: Verifies zip entry timestamps correct
- testMavenWarningScenario: Reproduces Maven warning about file timestamps
- testMultipleFilesTimestampConsistency: Verifies consistent timestamps

Test Results:
- All 6 tests pass with fixes applied
- Tests are Java 8 compatible using helper methods
- Clear failure messages with timestamp diffs for debugging

Fixes apache#387

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from 0b0ea6e to 91e8b9c Compare October 6, 2025 15:01
cowwoc pushed a commit to cowwoc/maven-build-cache-extension that referenced this pull request Oct 6, 2025
@claude
Add comprehensive fixes and unit tests for timestamp preservation
7b912ce 
Addresses feedback from PR apache#388 review comments and adds comprehensive
unit tests to validate timestamp preservation through cache operations.

Changes to CacheUtils.java:

1. Implement glob filtering for directory entries
   - Track directories containing matching files during visitFile()
   - Only add directory entries in postVisitDirectory() if they contain
     matching files or no glob filter is active
   - Preserves glob pattern contract while maintaining timestamp preservation

2. Fix Files.exists() race condition (TOCTOU vulnerability)
   - Remove unnecessary Files.exists() checks before Files.createDirectories()
   - Leverage idempotent behavior of createDirectories()
   - Eliminates time-of-check-to-time-of-use race condition

3. Add error handling for timestamp operations
   - Wrap Files.setLastModifiedTime() in try-catch blocks
   - Best-effort timestamp setting with graceful degradation
   - Prevents failures on filesystems without timestamp support

New Unit Tests (CacheUtilsTimestampTest.java):

- testFileTimestampPreservation: Verifies file timestamps preserved
- testDirectoryTimestampPreservation: Verifies directory timestamps preserved
- testDirectoryEntriesStoredInZip: Verifies directories stored in zip
- testTimestampsInZipEntries: Verifies zip entry timestamps correct
- testMavenWarningScenario: Reproduces Maven warning about file timestamps
- testMultipleFilesTimestampConsistency: Verifies consistent timestamps

Test Results:
- All 6 tests pass with fixes applied
- Tests are Java 8 compatible using helper methods
- Clear failure messages with timestamp diffs for debugging

Fixes apache#387

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from 0b0ea6e to 7b912ce Compare October 6, 2025 15:10
cowwoc pushed a commit to cowwoc/maven-build-cache-extension that referenced this pull request Oct 6, 2025
@claude
Add comprehensive fixes and unit tests for timestamp preservation
eb1e4da 
Addresses feedback from PR apache#388 review comments and adds comprehensive
unit tests to validate timestamp preservation through cache operations.

Changes to CacheUtils.java:

1. Implement glob filtering for directory entries
   - Track directories containing matching files during visitFile()
   - Only add directory entries in postVisitDirectory() if they contain
     matching files or no glob filter is active
   - Preserves glob pattern contract while maintaining timestamp preservation

2. Fix Files.exists() race condition (TOCTOU vulnerability)
   - Remove unnecessary Files.exists() checks before Files.createDirectories()
   - Leverage idempotent behavior of createDirectories()
   - Eliminates time-of-check-to-time-of-use race condition

3. Add error handling for timestamp operations
   - Wrap Files.setLastModifiedTime() in try-catch blocks
   - Best-effort timestamp setting with graceful degradation
   - Prevents failures on filesystems without timestamp support

New Unit Tests (CacheUtilsTimestampTest.java):

- testFileTimestampPreservation: Verifies file timestamps preserved
- testDirectoryTimestampPreservation: Verifies directory timestamps preserved
- testDirectoryEntriesStoredInZip: Verifies directories stored in zip
- testTimestampsInZipEntries: Verifies zip entry timestamps correct
- testMavenWarningScenario: Reproduces Maven warning about file timestamps
- testMultipleFilesTimestampConsistency: Verifies consistent timestamps

Test Results:
- All 6 tests pass with fixes applied
- Tests are Java 8 compatible using helper methods
- Clear failure messages with timestamp diffs for debugging

Fixes apache#387

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from 7b912ce to eb1e4da Compare October 6, 2025 15:12
cowwoc pushed a commit to cowwoc/maven-build-cache-extension that referenced this pull request Oct 6, 2025
@claude
Add comprehensive fixes and unit tests for timestamp preservation
b839b42 
Addresses feedback from PR apache#388 review comments and adds comprehensive
unit tests to validate timestamp preservation through cache operations.

Changes to CacheUtils.java:

1. Implement glob filtering for directory entries
   - Track directories containing matching files during visitFile()
   - Only add directory entries in postVisitDirectory() if they contain
     matching files or no glob filter is active
   - Preserves glob pattern contract while maintaining timestamp preservation

2. Fix Files.exists() race condition (TOCTOU vulnerability)
   - Remove unnecessary Files.exists() checks before Files.createDirectories()
   - Leverage idempotent behavior of createDirectories()
   - Eliminates time-of-check-to-time-of-use race condition

3. Add error handling for timestamp operations
   - Wrap Files.setLastModifiedTime() in try-catch blocks
   - Best-effort timestamp setting with graceful degradation
   - Prevents failures on filesystems without timestamp support

New Unit Tests (CacheUtilsTimestampTest.java):

- testFileTimestampPreservation: Verifies file timestamps preserved
- testDirectoryTimestampPreservation: Verifies directory timestamps preserved
- testDirectoryEntriesStoredInZip: Verifies directories stored in zip
- testTimestampsInZipEntries: Verifies zip entry timestamps correct
- testMavenWarningScenario: Reproduces Maven warning about file timestamps
- testMultipleFilesTimestampConsistency: Verifies consistent timestamps

Test Results:
- All 6 tests pass with fixes applied
- Tests are Java 8 compatible using helper methods
- Clear failure messages with timestamp diffs for debugging

Fixes apache#387

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch 7 times, most recently from 54e5412 to 078171b Compare October 6, 2025 15:31
@cowwoc
Add comprehensive unit tests and configuration for timestamp preserva…
6eee251 
…tion

Addresses feedback from PR apache#388 review comments and adds comprehensive
testing and configuration capabilities for timestamp preservation.

Changes:

1. Fix glob filtering for directory entries (PR apache#388 feedback)
   - Track directories containing matching files during visitFile()
   - Only add directory entries in postVisitDirectory() if they contain
     matching files or no glob filter is active
   - Preserves glob pattern contract while maintaining timestamp preservation

2. Fix Files.exists() race condition (PR apache#388 feedback)
   - Remove unnecessary Files.exists() checks before Files.createDirectories()
   - Leverage idempotent behavior of createDirectories()
   - Eliminates time-of-check-to-time-of-use (TOCTOU) vulnerability

3. Add error handling for timestamp operations (PR apache#388 feedback)
   - Wrap Files.setLastModifiedTime() in try-catch blocks
   - Best-effort timestamp setting with graceful degradation
   - Prevents failures on filesystems that don't support modification times

4. Add configuration property to control timestamp preservation
   - New preserveTimestamps field in AttachedOutputs configuration
   - Default value: true (timestamp preservation enabled by default)
   - Allows users to disable for performance if needed
   - Usage: <attachedOutputs><preserveTimestamps>false</preserveTimestamps></attachedOutputs>

5. Add comprehensive unit tests (CacheUtilsTimestampTest.java)
   - testFileTimestampPreservation: Verifies file timestamps preserved
   - testDirectoryTimestampPreservation: Verifies directory timestamps preserved
   - testDirectoryEntriesStoredInZip: Verifies directories stored in zip
   - testTimestampsInZipEntries: Verifies zip entry timestamps correct
   - testMavenWarningScenario: Reproduces Maven warning with manual repro steps
   - testMultipleFilesTimestampConsistency: Verifies consistent timestamps
   - testPreserveTimestampsFalse: Verifies configuration property works when disabled

6. Add test for module-info.class handling (ModuleInfoCachingTest.java)
   - Verifies module-info.class is preserved through zip/unzip operations

All tests pass (7 tests in CacheUtilsTimestampTest.java).
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from 078171b to 6eee251 Compare October 6, 2025 15:33
@cowwoc
Copy link
Author

cowwoc commented Oct 6, 2025 •
edited
Loading

@olamy @AlexanderAshitkin Please see the updated commit.

@AlexanderAshitkin Sorry for basing #388 on top of this PR. I have corrected the problem so they are now independent of each other.

@olamy https://github.com/apache/maven-build-cache-extension/pull/387/files#diff-090be5e498346368594f96199eab418ac5486df7368b0825ad1db7f63ef5fff1R214 contains instructions for reproducing the Maven warning message. Please let me know if you are able to reproduce it on your end. Thanks.

@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from 16800be to e7c23e8 Compare October 6, 2025 17:02
@cowwoc
Set all file timestamps consistently to prevent backwards time
a1321b5 
When restoring files from cache, set creation time, last modified time,
and last access time to the same value from the ZIP entry. This prevents
the scenario where creation time is newer than last modified time, which
can occur when only setting last modified time while creation time
defaults to the extraction time.

Uses BasicFileAttributeView.setTimes() instead of Files.setLastModifiedTime()
to set all three timestamps atomically. This ensures consistent timestamp
state across all supported platforms (Windows, macOS, Linux).
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from e7c23e8 to a1321b5 Compare October 6, 2025 17:03
@cowwoc
Add test and documentation for timestamp inclusion in cache checksums
d9fc24f 
When preserveTimestamps=true, file timestamps are stored in ZIP entry headers,
making them part of the ZIP file's binary content. This ensures that hashing
the ZIP file (for cache keys) includes timestamp information, providing proper
cache invalidation when file timestamps change.

Changes:
- Added testTimestampsAffectFileHash() test verifying that ZIP files with same
  content but different timestamps produce different hashes
- Added JavaDoc documentation in CacheUtils.zip() explaining that timestamps
  affect cache checksums when preservation is enabled
- Behavior is analogous to Git's inclusion of file mode in tree hashes

This addresses the architectural correctness concern that metadata preserved
during cache restoration should also be part of the cache key computation.
@cowwoc cowwoc force-pushed the fix-timestamp-preservation branch from 467c204 to d9fc24f Compare October 6, 2025 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@AlexanderAshitkin AlexanderAshitkin AlexanderAshitkin approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@cowwoc @olamy @AlexanderAshitkin