Skip to content

Commit 6812e0c

Browse files
jsbattigclaude
andcommitted
Implement Copy-on-Write (CoW) cloning support and migration middleware
Major features added: - Migration middleware system with real-time detection and state tracking - @requires_qdrant_access decorator applied to 8 CLI commands - force-flush command for CoW operations with multi-filesystem examples - Comprehensive E2E test for complete CoW clone workflow - Async-safe operations and deadlock prevention - Enhanced CLI help with BTRFS, ZFS, and XFS examples - Full automation integration with configurable CoW test inclusion Technical improvements: - Snapshot API integration for reliable data flushing - Project isolation through local storage directories - Robust error handling for test environments - Aggressive setup pattern for better test performance Version bump: 1.0.0.7 → 1.1.0.0 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent aa30767 commit 6812e0c

15 files changed

+2529
-8
lines changed

.github/workflows/main.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,11 @@ jobs:
8383
--ignore=tests/test_chunker_docstring_fix.py \
8484
--ignore=tests/test_prompt_formatting_issues.py \
8585
--ignore=tests/test_parallel_voyage_performance.py \
86+
--ignore=tests/test_branch_transition_logic_fix.py \
87+
--ignore=tests/test_compare_search_methods.py \
88+
--ignore=tests/test_debug_branch_isolation.py \
89+
--ignore=tests/test_search_with_branch_topology_fix.py \
90+
--ignore=tests/test_cow_clone_e2e_full_automation.py \
8691
-m "not slow and not e2e and not real_api" \
8792
--cov=src/code_indexer --cov-report=xml --cov-report=term
8893

CLAUDE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,4 +51,5 @@
5151
- When working on improvements for the smart indexer, always consider the --reconcile function (non git-aware) and ensure consistency across both indexing processes. Treat the --reconcile aspect as equally important, maintaining feature parity and functionality, except for semantically specific module considerations.
5252
- NEVER, EVER, remove functionality related to our enhanced processing of git projects. The git-awarness aspects, how we optimize processing branches, and keeping track of relationships, deduplication of indexing is what make this project unique. If you ever go into a refactoring rabbit hole and you will start removing functionality to that enables this capability you must stop, immediately, and ask if that's the true intent of the work you been asked to do.
5353
- When working on fixing quick feedback unit tests, or fast tests, always use ./ci-github.sh. This shell file is specifically tuned to run test that run fast, so they can be run efficiently from within Claude Code as a first layer of protection ensuring our tests pass and we didn't introduce regressions.
54-
- When indexing, progress reporting is done real-time, in a single line at the bottom, showing a progress bar, and right next to it we show speed metrics and current file being processed. Don't change this approach without confirmation from the user. This is how it is, and it should be for all indexing operations, we don't show feedback scrolling the console, EVER, NEVER, EVER. Ask for confirmation if you are about to change this behavior. If the user ask you to change it, ask question, confirm the user is sure it wants to remove the single line, fixed to the bottom, progress bar, speed and currently file being processed.
54+
- When indexing, progress reporting is done real-time, in a single line at the bottom, showing a progress bar, and right next to it we show speed metrics and current file being processed. Don't change this approach without confirmation from the user. This is how it is, and it should be for all indexing operations, we don't show feedback scrolling the console, EVER, NEVER, EVER. Ask for confirmation if you are about to change this behavior. If the user ask you to change it, ask question, confirm the user is sure it wants to remove the single line, fixed to the bottom, progress bar, speed and currently file being processed.
55+
- When asking to bump version, you will always check readme in case there's references to the version number, and you will always updte the release notes files with the latest changes

RELEASE_NOTES.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,42 @@
11
# Code Indexer Release Notes
22

3+
## Version 1.1.0.0 (2025-01-05)
4+
5+
### 🚀 Major Feature: Copy-on-Write (CoW) Clone Support
6+
7+
#### **Migration Middleware System**
8+
- **Real-time Migration Detection**: Automatically detects when indexed projects are CoW clones and handles migration gracefully
9+
- **Migration State Tracking**: Persistent state tracking across system restarts with async-safe operations
10+
- **Decorator Integration**: `@requires_qdrant_access` decorator automatically applied to all Qdrant-dependent CLI commands
11+
- **Deadlock Prevention**: Fixed critical async deadlock in migration state management
12+
13+
#### **CoW Operations Support**
14+
- **Force-flush Command**: New `force-flush` command ensures data consistency before CoW cloning
15+
- **Multi-filesystem Support**: Comprehensive examples for BTRFS, ZFS, and XFS filesystems
16+
- **Project Isolation**: Proper handling of local storage directories for cloned projects
17+
- **Configuration Migration**: Automatic config fixing for cloned projects
18+
19+
#### **Enhanced CLI Integration**
20+
- **8 Commands Protected**: Applied migration middleware to index, query, claude, watch, status, optimize, schema, and start commands
21+
- **Comprehensive Help**: Added detailed CoW cloning examples and workflow documentation
22+
- **Error Handling**: Robust error handling for missing directories and test environments
23+
24+
#### **Testing Infrastructure**
25+
- **E2E CoW Test**: Comprehensive 10-phase end-to-end test for complete CoW clone workflow
26+
- **Aggressive Setup Pattern**: Tests use shared services for better performance while maintaining isolation
27+
- **Full Automation Integration**: CoW tests included in full-automation.sh by default
28+
29+
### 📊 Technical Implementation
30+
- **Async-safe Operations**: All migration operations designed for concurrent access
31+
- **Snapshot API Integration**: Uses Qdrant snapshot API for reliable data flushing
32+
- **Project Path Flexibility**: Supports both current directory and explicit project path specifications
33+
- **Cleanup Management**: Automatic cleanup of temporary snapshots and migration artifacts
34+
35+
### 🔧 Developer Experience
36+
- **Example Workflows**: Complete documentation of CoW cloning processes for different filesystems
37+
- **Integration Testing**: Comprehensive test coverage for migration scenarios
38+
- **Performance Optimized**: Minimal overhead for non-CoW operations
39+
340
## Version 1.0.0.7 (2025-01-04)
441

542
### 🧪 Testing Infrastructure Improvements

ci-github.sh

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,7 @@ if pytest tests/ \
130130
--ignore=tests/test_compare_search_methods.py \
131131
--ignore=tests/test_debug_branch_isolation.py \
132132
--ignore=tests/test_search_with_branch_topology_fix.py \
133+
--ignore=tests/test_cow_clone_e2e_full_automation.py \
133134
-m "not slow and not e2e and not real_api" \
134135
--cov=src/code_indexer --cov-report=xml --cov-report=term; then
135136
print_success "Unit tests passed"
@@ -167,6 +168,7 @@ echo " • Docker Compose validation tests (require Docker)"
167168
echo " • Idempotent setup tests (require Docker services)"
168169
echo " • Branch topology E2E tests (require Git and indexing services)"
169170
echo " • Schema migration E2E tests (require Qdrant and embedding services)"
171+
echo " • CoW clone E2E tests (require CoW filesystem and real services)"
170172
echo " • Optimized example tests (require Docker and VoyageAI API)"
171173
echo " • Comprehensive git workflow tests (require full service stack)"
172174
echo " • Git-aware watch E2E tests (require service dependencies)"

full-automation.sh

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -89,11 +89,26 @@ fi
8989

9090
# 6. Run tests with coverage
9191
print_step "Running tests with coverage"
92-
if pytest tests/ --cov=src/code_indexer --cov-report=xml --cov-report=term; then
93-
print_success "All tests passed"
92+
93+
# Check if COW_CLONE_E2E_TESTS is set to exclude the slow CoW clone tests
94+
if [[ "${COW_CLONE_E2E_TESTS:-}" == "false" ]]; then
95+
print_warning "Skipping CoW clone E2E tests (COW_CLONE_E2E_TESTS=false)"
96+
# Exclude the specific CoW clone test to speed up the run
97+
if pytest tests/ --ignore=tests/test_cow_clone_e2e_full_automation.py --cov=src/code_indexer --cov-report=xml --cov-report=term; then
98+
print_success "All tests passed (excluding CoW clone E2E test)"
99+
else
100+
print_error "Tests failed"
101+
exit 1
102+
fi
94103
else
95-
print_error "Tests failed"
96-
exit 1
104+
print_step "Including CoW clone E2E tests (may take several minutes)"
105+
print_step "To skip them: export COW_CLONE_E2E_TESTS=false"
106+
if pytest tests/ --cov=src/code_indexer --cov-report=xml --cov-report=term; then
107+
print_success "All tests passed (including CoW clone E2E)"
108+
else
109+
print_error "Tests failed (including CoW clone E2E)"
110+
exit 1
111+
fi
97112
fi
98113

99114
# 7. Build package

0 commit comments

Comments
 (0)