feature: implement chained integration branches for parallel groups

[BasselBlal]

Description

This PR implements an advanced "chained" integration strategy for parallel task groups (batches) in the execution engine. Previously, tasks in parallel batches were isolated from each other until a final merge phase, which meant that a task in Group 2 could not see or build upon files created by Group 1.

With this change, parallel.ts now creates an integration branch after each batch completes. This integration branch:

1. Is created from the current chain's base.
2. Merges all successful branches from the current batch.
3. Becomes the base branch for the next batch.

This ensures that tasks are executed in a sequential-parallel hybrid flow:
Group 0 -> Integration 1 -> Group 1 (sees Group 0 work) -> Integration 2 -> ... -> Final Merge

Key Changes

cli/src/execution/parallel.ts

• Introduced currentBaseBranch to track the tip of the integration chain.
• Namespacing: Integration branches are now prefixed with the sanitized base branch name (e.g., ralphy/feature-database-integration-group-1) to prevent collisions when running multiple instances on different features.
• Robust Merge Logic:
  • Branches successfully integrated are removed from the pending list (completedBranches).
  • The final merge phase combines the integration chain tip (if it moved) AND any remaining pending branches (e.g., from failed integration attempts).
  • Fail-Stop Protection: If creating an integration branch fails (breaking the dependency chain), execution stops immediately.

cli/src/git/merge.ts

• Updated createIntegrationBranch to accept an optional prefix.
• Sanitizes the prefix (replacing non-alphanumeric characters with dashes) for safe branch naming.

Related Issues / Based on

• Addresses the need for sequential context in parallel executions (PR #14 logic ported to CLI).

[vercel]

@BasselBlal is attempting to deploy a commit to the Goshen Labs Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Overview

Greptile Summary

This PR implements chained integration branches for parallel task execution, enabling sequential-parallel hybrid workflows where later task groups can build upon work from earlier groups. The implementation creates intermediate integration branches after each batch completes, merges successful branches into them, and uses each integration branch as the base for the next batch.

Key improvements:

• Tracks currentBaseBranch to maintain the integration chain tip
• Prefixes integration branches with sanitized base branch name to prevent collisions
• Removes merged branches from completedBranches after integration to prevent duplicate merges
• Implements fail-fast behavior when integration fails to preserve dependency chains
• Properly cleans up integration branches in finally block
• Final merge phase combines integration chain tip and any remaining pending branches

Architecture:
The flow creates a chain: Group 0 → Integration 1 (base for Group 1) → Integration 2 (base for Group 2) → Final Merge to main

The implementation addresses previous review comments by ensuring branches are properly tracked and removed after integration, and stopping execution if integration fails to prevent broken dependency chains.

Confidence Score: 4/5

• This PR is safe to merge with proper testing to verify the integration chain behavior
• The implementation properly addresses previous review feedback including branch deduplication and fail-fast behavior. The core logic for chained integration is sound, though the complexity of the parallel execution flow warrants thorough testing to ensure integration branches work correctly across different scenarios
• No files require special attention - both changes are well-structured

Important Files Changed

Filename	Overview
cli/src/execution/parallel.ts	Implements chained integration branches for parallel task groups with proper cleanup and fail-fast behavior
cli/src/git/merge.ts	Adds prefix parameter to createIntegrationBranch with proper sanitization for branch namespacing

Sequence Diagram

sequenceDiagram
    participant Main as Main Branch
    participant Exec as Parallel Executor
    participant B0_1 as Batch 0 Task 1
    participant B0_2 as Batch 0 Task 2
    participant Int1 as Integration Branch 1
    participant B1_1 as Batch 1 Task 1
    participant B1_2 as Batch 1 Task 2
    participant Int2 as Integration Branch 2
    
    Note over Exec: currentBaseBranch = main
    Note over Exec: iteration = 1
    
    Exec->>B0_1: Run task (base: main)
    Exec->>B0_2: Run task (base: main)
    B0_1-->>Exec: Creates branch-0-1
    B0_2-->>Exec: Creates branch-0-2
    
    Note over Exec: Batch 1 complete
    
    Exec->>Int1: Create integration branch from main
    Exec->>Int1: Merge branch-0-1, branch-0-2
    Note over Exec: currentBaseBranch = integration-1
    Note over Exec: Remove branch-0-1, branch-0-2 from completedBranches
    
    Note over Exec: iteration = 2
    
    Exec->>B1_1: Run task (base: integration-1)
    Exec->>B1_2: Run task (base: integration-1)
    B1_1-->>Exec: Creates branch-1-1
    B1_2-->>Exec: Creates branch-1-2
    
    Note over Exec: Batch 2 complete
    
    Exec->>Int2: Create integration branch from integration-1
    Exec->>Int2: Merge branch-1-1, branch-1-2
    Note over Exec: currentBaseBranch = integration-2
    Note over Exec: Remove branch-1-1, branch-1-2 from completedBranches
    
    Note over Exec: Final merge phase
    
    Exec->>Main: Merge integration-2 into main
    Exec->>Exec: Cleanup integration-1, integration-2

Loading

[greptile-apps]

_{1 file reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

[dosubot]

Related Documentation

2 document(s) may need updating based on files changed in this PR:

Goshen Labs's Space

Copilot AI Engine Integration

View Suggested Changes

@@ -59,6 +59,13 @@
 ---
 
 ### Design Decisions
+
+#### Parallel Execution: Chained Integration Branches
+- In parallel mode, ralphy now uses a **chained integration branch** strategy for executing task groups (batches). After each batch of parallel tasks completes, their results are merged into a dedicated integration branch, which is then used as the base for the next batch. This ensures that later batches can see and build upon the work of earlier batches, providing a sequential-parallel hybrid execution flow.
+- Integration branches are namespaced using the sanitized base branch name (e.g., `ralphy/feature-database-integration-group-1`) to avoid collisions when running multiple instances on different features.
+- If any integration branch creation fails, execution stops immediately to preserve the dependency chain.
+- At the end of parallel execution, the final integration branch (containing all merged work) and any remaining successful branches are merged back into the original base branch.
+- This approach ensures that tasks in later groups have access to files and changes produced by earlier groups, improving reliability and correctness for workflows that require sequential context.
 
 #### Parallel Execution Reliability and Fallback
 - The CLI checks if git worktrees are available and usable. If not, or if worktree creation fails for any reason, it automatically falls back to sandbox mode for parallel execution. This ensures robust operation across different repository configurations and platforms.

[Accept] [Decline]

Parallel Execution with Group Dependencies

View Suggested Changes

@@ -21,7 +21,7 @@
   ]
 }
 ```
-Here, "Create User model" and "Create Post model" run in parallel (group 1), and "Add relationships" runs after both are complete (group 2) [source](https://github.com/michaelshimeles/ralphy/blob/fc2df589969b5fe16d31eccb4e7ff91314e31776/README.md#L148-L185). Parallel group handling works identically for both YAML and JSON PRD files.
+Here, "Create User model" and "Create Post model" run in parallel (group 1), and "Add relationships" runs after both are complete (group 2). Parallel group handling works identically for both YAML and JSON PRD files.
 
 ### The Branching Conflict Problem
 Originally, each parallel task branched directly from the base branch (e.g., `main`). This approach caused two major issues:
@@ -39,30 +39,32 @@
 ```
 All tasks branch from `main`, so `task-3` does not see changes from `task-1` or `task-2`.
 
-### Solution: Integration Branches
-To resolve these issues, Ralphy creates an integration branch after each parallel group completes and merges its branches. The integration branch aggregates all changes from the completed group. Subsequent groups then branch from the latest integration branch, ensuring they see all prior commits and reducing merge conflicts.
+### Solution: Chained Integration Branches
+To resolve these issues, Ralphy now uses a **chained integration branch** strategy for parallel groups. After each parallel group (batch) completes, an integration branch is created from the current tip of the integration chain. All successful branches from the batch are merged into this integration branch. This integration branch then becomes the base for the next group, ensuring that each subsequent group sees all changes from previous groups.
 
-The integration branch is named using the convention `ralphy/integration-group-{groupNum}`. After merging all branches from a group, Ralphy creates the integration branch from the updated base (which now includes all merged changes) [source](https://github.com/michaelshimeles/ralphy/blob/fc2df589969b5fe16d31eccb4e7ff91314e31776/cli/src/git/merge.ts#L16-L172).
+Integration branches are now named using a namespaced convention based on the original base branch, e.g., `ralphy/feature-database-integration-group-1`. This prevents branch name collisions when running multiple instances on different base branches.
 
-#### Diagram: Branch Relationships After the Fix
+#### Diagram: Chained Integration Branch Relationships
 ```mermaid
 graph TD
     A["main (base branch)"]
     B["task-1 (group 1)"] --> A
     C["task-2 (group 1)"] --> A
-    E["integration-group-1"] --> B
+    E["ralphy/feature-database-integration-group-1"] --> B
     E --> C
     D["task-3 (group 2)"] --> E
+    F["ralphy/feature-database-integration-group-2"] --> D
 ```
-`task-3` now branches from `integration-group-1`, which contains all changes from `task-1` and `task-2`.
+`task-3` now branches from the integration branch created after group 1, which contains all changes from `task-1` and `task-2`. Each integration branch is chained, so every group sees the cumulative work of all previous groups.
 
 ### Step-by-Step Example
-1. **Group 1 Execution:** Tasks `task-1` and `task-2` (group 1) each run in their own branch, both branched from `main`.
-2. **Group 1 Merge:** After completion, both branches are merged into `main`. An integration branch `ralphy/integration-group-1` is created from the updated `main`.
-3. **Group 2 Execution:** Task `task-3` (group 2) branches from `ralphy/integration-group-1`, inheriting all changes from group 1.
-4. **Group 2 Merge:** After completion, `task-3` is merged into `ralphy/integration-group-1` (or back into `main` if it is the final group), and a new integration branch is created if more groups remain.
+1. **Group 1 Execution:** Tasks `task-1` and `task-2` (group 1) each run in their own branch, both branched from the current base (initially `main`).
+2. **Group 1 Integration:** After completion, both branches are merged into a new integration branch, e.g., `ralphy/feature-database-integration-group-1`, created from the current base.
+3. **Group 2 Execution:** Task `task-3` (group 2) branches from `ralphy/feature-database-integration-group-1`, inheriting all changes from group 1.
+4. **Group 2 Integration:** After completion, `task-3` is merged into a new integration branch, e.g., `ralphy/feature-database-integration-group-2`, which becomes the base for any subsequent groups.
+5. **Final Merge:** After all groups are processed, the tip of the integration chain and any remaining branches are merged back into the original base branch.
 
-This workflow ensures that each group’s tasks have full visibility of all prior group commits, and that merge conflicts are minimized because overlapping changes are integrated before new work begins [source](https://github.com/michaelshimeles/ralphy/blob/fc2df589969b5fe16d31eccb4e7ff91314e31776/cli/src/git/merge.ts#L16-L172).
+This workflow ensures that each group’s tasks have full visibility of all prior group commits, and that merge conflicts are minimized because overlapping changes are integrated before new work begins. The namespaced integration branch naming also prevents collisions across different feature branches or concurrent runs.
 
 ### Summary
-By sequencing parallel task execution with integration branches, Ralphy guarantees that dependencies between groups are respected, prior changes are visible to subsequent work, and merge conflicts are significantly reduced. This approach is essential for reliable, scalable automation of complex, multi-step development workflows.
+By sequencing parallel task execution with chained, namespaced integration branches, Ralphy guarantees that dependencies between groups are respected, prior changes are visible to subsequent work, and merge conflicts are significantly reduced. This approach is essential for reliable, scalable automation of complex, multi-step development workflows.

[Accept] [Decline]

Note: You must be authenticated to accept/decline updates.

^{How did I do? Any feedback?}  

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[michaelshimeles]

@BasselBlal will test this