Add concurrent steps support to workflow definitions#191
Merged
Conversation
Enable workflows to specify nested arrays of step IDs to indicate steps that can be executed in parallel. The meta template renders concurrent step groups as "Concurrent Steps" with numbered "Background Task" items. Changes: - Schema: Added STEP_ID_SCHEMA, CONCURRENT_STEPS_SCHEMA, WORKFLOW_STEP_ENTRY_SCHEMA - Workflow steps accept either string (sequential) or array (concurrent) - Single-item arrays indicate steps with multiple parallel instances - Parser: Added WorkflowStepEntry dataclass for sequential/concurrent groups - Updated Workflow to use step_entries with backward-compatible steps property - Added get_step_entry_position_in_workflow() and get_concurrent_step_info() - Generator: Build step_entries with concurrency info in workflow context - Template: Render concurrent steps with "Background Task N" formatting - Tests: Added test fixture and 9 new unit tests - Docs: Updated architecture.md and CHANGELOG.md https://claude.ai/code/session_011qMp5U4AFQLMbJQvCE8Ck4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for concurrent (parallel) step execution in workflow definitions, allowing workflows to specify which steps can run in parallel while maintaining full backward compatibility with existing sequential workflows.
Key Changes
Parser & Data Model
WorkflowStepEntrydataclass to represent workflow step entries that can be either sequential (single step) or concurrent (multiple steps)Workflowclass to usestep_entriesinternally while maintaining a backward-compatiblestepsproperty that flattens all step IDsWorkflow:get_step_entry_for_step()andget_entry_index_for_step()Job Definition API
get_step_entry_position_in_workflow()method to get entry-based position of a step (useful for concurrent groups where multiple steps share the same position)get_concurrent_step_info()method to get a step's position within its concurrent group (returns tuple of position and group size)Schema & Validation
[fetch_campaign_data]for per-campaign execution)[task_a, task_b, task_c])Code Generation
_build_meta_skill_context()to includestep_entrieswith detailed concurrency informationDocumentation & Examples
concurrent_steps_jobdemonstrating a workflow with parallel research phaseBackward Compatibility
workflow.stepsproperty provides flattened list of all step IDs for backward compatibilityTesting
step_entriesstructure and backward-compatiblestepspropertyhttps://claude.ai/code/session_011qMp5U4AFQLMbJQvCE8Ck4