-
Notifications
You must be signed in to change notification settings - Fork 6
Description
Summary
Currently, floxy implements saga pattern where failure of one parallel step causes all siblings to stop/rollback and marks workflow as failed. Need option for batch processing mode where parallel steps run independently.
Current Behavior
wf := floxy.NewBuilder("batch-job", 1).
Fork("process-items",
func(b *floxy.Builder) { b.Step("item1", "handler") },
func(b *floxy.Builder) { b.Step("item2", "handler") },
func(b *floxy.Builder) { b.Step("item3", "handler") },
).
Join("collect", floxy.JoinStrategyAll).
Build()If item2 fails:
item1anditem3are stopped (Fork) or rolled back (Parallel)- Workflow status =
failed - Cannot retrieve results from
item1anditem3
Expected Behavior
If item2 fails:
item1anditem3continue executionitem1status =completed,item3status =completed,item2status =failed- Workflow status =
partial(orpartial_success,completed_with_errors) - Can retrieve results from successful steps
Use Case
Batch processing: Running 5 parallel long-running tasks (e.g., installing Arch Linux on 5 machines, each takes ~1 hour). Each task is independent. If 2 installations fail, the 3 successful installations are still valuable and should be retrievable without full rollback.
What I've Tried
DLQEnabled: Stops all active steps, requires manual interventionJoinStrategyAny: Join step doesn't fail, but workflow still triggers rollbackParallelinstead ofFork: Siblings complete, but still performs rollback- No
OnFailurehandlers: Steps complete but workflow still rolls back and marks asfailed
None of these provide batch processing semantics where independent tasks can partially succeed.
Question
Is there a way to achieve this with current floxy features, or would this require a new mode/option? Happy to contribute a PR if this aligns with floxy's design goals.