Skip to content

Feature Request: Support partial success for parallel steps (non-saga mode) #27

@sosiska

Description

@sosiska

Summary

Currently, floxy implements saga pattern where failure of one parallel step causes all siblings to stop/rollback and marks workflow as failed. Need option for batch processing mode where parallel steps run independently.

Current Behavior

wf := floxy.NewBuilder("batch-job", 1).
    Fork("process-items",
        func(b *floxy.Builder) { b.Step("item1", "handler") },
        func(b *floxy.Builder) { b.Step("item2", "handler") },
        func(b *floxy.Builder) { b.Step("item3", "handler") },
    ).
    Join("collect", floxy.JoinStrategyAll).
    Build()

If item2 fails:

  • item1 and item3 are stopped (Fork) or rolled back (Parallel)
  • Workflow status = failed
  • Cannot retrieve results from item1 and item3

Expected Behavior

If item2 fails:

  • item1 and item3 continue execution
  • item1 status = completed, item3 status = completed, item2 status = failed
  • Workflow status = partial (or partial_success, completed_with_errors)
  • Can retrieve results from successful steps

Use Case

Batch processing: Running 5 parallel long-running tasks (e.g., installing Arch Linux on 5 machines, each takes ~1 hour). Each task is independent. If 2 installations fail, the 3 successful installations are still valuable and should be retrievable without full rollback.

What I've Tried

  • DLQEnabled: Stops all active steps, requires manual intervention
  • JoinStrategyAny: Join step doesn't fail, but workflow still triggers rollback
  • Parallel instead of Fork: Siblings complete, but still performs rollback
  • No OnFailure handlers: Steps complete but workflow still rolls back and marks as failed

None of these provide batch processing semantics where independent tasks can partially succeed.

Question

Is there a way to achieve this with current floxy features, or would this require a new mode/option? Happy to contribute a PR if this aligns with floxy's design goals.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions