Skip to content

fix(storage): replace .returningAll() with insert+select for SQLite compat#2541

Closed
viktormarinho wants to merge 2 commits intomainfrom
feat/sqlite-compat
Closed

fix(storage): replace .returningAll() with insert+select for SQLite compat#2541
viktormarinho wants to merge 2 commits intomainfrom
feat/sqlite-compat

Conversation

@viktormarinho
Copy link
Contributor

@viktormarinho viktormarinho commented Mar 2, 2026

Summary

  • kysely-bun-worker (SQLite) does not support the RETURNING clause
  • Replace all INSERT/UPDATE … RETURNING * with a separate follow-up SELECT so storage operations work on both SQLite and PostgreSQL

Files changed:

  • apps/mesh/src/storage/threads.ts
  • packages/mesh-plugin-workflows/server/storage/workflow-collection.ts
  • packages/mesh-plugin-workflows/server/storage/workflow-execution.ts

Test plan

  • Run existing workflow engine tests: bun test packages/mesh-plugin-workflows
  • Verify thread creation works end-to-end against SQLite (bun run dev)
  • Verify workflow create/update/delete/cancel/resume still work against PostgreSQL

Known issues flagged in review

The SELECT+UPDATE pattern introduces TOCTOU windows that the original atomic RETURNING prevented. Specifically: cancelExecution, claimExecution, createStepResult, and updateExecution can return incorrect results under concurrent load. These regressions should be addressed with numUpdatedRows checks in a follow-up — tracked separately to keep this PR scoped to the compat fix.

Split from #2528

🤖 Generated with Claude Code


Summary by cubic

Restored atomic INSERT/UPDATE … RETURNING queries across storage to fix concurrency issues and confirm compatibility with SQLite 3.35+ and PostgreSQL.

  • Bug Fixes
    • Reverted insert/update + select back to RETURNING in threads, workflow collection, and workflow execution.
    • Fixed concurrent correctness: step-result claiming and conditional updates now respect affected rows atomically.
    • Confirmed BunWorkerDialect supports RETURNING; no SQLite fallback needed.

Written for commit d8856d4. Summary will update on new commits.

…ompat

kysely-bun-worker (SQLite) does not support the RETURNING clause.
Replace all INSERT/UPDATE … RETURNING usages with a separate follow-up
SELECT so the codebase runs on both SQLite and PostgreSQL.

Affected:
- apps/mesh/src/storage/threads.ts
- packages/mesh-plugin-workflows/server/storage/workflow-collection.ts
- packages/mesh-plugin-workflows/server/storage/workflow-execution.ts

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 2, 2026

🧪 Benchmark

Should we run the Virtual MCP strategy benchmark for this PR?

React with 👍 to run the benchmark.

Reaction Action
👍 Run quick benchmark (10 & 128 tools)

Benchmark will run on the next push after you react.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 2, 2026

Release Options

Should a new version be published when this PR is merged?

React with an emoji to vote on the release type:

Reaction Type Next Version
👍 Prerelease 2.123.5-alpha.1
🎉 Patch 2.123.5
❤️ Minor 2.124.0
🚀 Major 3.0.0

Current version: 2.123.4

Deployment

  • Deploy to production (triggers ArgoCD sync after Docker image is published)

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/mesh-plugin-workflows/server/storage/workflow-execution.ts">

<violation number="1" location="packages/mesh-plugin-workflows/server/storage/workflow-execution.ts:275">
P2: cancelExecution always returns true even when the UPDATE modifies 0 rows, so callers may think an execution was cancelled when it was not. Return success based on the update result.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

… support it

The original PR replaced `INSERT/UPDATE ... RETURNING` with separate
SELECT-after-write statements under the assumption that BunWorkerDialect
doesn't support RETURNING. This assumption is incorrect:

- SQLite has supported RETURNING since 3.35.0 (March 2021)
- BunWorkerDialect passes the full test suite including the
  concurrent stress tests with the original RETURNING queries

The SELECT+write pattern broke concurrent correctness:
- createStepResult: pre-check SELECT + INSERT + final SELECT meant
  all concurrent handlers passed the pre-check before any INSERT
  committed, causing all to return non-null (every handler thought it
  claimed the step), leading to exponential completed-event cascades
- Other mutations (cancelExecution, claimExecution, updateExecution)
  lost atomicity — returning true/non-null even when the conditional
  UPDATE affected 0 rows

Restore the original atomic RETURNING queries for all three files.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@viktormarinho
Copy link
Contributor Author

does not make sense

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/mesh-plugin-workflows/server/storage/workflow-execution.ts">

<violation number="1">
P1: These new `.returningAll()` calls reintroduce RETURNING, which the bun/SQLite dialect used here doesn’t reliably support. That means writes can succeed but these methods return `null`/`false`, and SQLite compatibility regresses. Replace with the intended insert/update + follow‑up `SELECT` pattern to fetch the row.</violation>
</file>

<file name="apps/mesh/src/storage/threads.ts">

<violation number="1">
P1: This reintroduces `RETURNING` in thread creation, which SQLite (kysely-bun-worker) does not support. Thread creation will fail on SQLite; use insert + follow-up select instead.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant