Skip to content

Optimize MySQL and PostgreSQL CI containers for test performance#9511

Open
awln-temporal wants to merge 1 commit intomainfrom
optimize-ci-db-perf
Open

Optimize MySQL and PostgreSQL CI containers for test performance#9511
awln-temporal wants to merge 1 commit intomainfrom
optimize-ci-db-perf

Conversation

@awln-temporal
Copy link
Contributor

Summary

  • Disable durability guarantees (fsync, doublewrite, binlog, full page writes) on MySQL and PostgreSQL CI containers
  • Mount data directories on tmpfs to eliminate disk I/O entirely
  • These settings only affect crash recovery (durability), not transaction isolation or consistency

MySQL flags added

  • innodb-flush-log-at-trx-commit=0 - don't fsync redo log on every commit
  • innodb-flush-method=nosync - skip fsync syscall
  • innodb-doublewrite=0 - skip doublewrite buffer
  • innodb-buffer-pool-size=512M - larger in-memory cache (up from 128M default)
  • innodb-log-buffer-size=64M - larger redo log buffer
  • sync-binlog=0 / skip-log-bin - disable binary logging
  • tmpfs at /var/lib/mysql (2GB)

PostgreSQL flags added

  • fsync=off - skip all fsync calls
  • synchronous_commit=off - don't wait for WAL flush on commit
  • full_page_writes=off - skip full page images in WAL
  • wal_level=minimal / max_wal_senders=0 - minimal WAL overhead
  • shared_buffers=256MB - larger shared memory cache
  • work_mem=16MB - more memory for sorts/joins
  • tmpfs at /var/lib/postgresql/data (2GB)

Test plan

  • Local A/B test with smoke tests (MySQL: 75s -> 71s, PostgreSQL: 72s -> 71s — modest gains expected locally due to macOS Docker VM and fast NVMe)
  • CI A/B comparison: compare this PR's job durations against recent main runs (baseline captured from runs 23058493365 and 23057581982)
  • Verify no test failures introduced by the config changes

Made with Cursor

Disable durability guarantees (fsync, doublewrite, binlog, full page
writes) and mount data directories on tmpfs for the CI-only MySQL and
PostgreSQL containers. These settings eliminate synchronous disk I/O on
every transaction commit, which is the primary bottleneck for
write-heavy functional tests on CI runners with network-attached
storage.

These flags only affect crash recovery (durability), not transaction
isolation or consistency. Since CI containers are never restarted after
a crash, the durability loss has zero practical impact.

Made-with: Cursor
@awln-temporal awln-temporal requested review from a team as code owners March 13, 2026 16:57
@awln-temporal awln-temporal added the test-all-dbs Request PR checks to test all databases label Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test-all-dbs Request PR checks to test all databases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant