Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test ingest_benchmark with different stripe size and also PostgreSQL version 17 #10510

Merged
merged 1 commit into from
Jan 27, 2025

Conversation

Bodobolero
Copy link
Contributor

@Bodobolero Bodobolero commented Jan 27, 2025

We want to verify if pageserver stripe size has an impact on ingest performance.
We want to verify if ingest performance has improved or regressed with postgres version 17.

Summary of changes

  • Allow to create new project with different postgres versions
  • allow to pre-shard new project with different stripe sizes instead of relying on storage manager to shard_split the project once a threshold is exceeded

Replaces #10509

Test run https://github.com/neondatabase/neon/actions/runs/12986410381

@Bodobolero Bodobolero requested a review from a team as a code owner January 27, 2025 09:58
@erikgrinaker
Copy link
Contributor

We have two other changes deployed to staging that we need to do performance testing for (L0 backpressure and parallel reads). Changing the stripe size too would make it hard to measure the impact of each change in isolation. Should we hold off on this until we've stabilized staging a bit?

Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM.

@Bodobolero
Copy link
Contributor Author

Bodobolero commented Jan 27, 2025

We have two other changes deployed to staging that we need to do performance testing for (L0 backpressure and parallel reads). Changing the stripe size too would make it hard to measure the impact of each change in isolation. Should we hold off on this until we've stabilized staging a bit?

@erikgrinaker
The change still tests the new_project as before.
It just adds additional variants in the strategy.
We also will continue to report the non-presharded, default stripe size new_project in the weekly perf report

Copy link

github-actions bot commented Jan 27, 2025

7429 tests run: 7043 passed, 0 failed, 386 skipped (full report)


Flaky tests (8)

Postgres 17

Postgres 16

Postgres 14

Code coverage* (full report)

  • functions: 33.5% (8496 of 25347 functions)
  • lines: 49.3% (71488 of 144978 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
f2bfe5d at 2025-01-27T20:41:58.358Z :recycle:

@Bodobolero Bodobolero added this pull request to the merge queue Jan 27, 2025
Merged via the queue into main with commit c8fbbb9 Jan 27, 2025
94 of 105 checks passed
@Bodobolero Bodobolero deleted the bodobolero/ingest_benchmark_improvements branch January 27, 2025 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants