Benchmark Prepared statement and expected Improvement

[harshit-gangal]

Description

Added a benchmark test for prepared statements covering insert, select, update, and delete operations. Utilized this test to analyze heap allocations, revealing that approximately 20% of allocations are consumed in generating a query plan, which can be avoided during prepared statement execution.

goos: darwin
goarch: arm64
pkg: vitess.io/vitess/go/test/endtoend/preparestmt
cpu: Apple M1 Max
                      │ /Users/planetscale/benchmark/prepare-no-change.txt │
                      │                       sec/op                       │
PreparedStmt/Insert-4                                          490.9µ ± 2%
PreparedStmt/Select-4                                          305.8µ ± 0%
PreparedStmt/Update-4                                          342.2µ ± 2%
PreparedStmt/Delete-4                                          311.3µ ± 1%
geomean                                                        355.6µ

Implemented an optimization to bypass query parsing and normalization for already-prepared statements. Benchmark results demonstrated significant query execution performance gains:

• 6.40% improvement in query execution time for basic select queries
• 9.08% improvement in query execution time for join-based select queries
• 3.62% improvement in query execution time for delete queries
• No statistically significant change in query execution time for insert and update queries

The overall geometric mean improvement in query execution across all operations was 4.94%, with potential for further gains as query size increases.

goos: darwin
goarch: arm64
pkg: vitess.io/vitess/go/test/endtoend/preparestmt
cpu: Apple M1 Max
                                        │ nochange-2s-v3.txt │         prepared-2s-v3.txt         │
                                        │       sec/op       │    sec/op     vs base              │
PreparedStmt/Insert-8                            506.5µ ± 9%   490.1µ ± 14%       ~ (p=0.065 n=6)
PreparedStmt/Select-8                            306.7µ ± 5%   287.1µ ±  5%  -6.40% (p=0.002 n=6)
PreparedStmt/Join_Select:Simple_Route-8          300.5µ ± 4%   273.3µ ±  3%  -9.08% (p=0.002 n=6)
PreparedStmt/Update-8                            351.0µ ± 5%   343.2µ ±  3%       ~ (p=0.240 n=6)
PreparedStmt/Delete-8                            318.9µ ± 4%   307.3µ ±  4%  -3.62% (p=0.015 n=6)
geomean                                          349.6µ        332.4µ        -4.94%

Related Issue(s)

Checklist

• Tests were added or are not required
• Did the new or modified tests pass consistently locally and on CI?
• Documentation was added or is not required

[vitess-bot]

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

• Ensure that the Pull Request has a descriptive title.
• Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

• Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

• Apply the release notes (needs details) label if users need to know about this change.
• New features should be documented.
• There should be some code comments as to why things are implemented the way they are.
• There should be a comment at the top of each new or modified test to explain what the test does.

New flags

• Is this flag really necessary?
• Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

• Each item in Jobs should be named in order to mark it as required.
• If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

• Protobuf changes should be wire-compatible.
• Changes to _vt tables and RPCs need to be backward compatible.
• RPC changes should be compatible with vitess-operator
• If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
• vtctl command output order should be stable and awk-able.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@8fe04c5). Learn more about missing BASE report.
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #17449   +/-   ##
=======================================
  Coverage        ?   67.68%           
=======================================
  Files           ?     1584           
  Lines           ?   254505           
  Branches        ?        0           
=======================================
  Hits            ?   172268           
  Misses          ?    82237           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[harshit-gangal]

A follow-up PR will include the implementation and final performance improvement results.

Temporary changes from this commit were used to evaluate the potential improvement.

[frouioui]

How do you get the profiles you've pasted in this PR's description? Do you have to query vtgate's http endpoint during while the test is running?

[harshit-gangal]

Yes, that is correct.

[frouioui]

It could be worth adding a comment on how to get the profiles, so we don't have to look into how the next time.

[harshit-gangal]

The curl commands are pretty standard. I will add them up in the follow up PR.