Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: add page_trace API for debugging #10293

Merged
merged 17 commits into from
Jan 15, 2025
Merged

pageserver: add page_trace API for debugging #10293

merged 17 commits into from
Jan 15, 2025

Conversation

jcsp
Copy link
Collaborator

@jcsp jcsp commented Jan 7, 2025

Problem

When a pageserver is receiving high rates of requests, we don't have a good way to efficiently discover what the client's access pattern is.

Closes: #10275

Summary of changes

  • Add /v1/tenant/x/timeline/y/page_trace?size_limit_bytes=...&time_limit_secs=... API, which returns a binary buffer.
  • Add pagectl page-trace tool to decode and analyze the output.

Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
pageserver/src/tenant/timeline.rs Outdated Show resolved Hide resolved
Copy link

github-actions bot commented Jan 7, 2025

7315 tests run: 6939 passed, 0 failed, 376 skipped (full report)


Flaky tests (3)

Postgres 17

Postgres 16

Code coverage* (full report)

  • functions: 33.7% (8422 of 24972 functions)
  • lines: 49.2% (70381 of 143061 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
6a3dd3a at 2025-01-15T17:05:26.344Z :recycle:

Copy link
Contributor

@problame problame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

I think this is safe to deploy, barring the check_permission problem.

Nits can be addressed in a follow-up.

pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
@erikgrinaker
Copy link
Contributor

erikgrinaker commented Jan 13, 2025

@problame I cleaned this up a bit, should be ready for final review.

@erikgrinaker erikgrinaker marked this pull request as ready for review January 13, 2025 17:15
@erikgrinaker erikgrinaker requested a review from a team as a code owner January 13, 2025 17:15
@erikgrinaker erikgrinaker requested review from skyzh and removed request for skyzh January 13, 2025 17:15
@erikgrinaker erikgrinaker requested a review from problame January 13, 2025 17:56
@erikgrinaker
Copy link
Contributor

@jcsp I can't request a review from you (since you're the author), but if you could give this an LGTM I can merge based on my approval.

pageserver/src/http/routes.rs Outdated Show resolved Hide resolved
@problame problame disabled auto-merge January 15, 2025 15:28
@erikgrinaker erikgrinaker requested a review from problame January 15, 2025 16:27
@problame problame enabled auto-merge January 15, 2025 17:36
@problame problame added this pull request to the merge queue Jan 15, 2025
Merged via the queue into main with commit fb0e2ac Jan 15, 2025
84 of 88 checks passed
@problame problame deleted the jcsp/pagetrace branch January 15, 2025 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Access pattern observation in keyspace ("pagetrace")
3 participants