Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance: use first h2 frame header detect instead of read first frame from whole payload #705

Merged
merged 1 commit into from
Dec 27, 2024

Conversation

yuweizzz
Copy link
Contributor

fix #700

the h2 response detect will read whole payload, which may cost too much memory.

@cfc4n
Copy link
Member

cfc4n commented Dec 25, 2024

@chilli13 Can you test the performance?

pkg/event_processor/http2_request.go Show resolved Hide resolved
pkg/event_processor/http2_response.go Show resolved Hide resolved
pkg/event_processor/http2_response.go Show resolved Hide resolved
@chilli13
Copy link

@chilli13 Can you test the performance?

The optimization effect on memory consumption is quite obvious.
50MBps http2 traffic
taskset -c 8 ./bin/ecapture tls

before optimism (master)

=====================
Process ecapture resource consumption statistics
CPU Average: 128.20%
CPU Max: 152.00%
Memory Average: 629.41 MB
Memory Max: 694.56 MB
=====================

after optimism (h2-patch)

=====================
Process ecapture resource consumption statistics
CPU Average: 72.60%
CPU Max: 81.00%
Memory Average: 389.11 MB
Memory Max: 508.99 MB
=====================

@yuweizzz
Copy link
Contributor Author

Why the data[4] "works":

data[4] actually read the frame flags, look back for flags type:

const (
	// Data Frame
	FlagDataEndStream Flags = 0x1
	FlagDataPadded    Flags = 0x8

	// Headers Frame
	FlagHeadersEndStream  Flags = 0x1
	FlagHeadersEndHeaders Flags = 0x4
	FlagHeadersPadded     Flags = 0x8
	FlagHeadersPriority   Flags = 0x20

	// Settings Frame
	FlagSettingsAck Flags = 0x1

	// Ping Frame
	FlagPingAck Flags = 0x1

	// Continuation Frame
	FlagContinuationEndHeaders Flags = 0x4

	FlagPushPromiseEndHeaders Flags = 0x4
	FlagPushPromisePadded     Flags = 0x8
)

those value uint32(data[4])>>7 will always pass the check.

@cfc4n cfc4n added the improve label Dec 27, 2024
Copy link
Member

@cfc4n cfc4n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks.

@cfc4n cfc4n merged commit 252f1c2 into gojue:master Dec 27, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

performance issue: (*HTTP2Response).detect cost too much memory
3 participants