-
-
Notifications
You must be signed in to change notification settings - Fork 33.6k
[3.14] gh-119452: Read/write CGI data using worker threads #142181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 3.14
Are you sure you want to change the base?
Conversation
This reads/writes data as available, making the CGI application responsible for managing any timeouts when receiving data from clients. Data is read in chunks of bounded size, and passed on immediately (except stderr, which is combined into a single message as before). This does need 3 threads. (As does process.communicate.)
|
@serhiy-storchaka See here. |
|
This approach has great advantages, I like it, but creating a thread is not cheap. This is also much bigger change, so we should evaluate all the pros and cons. Can the same |
Sure, but can you make this cheaper? Note that
On Windows, |
| # already closed? | ||
| pass | ||
| if self.command.lower() == "post" and nbytes > 0: | ||
| data = self.rfile.read(nbytes) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue was only around the memory allocation of an untrusted nbytes value here. introducing a thread seems unnecessary. There wasn't a resource exhaustion problem with this blocking for the issue at hand.
The memory-address-space-DoS consumption point is to only allocate an amount close in magnitude to the total data that actually arrives rather than the untrusted number up front. A simple loop reading in chunk increments as the earlier PR #119455 did, but without a select at all would handle the issue fine. (ie: just remove the select from the earlier pr entirely)
This reads/writes data as available, making the CGI application responsible for managing any timeouts when receiving data from clients.
Data is read in chunks of bounded size, and passed on immediately (except stderr, which is combined into a single message as before).
This does need 3 threads. (As does process.communicate.)