add max_handlers_open option for httpc profile #9712
Conversation
Whaileee
added
testing
CT Test Results 2 files 22 suites 25m 9s ⏱️ Results for commit 339c402. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts
// Erlang/OTP Github Action Bot |
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
2 times, most recently
from
2a8a4ad to
4e3f24f
Compare
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
from
4e3f24f to
92f73a1
Compare
|
@github-actions disable-cache |
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
2 times, most recently
from
465413d to
5e33cc0
Compare
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
2 times, most recently
from
7acd444 to
2e53f7f
Compare
|
solves #8841 |
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
from
2e53f7f to
f1603f9
Compare
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
from
bd74dab to
2ca8d94
Compare
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
2 times, most recently
from
6b02113 to
2974eac
Compare
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
3 times, most recently
from
f143915 to
d8a005b
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a new option max_connections_open for the HTTP client profile to limit the maximum number of concurrent HTTP handlers that can be open simultaneously. This feature helps prevent bandwidth exhaustion when multiple requests are processed concurrently and remote servers close connections before requests complete.
- Adds
max_connections_openoption to limit concurrent HTTP handlers - Implements request queuing mechanism when handler limit is reached
- Adds comprehensive test coverage for high-load scenarios with both HTTP and HTTPS
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
system/doc/general_info/upcoming_incompatibilities.md |
Documents the new option as an upcoming feature |
lib/inets/src/http_client/httpc_internal.hrl |
Adds max_connections_open field to options record |
lib/inets/src/http_client/httpc.erl |
Implements option validation and documentation |
lib/inets/src/http_client/httpc_manager.erl |
Core logic for handler limiting and request queuing |
lib/inets/test/httpc_SUITE.erl |
Comprehensive test suite for high-load scenarios |
lib/inets/test/http_test_lib.erl |
Minor test infrastructure update |
lib/inets/src/http_client/httpc_response.erl |
Cache renewal marker |
lib/inets/src/http_client/httpc_request.erl |
Cache renewal marker |
lib/inets/src/http_client/httpc_handler.erl |
Cache renewal marker |
lib/inets/src/http_client/httpc_cookie.erl |
Cache renewal marker |
Comments suppressed due to low confidence (2)
system/doc/general_info/upcoming_incompatibilities.md:259
- The option name in documentation should match the actual implementation. Use
max_connections_openconsistently throughout.
New option in Inets' http client `httpc:set_options([{max_connections_open, N}])`
lib/inets/src/http_client/httpc.erl:1649
- [nitpick] Function name is inconsistent with the option name
max_connections_open. Consider renaming tovalidate_max_sessions_max_handlers_openor updating the option name for consistency.
validate_max_sessions_max_connections_open(MaxSessions, MaxConnectionsOpen)
d8a005b to
6a642d8
Compare
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
3 times, most recently
from
124c4d4 to
7a7af8f
Compare
lib/inets/test/httpc_SUITE.erl
Outdated
| @@ -312,7 +323,8 @@ do_init_per_group(Group, Config0) -> | |||
| _ -> | |||
| Config0 | |||
| end, | |||
| Config = proplists:delete(port, Config1), | |||
| Config = proplists:delete(port, Config1), | |||
lib/inets/test/httpc_SUITE.erl
Outdated
| Config = [{iterations, 10} | Config0], | ||
| "ERIERL-937, OTP-19587). Transferred data size needs to be significant," | ||
| "so that socket would close, in the middle of a transfer," | ||
| "without max_connections_open option set"}, {timetrap, timer:minutes(15)}]. |
There was a problem hiding this comment.
I'm a bit confused with this testcase name and description.
It is with max_connections_open but comment refers to without case.
Is this remote socket close reproduced with this test code?
Maybe explain better and separately: what the problem was? what the testcase is doing?
There was a problem hiding this comment.
The testcase checks proper behavior of the option, and it will fail if the option doesn't work, and this scenario will also fail if the option is not set. That was my intention in the description
| @@ -2140,8 +2203,9 @@ queue_check() -> | |||
| end. | |||
|
|
|||
| request(Config) -> | |||
| Request = {url(group_name(Config), "/httpc_SUITE/foo", Config), []}, | |||
| httpc:request(get, Request, [],[{sync,true}, {body_format,binary}], ?profile(Config)). | |||
| Id = integer_to_list(?config(client_id, Config)), | |||
Whaileee
force-pushed
the
whaileee/inets/httpc/socket_closed_remotely/GH-8841/OTP-19587
branch
from
7a7af8f to
339c402
Compare
No description provided.