Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically set GOMAXPROCS according to available CPU quota #2831

Closed
wants to merge 1 commit into from

Conversation

raptorsun
Copy link
Contributor

This PR allows node_exporter to automatically set GOMAXPROCS according to available CPU quota if --runtime.gomaxprocs < 1, using the go.uber.org/automaxprocs package.
This is useful for running node_exporter on containerized environments with CPU resource limits.
This makes the node_exporter take advantage of its allocated CPU quota while avoiding being throttled due to over consumption of CPU burst budgets.

Here are the logs showing how it behavior under various configurations:

  • When --runtime.gomaxprocs is set > 1, GOMAXPROCS takes its value regardless of the situation.
$docker run --cpu-period=10000 --cpu-quota=36000   node_exporter --collector.disable-defaults --log.level=debug --runtime.gomaxprocs=5
ts=2023-10-23T15:22:04.327Z caller=node_exporter.go:193 level=info msg="Starting node_exporter" version="(version=1.6.1, branch=auto_maxprocs, revision=86ed8cdc6b1ba328e7ac4a9b1680129e9ab3d309)"
ts=2023-10-23T15:22:04.327Z caller=node_exporter.go:194 level=info msg="Build context" build_context="(go=go1.20.3, platform=linux/amd64, user=hsun@WORKPC-RH1, date=20231023-13:16:46, tags=netgo osusergo static_build)"
ts=2023-10-23T15:22:04.327Z caller=node_exporter.go:211 level=debug msg="Go MAXPROCS" procs=5
ts=2023-10-23T15:22:04.327Z caller=node_exporter.go:111 level=info msg="Enabled collectors"
ts=2023-10-23T15:22:04.328Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100
ts=2023-10-23T15:22:04.328Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100
  • When --runtime.gomaxprocs is set < 1, GOMAXPROCS is automatically set to optimal value.
$docker run --cpu-period=10000 --cpu-quota=36000   node_exporter --collector.disable-defaults -
-log.level=debug --runtime.gomaxprocs=0
ts=2023-10-23T15:21:03.075Z caller=node_exporter.go:193 level=info msg="Starting node_exporter" version="(version=1.6.1, branch=auto_maxprocs, revision=86ed8cdc6b1ba328e7ac4a9b1680129e9ab3d309)"
ts=2023-10-23T15:21:03.075Z caller=node_exporter.go:194 level=info msg="Build context" build_context="(go=go1.20.3, platform=linux/amd64, user=hsun@WORKPC-RH1, date=20231023-13:16:46, tags=netgo osusergo static_build)"
ts=2023-10-23T15:21:03.076Z caller=node_exporter.go:202 level=debug msg="maxprocs: Updating GOMAXPROCS=3: determined from CPU quota"
ts=2023-10-23T15:21:03.076Z caller=node_exporter.go:211 level=debug msg="Go MAXPROCS" procs=3
ts=2023-10-23T15:21:03.076Z caller=node_exporter.go:111 level=info msg="Enabled collectors"
ts=2023-10-23T15:21:03.077Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100
ts=2023-10-23T15:21:03.077Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100
    • When --runtime.gomaxprocs is not set , GOMAXPROCS is set to its default value 1.
$docker run --cpu-period=10000 --cpu-quota=36000   node_exporter --collector.disable-defaults --log.level=debug
ts=2023-10-23T15:21:35.110Z caller=node_exporter.go:193 level=info msg="Starting node_exporter" version="(version=1.6.1, branch=auto_maxprocs, revision=86ed8cdc6b1ba328e7ac4a9b1680129e9ab3d309)"
ts=2023-10-23T15:21:35.111Z caller=node_exporter.go:194 level=info msg="Build context" build_context="(go=go1.20.3, platform=linux/amd64, user=hsun@WORKPC-RH1, date=20231023-13:16:46, tags=netgo osusergo static_build)"
ts=2023-10-23T15:21:35.111Z caller=node_exporter.go:211 level=debug msg="Go MAXPROCS" procs=1
ts=2023-10-23T15:21:35.111Z caller=node_exporter.go:111 level=info msg="Enabled collectors"
ts=2023-10-23T15:21:35.111Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100
ts=2023-10-23T15:21:35.111Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100

…ntime.gomaxprocs < 1.

Signed-off-by: Haoyu Sun <hasun@redhat.com>
@SuperQ
Copy link
Member

SuperQ commented Oct 23, 2023

You probably don't want to do this, as the exporter is running with 1 CPU to avoid kernel race conditions.

@SuperQ SuperQ closed this Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants