Skip to content

Commit 166a4ea

Browse files
authored
Add note for 2 sockets platform (#330)
1 parent 99f9a32 commit 166a4ea

File tree

1 file changed

+17
-0
lines changed

1 file changed

+17
-0
lines changed

llm_bench/python/README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,23 @@ Add the option `--torch_compile_backend` with the desired backend: `pytorch` or
122122
python ./benchmark.py -m models/llama-2-7b-chat/pytorch -d CPU --torch_compile_backend openvino
123123
```
124124

125+
## Run on 2 sockets platform
126+
127+
benchmark.py sets openvino.properties.streams.num(1) by default
128+
129+
| OpenVINO version | Behaviors |
130+
|:--------------------|:------------------------------------------------|
131+
| Before 2024.0.0 | streams.num(1) <br>execute on 2 sockets. |
132+
| 2024.0.0 | streams.num(1) <br>execute on the same socket as the APP is running on. |
133+
134+
numactl on Linux or --load_config for benchmark.py can be used to change the behaviors.
135+
136+
For example, --load_config config.json as following in OpenVINO 2024.0.0 will result in streams.num(1) and execute on 2 sockets.
137+
```
138+
{"INFERENCE_NUM_THREADS":<NUMBER>}
139+
```
140+
`<NUMBER>` is the number of total physical cores in 2 sockets
141+
125142
## Additional Resources
126143
### 1. NOTE
127144
> If you encounter any errors, please check **[NOTES.md](./doc/NOTES.md)** which provides solutions to the known errors.

0 commit comments

Comments
 (0)