File tree Expand file tree Collapse file tree 1 file changed +17
-0
lines changed Expand file tree Collapse file tree 1 file changed +17
-0
lines changed Original file line number Diff line number Diff line change @@ -122,6 +122,23 @@ Add the option `--torch_compile_backend` with the desired backend: `pytorch` or
122
122
python ./benchmark.py -m models/llama-2-7b-chat/pytorch -d CPU --torch_compile_backend openvino
123
123
```
124
124
125
+ ## Run on 2 sockets platform
126
+
127
+ benchmark.py sets openvino.properties.streams.num(1) by default
128
+
129
+ | OpenVINO version | Behaviors |
130
+ | :--------------------| :------------------------------------------------|
131
+ | Before 2024.0.0 | streams.num(1) <br >execute on 2 sockets. |
132
+ | 2024.0.0 | streams.num(1) <br >execute on the same socket as the APP is running on. |
133
+
134
+ numactl on Linux or --load_config for benchmark.py can be used to change the behaviors.
135
+
136
+ For example, --load_config config.json as following in OpenVINO 2024.0.0 will result in streams.num(1) and execute on 2 sockets.
137
+ ```
138
+ {"INFERENCE_NUM_THREADS":<NUMBER>}
139
+ ```
140
+ ` <NUMBER> ` is the number of total physical cores in 2 sockets
141
+
125
142
## Additional Resources
126
143
### 1. NOTE
127
144
> If you encounter any errors, please check ** [ NOTES.md] ( ./doc/NOTES.md ) ** which provides solutions to the known errors.
You can’t perform that action at this time.
0 commit comments