需要配合vllm的定制版使用，项目地址： https://github.com/SomeoneKong/vllm/tree/partial_mode
vllm服务启动参数示例：
python -m vllm.entrypoints.openai.api_server --trust-remote-code --model model_path --served-model-name model_name --port port --max-logprobs 100
-
Notifications
You must be signed in to change notification settings - Fork 0
License
SomeoneKong/llm_decode_space
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published