Which one is faster for token generation? LLama.cpp v.s. SGLang

[ghostplant]

Is there any benchmark comparison based on the same batch size (= 1 for token generation) and same quant.ed type?

[zhyncs]

For batch size 1 use case, you can enable torch compile in SGLang. It's even faster than gpt-fast. So I believe SGLang is faster. Welcome to give us feedback and benchmark results in your use case.