How to use SGLang to conduct an end-to-end evaluation benchmarking on a real LLM model?

[zlwu92]

Hi,

I'm a beginner in using SGLang and learning with LLM.

And I am wondering how to use SGLang to conduct an end-to-end evaluation benchmarking on a real LLM model, such Meta-Llama-3.1-8B-Instruct.

I want to test the overall inference performance and the breakdown percentage for comparing both w/ structured generation or w/o structured generation.

Could you please provide some python code or guidance to teach me how to set up an experiment script?