Loouis

The document compares the throughput performance of three large model inference engines: VLLM, SGLang, and LMDeploy, measured in output tokens per second for short input and long output scenarios. The results are presented in a table showing performance across different concurrency levels (1, 2, 4, 8, 16, 32, 64, and 128). LMDeploy consistently outperforms the other two engines at all concurrency levels, with the highest throughput recorded at 1123.07 tokens/s for 64 concurrent requests. The tests were conducted using the Qwen2.5-14B-Instruct-AWQ model on a hardware setup of E5 2680v4 with a 2080ti GPU.