[Performance]: vLLM v0.6.3 to v0.7.1, improved tps by around 1.6x

[wonjerry]

Proposal to improve performance

Hi, I recently upgraded from vLLM v0.6.3 to v0.7.1, and I noticed that my RPS (requests per second) improved by around 1.6x — even though I’m still using the v0 engine, not v1.

I saw in the v0.7.1 release notes that it mentions “~3x the generation throughput, ~10x the memory capacity for tokens, and horizontal context scalability with pipeline parallelism.” Could you help me understand what specific optimizations or changes in v0.7.1 might have contributed to this performance gain, even without switching to the v1 engine?

Thanks in advance!

Report of performance regression

No response

Misc discussion on performance

No response

Your current environment (if you think it is necessary)

The output of `python collect_env.py`

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

What model are you using?

[wonjerry]

@DarkLight1337 I conducted the tests based on our in-house custom model. I did not use other public models such as LLaMA or DeepSeek. The tests were performed using the same model(in-house).

[DarkLight1337]

It's difficult to tell what caused the performance improvement without further details. Is it a text-only model or multimodal model?

[wonjerry]

@DarkLight1337 The model I used is a text-only model.

[DarkLight1337]

Did you enable tensor/pipeline parallelism?

[DarkLight1337]

I think that the optimizations in the release notes you mentioned are specific to DeepSeek like archs only . Not sure which model architecture you are following.

[wonjerry]

@DarkLight1337 I did not enable TP/PP parallelism.

As far as I understand, this model is not based on a DeepSeek-like architecture, but I’ll double-check to be sure. Thanks!