Skip to content

prefill metadata splitting #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 325 commits into
base: sage/dbo-full-cudagraphs
Choose a base branch
from
Draft

prefill metadata splitting #112

wants to merge 325 commits into from
+42,245 −66,180

Conversation

LucasWilkinson
Copy link

@LucasWilkinson LucasWilkinson commented Sep 5, 2025 •
edited by github-actions bot
Loading

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

DarkLight1337 and others added 30 commits September 17, 2025 12:19
@DarkLight1337
[Misc] Avoid use of deprecated AutoModelForVision2Seq (vllm-project…
e120533 
…#25065)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@danielafrimi
Add RADIO Vision Encoder Support to vLLM (vllm-project#24595)
252ada5 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: root <root@cw-dfw-h100-001-305-026.cm.cluster>
@bigPYJ1151
[Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (…
9fccd04 
…vllm-project#25046)

Signed-off-by: jiang1.li <jiang1.li@intel.com>
@Aidyn-A
Apply fixes for CUDA 13 (vllm-project#24599)
bfe9380 
Signed-off-by: Aidyn-A <aidyn.b.aitzhan@gmail.com>
@dolpm @jeejeelee
[fix] lora benchmarks pass no_lora_flag_cpu (vllm-project#23774)
1b962e2 
Signed-off-by: Dylan Maloy <34420038+dolpm@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
@sighingnow
[Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP imple…
dd6a910 
…mentation. (vllm-project#24957)

Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
@samzong
[Docs] improve code formatting and comments for eliminate griffe buil…
47f670b 
…d warning. (vllm-project#25010)

Signed-off-by: samzong <samzong.lu@gmail.com>
@MatthewBonanni
Remove old cutlass mla (vllm-project#23961)
8f3616f 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
@samzong
[Docs] vllm/benchmarks/datasets.py fix docstring param format. (vllm-…
4a2d33e 
…project#24970)

Signed-off-by: samzong <samzong.lu@gmail.com>
@mgoin
[CI Bugfix] Fix failing test_invalid_env (vllm-project#25078)
087c6ff 
Signed-off-by: mgoin <mgoin64@gmail.com>
@WoosukKwon
[V0 Deprecation] Remove V0 Core tests (vllm-project#25082)
4b946d6 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@simon-mo @jeejeelee
cleanup: remove adapter commons (vllm-project#25045)
4aa8c7b 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
@simon-mo
Remove unused find_cuda_init helper script (vllm-project#25044)
d6a518f
@WoosukKwon
[V0 Deprecation] Remove unused output processor util (vllm-project#25023
99cc41a 
)

Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
@mgoin
Change log level from info to debug for IOProcessor (vllm-project#24999)
8b32464 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
@WoosukKwon
[CI] Revert back prepare_prompts and check_answers (vllm-project#25087)
eb68c2d 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@WoosukKwon
[V0 Deprecation] Remove V0 tests in test_sequence.py (vllm-project#25088
9d442b7 
)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@mgoin
[CI Bugfix] Fix failing test_model_load_with_params tests due to toke…
e3db5eb 
…nizer refactor (vllm-project#25086)

Signed-off-by: mgoin <mgoin64@gmail.com>
@afeldman-nm @JosephMarinier
[V1] Logits processor docs (vllm-project#22919)
7ae9887 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Joseph Marinier <Joseph.Marinier@gmail.com>
@ApostaC
[Misc] Update owners for KV connector and V1 offloading (vllm-project…
ee5fd49 
…#25041)

Signed-off-by: ApostaC <yihua98@uchicago.edu>
@LucasWilkinson
cleanup
bd04165 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@mmangkad
[Bugfix] Update import path for bc_linter_include (vllm-project#24766)
8831315 
Signed-off-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
@ahao-anyscale
[BUG] Exclude .pth files when pulling remote files (vllm-project#25092)
f20c3b0 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
@czhu-cohere
[Kernel] Faster pre-processing time for W4A8 (vllm-project#23972)
3c068c6 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
@qandrew
[gpt-oss][2] fix types for streaming (vllm-project#24556)
bff2e5f 
Signed-off-by: Andrew Xia <axia@meta.com>
@alexm-redhat @mgoin
[Bugfix][B200] Fix cutlass_mla hang (vllm-project#24966)
fedb75f 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@dllehr-amd
Aiter mha fp8 fix (vllm-project#24991)
1a456c7 
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
@mgoin
Disable failing GPT-OSS Eval (Blackwell) for now (vllm-project#25107)
9f882d8 
Signed-off-by: mgoin <mgoin64@gmail.com>
@elvischenv @mgoin
[Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (v…
e67a79d 
…llm-project#24600)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@karan @gemini-code-assist
Add a batched auto tune script (vllm-project#25076)
2a4d641 
Signed-off-by: Karan Goel <karangoel@google.com>
Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
alecsolder and others added 30 commits September 22, 2025 23:38
@alecsolder @yeqcharlotte
[Frontend] Responses API MCP tools for built in tools and to pass thr…
45d7d85 
…ough headers (vllm-project#24628)

Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
@ProExpertProg
[Bugfix] fix custom op test (vllm-project#25429)
d588cd2 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@russellb
[Core] Drop overly aggressive whisper assertion (vllm-project#25408)
f31ff87 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@NickLucche
[Bugfix] Fix missing clear_connector_metadata (vllm-project#25397)
0901970 
Signed-off-by: NickLucche <nlucches@redhat.com>
@MatthewBonanni @robertgshaw2-redhat @Bam4d
[BugFix] [DP/EP] Fix slow execution when BS <= DP (vllm-project#25407)
ac0048c 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Chris Bamford <chrisbam4d@gmail.com>
@alexm-redhat
[Performance] Remove input pads in cutlass_mla and optimize v_proj ou…
0b7bed9 
…tput handling (vllm-project#25184)

Signed-off-by: Alexander Matveev <amatveev@redhat.com>
@yewentao256
[Perf] Apply torch.compile for per_block_cast_to_fp8 (vllm-project#…
9949aa2 
…24611)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
@Isotr0py
[V0 deprecation] Remove platform v1 controling interface (vllm-projec…
6fa78d8 
…t#25410)

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@Isotr0py
[V0 deprecation] Remove _set_default_args_v0 function (vllm-project…
c625f90 
…#25409)

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@yewentao256
[Bug] Fix Long Context OOM Issue (vllm-project#25290)
4741239 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
@tlrmchlsmth
Merge branch 'main' into lwilkinson/dbo-prefill
4e9e2ce 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
@LucasWilkinson
fix pre-commit
356ddcb 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@JJJYmmm
[feat] Support MRoPE + YaRN (vllm-project#25384)
fc97733 
Signed-off-by: liuye.hj <liuye.hj@alibaba-inc.com>
Co-authored-by: liuye.hj <liuye.hj@alibaba-inc.com>
@jikunshang
[XPU] Fix compile_size is None case. (vllm-project#25433)
f225ea7 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
@luccafong
[benchmarks]allow skip ready check for bench serve (vllm-project#25420)
eea1783 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
@mgoin
[Bugfix] Remove contiguous output req for context parallel MLA (vllm-…
78237e4 
…project#25414)

Signed-off-by: Michael Goin <mgoin64@gmail.com>
@windsonsea
[Docs] Fix griffe warnings in vllm/lora/ops (vllm-project#25369)
fafbe11 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
@varun-sundar-rabindranath
[DP/EP][GPTOSS] Use triton matmul-ogs kernels for GPTOSS DP/EP (vllm-…
e8db44f 
…project#24588)

Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
@xuechendi
[NIXL][OOT platform] support nixl_connector with oot platform and oth…
5774b0a 
…er nixl_backend (vllm-project#25121)

Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
@DarkLight1337
[Model] Enable DP for ViT in Qwen2-VL (vllm-project#25445)
c98be0a 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@minosfuture
Handle triton kernel import exception (vllm-project#25319)
ba8d216 
Signed-off-by: Ming Yang <minos.future@gmail.com>
@Zhikaiiii
[Frontend] Add a new xml-based tool parser for qwen3-coder (vllm-proj…
9383cd6 
…ect#25028)

Signed-off-by: Zhikaiiii <1658973216@qq.com>
@DarkLight1337
[Misc] Move DP for ViT code inside model executor dir (vllm-project#2…
babad6e 
…5459)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@ahartel
[Test]: Hermes tool parser stream output error in Qwen3 case (vllm-pr…
4322c55 
…oject#25203)

Signed-off-by: Andreas Hartel <andreas.hartel@aleph-alpha.com>
@Isotr0py
[Bugfix] Fix idefics3 tie_word_embeddings (vllm-project#25454)
231c2c6 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@jeejeelee
[Core] Optimize LoRA weight loading (vllm-project#25403)
273690a 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@vllmellm
[docs] Benchmark Serving Incorrect Arg (vllm-project#25474)
0d9fe26 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
@Isotr0py
[CI/Build] Fix disabled v1 attention backend selection test (vllm-pro…
b6a136b 
…ject#25471)

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@tlrmchlsmth
Merge branch 'main' into lwilkinson/dbo-prefill
4c631db 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
@LucasWilkinson
precommit fix
f0bc569 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NickLucche NickLucche Awaiting requested review from NickLucche NickLucche will be requested when the pull request is marked ready for review NickLucche is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin will be requested when the pull request is marked ready for review mgoin is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat will be requested when the pull request is marked ready for review robertgshaw2-redhat is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 will be requested when the pull request is marked ready for review yewentao256 is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth will be requested when the pull request is marked ready for review tlrmchlsmth is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao will be requested when the pull request is marked ready for review youkaichao is a code owner

@njhill njhill Awaiting requested review from njhill njhill will be requested when the pull request is marked ready for review njhill is a code owner

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat alexm-redhat will be requested when the pull request is marked ready for review alexm-redhat is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg will be requested when the pull request is marked ready for review ProExpertProg is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.