Skip to content

Alt Schedules #111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 518 commits into
base: sage/dbo-full-cudagraphs
Choose a base branch
from
Draft

Alt Schedules #111

wants to merge 518 commits into from
+56,896 −65,903

Conversation

LucasWilkinson
Copy link

@LucasWilkinson LucasWilkinson commented Sep 4, 2025 •
edited by github-actions bot
Loading

…bo-full-cudagraphs

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@LucasWilkinson LucasWilkinson force-pushed the lwilkinson/dbo-alt-schedules branch from 5ba544b to 759e737 Compare September 4, 2025 19:48
@LucasWilkinson LucasWilkinson mentioned this pull request Sep 5, 2025
Closed
5 tasks
wuxibin89 and others added 28 commits September 23, 2025 21:49
@wuxibin89 @youkaichao
[BugFix] Register expert_map as named buffer for wake_up and sleep (v…
61d1b35 
…llm-project#25458)

Signed-off-by: wuxibin <wuxibin@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
@chaunceyjiang @markmc
[P/D] Support NIXL connector to disconnect during a clean shutdown (v…
f05a4f0 
…llm-project#24423)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
@panpan0000 @NickLucche
[Docs] NixlConnector quickstart guide (vllm-project#24249)
da5e7e4 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
@faaany
[XPU] Fix MOE DP accuracy issue on XPU (vllm-project#25465)
4c966e4
@mgoin
[UX] Change kv-cache-memory log level to debug (vllm-project#25479)
2c58742 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
@tdoublep
[V1] Remove V0 code paths for Hybrid models (vllm-project#25400)
a903669 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@LucasWilkinson @SageMoore
@yewentao256 @tlrmchlsmth
[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support…
cc1dc7e 
… and Prefill support (vllm-project#24845)

Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
@hmellor
Add backward compatibility for GuidedDecodingParams (vllm-project#2…
875d6de 
…5422)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@bnellnm @mgoin
[Kernels] Support blocked fp8 quantization for compressed tensors MoE (…
f11e3c5 
…vllm-project#25219)

Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@rivos-shreeasish
[BugFix] Fix UB in per_token_group_quant.cu (vllm-project#24913)
2357480 
Signed-off-by: Shreeasish Kumar <shreeasish@rivosinc.com>
@yewentao256
[Log] Optimize kv cache memory log from Bytes to GiB (vllm-project#25204
846197f 
)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
@minosfuture @rahul-tuli
@claude @yewentao256 @houseroad @yeqcharlotte
Use macro guard CUDA functions for back compatibility in grouped_topk…
527821d 
…_kernel.cu (vllm-project#25346)

Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
@bringlein @cyang49 @yewentao256
[V1][Kernel] Add triton implementation for reshape_and_cache_flash (v…
100b630 
…llm-project#24503)

Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
@wdhongtw
[Misc] Reduce initialization time of auto_tune (vllm-project#23682)
24e8222 
Signed-off-by: Weida Hong <wdhongtw@google.com>
@ekagra-ranjan @ywang96
[Spec Decode][CI] Add e2e test for examples/spec_decode.py and prev…
867ecdd 
…ent breaking Acceptance Length (vllm-project#24531)

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
@jeejeelee
[Core] Ensure LoRA linear respect the base_layer's tp_size and tp_rank (
5abb117 
vllm-project#25487)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@amd-hhashemi
[ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (vllm-pr…
a3a7828 
…oject#24988)

Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
@Amir-19 @mgoin
[core] add nccl symmetric memory for all reduce (vllm-project#24532)
8c1c81a 
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@ElizaWszola @ProExpertProg @mgoin
[Performance] Move apply_w8a8_block_fp8_linear to an op class (vllm-p…
6340025 
…roject#24666)

Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
@mgoin
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEW…
24fab45 
…ISE (vllm-project#25444)

Signed-off-by: mgoin <mgoin64@gmail.com>
@jiahanc
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (…
d5944d5 
…vllm-project#25406)

Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
@mgoin
[Bugfix] Lower gpt-oss max cudagraph size to 992 to be compatible wit…
a8ffc4f 
…h FA3 (vllm-project#25508)

Signed-off-by: mgoin <mgoin64@gmail.com>
@ilmarkov @mgoin
Enable symmetric memory all reduce by default only enabling for TP (v…
8bdd8b5 
…llm-project#25070)

Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@yewentao256
[CI] Fix Pre-commit Issue (vllm-project#25497)
8b8a8af 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
@alecsolder
[Bugfix] gpt-oss container tool output bug (vllm-project#25485)
c828d1b 
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
@chaunceyjiang
[Build] Update Xgrammar to 0.1.25 (vllm-project#25467)
08275ec 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@gshtras
[Bugfix] Fix for the import error from vllm-project#24588 (vllm-proje…
690f948 
…ct#25481)

Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
@Isotr0py
[CI/Build] Fix and re-enable v1 PP test on CI (vllm-project#25496)
ae00292 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
aarnphm and others added 26 commits September 29, 2025 23:26
@aarnphm
[V0 Deprecation] Remove vllm.worker and update according imports (v…
6a113d9 
…llm-project#25901)
@qthequartermasterman
Test Prompt Embeds/LoRA compatibility and Enable LoRA Support for OPT…
78a47f8 
… Models (vllm-project#25717)

Signed-off-by: Andrew Sansom <andrew@protopia.ai>
@yewentao256 @gemini-code-assist
[Bug] Fix Weight Loading for Block FP8 Cutlass SM90 (vllm-project#25909)
89e4050 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@zhuohan123
[Benchmark] Support benchmark throughput for external launcher DP (vl…
d3bd171 
…lm-project#25913)

Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
@hmellor
MoveVllmConfig from config/__init__.py to config/vllm.py (vllm-…
61aedb5 
…project#25271)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@LucasWilkinson
[BugFix] Fix DP/EP hang (vllm-project#25906)
23194d8 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@acisseJZhong
[BugFix] Pass config_format via try_get_generation_config (vllm-proje…
e47433b 
…ct#25912)
@zhoukezi
[Model][Bugfix] Fix MiDashengLM audio encoder mask by removing incorr…
2e1b8bc 
…ect `logical_not` (vllm-project#25925)

Signed-off-by: zhoukz <me@zhoukz.com>
@simondanielsson
[Bugfix]: Clean up chunked prefill logging when using whisper (vllm-p…
e23cacd 
…roject#25075)

Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
@zyongye @heheda12345
@youkaichao @LucasWilkinson @robertgshaw2-redhat @yewentao256 @mgoin @luccafong @NickLucche @IwakuraRein @MatthewBonanni @mxz297 @Barry-Delaney
[New Model] DeepSeek-V3.2 (Rebased to Main) (vllm-project#25896)
fa7e254 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
@a120092009
[Doc] Add Cambricon MLU support (vllm-project#25942)
8d0afa9 
Signed-off-by: a120092009 <zhaoty0121@gmail.com>
@sergiopaniego @hmellor
Updated TRL integration docs (vllm-project#25684)
1ad3aca 
Signed-off-by: sergiopaniego <sergiopaniegoblanco@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@CSWYF3634076
[Bugfix][Model]fix ernie45 moe gate&bias dtype to float32 (vllm-proje…
ef6e0e7 
…ct#25936)

Signed-off-by: wangyafeng <wangyafeng@baidu.com>
@DarkLight1337
[Model] Move vision_feature_select_strategy into `resolve_visual_en…
d7e34b4 
…coder_outputs` (vllm-project#25938)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@lhtin
[perf] Use CPU tensor to reduce GPU->CPU sync (vllm-project#25884)
e184c9c 
Signed-off-by: Lehua Ding <lehuading@tencent.com>
@NickLucche @heheda12345
[NIXL] Add support for MLA caches with different latent dim (vllm-pro…
80608ba 
…ject#25902)

Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
@rzabarazesh @hmellor
[CI] Move applicable tests to CPU (vllm-project#24080)
bc546f7 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@ihb2032
[Fix] Improve CPU backend compatibility for RISC-V (vllm-project#25816)
bb6d430 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
@Josephasafg
[Kernel][Moe Configs] Add more tuned triton configs for ExpertsInt8 a…
35fe398 
…nd FP8 (vllm-project#25858)

Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
@sergiopaniego @hmellor
Add Hugging Face Inference Endpoints guide to Deployment docs (vllm-p…
099aaee 
…roject#25886)

Signed-off-by: sergiopaniego <sergiopaniegoblanco@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@Anionex
[Bugfix][Model] Fix inference for Hunyuan dense models (vllm-project#…
f4db5e6 
…25354)

Signed-off-by: anion <1005128408@qq.com>
Signed-off-by: Anion <123177548+Anionex@users.noreply.github.com>
@pavanimajety
[Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (vl…
ef28354 
…lm-project#25895)

Signed-off-by: Pavani Majety <pmajety@nvidia.com>
@DarkLight1337
[Bugfix] Token type and position embeddings fail to be applied to `in…
9f1c4ec 
…puts_embeds` (vllm-project#25922)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao
[bugfix][deepseek] fix flashmla kernel selection (vllm-project#25956)
a2e6fa7 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@LucasWilkinson
Squashed commit of nm/lwilkinson/dbo-alt-schedules changes relative t…
2682bb7 
…o origin/main

fixes

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fix

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fix

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fixes and formatting

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@LucasWilkinson
get deep ll to run
fcd015c 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@LucasWilkinson LucasWilkinson force-pushed the lwilkinson/dbo-alt-schedules branch from 79d0a01 to fcd015c Compare September 30, 2025 18:06
@LucasWilkinson
fix
6dc3492 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@LucasWilkinson
cleanup
2d7ca92 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@LucasWilkinson
cleanup
4effe25 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NickLucche NickLucche Awaiting requested review from NickLucche NickLucche will be requested when the pull request is marked ready for review NickLucche is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin will be requested when the pull request is marked ready for review mgoin is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat will be requested when the pull request is marked ready for review robertgshaw2-redhat is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 will be requested when the pull request is marked ready for review yewentao256 is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth will be requested when the pull request is marked ready for review tlrmchlsmth is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao will be requested when the pull request is marked ready for review youkaichao is a code owner

@njhill njhill Awaiting requested review from njhill njhill will be requested when the pull request is marked ready for review njhill is a code owner

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat alexm-redhat will be requested when the pull request is marked ready for review alexm-redhat is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg will be requested when the pull request is marked ready for review ProExpertProg is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.