Upstream merge 25 1 6 #350

gshtras · 2025-01-06T23:22:03Z

2025/1/6 merge

Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com> Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

…-project#10511) Signed-off-by: youkaichao <youkaichao@gmail.com>

…ols for multi-node distributed serving (vllm-project#11235) Co-authored-by: Michael Goin <michael@neuralmagic.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

…y default with OpenAI parse (vllm-project#10935) Signed-off-by: mgoin <michael@neuralmagic.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

…ject#11210) Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: ywang96 <ywang@example.com>

…ect#11225) Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>

…ct#11252) Signed-off-by: Isotr0py <2037008807@qq.com>

…#11264)

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>

Signed-off-by: Konrad Zawora <kzawora@habana.ai>

…t#11046)

…#10995) Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com> Co-authored-by: ilmarkov <markovilya197@gmail.com> Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

) Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>

vllm-project#11311) Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

…1313) Signed-off-by: mgoin <michael@neuralmagic.com>

…d_blocks` (vllm-project#11310)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…ect#11712)

Signed-off-by: xcnick <xcnick0412@gmail.com>

…graph-capture (vllm-project#11233) Signed-off-by: Yan Burman <yanburman@users.noreply.github.com> Signed-off-by: Ido Asraff <idoa@atero.ai>

…eVision (vllm-project#11717) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…11736) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Lu Fang <lufang@fb.com>

Co-authored-by: Lancer <maruixiang6688@gmail.com>

…ct#11752)

)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

Signed-off-by: lucast2021 <lucast2021@headroyce.org> Co-authored-by: lucast2021 <lucast2021@headroyce.org>

Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Roger Wang <ywang@roblox.com>

…oject#11771) Signed-off-by: youkaichao <youkaichao@gmail.com>

varun-sundar-rabindranath and others added 30 commits December 16, 2024 18:14

[misc] Layerwise profile updates (vllm-project#10242)

efbce85

Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com> Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

[core] overhaul memory profiling and fix backward compatibility (vllm…

551603f

…-project#10511) Signed-off-by: youkaichao <youkaichao@gmail.com>

[Docs] hint to enable use of GPU performance counters in profiling to…

35ffa68

…ols for multi-node distributed serving (vllm-project#11235) Co-authored-by: Michael Goin <michael@neuralmagic.com>

[ci][tests] add gh200 tests (vllm-project#11244)

c301616

Signed-off-by: youkaichao <youkaichao@gmail.com>

[torch.compile] fast inductor (vllm-project#11108)

88a412e

Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

fix gh200 tests on main (vllm-project#11246)

35bae11

Signed-off-by: youkaichao <youkaichao@gmail.com>

[CI] Add test case with JSON schema using references + use xgrammar b…

0064f69

…y default with OpenAI parse (vllm-project#10935) Signed-off-by: mgoin <michael@neuralmagic.com>

[Frontend] Add OpenAI API support for input_audio (vllm-project#11027)

66d4b16

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

[V1][VLM] Proper memory profiling for image language models (vllm-pro…

59c9b6e

…ject#11210) Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: ywang96 <ywang@example.com>

[Platform] platform agnostic for EngineArgs initialization (vllm-proj…

e88db68

…ect#11225) Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

[V1][Core] Use weakref.finalize instead of atexit (vllm-project#11242)

2bfdbf2

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

[Misc] Kernel Benchmark for RMSNorm (vllm-project#11241)

02222a0

Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>

[Misc] Allow passing logits_soft_cap for xformers backend (vllm-proje…

f9ecbb1

…ct#11252) Signed-off-by: Isotr0py <2037008807@qq.com>

[Bugfix] Fix request cancellation without polling (vllm-project#11190)

2d1b9ba

[Bugfix] Set temperature=0.7 in test_guided_choice_chat (vllm-project…

c77eb8a

…#11264)

[V1] Prefix caching for vision language models (vllm-project#11187)

bf8717e

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>

[Bugfix] Restore support for larger block sizes (vllm-project#11259)

866fa45

Signed-off-by: Konrad Zawora <kzawora@habana.ai>

[Bugfix] Fix guided decoding with tokenizer mode mistral (vllm-projec…

8b79f9e

…t#11046)

[MISC][XPU]update ipex link for CI fix (vllm-project#11278)

f04e407

[Bugfix] Fix broken phi3-v mm_processor_kwargs tests (vllm-project#11263

996aa70

) Signed-off-by: Isotr0py <2037008807@qq.com>

[CI][Misc] Remove Github Action Release Workflow (vllm-project#11274)

362cff1

[FIX] update openai version (vllm-project#11287)

f954fe0

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

[Bugfix] fix minicpmv test (vllm-project#11304)

ca5f54a

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

[V1] VLM - enable processor cache by default (vllm-project#11305)

fdea8ec

Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>

[Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (

5a9da2e

vllm-project#11311) Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

[Model] IBM Granite 3.1 (vllm-project#11307)

17ca964

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

[CI] Expand test_guided_generate to test all backends (vllm-project#1…

a30482f

…1313) Signed-off-by: mgoin <michael@neuralmagic.com>

[V1] Simplify prefix caching logic by removing `num_evictable_compute…

c6b0a7d

…d_blocks` (vllm-project#11310)

[VLM] Merged multimodal processor for Qwen2-Audio (vllm-project#11303)

6142ef0

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

bet0x and others added 26 commits January 4, 2025 06:29

Update bnb.md with example for OpenAI (vllm-project#11718)

d1d4939

[V1] Add RayExecutor support for AsyncLLM (api server) (vllm-proj…

fbf2564

…ect#11712)

[V1] Add kv cache utils tests. (vllm-project#11513)

d91457d

Signed-off-by: xcnick <xcnick0412@gmail.com>

[Core][Bugfix] Use correct device to initialize GPU data during CUDA-…

300acb8

…graph-capture (vllm-project#11233) Signed-off-by: Yan Burman <yanburman@users.noreply.github.com> Signed-off-by: Ido Asraff <idoa@atero.ai>

[VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-On…

eed11eb

…eVision (vllm-project#11717) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Bugfix] Fix precision error in LLaVA-NeXT (vllm-project#11735)

ba214df

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Model] Remove unnecessary weight initialization logic (vllm-project#…

65c0892

…11736) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com>

[Bugfix][V1] Fix test_kv_cache_utils.py (vllm-project#11738)

4783143

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

[MISC] Replace c10::optional with std::optional (vllm-project#11730)

4068f4b

Signed-off-by: Lu Fang <lufang@fb.com>

[distributed] remove pynccl's redundant stream (vllm-project#11744)

635b897

fix: [doc] fix typo (vllm-project#11751)

eba1717

Co-authored-by: Lancer <maruixiang6688@gmail.com>

[Frontend] Improve StreamingResponse Exception Handling (vllm-proje…

33fc1e2

…ct#11752)

[distributed] remove pynccl's redundant change_state (vllm-project#11749

9e764e7

)

[Doc] [1/N] Reorganize Getting Started section (vllm-project#11645)

402d378

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Bugfix] Remove block size constraint (vllm-project#11723)

408e560

[V1] Add BlockTable class (vllm-project#11693)

06bfb51

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

[Misc] Fix typo for valid_tool_parses (vllm-project#11753)

f8fcca1

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

[V1] Refactor get_executor_cls (vllm-project#11754)

022c5c6

[mypy] Forward pass function type hints in lora (vllm-project#11740)

9c74971

Signed-off-by: lucast2021 <lucast2021@headroyce.org> Co-authored-by: lucast2021 <lucast2021@headroyce.org>

k8s-config: Update the secret to use stringData (vllm-project#11679)

2a622d7

Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>

[VLM] Separate out profiling-related logic (vllm-project#11746)

996357e

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Doc][2/N] Reorganize Models and Usage sections (vllm-project#11755)

ee77fdb

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Bugfix] Fix max image size for LLaVA-Onevision (vllm-project#11769)

9279b9f

Signed-off-by: Roger Wang <ywang@roblox.com>

[doc] explain how to add interleaving sliding window support (vllm-pr…

4ca5d40

…oject#11771) Signed-off-by: youkaichao <youkaichao@gmail.com>

Merge remote-tracking branch 'upstream/main'

4773c29

format

267c1a1

shajrawi approved these changes Jan 7, 2025

View reviewed changes

Merge branch 'main' into upstream_merge_25_1_6

97067c0

gshtras merged commit 88e020d into main Jan 8, 2025
10 of 11 checks passed

gshtras deleted the upstream_merge_25_1_6 branch January 8, 2025 15:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upstream merge 25 1 6 #350

Upstream merge 25 1 6 #350

gshtras commented Jan 6, 2025

Upstream merge 25 1 6 #350

Upstream merge 25 1 6 #350

Conversation

gshtras commented Jan 6, 2025