Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add slice before matmut transformation for CB scenario #1261

Merged
merged 12 commits into from
Jan 4, 2025
Merged

Add slice before matmut transformation for CB scenario #1261

merged 12 commits into from
Jan 4, 2025

Conversation

olpipi
Copy link
Collaborator

@olpipi olpipi commented Nov 27, 2024 •
edited
Loading

CVS-154930
CVS-155533

@github-actions github-actions bot added category: continuous batching Continuous batching category: sampling Sampling / Decoding algorithms category: speculative decoding Speculative decoding no-match-files labels Nov 27, 2024
@ilya-lavrenov ilya-lavrenov self-assigned this Nov 27, 2024
ilya-lavrenov
ilya-lavrenov reviewed Nov 27, 2024
View reviewed changes
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/sampler.cpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
@ilya-lavrenov ilya-lavrenov added this to the 2025.0 milestone Nov 27, 2024
@olpipi olpipi marked this pull request as ready for review November 28, 2024 18:27
iefode
iefode reviewed Dec 4, 2024
View reviewed changes
src/cpp/src/utils/paged_attention_transformations.cpp Outdated Show resolved Hide resolved
src/cpp/src/utils.cpp Show resolved Hide resolved
src/cpp/src/sampler.cpp Outdated Show resolved Hide resolved
ilya-lavrenov
ilya-lavrenov reviewed Dec 4, 2024
View reviewed changes
src/cpp/src/utils/paged_attention_transformations.cpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/sampler.cpp Outdated Show resolved Hide resolved
src/cpp/src/sampler.cpp Outdated Show resolved Hide resolved
@github-actions github-actions bot added the category: LLM LLM pipeline (stateful, static) label Dec 6, 2024
@mlukasze mlukasze requested a review from iefode December 9, 2024 05:50
@olpipi olpipi force-pushed the slice_matmul_cb branch 2 times, most recently from 5fc1067 to c82d6d1 Compare December 12, 2024 11:58
@olpipi olpipi force-pushed the slice_matmul_cb branch 2 times, most recently from fa0a951 to 3a6066a Compare December 18, 2024 19:57
ilya-lavrenov
ilya-lavrenov reviewed Dec 20, 2024
View reviewed changes
src/cpp/src/speculative_decoding/speculative_decoding_impl.cpp Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/sampler.cpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Show resolved Hide resolved
src/cpp/src/model_runner.hpp Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the category: LLM LLM pipeline (stateful, static) label Jan 2, 2025
@olpipi olpipi requested a review from ilya-lavrenov January 2, 2025 17:00
@olpipi
Copy link
Collaborator Author

olpipi commented Jan 3, 2025

I just realized all my replies to new and old comments were not posted.
omg. Sorry for that

ilya-lavrenov
ilya-lavrenov approved these changes Jan 3, 2025
View reviewed changes
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/sequence_group.hpp Outdated Show resolved Hide resolved
src/cpp/src/sequence_group.hpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
src/cpp/src/model_runner.hpp Outdated Show resolved Hide resolved
@olpipi olpipi enabled auto-merge January 3, 2025 16:47
@olpipi olpipi added this pull request to the merge queue Jan 4, 2025
Merged via the queue into openvinotoolkit:master with commit 002f84f Jan 4, 2025
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iefode iefode iefode approved these changes

@ilya-lavrenov ilya-lavrenov ilya-lavrenov approved these changes

Assignees

@ilya-lavrenov ilya-lavrenov

@iefode iefode

Labels
category: continuous batching Continuous batching category: sampling Sampling / Decoding algorithms category: speculative decoding Speculative decoding no-match-files
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
3 participants
@olpipi @ilya-lavrenov @iefode