[Samples] merge LLM samples to "text_generation" folder #1411
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
category: GHA
olpipi
force-pushed
the
samples_movement
branch
2 times, most recently
from
9e7f861 to
3901fbb
Compare
Wovchena
reviewed
Dec 19, 2024
ilya-lavrenov
changed the title
ilya-lavrenov
requested changes
Dec 20, 2024
This was referenced Jan 3, 2025
github-merge-queue bot
pushed a commit
to openvinotoolkit/openvino
that referenced
this pull request
Jan 3, 2025
port: #28248 connected to: openvinotoolkit/openvino.genai#1411
github-merge-queue bot
pushed a commit
to openvinotoolkit/openvino
that referenced
this pull request
Jan 3, 2025
connected to: openvinotoolkit/openvino.genai#1411 Co-authored-by: Andrzej Kopytko <andrzejx.kopytko@intel.com>
olpipi
force-pushed
the
samples_movement
branch
from
b270500 to
a48de38
Compare
|
If there is no more major comments, I will make the similar changes to python samples |
olpipi
force-pushed
the
samples_movement
branch
from
a48de38 to
9a9d41c
Compare
olpipi
force-pushed
the
samples_movement
branch
from
6db3e88 to
3b54139
Compare
Wovchena
approved these changes
Jan 9, 2025
olpipi
force-pushed
the
samples_movement
branch
from
3b54139 to
1457292
Compare
3 tasks
|
@ilya-lavrenov re-review please. The PR cannot be merged without +1 from you |
| @@ -231,7 +231,7 @@ custom_streamer = CustomStreamer() | |||
| pipe.generate("The Sun is yellow because", max_new_tokens=15, streamer=custom_streamer) | |||
| ``` | |||
| For fully implemented iterable CustomStreamer please refer to [multinomial_causal_lm](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/samples/python/multinomial_causal_lm/README.md) sample. | |||
| For fully implemented iterable CustomStreamer please refer to [multinomial_causal_lm](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/samples/python/text_generation/README.md) sample. | |||
There was a problem hiding this comment.
I have also found that OpenVINO docs refer to GenAI samples locations
Please, prepare PRs to OpenVINO repo as well:
- releases/2024/6 branch to point to GenAI's releases/2024/6
- master branch to point to GenAI's master with updated locations.
There was a problem hiding this comment.
openvinotoolkit/openvino#28384 - master
openvinotoolkit/openvino#28383 - 2024/6
| Model examples to use for different samples: | ||
| chat_sample - meta-llama/Llama-2-7b-chat-hf | ||
| speculative_decoding_lm - meta-llama/Llama-2-13b-hf as main model and TinyLlama/TinyLlama-1.1B-Chat-v1.0 as draft model | ||
| other samples - meta-llama/Llama-2-7b-hf |
There was a problem hiding this comment.
it's drawn as plain text
which is hard to read
maybe we can show at the beginning how to convert model, while these recommendations about suggested model will be directly in the section with sample?
E.g. in common part:
optimim-cli export openvino --model <xx> <output_folder>
and in per-sample section:
chat sample:
recommended models: meta-llama/Llama-2-7b-chat-hf, etc
The main idea of this is to keep locality: users don't need to scroll up and down to read information about how to run samples.
This was referenced Jan 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.