LCM cpp demo #101
Merged
LCM cpp demo #101
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Wovchena
reviewed
Dec 22, 2023
Wovchena
requested changes
Dec 25, 2023
There was a problem hiding this comment.
I'm ok with master target branch, but you may want to retarget to releases
image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py
Outdated
Show resolved
Hide resolved
image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py
Outdated
Show resolved
Hide resolved
Wovchena
requested changes
Dec 26, 2023
There was a problem hiding this comment.
py scripts\convert_model.py -lcm "SimianLuo/LCM_Dreamshaper_v7" -t FP16 printed
Exception ignored in: <finalize object at 0x1fa1050bd20; dead>
Traceback (most recent call last):
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\weakref.py", line 590, in __call__
return info.func(*info.args, **(info.kwargs or {}))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 894, in _cleanup
cls._rmtree(name, ignore_errors=ignore_errors)
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
return _rmtree_unsafe(path, onerror)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 881, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
return _rmtree_unsafe(path, onerror)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe
onerror(os.scandir, path, sys.exc_info())
File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe
with os.scandir(path) as scandir_it:
^^^^^^^^^^^^^^^^
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\vzlobin\\AppData\\Local\\Temp\\tmp0btk8gby\\text_encoder\\openvino_model.bin'
Although it didn't fail. There's the same issue for stable_diffusion_1_5, but maybe you know what to do about it?
There was a problem hiding this comment.
There was a problem hiding this comment.
py scripts\convert_model.py -lcm "SimianLuo/LCM_Dreamshaper_v7" -t FP16printedException ignored in: <finalize object at 0x1fa1050bd20; dead> Traceback (most recent call last): File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\weakref.py", line 590, in __call__ return info.func(*info.args, **(info.kwargs or {})) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 894, in _cleanup cls._rmtree(name, ignore_errors=ignore_errors) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 881, in onerror cls._rmtree(path, ignore_errors=ignore_errors) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe onerror(os.scandir, path, sys.exc_info()) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe with os.scandir(path) as scandir_it: ^^^^^^^^^^^^^^^^ NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\vzlobin\\AppData\\Local\\Temp\\tmp0btk8gby\\text_encoder\\openvino_model.bin'Although it didn't fail. There's the same issue for stable_diffusion_1_5, but maybe you know what to do about it?
it looks like optimum issue
Wovchena
requested changes
Dec 29, 2023
Comment on lines
+1
to
+2
| #include <list> | ||
|
|
There was a problem hiding this comment.
Suggested change
| #include <list> |
Wovchena
approved these changes
Dec 29, 2023
|
@Wovchena @ilya-lavrenov thanks for review :) |
| model = OVLatentConsistencyModelPipeline.from_pretrained(args.lcm_weights,trust_remote_code=True, export=True, compile=False) | ||
| if args.type == "FP16": | ||
| model.half() | ||
| model.reshape(1, 512, 512, 1) |
There was a problem hiding this comment.
what if we don't use FP16? Looks like the model will be saved with dynamic shapes
|
|
||
| ov::Tensor text_embeddings(ov::element::f32, {1, MAX_LENGTH, HIDDEN_SIZE}); | ||
|
|
||
| compute_text_embeddings(pos_prompt, ov::Tensor(text_embeddings, {0, 0, 0}, {1, MAX_LENGTH, HIDDEN_SIZE})); |
There was a problem hiding this comment.
looks like we can embed (remove) this lambda since it's called only once
|
|
||
| std::vector<float> res_vec(half_dim), emb_cos(half_dim); | ||
| std::transform(emb_vec.begin(), emb_vec.end(), res_vec.begin(), [](float x){return std::sin(x);}); | ||
| std::transform(emb_vec.begin(), emb_vec.end(), emb_cos.begin(), [](float x){return std::cos(x);}); |
There was a problem hiding this comment.
looks like we can remove this multiple std::transform calls with simple for loop, which performs several operations at once
| ov::Tensor latent_model_input = randn_tensor(height, width, read_np_latent, seed); | ||
|
|
||
| float guidance_scale = 8.0; | ||
| ov::Tensor guidance_scale_embedding = get_w_embedding(guidance_scale, 256); |
There was a problem hiding this comment.
guidance_scale_embedding creation can be moved outside of the num_images loop
| * `-s, --seed arg` Number of random seed to generate latent (default: 42) | ||
| * `--num arg` Number of image output(default: 1) | ||
| * `--height arg` Height of output image (default: 512) | ||
| * `--width arg` Width of output image (default: 512) |
There was a problem hiding this comment.
it's stated that Only static model is currently supported for this sample. So, do we need height and width parameters then?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ToDo:
Only static model is supported for now