Use perf_client on models with string as input and variable shape #5414
-
|
I have a question about perf_client with ensemble_models: I have an ensemble model with this input config: input [ This model works properly when sending images with different sizes as encoded strings. How can I use perf_client on this? If I try
I get an error:
If I set the shape parameter:
It complains:
How to proceed with this? I am using perf_client from docker image tritonserver:22.12-py3-sdk Thanks! |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 3 replies
-
|
You have one variable dimension, but you're trying to set three. Your config either needs to have the input be [-1, -1, -1] or the shape needs to be one-dimensional, like |
Beta Was this translation helpful? Give feedback.
-
|
Great! It works now by using --shape INPUT:320. However, models complains about data decoding, I suppose because that send data is not a valid image. How should I send a test image using perf_client when input is with strings? |
Beta Was this translation helpful? Give feedback.
-
|
Depends on your model. What is it expecting? The way you wrote your config, it should have only one dimension. If, for example, your model expects the above dimensions, then you could change the INPUT dimensions in the config to "[320,240,3]" and run perf_client with your original |
Beta Was this translation helpful? Give feedback.
-
|
My model is expecting variable shape images, the only restriction I can set is that they are 3-channeled. However, if I set input to [-1,-1,3] I will have the same problem with perf_client as with -1, correct? |
Beta Was this translation helpful? Give feedback.
-
|
You'd set the config to [-1, -1, 3] and you'd add a flag to perf_client: |
Beta Was this translation helpful? Give feedback.
-
|
Yes, that would work for the shape, but what about the content itself? How
to send the image pixels?
El lun, 27 feb 2023, 18:02, dyastremsky ***@***.***> escribió:
… You'd set the config to [-1, -1, 3] and you'd add a flag to perf_client: --shape
INPUT:320,240,3.
—
Reply to this email directly, view it on GitHub
<#5414 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4BWMD5HH3N5UVCJRWQLFUTWZTM3BANCNFSM6AAAAAAVJFLNEQ>
.
You are receiving this because you authored the thread.Message ID:
<triton-inference-server/server/repo-discussions/5414/comments/5125742@
github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
If you need specific data needs, then you can follow the documentation for perf analyzer here. You can set data from an image, pre-defined values, or a directory. The default is random, but there are many ways to set your own data (or even use all zeroes, if that works for your use case). |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the doc link, there should be the answer for sure!
El lun, 27 feb 2023, 18:14, dyastremsky ***@***.***> escribió:
… If you need specific data needs, then you can follow the documentation for
perf analyzer here
<https://github.com/triton-inference-server/client/blob/main/src/c++/perf_analyzer/README.md#input-data>
.
You can set data from an image, pre-defined values, or a directory. The
default is random, but there are many ways to set your own data (or even
use all zeroes, if that works for your use case).
—
Reply to this email directly, view it on GitHub
<#5414 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4BWMD7WYDD6E45HMFFF44LWZTOI3ANCNFSM6AAAAAAVJFLNEQ>
.
You are receiving this because you authored the thread.Message ID:
<triton-inference-server/server/repo-discussions/5414/comments/5125875@
github.com>
|
Beta Was this translation helpful? Give feedback.
You have one variable dimension, but you're trying to set three. Your config either needs to have the input be [-1, -1, -1] or the shape needs to be one-dimensional, like
--shape INPUT:320.