how can i use triton_python_backend_utils?

[YooSungHyun]

Description
in trtionserver:23.05. and git clone https://github.com/triton-inference-server/python_backend -b r23.05

root@4248wefe0f480:~# python3

import triton_python_backend_utils
ModuleNotFoundError: No module named 'triton_python_backend_utils'

how can i use it? how can i doing pip install?
it just cp {workspace}/python_backend/src/resources/triton_python_backend_utils.py {model_path}/1/triton_python_backend_utils.py and use in model.py?

Triton Information
What version of Triton are you using?
23.05

Are you using the Triton container or did you build it yourself?
using tritonserver container

Expected behavior
A clear and concise description of what you expected to happen.

import done well

[kthui]

I think the triton_python_backend_utils will be available when the Python script is ran by Triton. Would you be able to try the quick start? The model.py on the quick start is a good example on importing triton_python_backend_utils.

[YooSungHyun]

@kthui thx for comment!

wow that is so amazing...!
in my python code, warning like this, and i don't have triton_python_backend_utils.py anywhere.

but, when running tritonserver, it work good...!

how is this possible?

[nnshah1]

triton_python_back_utils is added to the sys.path when the model is launched by Triton (in its own process) - that's why it is available during execution.

It is not installed in the path of your local python environment - but it could be copied out for linting purposes (note however it is not designed to be used outside of the Triton environment)

[kthui]

triton_python_backend_utils is a utility library provided by the Triton Python backend, see this comment for its purpose.

[Galoev]

How can I install this module (triton_python_backend_utils) so that I can have code auto-complete during development?

[nnshah1]

@rmccorm4 , @Tabrizian - Did we have a recipe for code complete during development?

[Tabrizian]

We don't have a recipe right now but we do have a few items to improve this. Related GH issue: #5961

[MatteoPagliani]

Hi @Tabrizian. Can you share your progress on this, please?