"Failed to create CUDAExecutionProvider" onnxruntime error on Google Colab

[hkhare42]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Really appreciate this library and ecosystem!

I'm trying to run the "time_in_zone" example on Google Colab (CUDAExecutionProvider). I pip installed the requirements.txt and in the next step, also pip installed inference-gpu library.

!nvidia-smi gives me the following showing that CUDA is available.

On running the inference_file_example.py, I get the following error:
UserWarning: Specified provider 'OpenVINOExecutionProvider' is not in available provider names.Available providers: 'TensorrtExecutionProvider, CUDAExecutionProvider, CPUExecutionProvider' 2024-07-22 18:17:36.983840877 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:640 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.

I tried going through the referenced link in the error and going over forums but wasn't able to resolve this. What additional step is needed to get CUDAExecutionProvider to work?

Additional

No response

[hkhare42]

Also sharing links to external threads that discuss similar issues:
roboflow/inference#324
microsoft/onnxruntime#11092
https://stackoverflow.com/questions/75267445/why-does-onnxruntime-fail-to-create-cudaexecutionprovider-in-linuxubuntu-20

[onuralpszr]

@hkhare42 hello o/ let me convert this issue to discussion and also could you try this codes I made small collab for you test and see are you in GPU

https://colab.research.google.com/drive/1Ltv7tKae7SOkM7bOjwPQ0x3kYMik5XkE?usp=sharing

[onuralpszr]

Hey @onuralpszr Any suggestions on what I can try out?

Do installation,next do manual install and make import and then run your code but put your code in collab instead of get from drive way so it can keep imports same and let me know how is it went

[hkhare42]

So the "Failed to create CUDAExecutionProvider" error is no longer popping up. So that's a positive. But it seems to be taking same time per iteration as it was when defaulting to the CPUExecutionProvider. Why do you think that's happening? Is that expected? Please refer to the notebook link above.

[onuralpszr]

So the "Failed to create CUDAExecutionProvider" error is no longer popping up. So that's a positive. But it seems to be taking same time per iteration as it was when defaulting to the CPUExecutionProvider. Why do you think that's happening? Is that expected? Please refer to the notebook link above.

After import all could you check is it gpu or cpu like in the collab I made in your collab ?

[onuralpszr]

Like this

[hkhare42]

It says 'GPU'