You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Take a look at #15. Minigpt4 model is composed of two models (vision and text). The vision model does not support GPU usage, but the text model (vicuna) does.
Try enabling LLAMA_CUBLAS and see if you can run part of the model on the GPU. I haven't tested these flags before, but I would assume that they would work.
Hi,
I enabled the cublas compilation option.
The problem is that not charge o process all in GRAM memory?
What is the best line command to construct and execute in a CUDA 3090 with 24GB GRAM in the more fast posibility for each model?
The text was updated successfully, but these errors were encountered: