Can it be used for Arabic language?

[ameen-samhan]

It worked out for English but when I tried it with Arabic it the returned answer was ???????? and even when I typed the question the Arabic letters where not connected and reversed even though i checked that the txt files is encoding utf8.

[WaleedAlfaris]

This is what I am currently looking into. If you find an answer please let me know

[selfmeister]

Are there Language Models trained in Arabic? Than you download a model that is compatible with GPT4all or LlamaCpp (.bin/.gguf for CPU), move it into models folder and change information in .env file accordingly. worked for my German model.

[WaleedAlfaris]

This is what I am currently trying. There is an English/Arabic multilingual LLM called JAIS which I am attempting to run. Will update when I know more

HF URL: https://huggingface.co/inception-mbzuai/jais-13b

[OdyAsh]

Got any updates @WaleedAlfaris ? I currently have personal arabic files which I'd like to act as a source for a Q&A app that I'm developing. Would be great to tell us if you've found a relative/working Arabic model for private-gpt! 🙌

[WaleedAlfaris]

Hey @OdyAsh . Unfortunately, I have not found an Arabic LLM that performs well. In the end, we went with translating the docs and using Mistral/OpenAI. If you do find one, please let me know.