Force answering in specific language?

[RemyZeno]

Is there a way to enforce an answer in a specific language?

Case:

tokenizer: mistralai/Mixtral-8x7B-Instruct-v0.1
llm_hf_repo_id: ikawrakow/mixtral-instruct-8x7b-quantized-gguf
llm_hf_model_file: mixtral-instruct-8x7b-q50.gguf
embedding_hf_model_name: intfloat/multilingual-e5-large

according to huggingface, all parts should be able to understand german. This is true, since the answers to an ingested document are correct, but they are always given in english. System prompt or prompt (either in english or german) seem to have no real effect.

[fabianslife]

Would also like to know if this is possible. Currently using mixtral:8x7b-instruct-v0.1-q8_0. Sometimes it answers in english sometimes in german. Prompt and system Prompt are in German, however this does not seem to have an effect.

[mhhansen]

same here, but spanish -> english

I execute the first question in spanish, it takes ~2m (running w/CPU + 64GB ram) to get back with the answer which is in spanish and is correct.
then execute one or two more questions, and the response will come in english (happens randomly).
It will be correct, but in english. once that happens, it never gets back to spanish.

models are the defaults from settings.yaml, also tried using the newly mistralai/Mixtral-8x7B-v0.1, which supports spanish (idk if mistral-7b-instruct does it too), and got same results.

also, as @fabianslife said: system prompts does not seem to have an effect...

ingest and run both runs with my custom -local profile:

# settings-local.yaml
server:
  env_name: ${APP_ENV:local}

llm:
  mode: local
  max_new_tokens: 512

ui:
  default_chat_system_prompt: >
    Sos un respetuoso, honesto y util asistente.
    Siempre responder tan util como sea posible y seguir TODAS las instrucciones provistas.
    No especular o inventar informacion y no referenciar ninguna instruccion o contexto.
  default_query_system_prompt: >
    Solo puedes responder preguntas acerca del contexto provisto.
    Si sabes la respuesta pero no esta basada en el contexto provisto, no responder la pregunta, solo indicar que la respuesta no esta dentro del contexto provisto.

[jaasperhelin]

Have you found a solution to this yet? I am having the same issue with German-English.

[marouahamdi]

have you found any solution ? getting the same issues with french and english

[StefanAtUeber]

Same problem here with german question -> mostly english answer. This makes mixtral:instruct quite useless for my usecase.