The model will not load when I ask certain Questions

[MrVolts]

As you can see it answered to hey but did not even try to respond to my second question

[ghost]

As you can see it answered to hey but did not even try to respond to my second question

Hi, Open llama 3B is more of an "auto complete" model compared to an instructional model. It's possible for the model to respond to subjective questioning, but you'll need to tell it what to do so that it knows how to handle such questions. One way is by communicating it in the prompt, "You're an A.I. What do you like to eat?"

-n 20 parameter in your ./main prompt limits the response, though it shouldn't limit it to no response at all.

Here's an example of the same prompt on my system, notice the model adds onto what I asked instead of addressing the question like an individual:

./main -m ~/llama.cpp/models/open-llama-3b-q8_0.bin --color -c 2048 --keep -1 -t 4 -b 10 -p "what do you like to eat?" -n 20        

main: build = 0 (unknown)                         
main: seed  = 1687982281
llama.cpp: loading model from /data/data/com.termux/files/home/llama.cpp/models/open-llama-3b-q8_0.bin
llama_model_load_internal: format     = ggjt v3 (latest)                                            llama_model_load_internal: n_vocab    = 32000    
llama_model_load_internal: n_ctx      = 2048     
llama_model_load_internal: n_embd     = 3200     
llama_model_load_internal: n_mult     = 240       
llama_model_load_internal: n_head     = 32        
llama_model_load_internal: n_layer    = 26       
llama_model_load_internal: n_rot      = 100     
llama_model_load_internal: ftype      = 7 (mostly Q8_0)
llama_model_load_internal: n_ff       = 8640     
llama_model_load_internal: model size = 3B        
llama_model_load_internal: ggml ctx size =    0.06 MB                                               llama_model_load_internal: mem required  = 4496.51 MB (+  682.00 MB per state)                      llama_new_context_with_model: kv self size  =  650.00 MB
system_info: n_threads = 4 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | VSX = 0 |                                        
sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.800000, mirostat = 0,
mirostat_lr = 0.100000, mirostat_ent = 5.00000
generate: n_ctx = 2048, n_batch = 10, n_predict = 20, n_keep = 8                                                                                      

what do you like to eat?                         
What is your favorite food to eat? What is your favorite food to cook?
If you

llama_print_timings:        load time =   624.29 ms
llama_print_timings:      sample time =    46.28 ms /    20 runs   (    2.31 ms per token,   432.18 tokens per second)
llama_print_timings: prompt eval time =   804.19 ms /     8 tokens (  100.52 ms per token,     9.95 tokens per second)
llama_print_timings: eval time =  3909.26 ms /    19 runs   (  205.75 ms per token,     4.86 tokens per second)
llama_print_timings:       total time =  4769.12 ms

I suggest removing or increasing the value for the -n parameter.

[SlyEcho]

I cannot replicate the issue with build 740 (447ccbe). It seems like the program is crashing but of course Windows doesn't tell you this.

What is the hash of your model file? Mine is 57f12972d25bae770e4f38a3f1d843d84c4444b59a9f6218e2a7ed4d63d3b2f9 and it seems to work.

I have the my files on HF as well, if you want to compare.

Hi, Open llama 3B is more of an "auto complete" model compared to an instructional model.

It will still generate something, for me it just generates more similar questions like the input.

The real tell is that it doesn't even reach the message llama.cpp: loading model from ... and the model info after that which makes me believe it is crashing.

[ghost]

It will still generate something, for me it just generates more similar questions like the input.

The real tell is that it doesn't even reach the message llama.cpp: loading model from ... and the model info after that which makes me believe it is crashing.

Yeah, I figured it's a combination of things, but it does look like a crash now that you mention it. I forgot how informative and helpful Windows OS can be ;)