Unable to get response from Lesson 04: Decision Making

[AseemWangoo]

Hey @pguso

• I was trying to run Lesson 04, but it always hangs the terminal and takes time to respond
• My model, (see image below)

• Temperature: Tried from 0.0 to 1.0

• The output is always

LESSON 04: Decision Making
==================================================
Decision: None

==================================================
All examples completed!

Anything I need to tweak?

[pguso]

Hey @AseemWangoo

Can you send me the exact download link to the model you are using?

Please also change the following in llm.py comment out line 16 and change in line 45 verbose to True. And then send me the output you are getting

[AseemWangoo]

Model Link: https://huggingface.co/bartowski/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf

Full Version Log:

##################################################
# AI Agent Examples - All Lessons
##################################################

==================================================
LESSON 04: Decision Making
==================================================
Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | REPACK = 1 | 
Model metadata: {'quantize.imatrix.entries_count': '224', 'general.quantization_version': '2', 'tokenizer.chat_template': "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}", 'tokenizer.ggml.eos_token_id': '128001', 'quantize.imatrix.file': '/models/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct.imatrix', 'tokenizer.ggml.bos_token_id': '128000', 'tokenizer.ggml.pre': 'llama-bpe', 'quantize.imatrix.chunks_count': '88', 'tokenizer.ggml.model': 'gpt2', 'llama.vocab_size': '128256', 'llama.attention.head_count_kv': '8', 'llama.context_length': '8192', 'llama.attention.head_count': '32', 'general.file_type': '15', 'llama.feed_forward_length': '14336', 'quantize.imatrix.dataset': '/training_data/groups_merged.txt', 'llama.rope.dimension_count': '128', 'llama.rope.freq_base': '500000.000000', 'llama.embedding_length': '4096', 'general.architecture': 'llama', 'llama.attention.layer_norm_rms_epsilon': '0.000010', 'general.name': 'Meta-Llama-3-8B-Instruct', 'llama.block_count': '32'}
Available chat formats from metadata: chat_template.default
Guessed chat format: llama-3
Llama.generate: 129 prefix-match hit, remaining 1 prompt tokens to eval
Llama.generate: 129 prefix-match hit, remaining 1 prompt tokens to eval
Decision: None

==================================================
All examples completed!
==================================================

[AseemWangoo]

Hey @pguso any chance the above logs were helpful?