Eval bug: LiquidAI/LFM2-8B-A1B-GGUF crashes #4
Description
Name and Version
build/bin/llama-cli --version
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: Apple M1 Max
ggml_metal_device_init: GPU family: MTLGPUFamilyApple7 (1007)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal3 (5001)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 51539.61 MB
version: 7727 (0e52270)
built with AppleClang 16.0.0.16000026 for Darwin arm64
git clone https://github.com/pwilkin/llama.cpp.git
git checkout autoparser
cmake -B build
cmake --build build --config Release -j 12
build/bin/llama-server --port 8080 --ctx-size 4096 -ngl 99 --jinja -hf LiquidAI/LFM2-8B-A1B-GGUF
curl http://localhost:8080/v1/chat/completions
-X POST
-H "Content-Type: application/json"
-H "Authorization: Bearer YOUR_API_KEY"
-d '{
"model": "LiquidAI/LFM2-8B-A1B-GGUF",
"messages": [
{
"role": "user",
"content": "What time is it now?"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Get the current time in a given timezone",
"parameters": {
"type": "object",
"properties": {
"timezone": {
"type": "string",
"description": "The timezone to get the current time for, e.g.,
"America/New_York""
}
},
"required": ["timezone"]
}
}
}
],
"tool_choice": "auto"
}'
curl: (52) Empty reply from server
/Users/m/tmp/llama.cpp/common/chat.cpp:1198: Legacy parsers have been deprecated
WARNING: Using native backtrace. Set GGML_BACKTRACE_LLDB for more info.
WARNING: GGML_BACKTRACE_LLDB may cause native MacOS Terminal.app to crash.
See: ggml-org#17869
0 libggml-base.0.9.5.dylib 0x000000010199c640 ggml_print_backtrace + 276
1 libggml-base.0.9.5.dylib 0x000000010199c82c ggml_abort + 156
2 llama-server 0x0000000100d43e44 _Z21common_chat_peg_parseRK16common_peg_arenaRKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEbRK18common_chat_syntax + 0
3 llama-server 0x0000000100c3cd24 _ZN17task_result_state15update_chat_msgERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEbRNS0_6vectorI20common_chat_msg_diffNS4_ISA_EEEEb + 216
4 llama-server 0x0000000100c4bf4c _ZN29server_task_result_cmpl_final6updateER17task_result_state + 68
5 llama-server 0x0000000100c57744 _ZN22server_response_reader4nextERKNSt3__18functionIFbvEEE + 288
6 llama-server 0x0000000100c57b58 _ZN22server_response_reader12wait_for_allERKNSt3__18functionIFbvEEE + 148
7 llama-server 0x0000000100c7f688 _ZN13server_routes23handle_completions_implERK15server_http_req16server_task_typeRKN8nlohmann16json_abi_v3_12_010basic_jsonINS5_11ordered_mapENSt3__16vectorENS8_12basic_stringIcNS8_11char_traitsIcEENS8_9allocatorIcEEEEbxydSD_NS5_14adl_serializerENS9_IhNSD_IhEEEEvEERKNS9_ISI_NSD_ISI_EEEE18task_response_type + 5336
8 llama-server 0x0000000100c9dffc _ZNSt3__110__function6__funcIZN13server_routes11init_routesEvE4$10NS_9allocatorIS3_EEFNS_10unique_ptrI15server_http_resNS_14default_deleteIS7_EEEERK15server_http_reqEEclESD + 240
9 llama-server 0x0000000100beaf3c _ZNSt3__110__function6__funcIZL10ex_wrapperNS_8functionIFNS_10unique_ptrI15server_http_resNS_14default_deleteIS4_EEEERK15server_http_reqEEEE3$0NS_9allocatorISD_EESB_EclESA + 76
10 llama-server 0x0000000100c07ef4 _ZNSt3__110__function6__funcIZNK19server_http_context4postERKNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEERKNS_8functionIFNS_10unique_ptrI15server_http_resNS_14default_deleteISD_EEEERK15server_http_reqEEEE3$0NS6_ISO_EEFvRKN7httplib7RequestERNSQ_8ResponseEEEclEST_SV + 460
11 llama-server 0x0000000100df8dcc _ZN7httplib6Server7routingERNS_7RequestERNS_8ResponseERNS_6StreamE + 3096
12 llama-server 0x0000000100dfa2a0 _ZN7httplib6Server15process_requestERNS_6StreamERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEiSB_ibRbRKNS3_8functionIFvRNS_7RequestEEEE + 3128
13 llama-server 0x0000000100dfad48 _ZN7httplib6Server24process_and_close_socketEi + 572
14 llama-server 0x0000000100c058a4 _ZN7httplib10ThreadPool6workerclEv + 456
15 llama-server 0x0000000100c0561c ZNSt3__114__thread_proxyB8ne180100INS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEN7httplib10ThreadPool6workerEEEEEEPvSB + 52
16 libsystem_pthread.dylib 0x000000018b2f3c0c _pthread_start + 136
17 libsystem_pthread.dylib 0x000000018b2eeb80 thread_start + 8
Operating systems
Mac
GGML backends
Metal
Hardware
Macbook M1 Max 64GB
Models
No response
Problem description & steps to reproduce
see above
First Bad Commit
No response