You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
when I run "python ./pyllama/quant_infer.py --wbits 4 --load ./output/654b.pt --text "what is life" --max_length 24 --cuda cuda:0" , I got the error below.
I converted llama 65B into 4int, with the file 654b.pt. Everything has followed the instruction here, anyone could help?
++++++++++
RuntimeError: Error(s) in loading state_dict for LLaMAForCausalLM:
Unexpected key(s) in state_dict: "model.layers.32.self_attn.q_proj.zeros", "model.layers.32.self_attn.q_proj.scales",
"model.layers.32.self_attn.q_proj.bias", "model.layers.32.self_attn.q_proj.qweight", "model.layers.32.self_attn.k_proj.zeros",
"model.layers.32.self_attn.k_proj.scales", "model.layers.32.self_attn.k_proj.bias", "model.layers.32.self_attn.k_proj.qweight",
"model.layers.32.self_attn.v_proj.zeros", "model.layers.32.self_attn.v_proj.scales", "model.layers.32.self_attn.v_proj.bias",
"model.layers.32.self_attn.v_proj.qweight", "model.layers.32.self_attn.o_proj.zeros", "model.layers.32.self_attn.o_proj.scales",
"model.layers.32.self_attn.o_proj.bias", "model.layers.32.self_attn.o_proj.qweight", "model.layers.32.self_attn.rotary_emb.inv_freq",
"model.layers.32.mlp.gate_proj.zeros", "model.layers.32.mlp.gate_proj.scales", "model.layers.32.mlp.gate_proj.bias", "model.layers.32.mlp.gate_proj.qweight",
"model.layers.32.mlp.down_proj.zeros", "model.layers.32.mlp.down_proj.scales", "model.layers.32.mlp.down_proj.bias", "model.layers.32.mlp.down_proj.qweight",
"model.layers.32.mlp.up_proj.zeros", "model.layers.32.mlp.up_proj.scales", "model.layers.32.mlp.up_proj.bias", "model.layers.32.mlp.up_proj.qweight",
"model.layers.32.input_layernorm.weight", "model.layers.32.post_attention_layernorm.weight", "model.layers.33.self_attn.q_proj.zeros",
"model.layers.33.self_attn.q_proj.scales", ......
......
size mismatch for model.layers.31.mlp.up_proj.scales: copying a param with shape torch.Size([22016, 1]) from checkpoint, the shape in current model is
torch.Size([11008, 1]).
size mismatch for model.layers.31.mlp.up_proj.bias: copying a param with shape torch.Size([22016]) from checkpoint, the shape in current model is
torch.Size([11008]).
size mismatch for model.layers.31.mlp.up_proj.qweight: copying a param with shape torch.Size([1024, 22016]) from checkpoint, the shape in current model is
torch.Size([512, 11008]).
size mismatch for model.layers.31.input_layernorm.weight: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is
torch.Size([4096]).
size mismatch for model.layers.31.post_attention_layernorm.weight: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current
model is torch.Size([4096]).
size mismatch for model.norm.weight: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for lm_head.weight: copying a param with shape torch.Size([32000, 8192]) from checkpoint, the shape in current model is torch.Size([32000,
4096]).
The text was updated successfully, but these errors were encountered:
when I run "python ./pyllama/quant_infer.py --wbits 4 --load ./output/654b.pt --text "what is life" --max_length 24 --cuda cuda:0" , I got the error below.
I converted llama 65B into 4int, with the file 654b.pt. Everything has followed the instruction here, anyone could help?
++++++++++
RuntimeError: Error(s) in loading state_dict for LLaMAForCausalLM:
Unexpected key(s) in state_dict: "model.layers.32.self_attn.q_proj.zeros", "model.layers.32.self_attn.q_proj.scales",
"model.layers.32.self_attn.q_proj.bias", "model.layers.32.self_attn.q_proj.qweight", "model.layers.32.self_attn.k_proj.zeros",
"model.layers.32.self_attn.k_proj.scales", "model.layers.32.self_attn.k_proj.bias", "model.layers.32.self_attn.k_proj.qweight",
"model.layers.32.self_attn.v_proj.zeros", "model.layers.32.self_attn.v_proj.scales", "model.layers.32.self_attn.v_proj.bias",
"model.layers.32.self_attn.v_proj.qweight", "model.layers.32.self_attn.o_proj.zeros", "model.layers.32.self_attn.o_proj.scales",
"model.layers.32.self_attn.o_proj.bias", "model.layers.32.self_attn.o_proj.qweight", "model.layers.32.self_attn.rotary_emb.inv_freq",
"model.layers.32.mlp.gate_proj.zeros", "model.layers.32.mlp.gate_proj.scales", "model.layers.32.mlp.gate_proj.bias", "model.layers.32.mlp.gate_proj.qweight",
"model.layers.32.mlp.down_proj.zeros", "model.layers.32.mlp.down_proj.scales", "model.layers.32.mlp.down_proj.bias", "model.layers.32.mlp.down_proj.qweight",
"model.layers.32.mlp.up_proj.zeros", "model.layers.32.mlp.up_proj.scales", "model.layers.32.mlp.up_proj.bias", "model.layers.32.mlp.up_proj.qweight",
"model.layers.32.input_layernorm.weight", "model.layers.32.post_attention_layernorm.weight", "model.layers.33.self_attn.q_proj.zeros",
"model.layers.33.self_attn.q_proj.scales", ......
......
size mismatch for model.layers.31.mlp.up_proj.scales: copying a param with shape torch.Size([22016, 1]) from checkpoint, the shape in current model is
torch.Size([11008, 1]).
size mismatch for model.layers.31.mlp.up_proj.bias: copying a param with shape torch.Size([22016]) from checkpoint, the shape in current model is
torch.Size([11008]).
size mismatch for model.layers.31.mlp.up_proj.qweight: copying a param with shape torch.Size([1024, 22016]) from checkpoint, the shape in current model is
torch.Size([512, 11008]).
size mismatch for model.layers.31.input_layernorm.weight: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is
torch.Size([4096]).
size mismatch for model.layers.31.post_attention_layernorm.weight: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current
model is torch.Size([4096]).
size mismatch for model.norm.weight: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for lm_head.weight: copying a param with shape torch.Size([32000, 8192]) from checkpoint, the shape in current model is torch.Size([32000,
4096]).
The text was updated successfully, but these errors were encountered: