ValueError: not enough values to unpack (expected 2, got 1)

[TDHHTXF]

i was using llava.server.cli to inference,and i use visionzip ,and got the bug:

Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00, 1.78s/it]
USER: test
Traceback (most recent call last):
File "/home/niumo/anaconda3/envs/llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/niumo/anaconda3/envs/llava/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/niumo/LLaVA/llava/serve/cli.py", line 129, in
main(args)
File "/home/niumo/LLaVA/llava/serve/cli.py", line 99, in main
output_ids = model.generate(
File "/home/niumo/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/niumo/LLaVA/llava/model/language_model/llava_llama.py", line 125, in generate
) = self.prepare_inputs_labels_for_multimodal(
File "/home/niumo/visionzip/visionzip/llava_arch.py", line 135, in prepare_inputs_labels_for_multimodal_visionzip
image_features = self.encode_images_visionzip(images)
File "/home/niumo/visionzip/visionzip/llava_arch.py", line 32, in encode_images_visionzip
image_features, keep_idx = self.get_model().get_vision_tower().forward(images)
ValueError: not enough values to unpack (expected 2, got 1)

[TDHHTXF]

in the "clip vision tower" model，"llava" returns image_features，
but"vision zip" returns hidden_states_save, all_indices

and i think this is the reason of the following bug:

image_features, keep_idx = self.get_model().get_vision_tower().forward(images)
ValueError: not enough values to unpack (expected 2, got 1)

and i try eval code you provide:

Load LLaVA Model (code from llava.eval.model_vqa_loader)

tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, args.model_base, model_name)

add VisionZip

from visionzip import visionzip
model = visionzip(model, dominant=54, contextual=10)

and get the same bug

[Yangsenqiao]

I recommend checking this issue: #19. I think you should ensure the code executes [this funcition](https://github.com/dvlab-research/VisionZip/blob/3151f6df4fca36a4cd1c985e5956e725b1f9ad02/visionzip/clip_encoder.py#L32) instead of the original forward method.

[cokeshao]

@TDHHTXF Hello, I have the same problem. Have you solved it?

[cokeshao]

I found the reason. My steps:

1. install LLaVA-NeXT environment.
2. install visionzip
3. uninstall LLaVA-NeXT and install the LLaVA environment.
   This will make the forward function replace fail.

[XinweiZhang1998]

I have the same issue, but can not be solved now.

[laiyingxin2]

same question~