why block size 4 is pretty slow? #35
Description
Thanks for your great work.
When I change the block_length in eval_llada_mini.sh from 32 to 4, the generation rate becomes pretty slow (e.g., 90 minutes for gsm8k_cot). Can you tell me why?
{{ message }}