Removes batch size from prefill attention calculation.#600

Merged

copybara-service[bot] merged 5 commits intomainfrom

patemotter_attn_calc_fix

Apr 19, 2024

Commits on Apr 19, 2024

Merge branch 'main' into patemotter_attn_calc_fix
patemotter
authored