Skip to content
Test SHARK

[WIP][Llama2] Add KVCache for prefill stage + interactive chat mode in llm_runner + StreamingLLM. #23

Sign in to view logs

[WIP][Llama2] Add KVCache for prefill stage + interactive chat mode in llm_runner + StreamingLLM.

[WIP][Llama2] Add KVCache for prefill stage + interactive chat mode in llm_runner + StreamingLLM. #23

Run time

Learn about OS pricing on GitHub Actions
Job Run time
test-shark (3.11, nodai-ubuntu-builder-large)
8m 47s
8m 47s