[Usage]: Could you guide a little bit more for EPIC usage?

[YEOMJINSEOP]

Your current environment

Hi, thanks for sharing EPIC code!

I wanna try to run EPIC (even it is only for functionality), However, it's not easy to know how to start EPIC.

Could you provide some scripts and guide to which file should i focus?

Best regards,
Jinseop Yeom

How would you like to use vllm

I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.

[shijuzhao]

Hi, you can start with the model file:

diff -y vllm/model_executor/models/llama.py vllm/model_executor/models/llama_org.py

It shows how CacheBlend and EPIC recompute KV cache.

And you can find the scripts for benchmarking in benchmark_ours/.