Benchmark info (#138)

casper-hansen · Nov 2, 2023 · 2bfa234 · 2bfa234
1 parent 0f3214b
commit 2bfa234
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -197,6 +197,9 @@ generation_output = model.generate(
 
 ## Benchmarks
 
+- GPU: RTX 3090
+- Command: `python examples/benchmark --model_path <hf_model>`
+
 | Model Name    | Version | Batch Size | Prefill Length | Decode Length | Prefill tokens/s | Decode tokens/s | Memory (VRAM)    |
 |---------------|---------|------------|----------------|---------------|------------------|-----------------|------------------|
 | Vicuna 7B     | GEMM    | 1          | 64             | 64            | 2618.88          | 125.428         | 4.57 GB (19.31%) |