Skip to content

Commit

Permalink
add infer time
Browse files Browse the repository at this point in the history
  • Loading branch information
techshoww committed Mar 3, 2025
1 parent 536734f commit 4ed84a7
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,14 @@ The street is marked with white lines, indicating parking or pedestrian areas. T
In summary, the image captures a moment in a historic urban setting with a red double-decker bus, an advertisement for Virgin Money, and a smiling woman standing on the sidewalk. The scene is characterized by classic architectural buildings and a calm street environment.<|im_end|>
```

## 模型速度
| Vision Encoder | Time to First Token (ms) |
|------|------|
| U16 PTQ | 824 |
| Mixed PTQ | |

Language Model Decode: 3.5 tokens/s .

## 关于 mrope
### 一、Qwen2.5-VL 中的 multimodal_rotary_embedding(mrope)和 rope 的区别
1. mrope的 position id 是三维的(temporal,height,width),rope 是一维的
Expand Down

0 comments on commit 4ed84a7

Please sign in to comment.