You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
what's the difference for speculative-eagle-topk speculative-num-draft-tokens ,from the example, by default it will use --speculative-eagle-topk 8 --speculative-num-draft-tokens 64, and from the help ,it means:
--speculative-num-draft-tokens SPECULATIVE_NUM_DRAFT_TOKENS
The number of token sampled from draft model in Speculative Decoding.
--speculative-eagle-topk {1,2,4,8}
The number of token sampled from draft model in eagle2 each step.
My question is:
which one is the value for tokens numbers which generated by draft model first? speculative-num-draft-tokens or speculative-eagle-topk? And what does mean for another one.
Thanks.
The text was updated successfully, but these errors were encountered:
what's the difference for speculative-eagle-topk speculative-num-draft-tokens ,from the example, by default it will use --speculative-eagle-topk 8 --speculative-num-draft-tokens 64, and from the help ,it means:
--speculative-num-draft-tokens SPECULATIVE_NUM_DRAFT_TOKENS
The number of token sampled from draft model in Speculative Decoding.
--speculative-eagle-topk {1,2,4,8}
The number of token sampled from draft model in eagle2 each step.
My question is:
which one is the value for tokens numbers which generated by draft model first? speculative-num-draft-tokens or speculative-eagle-topk? And what does mean for another one.
Thanks.
The text was updated successfully, but these errors were encountered: