Skip to content

Commit 7b957eb

Browse files
committed
Update docs
1 parent 0d629c2 commit 7b957eb

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

docs/METHODOLOGY.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,17 +14,21 @@ LOGIC utilizes log probability distributions as a unique "fingerprint" of a mode
1414

1515
The verifier supports two matching strategies:
1616

17-
#### Token ID Matching (Default)
17+
#### Token ID Matching (Primary Method)
1818

1919
The standard approach uses token IDs to align tokens between the original and verification responses:
2020

2121
- Most accurate for providers that return token IDs (e.g., vLLM with `return_tokens_as_token_ids`)
2222
- Ensures exact token-level correspondence
2323
- Recommended when both sample and verification sources support token IDs
2424

25+
With vLLM, we can request token IDs with the `return_tokens_as_token_ids` parameter. The OpenAI API, however, does not support this parameter.
26+
2527
#### Text-Based Matching (Fallback)
2628

27-
For providers that don't return token IDs (e.g., OpenRouter, some OpenAI configurations), use the `--text-only-matching` flag:
29+
For providers that don't return token IDs (e.g., OpenRouter, OpenAI, etc.), the system can fallback to text-based matching. In this approach, we will reconstruct the context up to the position of a given token and then query the verification model with that specific context. We will then compare the original log probabilities with the fresh ones using direct text matching.
30+
31+
We can use the `--text-only-matching` flag:
2832

2933
```bash
3034
uv run logprob-sample \

0 commit comments

Comments
 (0)