Skip to content

Commit d2eeaf9

Browse files
committed
Add documentation for model dispatch
1 parent 038a70a commit d2eeaf9

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

docs/reference/dispatch.md

+23
Original file line numberDiff line numberDiff line change
@@ -1 +1,24 @@
11
# Model-based prompt dispatching
2+
3+
4+
Different models often require different prompts to achieve a given task. They are, in essence, not different prompts in the sense that they are supposed to perform the same operation. In the same way we use `functools.singledispatch` to dispatch a functionality on the type of the first
5+
argument, it can be useful to dispatch the prompt on the model that is being used.
6+
7+
`prompts` provides a way to dispatch the prompt on the model:
8+
9+
10+
```python
11+
import prompts
12+
13+
14+
@prompts.template
15+
def a_simple_prompt(query: str):
16+
"""<s>{{ query }}</s>"""
17+
18+
@a_simple_prompt.register("google/gemma-2-9b")
19+
def a_simple_prompt_gemma(query: str):
20+
"""<bos>{{ query }}<eos>"""
21+
```
22+
23+
!!! note
24+
Choosing BOS and EOS based on the model is better achieved by using [special variables](special_tokens.md).

0 commit comments

Comments
 (0)