Thoughts on String Formatting

[jxnl]

Now that we have our own SDK we could do much more interesting things like template variables.
what are folks thoughts on something like

client.chat.completion.create(
    messages=[{
        "role": "user",
        "content": "Follow the instructions {{instructions}} to extract data out of {{data}}"
    }],
    messages_context={"instructions": "...", "data": "..."},
    response_model=ResponseModel
)

1. This would use jinja behind the scenes, but would effectively allow for template variables however if you never need them you don't have to set messages_context.

2. This would also allow us to use one messages_context for both formatting the messages and also providing it to the validation_context

class ResponseModel(BaseModel):
    ...

    def validate(self, v, info: str):
        do_something(v, info.context["data"], info.context["instructions"]) # from the messages_context
        return v

What are folks thoughts on something like this?

[palako]

When I moved from langchain to instructor, I found it liberating that I could just use plain python fstrings. Those template variables, in my opinion, don't add any extra value on top of what modern python allows, fstrings are actually really flexible beyond the basic use case. To me, referring to variables in double brackets only to have to pass their values later on was only a source of problems (which sadly python isn't great at helping with at coding time).

For that, my inclination would be to leave this as is.

[jxnl]

so you'd still be able use f strings, but for validation it could kill two birds with oen stone.

don't add any extra value on top of what modern python allows

the value is for logging you can track it all seperatetly, in many situations we have PII data, so we want to love the prompt but not the {data} thats is injected into the prompt.

[palako]

ah cool. I can see where you're going. It's a tricky one from an architectural perspective though, because now the prompt gets fully coupled with the validation objects. I, for example, use the same response_model objects in several prompts, and if I were to check variable names inside the validator other that what I pass as validation_context, it would break stuff all the time, and the validators code would get really messy. But I can see how for simple cases this can solve the problem you are going for.
The problem with logging PII is tricky. This solution kicks the can down the road by a level, allowing you to log in the validators, but at some point these templates get resolved, so if for example the validators use other objects or functions to encapsulate more complex code, either you keep passing everything as individual parameters, or you prevent those from logging.