Why are fields marked with exclude=True still sent to the LLM?

[sweepy84]

I may have misunderstood the purpose of exclude but it appears that even fields marked with exclude=True are still sent to the LLM as required, and the LLM responds accordingly with these values. It appears that these are then ignored by Instructor, but wouldn't it have been better to save on the tokens and not send them at all?

[covitof]

Hello.

exclude=True excludes the field from the output of model_json_dump(), not from the schema! Meaning

class UserDetail(BaseModel):
    age: int = Field(description="Age of the user", exclude=True)
    name: str
   
print(UserDetail.model_json_schema())
"""Prints the FULL schema, age included."""

print(UserDetail(age=33, name='aaa').model_dump_json())
"""Prints name only."""

Instructor obviously uses the schema definition to pass the LLM, hence "exclude" does not work to hide fields. All the workarounds I can think of are quire convoluted: e.g. one could only have the necessary fields in the schema definition and override the call method of the model where one includes the extra arguments. This means there is the extra step of calling this method after the initial model validation. Not sure if there's a better/faster way. I can show you my initial workaround (which requires a local version of instructor though.. which is not ideal I'd say)

[sweepy84]

thanks @covitof

I still don't understand the purpose of this parameter, if it's excluded from the generated model, why is it being sent to the LLM at all?

Surely this is not an oversight, as it seems strange to me given the precious nature of token usage. Unnecessary token usage would in my mind be of paramount importance. Developer should be in control of what is sent to the LLM.

I guess a solution would be to have 2 models (one for LLM and one for the application) both extending the same abstract model and easily convertible from one to the other.

Or fork Instructor and customise.

[jxnl]

A field exclude parameter comes from Pydantic. It is not a part of instructor. The point of exclude is to exclude it from the __repr__

If it's so important, why do you even have that attribute in your model?

[sweepy84]

The attribute is crucial for post-processing in our system, such as generating a unique UUID for database persistence. However, it's not needed for the LLM's processing. By controlling which attributes the LLM receives, we ensure it focuses only on relevant data. This not only improves efficiency but also maintains the functional separation of system components, keeping system-level details like UUIDs separate from the LLM's operations.

[jxnl]

you can either do

uuid: Optional[str] = Field(default=None)

or

do some inheritence, since exclude=True is a function of pydantic and not of the json_schema_model

or

>>> import pydantic
>>> 
>>> 
>>> class A(pydantic.BaseModel):
...     a: int 
...     _b: int
... 
>>> 
>>> A.model_json_schema()
{'properties': {'a': {'title': 'A', 'type': 'integer'}}, 'required': ['a'], 'title': 'A', 'type': 'object'}

[sweepy84]

thanks @jxnl - I think I will do some inheritance. Has the added benefit of clarity, I know when I look at the class that everything is going to the LLM.

[leobeeson]

The purpose of fields with exclude=True being sent to the LLM is quiet powerful. For a simple example:

import instructor
from pydantic import BaseModel, Field
from openai import OpenAI
from datetime import date

client = instructor.patch(OpenAI())

class DateRangeWithInstruction(BaseModel):
    instruction: str = Field(
        description="Generate the start and end date of World War II.", exclude=True
    )
    start_date: date
    end_date: date


def extract_entities(
        model = "gpt-4-0125-preview"
        ) -> DateRangeWithInstruction:
    messages=[
        {"role": "user", "content": "Generate a date range."},
    ]
    response = client.chat.completions.create(
        model=model,
        response_model=DateRangeWithInstruction,
        messages = messages,
    )
    return response

date_range_inst = extract_entities()

print(date_range_inst.model_dump_json())
#> {"start_date":"1939-09-01","end_date":"1945-09-02"}

Although you probably wont want to embed the actual task as above, it serves as an example of how you can embed specific instructions, examples, chain of though, etc. straight within the schema, instead of having to use argstrings or more complex data models when it's not necessary.