Implement common utilities for guardrails

[lordofthejars]

I expect a lot of boilerplate code on guardrails in several projects, for example, validate that the output is a JSON, or that the JSON follows a JSON schema, or the same for YAML. For sanitizing output, for example, the output is a JSON, but at the end, it has some markdown characters, even though we explicitly said to the model to not do it.

So maybe we could implement a new module (or a package into core) where we offer some standalone classes for some of these common features:

• Is the output a JSON
• Is the JSON follow this JSON Schema
• Removes everything from an output that is not a JSON
• Prompt-Injection

These are the first things that come to my mind but I am sure there can be more.

Maybe the module could be called guardrail-json another guardrail-yaml, guardrail-prompt-injection,... so they are only the utilities, everyone is free to use them or not, but at least we'll help users to not repeat the same checks over and over again.

[geoand]

cc @cescoffier @mariofusco

[cescoffier]

I have been thinking about it since day one and have added ideas to the list (with the streaming accumulator). So +1. I don't know if we need to provide guardrail implementation (with some customization) or if we only need to provide utility methods.

[lordofthejars]

My point would be to have two classes: the utility and the guardrail.

The Guardrail uses the utility class to implement the logic; in this way, we let users decide whether to use the Guardrail or implement their own guardrail with these utilities classes (for example, composing different logic in the same guardrail).