feat: Function calling with output schema enforcement

[jeffreyftang]

Introduces function-calling capabilities with JSON schema enforcement on output.

Example using Mistral-7B-Instruct-v0.3:

curl 127.0.0.1:8080/generate -X POST -d '{
   "inputs": "WHat is the current temperature of New York, San Francisco and Chicago?",
   "parameters": {
       "max_new_tokens": 512,
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_current_weather",
                    "description": "Get the current weather in a given location",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "location": {
                                "type": "string",
                                "description": "The city and state, e.g. San Francisco, CA"
                            },
                            "unit": {
                                "type": "string",
                                "enum": [
                                    "celsius",
                                    "fahrenheit"
                                ]
                            }
                        }
                    }
                }
            }
        ]
   }
}' -H 'Content-Type: application/json'


{"generated_text":"[{ \"id\": \"00000173499999999683\", \"type\": \"function\", \"function\": { \"name\": \"get_current_weather\", \"arguments\": { \"location\": \"New York\" } } },{ \"id\": \"00000173499999999684\", \"type\": \"function\", \"function\": { \"name\": \"get_current_weather\", \"arguments\": { \"location\": \"San Francisco\" } } },{ \"id\": \"00000173499999999685\", \"type\": \"function\", \"function\": { \"name\": \"get_current_weather\", \"arguments\": { \"location\": \"Chicago\" } } }]"}

[llama-shepard]

@tgaddair @magdyksaleh

Waiting for this merge to try out Function Calling in Lorax

[jeffreyftang]

Failing test doesn't seem related, but will have to dig more.

[llama-shepard]

@magdyksaleh Still waiting for this feature. I want to deploy Mistral Nemo and Llama 3.1 models with function calling

[llama-shepard]

I have been waiting for function calling feature for long time and got exited for this PR. But this PR is still not merged.

@tgaddair @jeffreyftang

[jeffreyftang]

@llama-shepard Sorry, I've been preoccupied with other obligations. What's remaining on this PR is to verify that pulling in chat templates will provide support for models other than mistral 3 (which is already working).

[nikhilh]

This would be extremely useful for me. When can we expect this to be merged?