how to make dataset format for fine tuning llm for langchain's agent (e.g create_sql_agent)?

[octadion]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

# fine tuning method & dataset format

Description

i want to fine tune opensource llm like llama 3 to specific langchain agent format like langchain create_sql_agent, etc. how to do that? and how's the dataset format looks like? anyone ever do this? thank you!

System Info

"pip freeze"
platform (windows)
python version (3.0)
langchain:
langchain==0.0.352
langchain-community==0.0.13
langchain-core==0.1.13
langchain-experimental==0.0.42
langchain-google-genai==1.0.1
langchain-openai==0.0.2.post1
langserve==0.0.36
langsmith==0.0.83

[dosubot[bot]]

To fine-tune an open-source LLM like LLaMA 3 to a specific LangChain agent format, such as LangChain's create_sql_agent, you need to follow these steps:

1. Prepare the Dataset: The dataset should include examples of inputs and expected outputs that the agent should handle. For create_sql_agent, this would typically involve SQL queries and the corresponding responses. The dataset format should be in a structured format like JSON, where each entry includes the input prompt and the expected output.

   Example dataset format:

   [
       {
           "input": "SELECT * FROM users WHERE age > 30;",
           "output": "Here are the users older than 30: [list of users]"
       },
       {
           "input": "What is the total revenue for the last quarter?",
           "output": "The total revenue for the last quarter is $1,000,000."
       }
   ]

2. Fine-Tuning the Model: Use a framework like Hugging Face's transformers to fine-tune the LLaMA model on your dataset. Below is a simplified example of how you might do this:

   from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
   
   # Load the pre-trained model and tokenizer
   model_name = "LLaMA-3"
   model = AutoModelForCausalLM.from_pretrained(model_name)
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   
   # Load your dataset
   dataset = [
       {"input": "SELECT * FROM users WHERE age > 30;", "output": "Here are the users older than 30: [list of users]"},
       {"input": "What is the total revenue for the last quarter?", "output": "The total revenue for the last quarter is $1,000,000."}
   ]
   
   # Tokenize the dataset
   def tokenize_function(examples):
       return tokenizer(examples["input"], padding="max_length", truncation=True)
   
   tokenized_datasets = dataset.map(tokenize_function, batched=True)
   
   # Define training arguments
   training_args = TrainingArguments(
       output_dir="./results",
       evaluation_strategy="epoch",
       learning_rate=2e-5,
       per_device_train_batch_size=8,
       per_device_eval_batch_size=8,
       num_train_epochs=3,
       weight_decay=0.01,
   )
   
   # Initialize the Trainer
   trainer = Trainer(
       model=model,
       args=training_args,
       train_dataset=tokenized_datasets["train"],
       eval_dataset=tokenized_datasets["test"],
   )
   
   # Train the model
   trainer.train()

3. Integrate with LangChain: Once the model is fine-tuned, you can integrate it with LangChain's create_sql_agent function. Here is an example of how to do this:

   from langchain_community.agent_toolkits import SQLDatabaseToolkit, create_sql_agent
   from langchain_community.utilities.sql_database import SQLDatabase
   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   # Load the fine-tuned model
   class FineTunedLLaMA:
       def __init__(self, model_name):
           self.model = AutoModelForCausalLM.from_pretrained(model_name)
           self.tokenizer = AutoTokenizer.from_pretrained(model_name)
   
       def generate(self, prompt):
           inputs = self.tokenizer(prompt, return_tensors="pt")
           outputs = self.model.generate(**inputs)
           return self.tokenizer.decode(outputs[0], skip_special_tokens=True)
   
   llm = FineTunedLLaMA("path/to/fine-tuned-llama")
   
   # Create the SQL agent
   db = SQLDatabase.from_uri("sqlite:///Chinook.db")
   toolkit = SQLDatabaseToolkit(db=db, llm=llm)
   agent_executor = create_sql_agent(llm=llm, toolkit=toolkit)
   
   # Test the agent
   assert agent_executor.run("SELECT * FROM users WHERE age > 30;") == "Here are the users older than 30: [list of users]"

Additionally, you can use the provided SQL prompt templates from LangChain to ensure your model generates the correct SQL queries. Here is an example of the SQL prompt template:

SQL_PREFIX = """You are an agent designed to interact with a SQL database.
Given an input question, create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.
Unless the user specifies a specific number of examples they wish to obtain, always limit your query to at most {top_k} results.
You can order the results by a relevant column to return the most interesting examples in the database.
Never query for all the columns from a specific table, only ask for the relevant columns given the question.
You have access to tools for interacting with the database.
Only use the below tools. Only use the information returned by the below tools to construct your final answer.
You MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.

DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.

If the question does not seem related to the database, just return "I don't know" as the answer.
"""

SQL_SUFFIX = """Begin!

Question: {input}
Thought: I should look at the tables in the database to see what I can query.  Then I should query the schema of the most relevant tables.
{agent_scratchpad}"""

This process involves preparing a dataset, fine-tuning the model, and then integrating the fine-tuned model with LangChain's create_sql_agent function using the appropriate SQL prompt templates ^[1][2][3].

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[GameModes]

@dosu can you still use the agent_type tool-calling in this method if you use a supported model such as OpenAI or Gemini?