Skip to content

Batches

Jump to bottom
sisbell edited this page May 8, 2023 · 1 revision

Prompt

Batches are useful when you have a number of prompts that you want to generate output for.

First create a prompt file. Borrowing an example prompt from DeepLearning.AI, create a prompt file. Name the file whatever you like. In our case, it's product.prompt.

Your task is to generate a short summary of a product
review from an ecommerce site to give feedback to the
pricing department, responsible for determining the
price of the product.

Summarize the review below, delimited by triple
backticks, in at most 30 words, and focusing on any aspects
that are relevant to the price and perceived value.

Review: ```${prod_review}```

Data File

Next create a data file called product.json. This file contains your batch inputs.

Note how ${prod_review} in the prompt file matches the name of prod_review key in the data file. This is how the tool does the substitution for creating the calculated prompt. You can configure multiple variables in the prompt and data file.

{
  "prod_review" : [
    "Got this panda plush toy for my daughter's birthday, who loves it and takes it everywhere. It's soft and  super cute, and its face has a friendly look...",
    "Needed a nice lamp for my bedroom, and this one had additional storage and not too high of a price point. Got it fast - arrived in 2 days. The string to.."
   ]
}

You could also include another set of inputs to your prompt. Note that the size of the arrays must be the same.

{
  "prod_review" : [
    "Got this panda plush toy for my daughter's birthday, who loves it and takes it everywhere. It's soft and  super cute, and its face has a friendly look...",
    "Needed a nice lamp for my bedroom, and this one had additional storage and not too high of a price point. Got it fast - arrived in 2 days. The string to.."
   ],
  "some_action" : [
    "Expand this statement",
    "Reduce this statement"
  ]
}

The first prompt for

${some_action}: ${prod_review}`

would calculate to

Expand this statement: Got this panda plush toy for my daughter's birthday, who loves it and takes it everywhere. It's soft and  super cute, and its face has a friendly look...

The second prompt would be

Reduce this statement: Needed a nice lamp for my bedroom, and this one had additional storage and not too high of a price point. Got it fast - arrived in 2 days. The string to..

Project File

To continue the example, create a project file called project.yaml. Note that the prompt and data file point to the files we previously created. You may include more than one batch object in the array.

---
projectName: product-summary
projectVersion: '2.8'
apiKeyFile: "../../api_key"
blocks:
  # Shows how to create a batch block
  - blockId: product-1
    pluginName: BatchGptPlugin
    blockRuns: 2
    configuration:
      requestParams:
        model: gpt-3.5-turbo
        temperature: 0.3
        top_p: 1
        max_tokens: 500
    executions:
      - id: batch-1
        dataFile: product.json
        prompt: product.prompt
        systemMessageFile: "../system-message.txt"

  - blockId: product-2
    pluginName: BatchGptPlugin
    configuration:
      requestParams:
        model: gpt-3.5-turbo
        temperature: .2
        top_p: .4
        max_tokens: 200
    executions:
      - id: batch-2
        dataFile: product.json
        prompt: product.prompt
      - id: batch-3
        dataFile: product.json
        prompt: product.prompt

Configuration

Field Description
configuration.requestParams configuration parameters that are sent to OpenAI. You may add any legal parameters that OpenAI uses. Required field.
executions[n].dataFile path of your data file. This is the input into the prompt. Required field.
executions[n].id unique id of the batch job. Required field.
executions[n].prompt path of your prompt template file. Required field.

Output

The output looks something like below. I've truncated the actual results to improve readability.

{
  "projectName": "product-summary",
  "projectVersion": "2.8",
  "blockId": "product-1",
  "blockRuns": [
    {
      "blockRun": 1,
      "blockResults": [
        {
          "input": {
            "prod_review": "Got this panda plush toy for my daughter's birthday..."
          },
          "output": "The panda plush toy is soft, cute, and has a friendly look, but the reviewer thinks..."
        },
        {
          "input": {
            "prod_review": "Needed a nice lamp for my bedroom, and this one had..."
          },
          "output": "The lamp has additional storage and is reasonably priced. The company has excellent customer..."
        },
        {
          "input": {
            "prod_review": "Got this panda plush toy for my daughter's birthday.."
          },
          "output": "The panda plush toy is soft, cute, and loved by the recipient. However, the price may be too high..."
        },
        {
          "input": {
            "prod_review": "Needed a nice lamp for my bedroom, and this one had..."
          },
          "output": "The customer found the lamp to be a good value with additional storage and fast shipping. The company's..."
        }
      ]
    }
  ]
}

CLI

To run the batch command with blockId product-1

air run -b product-1

The number of calls to OpenAI will be

block_runs * {number of array items in data file}.

In the case above, its

2 * 2 = 4

Clone this wiki locally