Character Extraction Agent using Nevermined's Payments API (TypeScript)

A TypeScript-based agent that extracts detailed character information from a film script using OpenAI's API via LangChain. Seamlessly integrated with Nevermined's Payments API, this agent handles task requests and billing efficiently within the Nevermined ecosystem.

---

Related Projects

This project is part of a larger workflow that explores the interconnection between agents and how can they communicate and work together. Please, refer to these projects in order to have a full view of the whole process

1. Movie Orchestrator Agent:

   • Coordinates the entire workflow, ensuring smooth task execution across agents.

2. Movie Script Generator Agent:

   • Generates movie scripts based on input ideas.

3. Character Extractor Agent:

   • Extracts character descriptions from movie scripts for further processing.

4. Image Generator Agent:

   • Generates realistic character images based on their descriptions.

Workflow Diagram:

---

Table of Contents

1. Introduction
2. Getting Started
   • Installation
   • Running the Agent
3. Project Structure
4. Integration with Nevermined Payments API
5. How to Create Your Own Agent
   • 1. Subscribing to Task Requests
   • 2. Handling Task Lifecycle
   • 3. Extracting Characters with LangChain
   • 4. Validating Steps and Sending Logs

---

Introduction

The Character Extraction Agent is a powerful application that extracts detailed character descriptions from a film script, including their physical attributes, attire, roles, and personality traits. The output, a JSON array of characters, is designed to be consumed by downstream agents, such as image generators, for further processing.

This agent is integrated with Nevermined's Payments API, which provides:

• Task lifecycle management: Efficiently handle task creation, updates, and completion.
• Billing integration: Utilize subscription plans (DIDs) to manage balance and execute tasks.
• Event subscription: React to task updates in real-time without requiring additional server setup.

The Character Extraction Agent is designed to work in a pipeline, receiving a script generated by the Script Generator Agent and preparing the data for the Image Generator Agent.

---

Getting Started

Installation

1. Clone the repository:

   git clone https://github.com/your-org/character-extraction-agent.git
   cd character-extraction-agent

2. Install dependencies:

   npm install

3. Configure environment variables:

   • Copy the .env.example file to .env:

     cp .env.example .env

   • Populate the .env file with the following details:

     NVM_API_KEY=YOUR_NVM_API_KEY
     OPENAI_API_KEY=YOUR_OPENAI_API_KEY
     NVM_ENVIRONMENT=testing # or staging/production
     AGENT_DID=YOUR_AGENT_DID

4. Build the project:

   npm run build

---

Running the Agent

Run the agent with the following command:

npm start

The agent will subscribe to the Nevermined task system and begin listening for task updates.

---

Project Structure

character-extraction-agent/
├── src/
│   ├── main.ts               # Main entry point for the agent
│   ├── characterExtractor.ts # Character extraction logic using LangChain
├── .env.example              # Example environment variables file
├── package.json              # Project dependencies and scripts
├── tsconfig.json             # TypeScript configuration

Key Components:

1. main.ts: Handles task lifecycle, from receiving steps to sending back results.
2. characterExtractor.ts: Implements the logic for extracting characters using LangChain and OpenAI.
3. .env: Stores sensitive configuration details like API keys and environment.

---

Integration with Nevermined Payments API

This agent leverages the Nevermined Payments API to handle tasks and billing. Below is an overview of how it works:

1. Initialize the Payments Instance:

   import { Payments, EnvironmentName } from "@nevermined-io/payments";
   
   const payments = Payments.getInstance({
     nvmApiKey: process.env.NVM_API_KEY!,
     environment: process.env.NVM_ENVIRONMENT as EnvironmentName,
   });
   
   if (!payments.isLoggedIn) {
     throw new Error("Failed to authenticate with Nevermined.");
   }

2. Subscribe to Task Updates:

   const subscriptionOpts = {
     joinAccountRoom: false,
     joinAgentRooms: [process.env.AGENT_DID!],
     subscribeEventTypes: ["step-updated"],
     getPendingEventsOnSubscribe: false,
   };
   
   await payments.query.subscribe(run, subscriptionOpts);

3. Task Lifecycle:

   • Fetch the step details using getStep.
   • Process the task (extract characters).
   • Update the task status using updateStep.

For detailed integration steps, refer to the official documentation.

---

How to Create Your Own Agent

1. Subscribing to Task Requests

Subscribing to task updates is the core of the agent’s functionality:

await payments.query.subscribe(run, {
  joinAccountRoom: false,
  joinAgentRooms: [process.env.AGENT_DID!],
  subscribeEventTypes: ["step-updated"],
  getPendingEventsOnSubscribe: false,
});

Key Points:

• The agent listens for task updates addressed to its DID.
• When a task is received, the run function is executed as the callback.

---

2. Handling Task Lifecycle

The run function orchestrates the lifecycle of a task:

async function run(data: any) {
  const step = await payments.query.getStep(data.step_id);
  if (step.step_status !== "Pending") return;

  const script = step.input_query || "";
  const characters = await characterExtractor.extractCharacters(script);

  await payments.query.updateStep(step.did, {
    ...step,
    step_status: "Completed",
    output_artifacts: characters,
  });
}

---

3. Extracting Characters with LangChain

The character extraction logic resides in characterExtractor.ts:

import { ChatOpenAI } from "@langchain/openai";
import { JsonOutputParser } from "@langchain/core/output_parsers";

const prompt = ChatPromptTemplate.fromTemplate(`
  Extract characters from this script. Return a JSON array:
  {script}
`);

const llm = new ChatOpenAI({
  model: "gpt-3.5-turbo",
  apiKey: process.env.OPENAI_API_KEY!,
});

export class CharacterExtractor {
  async extractCharacters(script: string): Promise<object[]> {
    const result = await prompt.invoke({ script });
    return result;
  }
}

---

4. Validating Steps and Sending Logs

The agent uses logMessage to send logs back to Nevermined:

async function logMessage(logMessage: TaskLogMessage) {
  if (logMessage.level === "error") logger.error(logMessage.message);
  else logger.info(logMessage.message);

  await payments.query.logTask(logMessage);
}

Use this to track progress and communicate task statuses.

---

Position in the Workflow

The Character Extraction Agent typically operates after the Script Generator Agent and before the Image Generator Agent, forming a key link in the processing pipeline.

1. Script Generator Agent: Generates the script.
2. Character Extraction Agent: Extracts characters and their attributes.
3. Image Generator Agent: Creates visual representations of the characters.

---

License

Copyright 2024 Nevermined AG

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.