🚀 Building an AI Computer-Using Agent with Scrapybara and OpenAI

We explore how modern AI agents can control real computers, navigate the web, and execute multi-step tasks using Scrapybara and OpenAI’s Computer Use (CUA) models. You’ll see live demos, architecture breakdowns, session handling, and even a mock mode you can run locally.

🎯 In this video, we’ll explore

---

🎤 1️⃣ Intro, Architecture, Limitations

• 🧠 How AI Agents Control Computers: Building with Scrapybara and CUA Models
• 🤖 From Chatbots to Computer-Using Agents: A Practical Overview
• 🛠️ Building an AI Assistant That Can Use a Computer: Tools, Models, and Limits

---

🖥️ 2️⃣ Demo: GitHub + React Search

• 🔍 Live Demo: AI Navigating GitHub via Computer Use
• 🎯 Running the Agent: Real-Time Screen Understanding and Browser Control

---

🗂️ 3️⃣ History Persistence & Architecture

• 🖼️ Maintaining Session Context: How Screenshot + JSON Histories Work
• 🔄 Passing History to Scrapybara: What Works and What Doesn’t

---

⚡ 4️⃣ Demo 2: Lightning Tools Products

• 🔁 Multi-Run Interaction: Executing new prompts and managing chats
• 🧭 Product Navigation: Browsing Lightning Tools products and selecting Social Squared

---

🧪 5️⃣ Mock Mode

• 🧩 Testing Without Scrapybara: Mocking Computer Use Interactions
• 💻 Local Mock Mode: Simulating AI Browser Control Without Credits

👉 Links & Resources

• Scrapybara
• Open AI Computer Use

---

🚀 Clone and Run

# Clone the repository
git clone https://github.com/Ashot72/computer-use-scrapybara-CUA

# Navigate into the project directory
cd computer-use-scrapybara-CUA

# Copy the environment template and add your actual API keys
cp env.local.example .env.local

# Install dependencies
pnpm install

# Start the development server
pnpm run dev

# The app will be available at http://localhost:3000

📺 Video: Watch on YouTube