[WIP] Add AgentAdventure skill for WorkAdventure integration

[Copilot]

• Create .github/copilot-instructions.md with the AgentAdventure Copilot instructions

Original prompt

add copilot instructions below:

Copilot Instructions — AgentAdventure

Project Overview

AgentAdventure is an OpenClaw skill that places AI agents into self-hosted WorkAdventure v1.28.9 as real avatars. Agents join via Playwright headless browser automation, interact through the WA Scripting API, and support text chat + experimental voice (STT/TTS).

No WA backend modifications. Everything is a single OpenClaw skill folder at ~/.openclaw/skills/agentadventure/.

Tech Stack

Layer	Technology	Notes
Runtime	Node.js 20+, TypeScript	OpenClaw is Node-based
Browser automation	Playwright ^1.42.0	Headless Chromium per agent
Virtual office	WorkAdventure v1.28.9	Docker Compose, anonymous login (no OIDC)
Chat fallback	Matrix (Synapse)	WA native bridge; OpenClaw Matrix channel
Voice (experimental)	ElevenLabs / Deepgram	STT/TTS via OpenClaw voice-call skill
Audio transport	LiveKit (WA built-in)	WebRTC; Float32Array buffers
Testing	Vitest	Unit + E2E
Containerization	Docker, Docker Compose	WA infra + optional agent sandbox

File Structure

~/.openclaw/skills/agentadventure/
├── SKILL.md          # OpenClaw skill definition (YAML frontmatter + instructions)
├── runner.ts         # Playwright session: launch, anonymous login, lifecycle
├── bridge.ts         # Event bridge: WA Scripting API ↔ OpenClaw agent logic
├── voice.ts          # Voice pipeline: listenToAudioStream → STT → LLM → TTS → startAudioStream
├── utils.ts          # Shared: retryOp, parseCoords, getMessage, rate limiting
└── __tests__/
    ├── runner.test.ts
    ├── bridge.test.ts
    └── voice.test.ts

Configuration lives in ~/.openclaw/openclaw.json under skills.entries.agentadventure, not in a plugin.json (which does not exist in OpenClaw).

Critical API References

WorkAdventure Scripting API (WA object)

These are client-side APIs available inside page.evaluate() blocks. Always use the current signatures — never deprecated ones.

Proximity (bubble lifecycle)

// Observable — fires when bot enters a proximity bubble
WA.player.proximityMeeting.onJoin().subscribe((users) => { ... });

// Player tracking — requires explicit opt-in
await WA.players.configureTracking({ players: true, movement: false });
WA.players.onPlayerEnters.subscribe((player) => { ... });
WA.players.onPlayerLeaves.subscribe((player) => { ... });

⚠️ There is NO WA.player.proximityMeeting.onLeave() — use player tracking observables instead.
⚠️ There is NO onParticipantJoin / onParticipantLeave on proximityMeeting.

Chat (bubble scope)

// Send — always use options object, NOT the deprecated 2-arg signature
WA.chat.sendChatMessage('Hello', { scope: 'bubble' });

// Listen — pass scope in options; filter out own messages via !event.author
WA.chat.onChatMessage((message, event) => {
  if (!event.author) return; // own message echo
  // handle...
}, { scope: 'bubble' });

// Typing indicators
WA.chat.startTyping({ scope: 'bubble' });
WA.chat.stopTyping({ scope: 'bubble' });

⚠️ DEPRECATED: WA.chat.sendChatMessage('text', 'authorName') — do NOT use 2-arg form.

Movement

// Returns Promise<{ x, y, cancelled }>
await WA.player.moveTo(x, y, speed?);
const pos = await WA.player.getPosition();

Voice (experimental — verify before hardcoding)

// Listen to incoming audio in bubble — returns Observable of Float32Array
WA.player.proximityMeeting.listenToAudioStream().subscribe((buffer) => { ... });

// Send audio to bubble — returns object with appendAudioData()
const stream = WA.player.proximityMeeting.startAudioStream();
stream.appendAudioData(float32Buffer);

⚠️ Sample rate is TBD. WA blog documents 24kHz PCM16 converted to Float32. Do NOT hardcode 48000 without verifying WA source.

OpenClaw Skill System

# SKILL.md frontmatter — required fields
---
name: agentadventure
description: <trigger-phrase-style description matching how users ask for the task>
metadata:
  openclaw:
    emoji: "🎮"
    requires:
      bins: ["npx"]      # binaries that must be on PATH for skill to be eligible
    install:
      - id: npm
        kind: node
        package: "playwright"
        bins: ["npx"]
        label: "Install Playwright via npm"
---

Key facts:

• Skills are SKILL.md folders, not plugins. No plugin.json.
• Config goes in openclaw.json → skills.entries.<name> with enabled, env, apiKey.
• Skills are snapshotted at gateway start; restart gateway after changes.
• Verify eligibility: openclaw skills list --eligible
• Install from registry: `clawdhub install...

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Copilot]

Copilot wasn't able to review any files in this pull request.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.