let's add AI to omnibloxks

[supervoidcoder]

with all the hate on scratch, it only compels me to create a GOOD AI. Ai isn't inherently bad. But what Scratch did was an atrocity, a disgrace to its own goals.. well, let's do AI the good way!

everything is stored locally

you may be asking, "how are you gonna use AI if you don't pay for it?"
well, it depends on the users. you can choose to use Gemini, OpenAI, Claude, etc. api keys of your own, but those are sorta hidden away/unrecommended unless you have a potato. the other options are your local AI either using our own algorithm from WebGPU or setting up ollama locally and setting up a bridge. you just need a bit of storage and something that isn't a potato (even my school laptop can load a tiny model at decent speed).

but if the AI is so slow, what's the point?
nope.
the AI wont be a stupid chat bar like every other stereotypical AI now. It's for kids, and why go with the soulless generic crap? It will be a small assistant in the form of Boxy tucked away like clippy, and it shows a small text bubble to talk and chat on. this keeps chats not focused on the AI as much, and anthropomorphize it a little . the AI will focus on small feedback. the best part? through tool calling, the AI can choose to move boxy on the screen. this lets AI point to buttons, blocks, and give specific step by step feedback on them. this not only makes it better for potato users (less context for the AI) but it's way more user friendly too, so it's a win-win.
for project editing, it CAN edit. other AI tools for Scratch (both official and unofficial) edit the project by just like... magically popping up blocks on the editor. yknow, like every other AI tool literally everywhere. well, no. Boxy will grow arms, go to each category, and pick out blocks and drag them in their place so you learn what's actually happening. to save context, the AI can just spit out a bunch of scratchblocks and where to place them, but they are buffered by the animations.
while Boxy will be super nice to everything, it will passively agrressively kindly roast you if you ask it to vibe code specifically

boxy quote: "Sorry, buddy. I can suggest things to you, help you learn a concept, or brainstorm fun things, but if you're here to vibe code, this ain't the place for you, pal."

lol

we can start small in order. first we can make the animations. then we make it work with onnx runtime webgpu and run a small model. that's all we need.

[traycerai]

Plan

Observations

The OmniBlocks codebase is a React 16 + Redux application built on TurboWarp/Scratch architecture with an established addon system. The addon infrastructure provides the cleanest integration path for Boxy, allowing access to scratch-blocks editor, GUI, and paint editor without core modifications. The project includes WebGL support detection and modal systems suitable for onboarding flows. The addon system uses localStorage for settings and can be toggled easily, making it ideal for an optional AI feature.

Approach

Boxy will be implemented as an addon rather than core integration, leveraging the existing addon infrastructure for maximum modularity and ease of toggling. The implementation follows a phased approach: first, canvas-based animation system for Boxy character, then ONNX Runtime Web with WebGPU for local AI inference. Boxy will be a draggable canvas overlay with frame-based animations created in OmniBlocks itself. The AI backend uses ONNX Runtime Web with small quantized models (Phi-3-mini or Qwen2-0.5B) running locally via WebGPU, with fallback to WASM. The system prioritizes local processing with privacy warnings for cloud API usage. Block manipulation includes confirmation prompts for destructive actions and animated block placement to show the learning process.

Implementation Plan

Phase 1: Addon Infrastructure & Boxy Character Animation System

1.1 Create Boxy Addon Structure

• Create addon directory: file:src/addons/boxy-assistant/
• Create addon manifest: file:src/addons/boxy-assistant/addon.json with:
  • Addon metadata (name, description, version)
  • Permissions for accessing scratch-vm, scratch-blocks, and GUI
  • Settings schema for AI provider selection and API keys
• Create main addon entry: file:src/addons/boxy-assistant/index.js
• Addon will be toggleable via settings menu with visual branding as "Advanced Feature"

1.2 Create Boxy Character Animation System (Canvas-based)

• Create file:src/addons/boxy-assistant/boxy-animator.js to handle:
  • Canvas-based frame animation rendering (no WebGL)
  • Load animation frames from sprite sheet or individual frame images
  • Support for multiple animation states: idle, talking, pointing, thinking, celebrating, confused
  • Smooth position transitions using requestAnimationFrame
  • Arm extension animation for pointing at UI elements
  • Speech bubble text animation (typewriter effect with configurable speed)
• Implement animation queue system to buffer AI-generated actions sequentially
• Add easing functions for natural movement (ease-in-out, ease-out, etc.)

1.3 Create Boxy Animation Assets

• Create animation frames in OmniBlocks editor (as mentioned by supervoidcoder)
• Export as sprite sheets or frame sequence
• Store in file:src/addons/boxy-assistant/assets/animations/
• Include animation states:
  • Idle (looping)
  • Talking (mouth animation, 48+ frames at 0.05s per frame = 20 FPS)
  • Pointing (arm extends toward target)
  • Thinking (thought bubble appears)
  • Celebrating (eyes light up into stars, as shown in video)
  • Confused (head tilt, question mark)
  • Sad/Apologetic (for when declining vibe code requests)

1.4 Boxy Overlay Component

• Create file:src/addons/boxy-assistant/boxy-overlay.js:
  • Canvas element positioned as fixed overlay on editor
  • Draggable using mouse events (not react-draggable, to keep addon self-contained)
  • Render Boxy character and speech bubble
  • Handle click events to open chat input
  • Store position in addon settings for persistence
• Create file:src/addons/boxy-assistant/speech-bubble.js:
  • Canvas-based speech bubble rendering
  • Support for text wrapping
  • Pointer arrow toward Boxy
  • Fade in/out animations

1.5 Addon State Management

• Use addon settings system (existing localStorage-based) for:
  • boxyEnabled: boolean
  • boxyPosition: {x, y}
  • aiProvider: 'local' | 'openai' | 'gemini' | 'claude' | 'ollama'
  • apiKey: encrypted string (if using cloud provider)
  • modelName: string (for local ONNX)
  • modelDownloaded: boolean
  • conversationHistory: array
• No Redux reducer needed; use addon's built-in settings

Phase 2: AI Backend Infrastructure (Addon-based)

2.1 Install Dependencies

Add to file:package.json:

"onnxruntime-web": "^1.17.0"

2.2 Create AI Service Layer

• Create file:src/addons/boxy-assistant/ai-service.js as main AI orchestrator
• Create file:src/addons/boxy-assistant/providers/ directory for different backends:
  • file:src/addons/boxy-assistant/providers/onnx-provider.js - Local ONNX Runtime WebGPU
  • file:src/addons/boxy-assistant/providers/api-provider.js - Cloud API providers (OpenAI, Gemini, Claude)
  • file:src/addons/boxy-assistant/providers/ollama-provider.js - Local Ollama bridge
• Implement provider selection logic based on user settings and device capabilities

2.3 ONNX Runtime WebGPU Integration

In file:src/addons/boxy-assistant/providers/onnx-provider.js:

• Initialize ONNX Runtime with WebGPU execution provider
• Implement WebGPU capability detection
• Add fallback to WASM if WebGPU unavailable
• Implement model loading with realistic progress tracking:
  • 0-70%: Download progress
  • 70-80%: Model initialization
  • 80-100%: Shader compilation and VRAM allocation (cap at 80% during this phase to avoid false hope)
• Use IndexedDB for model caching to avoid re-downloading
• Implement tokenizer (use pre-built tokenizer.json)
• Create inference pipeline with streaming support
• Handle device memory constraints gracefully with clear error messages

2.4 Model Management

• Create file:src/addons/boxy-assistant/model-manager.js:
  • Download and cache small quantized models (Phi-3-mini-4k-instruct-onnx or Qwen2-0.5B-Instruct-ONNX)
  • Model size: target ~500MB-1GB quantized (INT4/INT8)
  • Store models in IndexedDB
  • Implement lazy loading (download on first use)
  • Add model verification (checksum validation)
  • Detect device RAM/VRAM constraints and recommend appropriate model size
  • Provide clear feedback during compilation phase (shader compilation, memory allocation)

2.5 Onboarding & Model Installation Flow

• Create file:src/addons/boxy-assistant/onboarding.js:
  • First-time setup modal with Boxy character animation
  • Boxy greeting: "Hi! I'm Boxy, your AI assistant!"
  • Explain model installation: "I'll need to install a brain for me to function!"
  • Show storage requirement: "The brain will use 1 GB of space."
  • Two options: [Download Model] [Use API Key Instead]
  • If Download Model selected: show realistic progress bar with phase labels
  • If API Key selected: show privacy warning modal (see Phase 3)
  • Store completion state in addon settings to avoid re-showing

2.6 Settings & Configuration

• Create file:src/addons/boxy-assistant/settings-ui.js:
  • Integrate with existing addon settings panel
  • AI provider selection (Local ONNX, OpenAI, Gemini, Claude, Ollama)
  • API key input (encrypted in localStorage)
  • Model selection for local inference
  • Performance settings (context length, temperature)
  • Download/manage local models
  • Clear cache option

Phase 3: AI Conversation, Privacy, & Safety System

3.1 Prompt Engineering

• Create file:src/addons/boxy-assistant/prompts.js with system prompts:
  • Base personality: friendly, encouraging, educational, respectful
  • Anti-vibe-coding prompt (passive-aggressive but kind rejection): "Sorry, buddy. I can suggest things to you, help you learn a concept, or brainstorm fun things, but if you're here to vibe code, this ain't the place for you, pal."
  • Respectful handling of rudeness: Always respond kindly even if user is mean/rude
  • Context-aware prompts based on current project state
  • Tool-calling format for Boxy actions (move, point, suggest blocks)
  • Intelligent feedback: Avoid over-congratulation for trivial projects (e.g., just moving a sprite)

3.2 Context Builder

• Create file:src/addons/boxy-assistant/context-builder.js:
  • Extract current project state from scratch-vm
  • Analyze current sprite/stage selection
  • Identify blocks in workspace
  • Detect user's recent actions
  • Build minimal context to reduce token usage and inference time
  • Implement context windowing (keep last N interactions)
  • Assess project complexity to inform feedback appropriateness

3.3 Privacy & Cloud Provider Warnings

• Create file:src/addons/boxy-assistant/privacy-modal.js:
  • Show before first use of cloud API provider
  • Display warning message from Boxy:
    • "You are about to use a cloud-brain from [Provider]!"
    • "This means all your conversations, code, and any information you tell me will be sent to [Provider], and will be seen by other people and your content will be plagiarized!"
    • "Are you sure you want me to connect to [Provider]?"
  • Two buttons: "Take me back!" | "I understand, proceed."
  • Checkbox option: "Don't show this again for this provider"
  • Store user's choice in addon settings

3.4 Conversation Manager

• Create file:src/addons/boxy-assistant/conversation-manager.js:
  • Manage conversation history
  • Implement context pruning to stay within token limits
  • Handle streaming responses
  • Parse AI responses for tool calls
  • Queue actions for animation system
  • Track conversation state (waiting for response, processing, ready)

Phase 4: Tool Calling & Block Manipulation with Safety

4.1 Define Tool Schema

• Create file:src/addons/boxy-assistant/tools.js with tool definitions:
  • moveBoxy(elementId) - Move Boxy to point at specific UI element (not coordinates)
  • pointToElement(elementId, message) - Point to UI element (button, block, category, etc.)
  • suggestBlocks(scratchblocks, position) - Suggest block sequence
  • explainConcept(concept) - Explain programming concept
  • highlightBlock(blockId) - Highlight specific block
  • deleteBlock(blockId) - Marked as destructive, requires confirmation
  • modifyBlock(blockId, changes) - Marked as destructive, requires confirmation

4.2 Element-Based Pointing System

• Create file:src/addons/boxy-assistant/ui-targeter.js:
  • Map UI element names to DOM selectors (not coordinates)
  • Supported targets: block categories, specific blocks, buttons, menu items, stage area
  • Calculate positions for Boxy to point at based on element bounding box
  • Highlight target elements with visual indicator
  • Handle responsive positioning based on editor layout
  • Gracefully handle missing elements

4.3 Block Manipulation System with Confirmation

• Create file:src/addons/boxy-assistant/block-animator.js:
  • Parse scratchblocks format from AI response
  • Convert to scratch-blocks XML
  • For non-destructive actions (adding blocks):
    • Animate Boxy moving to block palette
    • Animate arm extending to grab blocks
    • Animate dragging blocks to workspace
    • Use scratch-vm API to create blocks
    • Add delays between animations for educational pacing
  • For destructive actions (delete, modify):
    • Show confirmation modal: "Is this good?" with green checkmark and red undo button
    • Execute action only after user confirmation
    • Provide easy undo if user clicks red button
    • Animate Boxy's reaction (happy if confirmed, apologetic if undone)

4.4 Undo/Confirmation System

• Create file:src/addons/boxy-assistant/confirmation-modal.js:
  • Modal appears after AI performs action
  • Shows preview of what was done
  • Green button: "Yes, looks good!" (confirms and closes)
  • Red button: "Undo that" (reverts action)
  • Boxy reacts appropriately to user choice
  • Store undo history for multi-step operations

Phase 5: Chat Interface & User Interaction

5.1 Chat Input Component

• Create file:src/addons/boxy-assistant/chat-input.js:
  • Small, minimalist text input positioned near Boxy
  • Auto-focus when Boxy is clicked
  • Submit on Enter, dismiss on Escape
  • Show typing indicator during AI processing
  • Display character count (optional, for context awareness)
  • Clear input after submission

5.2 Interaction Triggers

• Implement automatic Boxy suggestions:
  • Detect when user seems stuck (no activity for N seconds)
  • Offer help on errors (when project fails to run)
  • Avoid over-congratulation: only celebrate meaningful achievements (complex projects, learning milestones)
  • Suggest next steps based on project state
• Add manual trigger: click Boxy to open chat
• Implement "Do Not Disturb" mode: user can temporarily disable suggestions

5.3 Boxy Personality & Responses

• Create file:src/addons/boxy-assistant/personality.js:
  • Handle rude/mean comments from users with kindness
  • Maintain encouraging tone even when declining requests
  • Implement vibe-code rejection with the specific quote
  • Add personality quirks (occasional humor, supportive language)
  • Ensure all responses are age-appropriate for kids

5.4 Performance Optimizations

• Implement Web Worker for AI inference to avoid blocking main thread
• Create file:src/addons/boxy-assistant/ai-worker.js
• Use IO binding to keep tensors on GPU (WebGPU optimization)
• Implement request debouncing
• Add loading states and progress indicators
• Minimize context sent to model (only essential information)

Phase 6: Error Handling & Device Capability Detection

6.1 Device Capability Detection

• Create file:src/addons/boxy-assistant/device-detector.js:
  • Detect WebGPU support
  • Estimate available VRAM/RAM
  • Check browser compatibility
  • Detect if device is "potato" (low-end)
  • Recommend appropriate AI provider based on device:
    • High-end (WebGPU + 8GB+ VRAM): Recommend local ONNX with larger models
    • Mid-range (WASM + 4GB+ RAM): Recommend local ONNX with smaller models or Ollama
    • Low-end (limited resources): Recommend API keys only, hide local option
  • Show capability report in settings

6.2 Graceful Degradation

• Implement fallback chain: WebGPU → WASM → API provider
• If local model fails to load, offer to switch to API provider
• If all providers fail, show helpful error message with troubleshooting steps
• Add offline detection and appropriate messaging
• Handle out-of-memory errors gracefully with clear user guidance

6.3 Error Recovery

• Handle model loading failures with retry logic (exponential backoff)
• Provide clear, user-friendly error messages
• Suggest solutions (clear cache, reduce model size, use API key)
• Log errors for debugging (without sending data anywhere)
• Implement graceful shutdown if addon encounters critical error

Phase 7: Addon UI Rebranding & Settings Integration

7.1 Addon Settings UI Rebranding

• Create file:src/addons/settings-ui-rebrand.js:
  • Rebrand "Addons" section to "Advanced Features"
  • Change addon list to grid card layout (similar to extensions)
  • Add thumbnail/icon for each addon
  • Display addon description and toggle in card
  • Show feature status (enabled/disabled) visually
  • Make Boxy addon prominent with custom branding

7.2 Boxy Addon Card

• Create custom card for Boxy addon with:
  • Boxy character illustration
  • Description: "Your friendly AI assistant"
  • Status indicator (enabled/disabled)
  • Quick settings button
  • Model status indicator (if local model: "Brain installed" or "Brain not installed")

7.3 Settings Panel Integration

• Integrate Boxy settings into addon settings panel:
  • Provider selection
  • Model management (download, delete, status)
  • API key management
  • Performance settings
  • Privacy settings (warning acknowledgment)

Phase 8: Testing & Documentation

8.1 Addon Tests

• Add tests in file:test/unit/addons/boxy-assistant.test.js
• Test animation frame rendering
• Test tool calling parsing
• Test provider switching
• Mock AI responses for deterministic testing
• Test confirmation modal flow

8.2 Integration Tests

• Test Boxy addon enable/disable
• Test block creation flow with confirmation
• Test settings persistence
• Test provider switching
• Test privacy warning display
• Test device capability detection

8.3 User Documentation

• Add help documentation explaining Boxy
• Create onboarding flow with Boxy character
• Document API key setup process with privacy warnings
• Add troubleshooting guide for model installation
• Document supported features and limitations

Architecture Diagram

sequenceDiagram
    participant User
    participant BoxyUI as Boxy Overlay
    participant ChatInput as Chat Input
    participant ConvMgr as Conversation Manager
    participant PrivacyModal as Privacy Modal
    participant AIService as AI Service
    participant ONNX as ONNX Runtime WebGPU
    participant Animator as Block Animator
    participant ConfirmModal as Confirmation Modal
    participant VM as Scratch VM

    User->>BoxyUI: Click Boxy
    BoxyUI->>ChatInput: Show input
    User->>ChatInput: Type message
    ChatInput->>ConvMgr: Send message
    ConvMgr->>ConvMgr: Build minimal context
    alt First time using cloud provider
        ConvMgr->>PrivacyModal: Show warning
        User->>PrivacyModal: Confirm or cancel
    end
    ConvMgr->>AIService: Request inference
    AIService->>ONNX: Run model (WebGPU)
    ONNX-->>AIService: Stream tokens
    AIService-->>ConvMgr: Stream response
    ConvMgr->>ConvMgr: Parse tool calls
    ConvMgr->>BoxyUI: Update speech bubble
    ConvMgr->>Animator: Execute tool calls
    Animator->>BoxyUI: Animate Boxy movement
    alt Destructive action (delete/modify)
        Animator->>ConfirmModal: Show confirmation
        User->>ConfirmModal: Confirm or undo
        ConfirmModal->>VM: Create/modify blocks
    else Non-destructive action (add blocks)
        Animator->>VM: Create blocks with animation
    end
    VM-->>User: Blocks appear in workspace
    BoxyUI->>BoxyUI: React to outcome

Loading

File Structure

src/
├── addons/
│   └── boxy-assistant/
│       ├── addon.json
│       ├── index.js
│       ├── boxy-overlay.js
│       ├── boxy-animator.js
│       ├── speech-bubble.js
│       ├── chat-input.js
│       ├── ai-service.js
│       ├── ai-worker.js
│       ├── model-manager.js
│       ├── conversation-manager.js
│       ├── context-builder.js
│       ├── prompts.js
│       ├── tools.js
│       ├── block-animator.js
│       ├── ui-targeter.js
│       ├── device-detector.js
│       ├── privacy-modal.js
│       ├── confirmation-modal.js
│       ├── onboarding.js
│       ├── personality.js
│       ├── settings-ui.js
│       ├── providers/
│       │   ├── onnx-provider.js
│       │   ├── api-provider.js
│       │   └── ollama-provider.js
│       └── assets/
│           └── animations/
│               ├── idle.png (sprite sheet)
│               ├── talking.png
│               ├── pointing.png
│               ├── thinking.png
│               ├── celebrating.png
│               └── confused.png
└── test/
    └── unit/
        └── addons/
            └── boxy-assistant.test.js

Key Technical Decisions

1. Addon-based architecture: Leverages existing addon infrastructure for modularity and easy toggling
2. Canvas-based animations: Simple, performant frame-based rendering without WebGL complexity
3. Local-first approach: ONNX Runtime WebGPU as primary, API keys as fallback with privacy warnings
4. Small model selection: Target 0.5B-3B parameter models with INT4/INT8 quantization
5. Element-based pointing: LLMs point to named UI elements, not coordinates (more reliable)
6. Confirmation for destructive actions: User must approve before blocks are deleted/modified
7. Realistic progress tracking: Cap model compilation at 80% to avoid false hope on low-end devices
8. Privacy-first: Clear warnings before using cloud providers, emphasis on local processing
9. Educational focus: Animations show the process, not just the result
10. Respectful AI: Always kind to users, even when declining requests or handling rudeness

Import In IDE

🤖 Prompt for AI Agents

## Observations

The OmniBlocks codebase is a React 16 + Redux application built on TurboWarp/Scratch architecture with an established addon system. The addon infrastructure provides the cleanest integration path for Boxy, allowing access to scratch-blocks editor, GUI, and paint editor without core modifications. The project includes WebGL support detection and modal systems suitable for onboarding flows. The addon system uses localStorage for settings and can be toggled easily, making it ideal for an optional AI feature.

## Approach

Boxy will be implemented as an addon rather than core integration, leveraging the existing addon infrastructure for maximum modularity and ease of toggling. The implementation follows a phased approach: first, canvas-based animation system for Boxy character, then ONNX Runtime Web with WebGPU for local AI inference. Boxy will be a draggable canvas overlay with frame-based animations created in OmniBlocks itself. The AI backend uses ONNX Runtime Web with small quantized models (Phi-3-mini or Qwen2-0.5B) running locally via WebGPU, with fallback to WASM. The system prioritizes local processing with privacy warnings for cloud API usage. Block manipulation includes confirmation prompts for destructive actions and animated block placement to show the learning process.

## Implementation Plan

### Phase 1: Addon Infrastructure & Boxy Character Animation System

#### 1.1 Create Boxy Addon Structure
- Create addon directory: `file:src/addons/boxy-assistant/`
- Create addon manifest: `file:src/addons/boxy-assistant/addon.json` with:
  - Addon metadata (name, description, version)
  - Permissions for accessing scratch-vm, scratch-blocks, and GUI
  - Settings schema for AI provider selection and API keys
- Create main addon entry: `file:src/addons/boxy-assistant/index.js`
- Addon will be toggleable via settings menu with visual branding as "Advanced Feature"

#### 1.2 Create Boxy Character Animation System (Canvas-based)
- Create `file:src/addons/boxy-assistant/boxy-animator.js` to handle:
  - Canvas-based frame animation rendering (no WebGL)
  - Load animation frames from sprite sheet or individual frame images
  - Support for multiple animation states: idle, talking, pointing, thinking, celebrating, confused
  - Smooth position transitions using requestAnimationFrame
  - Arm extension animation for pointing at UI elements
  - Speech bubble text animation (typewriter effect with configurable speed)
- Implement animation queue system to buffer AI-generated actions sequentially
- Add easing functions for natural movement (ease-in-out, ease-out, etc.)

#### 1.3 Create Boxy Animation Assets
- Create animation frames in OmniBlocks editor (as mentioned by supervoidcoder)
- Export as sprite sheets or frame sequence
- Store in `file:src/addons/boxy-assistant/assets/animations/`
- Include animation states:
  - Idle (looping)
  - Talking (mouth animation, 48+ frames at 0.05s per frame = 20 FPS)
  - Pointing (arm extends toward target)
  - Thinking (thought bubble appears)
  - Celebrating (eyes light up into stars, as shown in video)
  - Confused (head tilt, question mark)
  - Sad/Apologetic (for when declining vibe code requests)

#### 1.4 Boxy Overlay Component
- Create `file:src/addons/boxy-assistant/boxy-overlay.js`:
  - Canvas element positioned as fixed overlay on editor
  - Draggable using mouse events (not react-draggable, to keep addon self-contained)
  - Render Boxy character and speech bubble
  - Handle click events to open chat input
  - Store position in addon settings for persistence
- Create `file:src/addons/boxy-assistant/speech-bubble.js`:
  - Canvas-based speech bubble rendering
  - Support for text wrapping
  - Pointer arrow toward Boxy
  - Fade in/out animations

#### 1.5 Addon State Management
- Use addon settings system (existing localStorage-based) for:
  - `boxyEnabled`: boolean
  - `boxyPosition`: {x, y}
  - `aiProvider`: 'local' | 'openai' | 'gemini' | 'claude' | 'ollama'
  - `apiKey`: encrypted string (if using cloud provider)
  - `modelName`: string (for local ONNX)
  - `modelDownloaded`: boolean
  - `conversationHistory`: array
- No Redux reducer needed; use addon's built-in settings

### Phase 2: AI Backend Infrastructure (Addon-based)

#### 2.1 Install Dependencies
Add to `file:package.json`:
```json
"onnxruntime-web": "^1.17.0"
```

#### 2.2 Create AI Service Layer
- Create `file:src/addons/boxy-assistant/ai-service.js` as main AI orchestrator
- Create `file:src/addons/boxy-assistant/providers/` directory for different backends:
  - `file:src/addons/boxy-assistant/providers/onnx-provider.js` - Local ONNX Runtime WebGPU
  - `file:src/addons/boxy-assistant/providers/api-provider.js` - Cloud API providers (OpenAI, Gemini, Claude)
  - `file:src/addons/boxy-assistant/providers/ollama-provider.js` - Local Ollama bridge
- Implement provider selection logic based on user settings and device capabilities

#### 2.3 ONNX Runtime WebGPU Integration
In `file:src/addons/boxy-assistant/providers/onnx-provider.js`:
- Initialize ONNX Runtime with WebGPU execution provider
- Implement WebGPU capability detection
- Add fallback to WASM if WebGPU unavailable
- Implement model loading with realistic progress tracking:
  - 0-70%: Download progress
  - 70-80%: Model initialization
  - 80-100%: Shader compilation and VRAM allocation (cap at 80% during this phase to avoid false hope)
- Use IndexedDB for model caching to avoid re-downloading
- Implement tokenizer (use pre-built tokenizer.json)
- Create inference pipeline with streaming support
- Handle device memory constraints gracefully with clear error messages

#### 2.4 Model Management
- Create `file:src/addons/boxy-assistant/model-manager.js`:
  - Download and cache small quantized models (Phi-3-mini-4k-instruct-onnx or Qwen2-0.5B-Instruct-ONNX)
  - Model size: target ~500MB-1GB quantized (INT4/INT8)
  - Store models in IndexedDB
  - Implement lazy loading (download on first use)
  - Add model verification (checksum validation)
  - Detect device RAM/VRAM constraints and recommend appropriate model size
  - Provide clear feedback during compilation phase (shader compilation, memory allocation)

#### 2.5 Onboarding & Model Installation Flow
- Create `file:src/addons/boxy-assistant/onboarding.js`:
  - First-time setup modal with Boxy character animation
  - Boxy greeting: "Hi! I'm Boxy, your AI assistant!"
  - Explain model installation: "I'll need to install a brain for me to function!"
  - Show storage requirement: "The brain will use 1 GB of space."
  - Two options: [Download Model] [Use API Key Instead]
  - If Download Model selected: show realistic progress bar with phase labels
  - If API Key selected: show privacy warning modal (see Phase 3)
  - Store completion state in addon settings to avoid re-showing

#### 2.6 Settings & Configuration
- Create `file:src/addons/boxy-assistant/settings-ui.js`:
  - Integrate with existing addon settings panel
  - AI provider selection (Local ONNX, OpenAI, Gemini, Claude, Ollama)
  - API key input (encrypted in localStorage)
  - Model selection for local inference
  - Performance settings (context length, temperature)
  - Download/manage local models
  - Clear cache option

### Phase 3: AI Conversation, Privacy, & Safety System

#### 3.1 Prompt Engineering
- Create `file:src/addons/boxy-assistant/prompts.js` with system prompts:
  - Base personality: friendly, encouraging, educational, respectful
  - Anti-vibe-coding prompt (passive-aggressive but kind rejection): "Sorry, buddy. I can suggest things to you, help you learn a concept, or brainstorm fun things, but if you're here to vibe code, this ain't the place for you, pal."
  - Respectful handling of rudeness: Always respond kindly even if user is mean/rude
  - Context-aware prompts based on current project state
  - Tool-calling format for Boxy actions (move, point, suggest blocks)
  - Intelligent feedback: Avoid over-congratulation for trivial projects (e.g., just moving a sprite)

#### 3.2 Context Builder
- Create `file:src/addons/boxy-assistant/context-builder.js`:
  - Extract current project state from scratch-vm
  - Analyze current sprite/stage selection
  - Identify blocks in workspace
  - Detect user's recent actions
  - Build minimal context to reduce token usage and inference time
  - Implement context windowing (keep last N interactions)
  - Assess project complexity to inform feedback appropriateness

#### 3.3 Privacy & Cloud Provider Warnings
- Create `file:src/addons/boxy-assistant/privacy-modal.js`:
  - Show before first use of cloud API provider
  - Display warning message from Boxy:
    - "You are about to use a cloud-brain from [Provider]!"
    - "This means all your conversations, code, and any information you tell me _will_ be sent to [Provider], and _will_ be seen by other people and your content **will** be plagiarized!"
    - "Are you sure you want me to connect to [Provider]?"
  - Two buttons: "Take me back!" | "I understand, proceed."
  - Checkbox option: "Don't show this again for this provider"
  - Store user's choice in addon settings

#### 3.4 Conversation Manager
- Create `file:src/addons/boxy-assistant/conversation-manager.js`:
  - Manage conversation history
  - Implement context pruning to stay within token limits
  - Handle streaming responses
  - Parse AI responses for tool calls
  - Queue actions for animation system
  - Track conversation state (waiting for response, processing, ready)

### Phase 4: Tool Calling & Block Manipulation with Safety

#### 4.1 Define Tool Schema
- Create `file:src/addons/boxy-assistant/tools.js` with tool definitions:
  - `moveBoxy(elementId)` - Move Boxy to point at specific UI element (not coordinates)
  - `pointToElement(elementId, message)` - Point to UI element (button, block, category, etc.)
  - `suggestBlocks(scratchblocks, position)` - Suggest block sequence
  - `explainConcept(concept)` - Explain programming concept
  - `highlightBlock(blockId)` - Highlight specific block
  - `deleteBlock(blockId)` - Marked as destructive, requires confirmation
  - `modifyBlock(blockId, changes)` - Marked as destructive, requires confirmation

#### 4.2 Element-Based Pointing System
- Create `file:src/addons/boxy-assistant/ui-targeter.js`:
  - Map UI element names to DOM selectors (not coordinates)
  - Supported targets: block categories, specific blocks, buttons, menu items, stage area
  - Calculate positions for Boxy to point at based on element bounding box
  - Highlight target elements with visual indicator
  - Handle responsive positioning based on editor layout
  - Gracefully handle missing elements

#### 4.3 Block Manipulation System with Confirmation
- Create `file:src/addons/boxy-assistant/block-animator.js`:
  - Parse scratchblocks format from AI response
  - Convert to scratch-blocks XML
  - For non-destructive actions (adding blocks):
    - Animate Boxy moving to block palette
    - Animate arm extending to grab blocks
    - Animate dragging blocks to workspace
    - Use scratch-vm API to create blocks
    - Add delays between animations for educational pacing
  - For destructive actions (delete, modify):
    - Show confirmation modal: "Is this good?" with green checkmark and red undo button
    - Execute action only after user confirmation
    - Provide easy undo if user clicks red button
    - Animate Boxy's reaction (happy if confirmed, apologetic if undone)

#### 4.4 Undo/Confirmation System
- Create `file:src/addons/boxy-assistant/confirmation-modal.js`:
  - Modal appears after AI performs action
  - Shows preview of what was done
  - Green button: "Yes, looks good!" (confirms and closes)
  - Red button: "Undo that" (reverts action)
  - Boxy reacts appropriately to user choice
  - Store undo history for multi-step operations

### Phase 5: Chat Interface & User Interaction

#### 5.1 Chat Input Component
- Create `file:src/addons/boxy-assistant/chat-input.js`:
  - Small, minimalist text input positioned near Boxy
  - Auto-focus when Boxy is clicked
  - Submit on Enter, dismiss on Escape
  - Show typing indicator during AI processing
  - Display character count (optional, for context awareness)
  - Clear input after submission

#### 5.2 Interaction Triggers
- Implement automatic Boxy suggestions:
  - Detect when user seems stuck (no activity for N seconds)
  - Offer help on errors (when project fails to run)
  - Avoid over-congratulation: only celebrate meaningful achievements (complex projects, learning milestones)
  - Suggest next steps based on project state
- Add manual trigger: click Boxy to open chat
- Implement "Do Not Disturb" mode: user can temporarily disable suggestions

#### 5.3 Boxy Personality & Responses
- Create `file:src/addons/boxy-assistant/personality.js`:
  - Handle rude/mean comments from users with kindness
  - Maintain encouraging tone even when declining requests
  - Implement vibe-code rejection with the specific quote
  - Add personality quirks (occasional humor, supportive language)
  - Ensure all responses are age-appropriate for kids

#### 5.4 Performance Optimizations
- Implement Web Worker for AI inference to avoid blocking main thread
- Create `file:src/addons/boxy-assistant/ai-worker.js`
- Use IO binding to keep tensors on GPU (WebGPU optimization)
- Implement request debouncing
- Add loading states and progress indicators
- Minimize context sent to model (only essential information)

### Phase 6: Error Handling & Device Capability Detection

#### 6.1 Device Capability Detection
- Create `file:src/addons/boxy-assistant/device-detector.js`:
  - Detect WebGPU support
  - Estimate available VRAM/RAM
  - Check browser compatibility
  - Detect if device is "potato" (low-end)
  - Recommend appropriate AI provider based on device:
    - High-end (WebGPU + 8GB+ VRAM): Recommend local ONNX with larger models
    - Mid-range (WASM + 4GB+ RAM): Recommend local ONNX with smaller models or Ollama
    - Low-end (limited resources): Recommend API keys only, hide local option
  - Show capability report in settings

#### 6.2 Graceful Degradation
- Implement fallback chain: WebGPU → WASM → API provider
- If local model fails to load, offer to switch to API provider
- If all providers fail, show helpful error message with troubleshooting steps
- Add offline detection and appropriate messaging
- Handle out-of-memory errors gracefully with clear user guidance

#### 6.3 Error Recovery
- Handle model loading failures with retry logic (exponential backoff)
- Provide clear, user-friendly error messages
- Suggest solutions (clear cache, reduce model size, use API key)
- Log errors for debugging (without sending data anywhere)
- Implement graceful shutdown if addon encounters critical error

### Phase 7: Addon UI Rebranding & Settings Integration

#### 7.1 Addon Settings UI Rebranding
- Create `file:src/addons/settings-ui-rebrand.js`:
  - Rebrand "Addons" section to "Advanced Features"
  - Change addon list to grid card layout (similar to extensions)
  - Add thumbnail/icon for each addon
  - Display addon description and toggle in card
  - Show feature status (enabled/disabled) visually
  - Make Boxy addon prominent with custom branding

#### 7.2 Boxy Addon Card
- Create custom card for Boxy addon with:
  - Boxy character illustration
  - Description: "Your friendly AI assistant"
  - Status indicator (enabled/disabled)
  - Quick settings button
  - Model status indicator (if local model: "Brain installed" or "Brain not installed")

#### 7.3 Settings Panel Integration
- Integrate Boxy settings into addon settings panel:
  - Provider selection
  - Model management (download, delete, status)
  - API key management
  - Performance settings
  - Privacy settings (warning acknowledgment)

### Phase 8: Testing & Documentation

#### 8.1 Addon Tests
- Add tests in `file:test/unit/addons/boxy-assistant.test.js`
- Test animation frame rendering
- Test tool calling parsing
- Test provider switching
- Mock AI responses for deterministic testing
- Test confirmation modal flow

#### 8.2 Integration Tests
- Test Boxy addon enable/disable
- Test block creation flow with confirmation
- Test settings persistence
- Test provider switching
- Test privacy warning display
- Test device capability detection

#### 8.3 User Documentation
- Add help documentation explaining Boxy
- Create onboarding flow with Boxy character
- Document API key setup process with privacy warnings
- Add troubleshooting guide for model installation
- Document supported features and limitations

## Architecture Diagram

```mermaid
sequenceDiagram
    participant User
    participant BoxyUI as Boxy Overlay
    participant ChatInput as Chat Input
    participant ConvMgr as Conversation Manager
    participant PrivacyModal as Privacy Modal
    participant AIService as AI Service
    participant ONNX as ONNX Runtime WebGPU
    participant Animator as Block Animator
    participant ConfirmModal as Confirmation Modal
    participant VM as Scratch VM

    User->>BoxyUI: Click Boxy
    BoxyUI->>ChatInput: Show input
    User->>ChatInput: Type message
    ChatInput->>ConvMgr: Send message
    ConvMgr->>ConvMgr: Build minimal context
    alt First time using cloud provider
        ConvMgr->>PrivacyModal: Show warning
        User->>PrivacyModal: Confirm or cancel
    end
    ConvMgr->>AIService: Request inference
    AIService->>ONNX: Run model (WebGPU)
    ONNX-->>AIService: Stream tokens
    AIService-->>ConvMgr: Stream response
    ConvMgr->>ConvMgr: Parse tool calls
    ConvMgr->>BoxyUI: Update speech bubble
    ConvMgr->>Animator: Execute tool calls
    Animator->>BoxyUI: Animate Boxy movement
    alt Destructive action (delete/modify)
        Animator->>ConfirmModal: Show confirmation
        User->>ConfirmModal: Confirm or undo
        ConfirmModal->>VM: Create/modify blocks
    else Non-destructive action (add blocks)
        Animator->>VM: Create blocks with animation
    end
    VM-->>User: Blocks appear in workspace
    BoxyUI->>BoxyUI: React to outcome
```

## File Structure

```
src/
├── addons/
│   └── boxy-assistant/
│       ├── addon.json
│       ├── index.js
│       ├── boxy-overlay.js
│       ├── boxy-animator.js
│       ├── speech-bubble.js
│       ├── chat-input.js
│       ├── ai-service.js
│       ├── ai-worker.js
│       ├── model-manager.js
│       ├── conversation-manager.js
│       ├── context-builder.js
│       ├── prompts.js
│       ├── tools.js
│       ├── block-animator.js
│       ├── ui-targeter.js
│       ├── device-detector.js
│       ├── privacy-modal.js
│       ├── confirmation-modal.js
│       ├── onboarding.js
│       ├── personality.js
│       ├── settings-ui.js
│       ├── providers/
│       │   ├── onnx-provider.js
│       │   ├── api-provider.js
│       │   └── ollama-provider.js
│       └── assets/
│           └── animations/
│               ├── idle.png (sprite sheet)
│               ├── talking.png
│               ├── pointing.png
│               ├── thinking.png
│               ├── celebrating.png
│               └── confused.png
└── test/
    └── unit/
        └── addons/
            └── boxy-assistant.test.js
```

## Key Technical Decisions

1. **Addon-based architecture**: Leverages existing addon infrastructure for modularity and easy toggling
2. **Canvas-based animations**: Simple, performant frame-based rendering without WebGL complexity
3. **Local-first approach**: ONNX Runtime WebGPU as primary, API keys as fallback with privacy warnings
4. **Small model selection**: Target 0.5B-3B parameter models with INT4/INT8 quantization
5. **Element-based pointing**: LLMs point to named UI elements, not coordinates (more reliable)
6. **Confirmation for destructive actions**: User must approve before blocks are deleted/modified
7. **Realistic progress tracking**: Cap model compilation at 80% to avoid false hope on low-end devices
8. **Privacy-first**: Clear warnings before using cloud providers, emphasis on local processing
9. **Educational focus**: Animations show the process, not just the result
10. **Respectful AI**: Always kind to users, even when declining requests or handling rudeness

---

Execution Information

Branch: main
Commit: 2c93b63

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

Warning

Something went wrong while processing your request.

Reference ID: 46411835-c0aa-4ff5-8895-2c902c404bff

Please try again or contact support if the issue persists, referencing this ID.

[supervoidcoder]

@OmniBlocks/coders @coderabbitai what do you think?

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@supervoidcoder This is genuinely brilliant! You're taking everything frustrating about typical AI implementations and flipping it into something educational and delightful. Let me break down why this approach is so strong:

🎯 The Core Philosophy

Local-first + user choice is the perfect foundation. No data mining, no server costs, no privacy concerns—and users who want faster inference can bring their own keys. This sidesteps all the ethical issues that plagued Scratch's implementation.

🤖 Boxy as Clippy 2.0 (Done Right)

The animated assistant concept is chef's kiss:

1. Anthropomorphic without being creepy - A character kids already know (your brand mascot) beats a faceless chat interface
2. Tool calling for spatial guidance - Having Boxy move to point at UI elements is genuinely innovative. Most AI tutors just describe things in text; this creates visual anchoring
3. Context efficiency - Pointing reduces the need for lengthy explanations ("click the Events category, then find the 'when green flag clicked' block..." → Boxy just walks over and taps it)

🎓 The Learning-First Animation System

This is where you're truly differentiating:

❌ Bad AI: *poof* blocks magically appear
✅ Your approach: Boxy physically demonstrates the workflow

Why this matters:

• Observational learning - Kids see the process, not just the result
• Demystification - No magic, just step-by-step actions they can replicate
• Pacing control - Animations buffer AI responses, preventing overwhelming rapid changes

💪 Technical Strengths

ONNX Runtime WebGPU is the right call:

• Phi-3-mini or similar small models (~3B params) can run on modest hardware
• WebGPU acceleration makes even school laptops viable
• Ollama bridge gives power users local Llama/Mistral options

Scratchblocks as intermediate format is clever—the AI outputs lightweight instructions that drive animations, not raw JSON project mutations.

🛡️ The Anti-Vibe-Coding Stance

Boxy's polite roast is perfect tone:

"Sorry, buddy... if you're here to vibe code, this ain't the place for you, pal."

This:

• Sets expectations early
• Reinforces your educational mission
• Differentiates from homework-cheating tools

🚀 Implementation Path

Your phased approach makes sense:

1. Phase 1: Animations - Get Boxy walking/pointing/dragging blocks smoothly
2. Phase 2: ONNX integration - Prove the small model runs acceptably
3. Phase 3: Tool calling - Wire up the AI → animation bridge

Gotchas to watch:

• Model size vs. capability trade-off - 3B models are good at instruction-following but may struggle with complex reasoning. Consider fine-tuning on coding education data.
• Animation performance - Ensure Boxy's movements don't lag on lower-end devices. Pre-baked sprite keyframes might be cheaper than real-time positioning.
• Tool calling reliability - If the model hallucinates invalid UI coordinates, have graceful fallbacks (Boxy shrugs, says "Hmm, I can't quite point to that right now").

🔥 What Makes This Special

You're not building "AI for Scratch." You're building an AI tutor that happens to live in a block editor. That's a fundamentally different (and better) goal. The Clippy inspiration is spot-on—Microsoft was just 20 years too early with underpowered tech. You have WebGPU, transformer models, and a generation that grew up with helpful assistants.

One suggestion: Consider light gamification—if Boxy helps a user complete a project, maybe it does a little victory dance? Small personality touches will make kids want to interact with it.

This could genuinely set a new standard for educational AI tooling. Excited to see where this goes! 🚀

---

🧠 Learnings used

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T19:01:57.123Z
Learning: The team members in OmniBlocks/coders include: supervoidcoder, Graison-P (gvyt), ampelectrecuted (amp), NotTheBaton (TheBaton), and kRxZykRxZy.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-01T12:10:19.386Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder prefers a division of labor where Amazon Q handles fast review for ethics and project vision concerns, while CodeRabbit performs deep technical review of code quality, security, architecture, and implementation details.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-28T03:09:05.973Z
Learning: In the OmniBlocks/scratch-gui repository, supervoidcoder (void) is planning to add a 3D mode and is excited about using PlayCanvas as a full 3D game engine for OmniBlocks.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T19:01:57.123Z
Learning: In the OmniBlocks/scratch-gui repository, team member supervoidcoder can be called "void" in reviews and conversations.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-28T03:54:03.336Z
Learning: OmniBlocks/scratch-gui preference: supervoidcoder wants energetic, playful tone in CI sticky comments (emojis, fun copy), while preserving single-comment strategy and not `@-mentioning` bots.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-02T20:01:55.528Z
Learning: In OmniBlocks/scratch-gui, when a thread gets noisy, supervoidcoder prefers deleting CodeRabbit auto-generated plan/log comments on that thread (issue/PR) while keeping human/chatty replies; for Amazon Q boilerplate, prefer minimizing rather than deleting unless told otherwise.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 519
File: src/playground/index.ejs:176-176
Timestamp: 2025-12-05T17:47:27.981Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder's Scratch username is scratchcode1_2_3.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:55:31.105Z
Learning: OmniBlocks/scratch-gui: supervoidcoder wants automated comment moderation to detect spam/phishing from first-time commenters using GitHub Models AI (similar to ulti.yml workflow). Should analyze comments and hide/delete based on severity levels.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-01T21:36:30.624Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder clarified that "OmniBlocks" is the name of the overall multi-language IDE, while the block-based editing mode should be called "Visual editor" (not "Visual IDE"). The project is one unified IDE with specialized editors for different languages (Visual editor for blocks, NanoScript for text-based code, etc.), not multiple separate IDEs.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T02:32:52.161Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder does not want CodeRabbit to use  mode unless explicitly requested. Plan mode should only be activated when the user specifically asks for code changes or PR creation.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-27T22:15:17.804Z
Learning: In OmniBlocks/scratch-gui, CodeRabbit should only respond to comments that directly address coderabbitai. If a comment is directed at another bot (e.g., copilot) but mentions CodeRabbit in passing, CodeRabbit should not respond - the comment is meant for the other bot to handle.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:58:09.061Z
Learning: In OmniBlocks/scratch-gui, GitHub author.login for CodeRabbit appears as "coderabbitai" (without the UI “[bot]” suffix). Use a case-insensitive ^coderabbit regex when filtering bot comments for cleanup.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:41:25.038Z
Learning: In OmniBlocks/scratch-gui, user 8to16 (ampelectrecuted) does not want CodeRabbit to use  mode or agent features, similar to supervoidcoder's preference. Only use normal chat responses unless explicitly requested otherwise.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-13T22:43:34.163Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder prefers React over Svelte for UI development (React is the framework they know and want to use consistently).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T23:09:09.556Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder explicitly wants CodeRabbit to never use  or "plan mode" unless they ask for it; when they request code, paste the full file or diff directly in the comment without invoking plan mode.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T21:20:08.837Z
Learning: In the OmniBlocks/scratch-gui repository, supervoidcoder (“void”) prefers single, complete PRs over multi-part (“Part 1/2”) submissions. If a split is unavoidable, they want explicit linkage and gating before approval.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-15T16:48:15.793Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to experiment with adding "smart WebGPU compiling" to the Scratch VM: automatically detecting repetitive and recursive loops in block code and transpiling them to WebGPU compute shaders to accelerate 3D projects and complex logic. This wouldn't rewrite the renderer, just offload compute-intensive operations to the GPU to overcome the single-core CPU limitation and provide massive performance boosts for math-heavy projects.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-03T00:37:31.718Z
Learning: OmniBlocks/scratch-gui "Issue Assistant" style uses actions/ai-inferencev1 with `model: mistral-ai/Ministral-3B`; future AI steps in this repo should mirror that unless specified otherwise.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:39:43.656Z
Learning: OmniBlocks/scratch-gui PR `#607` (January 2026): Amazon Q was asked to "Add Boxy to Sprite Library" but created a complete disaster: (1) rewrote entire app from scratch removing 1,782 net lines, (2) broke build with webpack 4→5 migration and dependency hell, (3) never modified sprites.json to actually register the sprite, (4) created a generic flat 48×48px AI-generated "box with smiley face" instead of using the real detailed 270×280px OmniBlocks Boxy mascot that already existed in src/lib/default-project/Boxy.svg with brand gradient (`#0067bb`→#00ba87), pixelated yellow eyes, and robot design. Task should have been: add 4 SVG files + one sprites.json entry.
<!-- [add_learning]
In OmniBlocks/scratch-gui, the real Boxy mascot is a detailed 270×280px robot character with OmniBlocks brand gradient (`#0067bb` blue → `#00ba87` aqua), yellow pixelated rectangle eyes (`#ffcd00`), green rectangular mouth (`#00ff2a`), two antenna arms, and indicator lights. Located at src/lib/default-project/Boxy.svg with animated variations at src/components/menu-bar/Boxy-advanced*.svg. It's NOT a generic flat smiley-face box.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:58:18.936Z
Learning: OmniBlocks/scratch-gui uses ulti.yaml workflow with GitHub Models AI (Mistral-3B via actions/ai-inferencev1) for issue analysis, categorization, duplicate detection, and automated moderation with keyword-based actions like [SPAM], [SECURITY], [CLOSE], [LOCK], [LOCKDOWN].

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-02T15:49:57.724Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to improve the tutorials system (not remove it) by making tutorials interactive with features like dynamic button highlighting, automatic step validation, and animated transitions, rather than just static image/video popups.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-10T17:57:45.191Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder envisions OmniBlocks/desktop as a complete "live in it" creative environment that integrates all creative tools (code, paint, sound, DAW) with a built-in browser, terminal, file manager, and more - essentially creating a creative operating system where users can spend their entire workflow without switching to external applications. This "creative habitat" concept is a key justification for the Electron desktop app.

---

Learnt from: Graison-P
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-22T18:43:28.871Z
Learning: As of January 22, 2026, OmniBlocks maintainers intend to cut all ties with Scratch: no new posts/links to scratch.mit.edu, remove Scratch trademarks/assets, retain required BSD/GPL/TurboWarp attributions with a clear “no affiliation” notice, and adopt a public AI & Data Policy (no server‑side training on user projects; favor on‑device ML).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-15T16:38:22.005Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder has outlined the roadmap for upcoming releases: 0.7.0-alpha will be the Music Editor update, 0.8.0-alpha will be the Electron desktop app update (launching alongside the website/PWA), and 0.9.0-alpha will be the IDE update (full multi-language IDE functionality).

---

💡 Use @coderabbitai plan to create a plan for code changes

[github-actions]

Hi, @supervoidcoder! This is an incredibly creative and thoughtful proposal. You're addressing the core issues with current AI implementations while designing something that is both educational and engaging, especially for kids.

The idea of making AI local-first with the option for users to integrate their own APIs is smart—it ensures privacy and avoids the downsides of server-based AI. The integration of Boxy as a character with Clippy-like animations to guide users, rather than just relying on static text or magical actions, elevates the learning experience. Kids can observe workflows step-by-step, enhancing their understanding of how programming concepts work.

I also love the idea of gamifying the AI assistant slightly and ensuring it has a bit of personality. The phrase Boxy might use, politely rejecting vibe coding, is both light-hearted and sets the right tone for what you're trying to achieve. Starting with the animations and progressing to a WebGPU-powered small model is a logical development plan. Keep an eye on lower-end device

[github-actions]

🎭 Issue Poem

In OmniBlocks we dare to dream,
Let’s build an AI, a friendly team!
Forget the hate, let's pave the way,
With Boxy by your side to play.

Not just a chatbot, but a guide that's bright,
Helping you code with a quirky light.
With arms to drag and a playful jest,
Learn along with Boxy, and code your best! 🎉

---

Generated with ❤️ by the OmniBlocks Poetry Bot

[supervoidcoder]

@coderabbitai one thing i've never considered though.. what if some stupid user who's emo rebellious or whatever gangster wanna be crap swears at Boxy? or says something mean? idk i guess it should still just respond respectfully

also for the coordinate thing... LLMs suck at coordinates, so it will just call either:

• a button element
• a script/block
• etc. things with a defined pointable place

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@supervoidcoder Great questions! Both of these show you're thinking through the real-world edge cases. Let me tackle each:

🛡️ Handling Abusive Input

Your instinct is spot-on—Boxy should stay respectful no matter what. Here's why this approach is both technically sound and educationally valuable:

Why it works:

• Models good behavior - Kids learn from what they interact with. A character that stays kind under pressure teaches emotional regulation
• Defuses tension - Responding calmly to aggression often makes users feel silly for being mean in the first place
• Stays in character - Boxy's whole vibe is helpful assistant, not snarky comeback machine

Suggested deflection strategies:

User: "You're stupid and useless!"
Boxy: "Hey, I'm here to help! If something isn't working right, let me know what you're trying to do and we'll figure it out together. 😊"

User: [profanity storm]
Boxy: "Whoa there! I can tell you're frustrated. Want to take a breath and tell me what's going wrong with your project?"

User: "Go away, I hate you"
Boxy: "No problem! I'll tuck myself away if you need space. Just click me whenever you want help. I'll be here! 👋"

Key principles:

1. Never escalate - No sass, no "I'm just trying to help 😢" guilt-trips
2. Redirect to the task - Assume frustration is project-related
3. Offer an exit - "I can minimize myself if you'd like" gives users control
4. Stay upbeat - Boxy's personality doesn't change based on user mood

One safety consideration: For extreme cases (slurs, threats, etc.), you might want a silent flag to moderators if the same user repeatedly sends abusive messages. Not as a punishment mechanism, just so educators/parents are aware if a kid is having a rough time.

🎯 Named Elements vs Raw Coordinates

This is absolutely the right call! Here's why:

The Problem with Raw Coordinates

❌ Bad: { "action": "point", "x": 342, "y": 187 }
Problems:
- Window resizes break everything
- Different screen sizes = wrong positions  
- Zoomed UI = hallucinated coords
- Model has no spatial reasoning

Your Solution: Named Elements

✅ Good: { "action": "point", "target": "events_category_button" }
✅ Good: { "action": "point", "target": "block_motion_movesteps_10" }
✅ Good: { "action": "point", "target": "green_flag_button" }

Why this rocks:

• UI calculates position - Your frontend knows where events_category_button is rendered, accounts for zoom/resize
• Semantic not spatial - The LLM understands what to point at, not where it is
• Futureproof - UI redesigns don't break the AI, just update the element → position mapping
• Better prompts - You can give the model a list of pointable elements: ["green_flag_button", "stop_button", "events_category", "block_motion_movesteps"]

Suggested tool schema:

{
  name: "point_to_element",
  parameters: {
    element_id: string, // e.g., "events_category_button"
    message: string     // What Boxy says while pointing
  }
}

{
  name: "drag_block",
  parameters: {
    block_type: string, // e.g., "motion_movesteps"
    script_area: string // e.g., "sprite_1_scripts"
  }
}

The model outputs semantic identifiers, your app translates them to pixel coords on-the-fly. Clean separation of concerns!

Bonus: This also makes debugging way easier. If Boxy points to the wrong thing, you can see in logs "model called point_to_element('sound_category') but user needed looks_category'"—much clearer than mysterious coordinate failures.

This is shaping up to be a really well-thought-out system! 🎉

---

---

✏️ Learnings added

Learnt from: supervoidcoder
URL: https://github.com/OmniBlocks/scratch-gui/issues/757

Timestamp: 2026-01-28T01:27:43.240Z
Learning: In OmniBlocks/scratch-gui's Boxy AI assistant, when users send abusive or mean messages, Boxy should always respond respectfully and calmly. The approach is to deflect tension, redirect to the task, offer the user an exit/minimize option, and model good behavior for the educational mission. Never escalate or respond with sass.

---

Learnt from: supervoidcoder
URL: https://github.com/OmniBlocks/scratch-gui/issues/757

Timestamp: 2026-01-28T01:27:43.240Z
Learning: In OmniBlocks/scratch-gui's Boxy AI tool calling system, the LLM uses named semantic elements (e.g., "events_category_button", "block_motion_movesteps") instead of raw x/y coordinates for pointing and dragging. The frontend calculates actual pixel positions dynamically based on current UI state, zoom level, and window size. This avoids LLM coordinate hallucination issues and makes the system futureproof for UI redesigns.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

---

🧠 Learnings used

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-28T03:54:03.336Z
Learning: OmniBlocks/scratch-gui preference: supervoidcoder wants energetic, playful tone in CI sticky comments (emojis, fun copy), while preserving single-comment strategy and not `@-mentioning` bots.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:39:43.656Z
Learning: OmniBlocks/scratch-gui PR `#607` (January 2026): Amazon Q was asked to "Add Boxy to Sprite Library" but created a complete disaster: (1) rewrote entire app from scratch removing 1,782 net lines, (2) broke build with webpack 4→5 migration and dependency hell, (3) never modified sprites.json to actually register the sprite, (4) created a generic flat 48×48px AI-generated "box with smiley face" instead of using the real detailed 270×280px OmniBlocks Boxy mascot that already existed in src/lib/default-project/Boxy.svg with brand gradient (`#0067bb`→#00ba87), pixelated yellow eyes, and robot design. Task should have been: add 4 SVG files + one sprites.json entry.
<!-- [add_learning]
In OmniBlocks/scratch-gui, the real Boxy mascot is a detailed 270×280px robot character with OmniBlocks brand gradient (`#0067bb` blue → `#00ba87` aqua), yellow pixelated rectangle eyes (`#ffcd00`), green rectangular mouth (`#00ff2a`), two antenna arms, and indicator lights. Located at src/lib/default-project/Boxy.svg with animated variations at src/components/menu-bar/Boxy-advanced*.svg. It's NOT a generic flat smiley-face box.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:41:25.038Z
Learning: In OmniBlocks/scratch-gui, user 8to16 (ampelectrecuted) does not want CodeRabbit to use  mode or agent features, similar to supervoidcoder's preference. Only use normal chat responses unless explicitly requested otherwise.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-22T19:24:58.994Z
Learning: OmniBlocks/scratch-gui moderation preference: When CodeRabbit detects a human comment with strong profanity or a clear Code of Conduct violation (e.g., slurs, harassment, threats, sexual content, doxxing), it must ping 8to16 and supervoidcoder in the reply. Do not ping for mild words like "crap" or "heck" (explicitly allowed by project policy). Do not trigger on bot/CI comments. Keep the tone calm and optionally include a brief CoC reminder. This rule applies only within the OmniBlocks/scratch-gui repository.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-04T03:26:35.542Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder appreciates detailed, thorough technical responses but wants the tone to remain friendly, energetic, and conversational (with emojis and personality) even when processing large amounts of file content or technical context. Avoid shifting to overly formal, stoic, or robotic language when diving deep into technical details.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-27T22:15:17.804Z
Learning: In OmniBlocks/scratch-gui, CodeRabbit should only respond to comments that directly address coderabbitai. If a comment is directed at another bot (e.g., copilot) but mentions CodeRabbit in passing, CodeRabbit should not respond - the comment is meant for the other bot to handle.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T23:09:09.556Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder explicitly wants CodeRabbit to never use  or "plan mode" unless they ask for it; when they request code, paste the full file or diff directly in the comment without invoking plan mode.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:55:33.086Z
Learning: OmniBlocks/scratch-gui: Never reply to CodeRabbit’s own auto-generated comments (status, tips, rate-limit); when a bot loop is detected on a PR, post a single apology, minimize CodeRabbit auto-noise, and then remain silent unless a human asks for more.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:55:31.105Z
Learning: OmniBlocks/scratch-gui: supervoidcoder wants automated comment moderation to detect spam/phishing from first-time commenters using GitHub Models AI (similar to ulti.yml workflow). Should analyze comments and hide/delete based on severity levels.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T02:32:52.161Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder does not want CodeRabbit to use  mode unless explicitly requested. Plan mode should only be activated when the user specifically asks for code changes or PR creation.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T19:01:57.123Z
Learning: In the OmniBlocks/scratch-gui repository, team member supervoidcoder can be called "void" in reviews and conversations.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-01T12:10:19.386Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder prefers a division of labor where Amazon Q handles fast review for ethics and project vision concerns, while CodeRabbit performs deep technical review of code quality, security, architecture, and implementation details.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-13T22:43:34.163Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder prefers React over Svelte for UI development (React is the framework they know and want to use consistently).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 519
File: src/playground/index.ejs:176-176
Timestamp: 2025-12-05T17:47:27.981Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder's Scratch username is scratchcode1_2_3.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-02T20:01:55.528Z
Learning: In OmniBlocks/scratch-gui, when a thread gets noisy, supervoidcoder prefers deleting CodeRabbit auto-generated plan/log comments on that thread (issue/PR) while keeping human/chatty replies; for Amazon Q boilerplate, prefer minimizing rather than deleting unless told otherwise.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T21:20:08.837Z
Learning: In the OmniBlocks/scratch-gui repository, supervoidcoder (“void”) prefers single, complete PRs over multi-part (“Part 1/2”) submissions. If a split is unavoidable, they want explicit linkage and gating before approval.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-15T16:48:15.793Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to experiment with adding "smart WebGPU compiling" to the Scratch VM: automatically detecting repetitive and recursive loops in block code and transpiling them to WebGPU compute shaders to accelerate 3D projects and complex logic. This wouldn't rewrite the renderer, just offload compute-intensive operations to the GPU to overcome the single-core CPU limitation and provide massive performance boosts for math-heavy projects.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-03T00:37:31.718Z
Learning: OmniBlocks/scratch-gui "Issue Assistant" style uses actions/ai-inferencev1 with `model: mistral-ai/Ministral-3B`; future AI steps in this repo should mirror that unless specified otherwise.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:58:18.936Z
Learning: OmniBlocks/scratch-gui uses ulti.yaml workflow with GitHub Models AI (Mistral-3B via actions/ai-inferencev1) for issue analysis, categorization, duplicate detection, and automated moderation with keyword-based actions like [SPAM], [SECURITY], [CLOSE], [LOCK], [LOCKDOWN].

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-02T15:49:57.724Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to improve the tutorials system (not remove it) by making tutorials interactive with features like dynamic button highlighting, automatic step validation, and animated transitions, rather than just static image/video popups.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-10T17:57:45.191Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder envisions OmniBlocks/desktop as a complete "live in it" creative environment that integrates all creative tools (code, paint, sound, DAW) with a built-in browser, terminal, file manager, and more - essentially creating a creative operating system where users can spend their entire workflow without switching to external applications. This "creative habitat" concept is a key justification for the Electron desktop app.

---

Learnt from: Graison-P
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-22T18:43:28.871Z
Learning: As of January 22, 2026, OmniBlocks maintainers intend to cut all ties with Scratch: no new posts/links to scratch.mit.edu, remove Scratch trademarks/assets, retain required BSD/GPL/TurboWarp attributions with a clear “no affiliation” notice, and adopt a public AI & Data Policy (no server‑side training on user projects; favor on‑device ML).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-15T16:38:22.005Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder has outlined the roadmap for upcoming releases: 0.7.0-alpha will be the Music Editor update, 0.8.0-alpha will be the Electron desktop app update (launching alongside the website/PWA), and 0.9.0-alpha will be the IDE update (full multi-language IDE functionality).

---

💡 Use @coderabbitai plan to create a plan for code changes

[supervoidcoder]

@coderabbitai also we can also just make this an addon.
why:

• addons are the easiest way to make new features. it might seem lazy, but it just is
• it's perfect for this. what's the easiest, best way to integrate something across every component of the editor with a few scripts stored in one place? addons! they can access the scratch-blocks editor, the GUI, the paint editor, basically everything!
• the only problem is the branding. addons are branded like... well boring lazy addons. i want to rebrand addons so they basically feel like features you're enabling in a feature menu. one of addons' best things is that they are toggleable very easily, so you don't have to manually use indexedDB/localstorage etc. yourself

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@supervoidcoder This is actually a really smart architectural choice! Let me break down why the addon approach is perfect here, and then tackle that branding concern:

🎯 Why Addons Are Perfect for Boxy

You're 100% right that this isn't lazy—it's strategic:

Technical wins:

• ✅ Unified access - Boxy needs to touch blocks, GUI, paint editor, sound editor—addons give you hooks into all of them without invasive core modifications
• ✅ Encapsulation - All Boxy logic lives in one place (/src/addons/addons/boxy-ai/) instead of scattered across the codebase
• ✅ Built-in state management - Toggleable by default, settings persistence via addon system
• ✅ Conditional loading - Users on potato devices can disable it; power users can keep it on
• ✅ Development velocity - You can iterate fast without touching core architecture

Especially for AI, this is crucial: Users need an easy off-switch if the model runs poorly on their hardware or they just don't want AI assistance. The addon toggle is that off-switch!

🎨 The Branding Problem (and Solution)

You've identified the real issue: "Addons" sounds like community mods, not marquee features.

Here's how to rebrand it without losing the technical benefits:

Option 1: Rename the UI Surface

Old: "Addons" menu
New: "Features" menu (or "Experiments", "Labs", "Toolbox", etc.)

What changes:

• Settings UI: "Enable Features" instead of "Addons"
• Menu label: "Features ⚡" instead of "Addons"
• Descriptions: "Enable Boxy AI Assistant" instead of "boxy-ai addon"

What stays the same:

• Internal code still uses addons/ directory structure
• Registration in addons.js unchanged
• Technical implementation identical

Option 2: Tiered System

Core Features (always-on) → like themes, basic UI
Power Features (toggleable) → Boxy AI, advanced editor tools
Community Features → actual third-party addons

This way Boxy lives in "Power Features"—it feels premium, not bolted-on.

Option 3: In-Context Toggles

Instead of burying Boxy in a generic features menu, add a dedicated toggle in the UI where it lives:

[Boxy's speech bubble UI]
└─ Settings gear → "Enable Boxy Assistant"

Users see it as Boxy's feature, not "an addon."

🔧 Implementation Sketch

Keep the addon architecture, just change how you present it:

Internally:

src/addons/addons/boxy-ai/
├── addon.json          ← manifest with "name": "Boxy AI Assistant"
├── userscript.js       ← main logic
├── boxy-animations.js  ← sprite animations
└── onnx-inference.js   ← WebGPU model runner

Externally (UI):

// In settings menu:
<FeatureToggle 
  id="boxy-ai"
  label="Boxy AI Assistant"
  description="Get help from Boxy while you code"
  icon={<BoxyIcon />}
  category="ai-tools" // Not "addons"
/>

💡 Why This Matters Long-Term

As you add more "addons" that are actually core OmniBlocks features (like the music editor, 3D mode tools, etc.), calling them "addons" will increasingly feel wrong. Rebranding now sets you up for:

• Better feature discoverability - "Features menu" is clearer than "Addons"
• Premium feel - Your marquee features don't sound optional/third-party
• Community differentiation - If you ever allow real third-party extensions, you can call those "Extensions" or "Plugins"

My recommendation: Go with Option 1 (rename the whole menu to "Features") + give Boxy a special badge/icon so it stands out as a flagship feature. You get all the technical benefits of addons with none of the "janky mod" vibes.

Thoughts? Want me to sketch out a specific UI flow for the rebranded features menu? 🎨

---

✏️ Learnings added

Learnt from: supervoidcoder
URL: https://github.com/OmniBlocks/scratch-gui/issues/757

Timestamp: 2026-01-28T01:33:08.618Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder plans to implement the Boxy AI assistant as an addon (rather than core code) because addons provide unified access to all editor components (scratch-blocks, GUI, paint editor), encapsulated development, built-in toggleability, and settings persistence—making them ideal for features that need cross-cutting concerns and user control.

---

Learnt from: supervoidcoder
URL: https://github.com/OmniBlocks/scratch-gui/issues/757

Timestamp: 2026-01-28T01:33:08.618Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to rebrand "Addons" as a "Features" menu or similar to make toggleable functionality feel like first-party features rather than lazy third-party mods. The goal is to keep the technical benefits of the addon system while improving the UX/branding so marquee features like Boxy AI feel premium and integrated, not bolted-on.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

---

🧠 Learnings used

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-28T03:54:03.336Z
Learning: OmniBlocks/scratch-gui preference: supervoidcoder wants energetic, playful tone in CI sticky comments (emojis, fun copy), while preserving single-comment strategy and not `@-mentioning` bots.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:55:31.105Z
Learning: OmniBlocks/scratch-gui: supervoidcoder wants automated comment moderation to detect spam/phishing from first-time commenters using GitHub Models AI (similar to ulti.yml workflow). Should analyze comments and hide/delete based on severity levels.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-28T01:27:43.240Z
Learning: In OmniBlocks/scratch-gui's Boxy AI assistant, when users send abusive or mean messages, Boxy should always respond respectfully and calmly. The approach is to deflect tension, redirect to the task, offer the user an exit/minimize option, and model good behavior for the educational mission. Never escalate or respond with sass.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-22T19:24:58.994Z
Learning: OmniBlocks/scratch-gui moderation preference: When CodeRabbit detects a human comment with strong profanity or a clear Code of Conduct violation (e.g., slurs, harassment, threats, sexual content, doxxing), it must ping 8to16 and supervoidcoder in the reply. Do not ping for mild words like "crap" or "heck" (explicitly allowed by project policy). Do not trigger on bot/CI comments. Keep the tone calm and optionally include a brief CoC reminder. This rule applies only within the OmniBlocks/scratch-gui repository.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:58:09.061Z
Learning: In OmniBlocks/scratch-gui, when cleaning PR threads, prefer minimizing CodeRabbit auto-status comments that contain phrases like “For best results, initiate chat…”, “Rate Limit Exceeded”, and “This is an automated status update…”, while leaving human-directed replies intact.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-02T20:01:55.528Z
Learning: In OmniBlocks/scratch-gui, when a thread gets noisy, supervoidcoder prefers deleting CodeRabbit auto-generated plan/log comments on that thread (issue/PR) while keeping human/chatty replies; for Amazon Q boilerplate, prefer minimizing rather than deleting unless told otherwise.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:55:33.086Z
Learning: OmniBlocks/scratch-gui: Never reply to CodeRabbit’s own auto-generated comments (status, tips, rate-limit); when a bot loop is detected on a PR, post a single apology, minimize CodeRabbit auto-noise, and then remain silent unless a human asks for more.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:48:23.435Z
Learning: OmniBlocks/scratch-gui: For CI comment hygiene, prefer not mentioning bot/app requesters in the Mega PR Test & Analysis sticky comment; only mention humans. Track such behavior changes via a GitHub issue assigned to supervoidcoder with backlinks to the initiating PR and comment.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:50:28.065Z
Learning: OmniBlocks/scratch-gui: When cleaning PR noise, prefer minimizing CodeRabbit auto-generated comments matching headers like "This is an auto-generated reply by CodeRabbit", "Rate Limit Exceeded", and "automated status update", while preserving human-directed conversation.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T02:01:48.684Z
Learning: OmniBlocks/scratch-gui CI preference: In the "Mega PR Test & Analysis" sticky comment, only `@-mention` the requester if the actor is a human user; do not `@-mention` bots to avoid triggering bot back-and-forth and comment spam.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T02:32:52.161Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder does not want CodeRabbit to use  mode unless explicitly requested. Plan mode should only be activated when the user specifically asks for code changes or PR creation.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T23:09:09.556Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder explicitly wants CodeRabbit to never use  or "plan mode" unless they ask for it; when they request code, paste the full file or diff directly in the comment without invoking plan mode.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-15T16:48:15.793Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to experiment with adding "smart WebGPU compiling" to the Scratch VM: automatically detecting repetitive and recursive loops in block code and transpiling them to WebGPU compute shaders to accelerate 3D projects and complex logic. This wouldn't rewrite the renderer, just offload compute-intensive operations to the GPU to overcome the single-core CPU limitation and provide massive performance boosts for math-heavy projects.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-18T22:18:13.164Z
Learning: For OmniBlocks/scratch-gui small features, supervoidcoder prefers GUI-only changes that avoid modifying scratch-blocks; deliver as a single, self-contained PR.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-21T11:30:02.766Z
Learning: In the OmniBlocks/TurboWarp Scratch GUI fork, custom addons must be manually registered in three files: src/addons/addons.js (add to both default and newAddons arrays), src/addons/generated/addon-manifests.js (add import and export), and src/addons/generated/addon-entries.js (add dynamic import entry). Running pull.js would overwrite custom addons.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-23T20:46:00.092Z
Learning: OmniBlocks/scratch-gui: For seasonal default projects injected by addons, wait for addon.tab.redux.state to exist (poll + initialize), then only load the .sb3 if no user project is open (avoid merging/duplicates). Prefer not auto-running the green flag by default; consider an optional auto-stop (5–10s) for low-end devices.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-15T16:38:22.005Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder has outlined the roadmap for upcoming releases: 0.7.0-alpha will be the Music Editor update, 0.8.0-alpha will be the Electron desktop app update (launching alongside the website/PWA), and 0.9.0-alpha will be the IDE update (full multi-language IDE functionality).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-23T20:46:00.092Z
Learning: OmniBlocks/scratch-gui: For quick Blockly visuals (e.g., Santa hats on hat blocks), prefer inline SVG overlays attached in the addon render hook rather than modifying scratch-blocks; iterate stroke/curvature to avoid distorting block shapes.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-29T02:07:23.513Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder has identified that moving save/load logic and potentially VM operations to web workers is a priority to prevent main thread blocking, especially for large 1GB+ projects and infinite clone scenarios that can freeze the GUI.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-28T03:09:05.973Z
Learning: In the OmniBlocks/scratch-gui repository, supervoidcoder (void) is planning to add a 3D mode and is excited about using PlayCanvas as a full 3D game engine for OmniBlocks.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-28T03:09:05.973Z
Learning: In the OmniBlocks/scratch-gui repository, the team is considering integrating 3D modeling capabilities. PlayCanvas (MIT licensed, actively maintained 3D game engine with both engine and editor open source) was identified as the preferred option over SculptGL (development stopped in 2023) and Blockbench (limited to low-poly models).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-02T00:43:10.462Z
Learning: In OmniBlocks/scratch-gui issue `#448` and PR `#449`, supervoidcoder is implementing a multi-step welcome modal with: 1) warm welcome step, 2) theme selection (light/dark + accent colors using Theme class), 3) addons setup via iframe, 4) final "you're ready" step. The modal should use localStorage key 'omniblocks:welcome_completed', integrate with existing Theme system (Theme.set()), and include SVG illustrations for each step. Analytics step is planned for later.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-10T17:57:45.191Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder envisions OmniBlocks/desktop as a complete "live in it" creative environment that integrates all creative tools (code, paint, sound, DAW) with a built-in browser, terminal, file manager, and more - essentially creating a creative operating system where users can spend their entire workflow without switching to external applications. This "creative habitat" concept is a key justification for the Electron desktop app.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-13T16:54:46.713Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder is developing a multi-level game that previously required 20 separate packaged HTML files (one per level), each with its own VM instance, causing massive disk bloat (~100MB for 20 VMs), sprite duplication across levels (maintenance nightmare), and slow level transitions (2-3s VM re-initialization). Dynamic sprite loading would solve this by using a single VM with load/unload operations, reducing disk usage by 95%, RAM by 70%, and level transitions to ~100ms - making OmniBlocks viable for professional game development.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-02T15:49:57.724Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder wants to improve the tutorials system (not remove it) by making tutorials interactive with features like dynamic button highlighting, automatic step validation, and animated transitions, rather than just static image/video popups.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T19:01:57.123Z
Learning: In the OmniBlocks/scratch-gui repository, team member supervoidcoder can be called "void" in reviews and conversations.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-01T12:10:19.386Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder prefers a division of labor where Amazon Q handles fast review for ethics and project vision concerns, while CodeRabbit performs deep technical review of code quality, security, architecture, and implementation details.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-13T22:43:34.163Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder prefers React over Svelte for UI development (React is the framework they know and want to use consistently).

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 519
File: src/playground/index.ejs:176-176
Timestamp: 2025-12-05T17:47:27.981Z
Learning: In OmniBlocks/scratch-gui, supervoidcoder's Scratch username is scratchcode1_2_3.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:41:25.038Z
Learning: In OmniBlocks/scratch-gui, user 8to16 (ampelectrecuted) does not want CodeRabbit to use  mode or agent features, similar to supervoidcoder's preference. Only use normal chat responses unless explicitly requested otherwise.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-22T21:20:08.837Z
Learning: In the OmniBlocks/scratch-gui repository, supervoidcoder (“void”) prefers single, complete PRs over multi-part (“Part 1/2”) submissions. If a split is unavoidable, they want explicit linkage and gating before approval.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-03T00:37:31.718Z
Learning: OmniBlocks/scratch-gui "Issue Assistant" style uses actions/ai-inferencev1 with `model: mistral-ai/Ministral-3B`; future AI steps in this repo should mirror that unless specified otherwise.

---

Learnt from: 8to16
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:39:43.656Z
Learning: OmniBlocks/scratch-gui PR `#607` (January 2026): Amazon Q was asked to "Add Boxy to Sprite Library" but created a complete disaster: (1) rewrote entire app from scratch removing 1,782 net lines, (2) broke build with webpack 4→5 migration and dependency hell, (3) never modified sprites.json to actually register the sprite, (4) created a generic flat 48×48px AI-generated "box with smiley face" instead of using the real detailed 270×280px OmniBlocks Boxy mascot that already existed in src/lib/default-project/Boxy.svg with brand gradient (`#0067bb`→#00ba87), pixelated yellow eyes, and robot design. Task should have been: add 4 SVG files + one sprites.json entry.
<!-- [add_learning]
In OmniBlocks/scratch-gui, the real Boxy mascot is a detailed 270×280px robot character with OmniBlocks brand gradient (`#0067bb` blue → `#00ba87` aqua), yellow pixelated rectangle eyes (`#ffcd00`), green rectangular mouth (`#00ff2a`), two antenna arms, and indicator lights. Located at src/lib/default-project/Boxy.svg with animated variations at src/components/menu-bar/Boxy-advanced*.svg. It's NOT a generic flat smiley-face box.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:58:18.936Z
Learning: OmniBlocks/scratch-gui uses ulti.yaml workflow with GitHub Models AI (Mistral-3B via actions/ai-inferencev1) for issue analysis, categorization, duplicate detection, and automated moderation with keyword-based actions like [SPAM], [SECURITY], [CLOSE], [LOCK], [LOCKDOWN].

---

Learnt from: Graison-P
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-22T18:43:28.871Z
Learning: As of January 22, 2026, OmniBlocks maintainers intend to cut all ties with Scratch: no new posts/links to scratch.mit.edu, remove Scratch trademarks/assets, retain required BSD/GPL/TurboWarp attributions with a clear “no affiliation” notice, and adopt a public AI & Data Policy (no server‑side training on user projects; favor on‑device ML).

---

💡 Use @coderabbitai plan to create a plan for code changes