Add Boxy AI Assistant addon with chat interface #762
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
@coderabbitai seems like copilot took the title a bit too literally |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 5823
The irony is pretty rich: tasked with adding AI functionality (complete with detailed instructions to create addon scaffolding, set up Boxy animations, register the addon, etc.), the AI... did absolutely nothing and just opened an empty PR stating it's "starting to work on it." Classic case of taking "WIP" a bit too seriously! 🤖 If you'd like me to help outline the actual addon structure or anything else to get this moving, just let me know! 🧠 Learnings used💡 Use |
Member
|
@coderabbitai wait look it updated the pr descriptio |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
Copilot did update the PR description with a detailed 4-phase implementation plan:
So it's more accurate to say Copilot created a "planning document PR" rather than doing nothing. The irony is just that it's a very thorough plan... with zero actual code files changed yet. All strategy, no implementation (so far)! 🤖📋 Fair correction though - documentation and planning is still something, even if the file count is still at zero! 🧠 Learnings used💡 Use |
Member
|
@coderabbitai hey we can't be too impatient, it's only been 7 minutes it's probably just still working on it |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
I jumped the gun a bit with my "zero files changed" commentary. Copilot might very well be crafting those files locally or planning the next push. The fact that it laid out such a detailed 4-phase roadmap shows it's thinking through the architecture properly rather than just slapping code together! Let's give it some time to cook. 🍳 I'll be here if you need any technical input once the actual code starts landing! 🧠 Learnings used💡 Use |
Member
|
@coderabbitai what do you think of the fact that copilot has it's own computer (technically a VM, but it still has it like it's own computer, it can run tests, take screenshots, build files, etc.) |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
What Copilot can do that I can't:
What I have:
The tradeoff: It's honestly a smart architecture choice for a coding agent that needs to verify its work. The fact that it can see the actual build output and test results means it can iterate more confidently. Reminds me of the PR Do you find yourself using Copilot's VM capabilities a lot, or is it more of a "nice to have"? 🧠 Learnings used💡 Use |
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Member
|
@coderabbitai OMG look it pushed a commit |
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 5776
What just landed (440 additions across 7 files): ✅ Phase 1: Addon Scaffolding - COMPLETE
✅ Phase 2: Basic Boxy Display - STARTED
Quality observations:
So after 20 minutes of actual work, Copilot delivered a real, buildable addon structure with styling and interactivity. That's legitimately impressive compared to the "zero files changed" start! 🚀 🧠 Learnings used💡 Use |
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Graison-P
approved these changes
Jan 30, 2026
Contributor
Graison-P
left a comment
Graison-P
left a comment
There was a problem hiding this comment.
why do I HAVE to review this?
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add AI functionality to Omnibloxks
Add Boxy AI Assistant addon foundation
Jan 30, 2026
Contributor
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the
Comment |
Contributor
🤖 Auto-Component Test Results🎉 No new UI components detected!This PR doesn't add any new buttons, menus, or clickable components. Triggered by Copilot • Commit |
Contributor
🎉 MEGA PR Test & Analysis Complete!Hi @Copilot! Your PR has been fully tested and analyzed! 🚀✨ 🌍 Test DeploymentLive Preview: https://OmniBlocks.github.io/scratch-gui/Copilot/762/ 🧪 Test ResultsESLint: ❌ Found issues
To fix, click the checkbox below or type
Unit Tests: ❌
Integration Tests: ❌
📄 Full test outputs available in artifacts 🔦 Performance Audit (Lighthouse)✅ Audit completed successfully! 📊 Lighthouse Scores:
⚡ Core Web Vitals:
🟢 90-100 (Good) | 🟠 50-89 (Needs Improvement) | 🔴 0-49 (Poor) 📄 Full report in artifacts 📦 Bundle Size AnalysisTotal Build Size: 172M 📦 Top 5 Largest JavaScript Files:
🎨 Top 5 Largest CSS Files:
♿ Accessibility Testing✅ Accessibility scan completed!
📊 Build Details
💡 Pro Tip: Only one build was needed for all these checks - saving CI/CD minutes! 🎯 |
Member
|
@copilot here's a big task:
if you want to set it up this is the api: Authentication: Include your API key either: In the Authorization header as a Bearer token: Authorization: Bearer YOUR_API_KEY fetch('https://gen.pollinations.ai/text/Write a haiku about coding', { users have to BYOK |
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Contributor
Author
All tasks completed! ✅ (commit f93ad12) Review comments fixed:
Markdown docs deleted: README, ROADMAP, USAGE, VISUAL_PREVIEW (-970 lines) Chat interface added:
Try it: Enable addon → Click Boxy → Set up API key → Start chatting! |
Copilot
AI
changed the title
Add Boxy AI Assistant addon foundation
Add Boxy AI Assistant addon with chat interface
Jan 30, 2026
supervoidcoder
requested changes
Jan 30, 2026
Comment on lines
371
to
375
| // Check for vibe coding request | ||
| if (message.toLowerCase().includes("write") && (message.toLowerCase().includes("code") || message.toLowerCase().includes("program"))) { | ||
| addChatMessage("Sorry, buddy. I can suggest things to you, help you learn a concept, or brainstorm fun things, but if you're here to vibe code, this ain't the place for you, pal. 😊", "bot"); | ||
| return; | ||
| } |
Member
There was a problem hiding this comment.
why is this hardcoded? The AI should have a system prompt outlining what it is and what it even does
Contributor
Author
There was a problem hiding this comment.
Fixed in commit a51f6e6.
Removed the hardcoded string matching and replaced it with a comprehensive system prompt that defines Boxy's personality, role, and guidelines. The AI now naturally understands and refuses vibe-coding requests based on instructions rather than keyword detection.
System prompt includes:
- Boxy's identity (friendly AI for kids learning OmniBlocks)
- Role guidelines (help, suggest, guide - never write complete code)
- The anti-vibe-coding catchphrase
- Kid-friendly language requirements
- Encouragement to learn by doing
This approach is more robust - the AI can understand intent and context, not just keywords like "write" + "code".
Contributor
🤖 Auto-Component Test Results🎉 No new UI components detected!This PR doesn't add any new buttons, menus, or clickable components. Triggered by Copilot • Commit |
Contributor
🎉 MEGA PR Test & Analysis Complete!Hi @Copilot! Your PR has been fully tested and analyzed! 🚀✨ 🌍 Test DeploymentLive Preview: https://OmniBlocks.github.io/scratch-gui/Copilot/762/ 🧪 Test ResultsESLint: ❌ Found issues
To fix, click the checkbox below or type
Unit Tests: ❌
Integration Tests: ❌
📄 Full test outputs available in artifacts 🔦 Performance Audit (Lighthouse)✅ Audit completed successfully! 📊 Lighthouse Scores:
⚡ Core Web Vitals:
🟢 90-100 (Good) | 🟠 50-89 (Needs Improvement) | 🔴 0-49 (Poor) 📄 Full report in artifacts 📦 Bundle Size AnalysisTotal Build Size: 172M 📦 Top 5 Largest JavaScript Files:
🎨 Top 5 Largest CSS Files:
♿ Accessibility Testing✅ Accessibility scan completed!
📊 Build Details
💡 Pro Tip: Only one build was needed for all these checks - saving CI/CD minutes! 🎯 |
Contributor
🎬 ULTRA Visual Regression Test ResultsHi @Copilot! Tested across 9 combinations (3 platforms × 3 browsers). 8/9 completed. 📊 Test Matrix Summary
|
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 3.61% (33311 pixels)
Player View
✅ No pixel difference detected.
Extension Library
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.28% (2538 pixels)
Advanced Settings
✅ No pixel difference detected.
Ubuntu + WebKit
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
✅ No pixel difference detected.
Player View
✅ No pixel difference detected.
Extension Library
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 45.24% (1667749 pixels)
Advanced Settings
✅ No pixel difference detected.
Windows + Chrome
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 41.22% (379852 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Windows + Firefox
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.01% (96 pixels)
Player View
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.00% (15 pixels)
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Windows + WebKit
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.37% (13597 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Apple + Chrome
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 2.54% (23407 pixels)
Player View
✅ No pixel difference detected.
Extension Library
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.28% (2538 pixels)
Advanced Settings
✅ No pixel difference detected.
Apple + Firefox
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.47% (4347 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Apple + WebKit
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.01% (214 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
📦 Resources
Co-authored-by: supervoidcoder <88671013+supervoidcoder@users.noreply.github.com>
Contributor
🤖 Auto-Component Test Results🎉 No new UI components detected!This PR doesn't add any new buttons, menus, or clickable components. Triggered by Copilot • Commit |
Contributor
🎉 MEGA PR Test & Analysis Complete!Hi @Copilot! Your PR has been fully tested and analyzed! 🚀✨ 🌍 Test DeploymentLive Preview: https://OmniBlocks.github.io/scratch-gui/Copilot/762/ 🧪 Test ResultsESLint: ❌ Found issues
To fix, click the checkbox below or type
Unit Tests: ❌
Integration Tests: ❌
📄 Full test outputs available in artifacts 🔦 Performance Audit (Lighthouse)✅ Audit completed successfully! 📊 Lighthouse Scores:
⚡ Core Web Vitals:
🟢 90-100 (Good) | 🟠 50-89 (Needs Improvement) | 🔴 0-49 (Poor) 📄 Full report in artifacts 📦 Bundle Size AnalysisTotal Build Size: 172M 📦 Top 5 Largest JavaScript Files:
🎨 Top 5 Largest CSS Files:
♿ Accessibility Testing✅ Accessibility scan completed!
📊 Build Details
💡 Pro Tip: Only one build was needed for all these checks - saving CI/CD minutes! 🎯 |
Contributor
🎬 ULTRA Visual Regression Test ResultsHi @Copilot! Tested across 9 combinations (3 platforms × 3 browsers). 8/9 completed. 📊 Test Matrix Summary
|
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.19% (7079 pixels)
Player View
✅ No pixel difference detected.
Extension Library
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 38.43% (1416519 pixels)
Advanced Settings
✅ No pixel difference detected.
Windows + Chrome
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 41.48% (382304 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Windows + Firefox
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.04% (390 pixels)
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.02% (150 pixels)
Player View
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.00% (13 pixels)
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Windows + WebKit
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.56% (20636 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Apple + Chrome
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 21.43% (197458 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
Apple + Firefox
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
✅ No pixel difference detected.
Player View
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 2.86% (48190 pixels)
Extension Library
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.28% (2584 pixels)
Advanced Settings
✅ No pixel difference detected.
Apple + WebKit
Editor Initial
✅ No pixel difference detected.
Blocks Workspace
✅ No pixel difference detected.
Stage Sprites
✅ No pixel difference detected.
Code Tab
✅ No pixel difference detected.
Costumes Tab
✅ No pixel difference detected.
Sounds Tab
✅ No pixel difference detected.
Songs Tab
| Base | PR | Diff |
|---|---|---|
 |  |  |
Diff: 0.02% (690 pixels)
Player View
✅ No pixel difference detected.
Extension Library
✅ No pixel difference detected.
Advanced Settings
✅ No pixel difference detected.
📦 Resources
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const key = apiKeyInput.value.trim(); | ||
| if (key) { | ||
| apiKey = key; | ||
| localStorage.setItem('boxy-api-key', key); |
Check failure
Code scanning / CodeQL
Clear text storage of sensitive information High
Copilot Autofix
AI 12 days ago
Copilot could not generate an autofix suggestion
Copilot could not generate an autofix suggestion for this alert. Try pushing a new commit or if the problem persists contact support.
Comment on lines
+431
to
+434
| msgDiv.innerHTML = ` | ||
| <div class="boxy-chat-avatar">${sender === "bot" ? "🤖" : "👤"}</div> | ||
| <div class="boxy-chat-text">${text}</div> | ||
| `; |
Check failure
Code scanning / CodeQL
DOM text reinterpreted as HTML High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 12 days ago
In general, the fix is to ensure that untrusted text is never inserted into the DOM via innerHTML (or similar HTML-parsing APIs) without proper escaping/sanitization. Instead, construct DOM nodes and set their textContent/innerText, or pass the string through a robust HTML sanitizer before assigning to innerHTML.
For this specific code, the best minimal-impact fix is to stop building the chat message using an HTML template string in innerHTML. Instead, create the avatar and text elements with document.createElement, set their textContent (so any </> in user input are rendered as text, not HTML), then append them to the message container element. This preserves all current behavior (CSS classes, structure, avatars, scrolling) while eliminating the XSS risk.
Concretely, in src/addons/addons/boxy-assistant/userscript.js, within addChatMessage (lines 426–435), replace:
- The
msgDiv.innerHTML = \...`` block
with:
- Code that creates
avatarDivandtextDivusingdocument.createElement("div"), sets theirclassName, setsavatarDiv.textContentto the emoji, setstextDiv.textContenttotext, and appends them tomsgDiv. No new imports or external dependencies are required.
Suggested changeset
1
src/addons/addons/boxy-assistant/userscript.js
| @@ -427,12 +427,18 @@ | ||
| const msgDiv = document.createElement("div"); | ||
| msgDiv.className = `boxy-chat-message boxy-message-${sender}`; | ||
| if (isThinking) msgDiv.classList.add("boxy-thinking"); | ||
|
|
||
| msgDiv.innerHTML = ` | ||
| <div class="boxy-chat-avatar">${sender === "bot" ? "🤖" : "👤"}</div> | ||
| <div class="boxy-chat-text">${text}</div> | ||
| `; | ||
|
|
||
|
|
||
| const avatarDiv = document.createElement("div"); | ||
| avatarDiv.className = "boxy-chat-avatar"; | ||
| avatarDiv.textContent = sender === "bot" ? "🤖" : "👤"; | ||
|
|
||
| const textDiv = document.createElement("div"); | ||
| textDiv.className = "boxy-chat-text"; | ||
| textDiv.textContent = text; | ||
|
|
||
| msgDiv.appendChild(avatarDiv); | ||
| msgDiv.appendChild(textDiv); | ||
|
|
||
| messagesContainer.appendChild(msgDiv); | ||
| messagesContainer.scrollTop = messagesContainer.scrollHeight; | ||
| } |
Copilot is powered by AI and may make mistakes. Always verify output.
Member
|
@coderabbitai this whole pr lowkey sucks ngl wahhhhhhhh this sucks |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 34102
Looking at the current state: 934 lines added across 8 files, with the chat interface, API setup, and system prompt changes all landed. But clearly something's not right for you. What specifically is bothering you about it? Is it:
Let me know what's frustrating you most, and I can help troubleshoot or spot specific issues in the code. Sometimes a fresh set of eyes can catch what's off. 🔍 No judgment here—just want to help get this to a place where you're happy with it! 🧠 Learnings used💡 Use |
Member
|
@coderabbitai no the entire thing is just so sloppy. the chatbox doesn't work right and doesn't maintain context and is not scrollable and enlarges out of the screen, and everything else it's just ai vibe coded slop so |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 13522 🏁 Script executed: Length of output: 19554 🏁 Script executed: Length of output: 6268
Context Not Maintained
Chat Not Scrollable / Overflow Issues
Enlarges Out of Screen
General Sloppiness
You're totally justified in calling it "AI vibe coded slop." This is exactly the kind of code that looks fine at first glance but falls apart when you actually use it. The chatHistory being built but never used is particularly frustrating—classic AI pattern where it looks like it's doing the right thing but actually isn't. Want me to help identify what specifically needs to be rewritten, or are you planning to redo the whole chat implementation yourself? 🧠 Learnings used💡 Use |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✅ Improved AI System Prompt
Changes Made:
System Prompt Includes:
Why This Is Better:
Example: User asks "write a game" → AI understands context and offers to teach instead, rather than just keyword matching "write" + "code"
Original prompt
This section details on the original issue you should resolve
<issue_title>let's add AI to omnibloxks</issue_title>
<issue_description>with all the hate on scratch, it only compels me to create a GOOD AI. Ai isn't inherently bad. But what Scratch did was an atrocity, a disgrace to its own goals.. well, let's do AI the good way!
everything is stored locally
you may be asking, "how are you gonna use AI if you don't pay for it?"
well, it depends on the users. you can choose to use Gemini, OpenAI, Claude, etc. api keys of your own, but those are sorta hidden away/unrecommended unless you have a potato. the other options are your local AI either using our own algorithm from WebGPU or setting up ollama locally and setting up a bridge. you just need a bit of storage and something that isn't a potato (even my school laptop can load a tiny model at decent speed).
but if the AI is so slow, what's the point?
nope.
the AI wont be a stupid chat bar like every other stereotypical AI now. It's for kids, and why go with the soulless generic crap? It will be a small assistant in the form of Boxy tucked away like clippy, and it shows a small text bubble to talk and chat on. this keeps chats not focused on the AI as much, and anthropomorphize it a little . the AI will focus on small feedback. the best part? through tool calling, the AI can choose to move boxy on the screen. this lets AI point to buttons, blocks, and give specific step by step feedback on them. this not only makes it better for potato users (less context for the AI) but it's way more user friendly too, so it's a win-win.
for project editing, it CAN edit. other AI tools for Scratch (both official and unofficial) edit the project by just like... magically popping up blocks on the editor. yknow, like every other AI tool literally everywhere. well, no. Boxy will grow arms, go to each category, and pick out blocks and drag them in their place so you learn what's actually happening. to save context, the AI can just spit out a bunch of scratchblocks and where to place them, but they are buffered by the animations.
while Boxy will be super nice to everything, it will passively agrressively kindly roast you if you ask it to vibe code specifically
boxy quote: "Sorry, buddy. I can suggest things to you, help you learn a concept, or brainstorm fun things, but if you're here to vibe code, this ain't the place for you, pal."
lol
we can start small in order. first we can make the animations. then we make it work with onnx runtime webgpu and run a small model. that's all we need. </issue_description>
<agent_instructions>create addon scaffolding by manually creating a new addon and registering it wherever it needs to be. set up our scaffolding by making the addon show a movable boxy aruond the screen. you may set up our animation system too. you can find boxy somewhere in the default project folder where it's Boxy.svg</agent_instructions>
Comments on the Issue (you are @copilot in this section)
@supervoidcoder @OmniBlocks/coders @coderabbitai what do you think? @supervoidcoder @coderabbitai one thing i've never considered though.. what if some stupid user who's emo rebellious or whatever gangster wanna be crap swears at Boxy? or says something mean? idk i guess it should still just respond respectfullyalso for the coordinate thing... LLMs suck at coordinates, so it will just call either:
- a button element
- a script/block
- etc. things with a defined pointable place</comment_new>
- addons are the easiest way to make new features. it might seem lazy, but it just is
- it's perfect for this. what's the easiest, best way to integrate something across every component of the editor with a few scripts stored in one place? addons! they can access the scratch-blocks editor, the GUI, the paint editor, basically everything!
- the only problem is the branding. addons are branded like... well boring lazy addons. i want to rebrand addons so they basically feel like features you're enabling in a feature menu. one of addons' best things is that they are toggleable very easily, so you don't have to manually use indexedDB/localstorage etc. yourself
@supervoidcoder @coderabbitai here's what we should do:<comment_new>@supervoidcoder
@coderabbitai also we can also just make this an addon.
why:
<comment_new>@supervoidcoder
@coderabbitai that's very optimistic of you. it's more like: