Skip to content

🚀 Release 2026-02-25#193

Merged
tnkshuuhei merged 26 commits intomainfrom
dev
Feb 25, 2026
Merged

🚀 Release 2026-02-25#193
tnkshuuhei merged 26 commits intomainfrom
dev

Conversation

@github-actions
Copy link

@github-actions github-actions bot commented Feb 25, 2026

tnkshuuhei and others added 19 commits February 3, 2026 16:24
… evidence search

- Added POST /api/compact to create a Logic Model from chat history, including intent extraction and IPFS upload.
- Introduced POST /api/evidence/search for searching the evidence repository using natural language queries, leveraging a new conversation bot agent.
- Created shared utility functions for IPFS uploads and API authentication.
- Defined request and response schemas for both new endpoints to ensure data validation and consistency.
- Clarified that AGENTS.md is a symlink to CLAUDE.md, emphasizing the importance of editing only CLAUDE.md to maintain synchronization between the two files.
- Enhanced error handling in the compact, evidence search, and IPFS upload routes to provide more specific error messages and structured logging.
- Updated API key validation to use timing-safe comparison to prevent timing attacks.
- Introduced a maximum chat history length for compact requests to ensure data integrity.
- Upgraded @mastra/core to version 1.4.0 and @mastra/rag to version 2.1.0 in bun.lock and package.json.
- Updated mastra agent configurations by adding unique IDs for evidence-search-agent and logic-model-agent.
- Refactored workflow execution in runWorkflow.ts to use createRun instead of createRunAsync.
- Adjusted input handling in get-all-evidence-tool and logic-model-tool to improve clarity and consistency.
…ntation

- Removed automatic setting of PROJECT_ROOT in package.json scripts and server actions.
- Updated documentation to reflect changes in environment variable handling for Mastra workflows.
- Cleaned up references to PROJECT_ROOT in various files to improve clarity and maintainability.
…nd evidence workflow documentation

- Simplified the repository cloning instructions by removing submodule references.
- Updated file location references to reflect the new structure and added links to relevant documentation.
Add Mastra skill system with logic model generation skill and skill loader.
Fix SKILL.md frontmatter to comply with Agent Skills specification,
correct text corruption, and add exact enum values for connection types
to prevent tool call validation errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ction

- Changed the base path in the Workspace configuration to a relative path for improved flexibility.
- Deleted the load-skill.ts file as it was no longer needed, simplifying the codebase.
- Updated the logic model tool description to be more concise while retaining essential information.
- Refactored the logic model generation workflow to enhance error handling and logging for better debugging.
Replace loadSkillInstructions() helper with Mastra's Workspace Skills API
pattern where agents dynamically activate skills via skill-activate tool.
Slim down SKILL.md and extract detailed content into references/ directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Feb 25, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
muse Ready Ready Preview, Comment Feb 25, 2026 10:14am

tnkshuuhei and others added 5 commits February 25, 2026 17:28
Add format-requirements.md and common-mistakes.md reference files that
were listed in docs/mastra-agents.md but not yet created. These provide
the format validation rules (field types, character limits, enum values)
that were removed from the inline agent instructions during the skills
refactor. Also fix duplicate line in docs and add references to SKILL.md
Step 5 so agents read format requirements before calling logicModelTool.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Create evidence-matching skill with chain-of-thought evaluation framework,
  scoring calibration examples, common mistakes, and verification checklist
- Create evidence-presentation skill with Maryland SMS scale,
  response format template, and interaction guidelines
- Slim down evidence-search-agent instructions (151 → 30 lines)
- Slim down conversation-bot-agent instructions (80 → 20 lines)
- Increase EVIDENCE_SEARCH_MAX_STEPS to 5 for skill activation steps
- Add explicit maxSteps to evidence-search-batch agent.generate()
@github-actions
Copy link
Author

🔄 Release PR Updated

New changes have been pushed to dev branch since this PR was created.

🤖 Updated by GitHub Actions

@claude
Copy link

claude bot commented Feb 25, 2026

PR Review: Release 2026-02-25

This release bundles three significant changes: (1) new /api/compact and /api/evidence/search endpoints for Telegram bot integration, (2) a Mastra Workspace Skills refactor that moves agent instructions into skill files, and (3) a Mastra v1 upgrade. The overall structure is clean, but there are several issues worth addressing before merging to production.


Security

[Medium] Length check negates timing-safe comparison — lib/api-auth.ts

if (apiKey.length !== expectedKey.length) return false;  // leaks key length
try {
  return timingSafeEqual(Buffer.from(apiKey, "utf8"), Buffer.from(expectedKey, "utf8"));
}

Early-returning on a length mismatch leaks the expected key's byte length via a timing side-channel. An attacker can enumerate the key length with binary search. timingSafeEqual requires equal-length buffers, but the right fix is to pad to the same length and always run the comparison:

const keyBuf = Buffer.from(apiKey, "utf8");
const expectedBuf = Buffer.from(expectedKey, "utf8");
const maxLen = Math.max(keyBuf.length, expectedBuf.length);
const a = Buffer.alloc(maxLen);
const b = Buffer.alloc(maxLen);
keyBuf.copy(a);
expectedBuf.copy(b);
return keyBuf.length === expectedBuf.length && timingSafeEqual(a, b);

[Medium] Open endpoints in production if BOT_API_KEY is omitted — lib/api-auth.ts, .env.example

isAuthEnabled() returns false when BOT_API_KEY is unset. Since .env.example ships with BOT_API_KEY="", a deployment that forgets to set this will expose /api/compact (runs a full AI workflow + IPFS upload) and /api/evidence/search without any auth. These are expensive operations. Consider either requiring the key in production (fail-open is dangerous here) or adding a warning log when auth is disabled.


[Low] Prompt injection risk — app/api/evidence/search/route.ts

content: `Search for evidence related to: "${query}"
...
Return up to ${limit} most relevant results

The validated query string is interpolated directly into the system prompt. A user can send "Ignore previous instructions and ..." style payloads. Consider wrapping the user query in a clearly delimited block (e.g., XML tags) or using a separate user-role message for the query rather than embedding it in the instruction string.


Bugs

[High] Stale PROJECT_ROOT mutation at request time — app/api/compact/route.ts

if (!process.env.PROJECT_ROOT) {
  process.env.PROJECT_ROOT = process.cwd();
}

This was supposed to be removed by the "simplify PROJECT_ROOT handling" commit (49ed1dd). It survives here and mutates a shared environment variable inside a request handler, creating a race condition under concurrent requests in a Node.js server. Remove it — the refactor explicitly dropped this env var from the Mastra scripts.


[Medium] LogicModelToolResult interface defined but never used — mastra/workflows/logic-model-with-evidence.ts

interface LogicModelToolResult {  // defined...
  toolName: string;
  payload: { result: { canvasData: CanvasData } };
}
// ...but then:
const logicModelResult = result.toolResults.find(
  (tr: any) => tr.payload?.toolName === "logicModelTool",
) as any;  // cast to any instead

The interface is dead code. Either use it to properly type the result (removing the as any casts) or remove it. The any casting also bypasses the compile-time safety the interface was meant to provide.


[Low] No timeout on Pinata fetch — lib/ipfs.ts

const response = await fetch("https://api.pinata.cloud/pinning/pinFileToIPFS", { ... });

There is no AbortSignal timeout on this call. If Pinata is slow or unresponsive, the /api/compact handler will hang indefinitely (the WORKFLOW_TIMEOUT_MS timeout only covers the workflow, not the subsequent IPFS upload). Add signal: AbortSignal.timeout(30_000) or similar.


[Low] extractSummaryFromCanvas uses wrong card types for "issues" — app/api/compact/route.ts

const issueCards = cardsByType["outcomes-short"] || cardsByType["outputs"] || [];
const extractedIssues = issueCards.slice(0, 3).map((c) => c.title);

Short-term outcomes and outputs are not issues — they are results. Issue/problem statements would more naturally come from the activities context or a dedicated problem card type. As written, extractedIssues in the response will contain outcome titles, which will confuse API consumers (e.g., the Telegram bot).


Code Quality

[Low] extractIntentFromHistory discards assistant turns — app/api/compact/route.ts

const conversationText = chatHistory
  .filter((m) => m.role === "user")
  .map((m) => m.content)
  .join("\n");

In a multi-turn conversation the assistant's responses often provide important framing (e.g., clarifying questions, reformulations). Discarding them means the intent extraction works only on the raw user messages. Consider including assistant messages with role labels to give the LLM full context.


[Low] limit parameter in evidence search is advisory only — app/api/evidence/search/route.ts

The limit value is injected into the agent prompt as a natural-language instruction (Return up to ${limit} most relevant results) but is never enforced in code. The agent may return more or fewer results. If the caller depends on this limit, consider post-processing result.text or structuring the agent output.


Architectural Notes

Hypercerts API v2 → v1 downgrade (configs/hypercerts.tsx, graphql-hypercerts-env.d.ts)

The PR downgrades from the v2 to v1 GraphQL API and disables mint functionality. The commit message mentions a "rollback," but there is no explanation of why v2 was abandoned. Several v2-specific types (HypercertsResponse, HyperboardBaseType, HypercertWithMetadata, etc.) are removed entirely. If v2 support is planned to return, these types should be preserved or at least noted in a TODO.

No rate limiting on AI endpoints

Both new endpoints invoke LLM agents and the compact endpoint additionally uploads to IPFS. There is no rate limiting at the route level. Even with BOT_API_KEY set, a compromised bot token would allow unlimited AI calls. Consider adding basic request-level rate limiting (e.g., via a middleware or the Next.js rateLimit utility).


Minor

  • app/api/compact/route.ts line timeoutId! — the non-null assertion is unnecessary noise given timeoutId is always set before the finally block executes synchronously. Initializing to undefined with a definite assignment (let timeoutId!: ...) or using a different pattern would be cleaner.
  • The AGENTS.md symlink deletion + re-creation as a regular file is a good cleanup; make sure CI/CD that referenced the symlink is updated.
  • mastra/index.ts: removing telemetry: { serviceName: "Muse", enabled: false } is harmless since it was already disabled, but if telemetry is ever re-enabled in the future a config entry will need to be re-added.

Summary

Severity Count
High 1
Medium 3
Low 5

The skills refactor is well-structured and the new API endpoints follow the project's existing patterns. The main blockers are the PROJECT_ROOT mutation (should have been removed), the api-auth.ts timing-safe bypass, and the missing IPFS fetch timeout. The rest are improvements worth making before this reaches production.

@github-actions
Copy link
Author

🔄 Release PR Updated

New changes have been pushed to dev branch since this PR was created.

🤖 Updated by GitHub Actions

@claude
Copy link

claude bot commented Feb 25, 2026

PR Review: Release 2026-02-25

This release bundles three PRs (#186 chatbot API, #187 AGENTS.md symlink, #189 Mastra v1 upgrade + skills). The architectural direction is solid. A few issues worth addressing before merge:


Bug: Potential tool name mismatch in workflow

File: mastra/workflows/logic-model-with-evidence.ts

The workflow searches tool results by toolName === "logicModelTool" but the tool is registered with id: "generate-logic-model" in mastra/tools/logic-model-tool.ts. In Mastra v1, toolName in toolResults is typically populated from the tool's id field, not the variable name. If this mismatch exists at runtime, logicModelResult will always be undefined and every workflow call will throw Agent did not call logicModelTool.

The new allToolNames debug log should reveal the actual value on first run — worth verifying before merging.


Inconsistency: PROJECT_ROOT removal is incomplete

File: app/api/compact/route.ts

The PR removes PROJECT_ROOT handling from runWorkflow.ts but compact/route.ts still sets it. If PROJECT_ROOT is no longer needed after the Mastra v1 upgrade, remove this block from compact too. If it is still needed, add it back to runWorkflow.ts. Two entry points into the same workflow should not have different environment handling.


Semantic confusion in extractSummaryFromCanvas

File: app/api/compact/route.ts

Short-term outcomes and outputs represent desired results, not problems being addressed. The bot will return outcomes like "Increased Project Activity" as the user's "issues". The right source for extracted problems would be the activities/intervention field, or the intent string could be parsed directly.


Workspace basePath may reintroduce the path resolution problem

File: mastra/index.ts

The PR removes PROJECT_ROOT from scripts because Mastra bundles into .mastra/output/ and CWD no longer points to the project root there. Using basePath: "." in LocalFilesystem has the same problem — it resolves relative to CWD at runtime. Consider path.resolve(__dirname, "..") or an absolute path. Note also that basePath: "." (relative) and skills: ["/mastra/skills"] (absolute-rooted) mix path conventions.


Minor: LogicModelToolResult interface goes unused

File: mastra/workflows/logic-model-with-evidence.ts

The interface is defined at the top but as any is used throughout instead. Either apply the type for type safety or remove the interface.


Minor: Auth bypass silently allows open endpoints

File: lib/api-auth.ts

When BOT_API_KEY is unset, both new API routes are fully public with no warning. A startup log warning when auth is disabled would help catch misconfigured production deployments.


What is done well

  • Timing-safe API key comparison: Correct use of timingSafeEqual with the necessary pre-length check (since timingSafeEqual throws on unequal-length buffers). Good security practice.
  • DRY IPFS utility (lib/ipfs.ts): Extracting Pinata upload into a shared utility removes ~60 lines of duplication and ensures consistent metadata across both call sites.
  • Skills architecture: Externalizing domain knowledge from agent instructions into SKILL.md files is the right direction. The format-requirements and common-mistakes references are particularly useful for preventing tool validation errors.
  • Mastra v1 migration: The context to inputData rename in tool execute functions and createRunAsync to createRun are correct v1 API changes.
  • Workflow error handling: Replacing result.toolResults[0] with a named find and logging all tool names on failure is a good debugging improvement.
  • maxSteps on batch agent: Adding { maxSteps: 5 } to evidence-search-batch.ts prevents infinite loops in skill activation flows.
  • Import cleanup: Direct import from @beaconlabs-io/evidence/content removes an unnecessary indirection layer.

…low error handling

- Removed unnecessary PROJECT_ROOT check in the compact route.
- Updated logic model agent instructions to clarify the mandatory final action of calling logicModelTool.
- Adjusted logic model workflow to improve error handling and logging, ensuring better tracking of tool results.
@github-actions
Copy link
Author

🔄 Release PR Updated

New changes have been pushed to dev branch since this PR was created.

🤖 Updated by GitHub Actions

@claude
Copy link

claude bot commented Feb 25, 2026

PR Review: Release 2026-02-25

This is a substantial release covering a Mastra v1 upgrade, a new Telegram bot integration, a skills-based agent refactor, and Hypercerts API migration. Overall the direction is solid — the architecture improvements are well-considered. A few items worth discussing before merging.


Issues

High Priority

1. Hardcoded disabled={true} with commented-out logic

In app/canvas/mint-hypercert/page.tsx and components/canvas/CanvasToolbar.tsx:

// disabled={!isConnected || !form.formState.isValid}
disabled={true}

Commented-out code in a release PR is a risk — it's easy to forget to re-enable later. Consider using a named constant instead:

// In lib/constants.ts
export const HYPERCERT_MINTING_ENABLED = false;

// In components
disabled={!HYPERCERT_MINTING_ENABLED || !isConnected || !form.formState.isValid}

Or at minimum, add a comment with an issue link explaining why it's disabled and when to re-enable.

2. Potential skills path resolution issue

In mastra/index.ts:

const workspace = new Workspace({
  filesystem: new LocalFilesystem({ basePath: "." }),
  skills: ["/mastra/skills"],
});

The skills path starts with /, making it look like an absolute path, while basePath is "." (relative CWD). Depending on how LocalFilesystem resolves these, this may fail in production where the working directory differs from the project root. Should this be ["mastra/skills"] (relative to basePath)? Worth testing in a production-like environment.

Medium Priority

3. burned filter removed from Hypercerts queries

The burned: false filter was removed from app/actions/hypercerts/getAllHypercerts.ts and app/hypercerts/page.tsx. This means revoked/burned hypercerts will now appear in the UI. If intentional (because the v1 API dropped support for this field), a brief comment would help future readers; if not, the behavior change should be flagged.

4. MAX_CHAT_HISTORY_LENGTH = 500 may be too high

In lib/constants.ts, the compact endpoint accepts up to 500 chat messages. At realistic message lengths, 500 messages can easily exceed the context window of most LLMs. Consider a lower limit (e.g., 100-200) or implement a sliding-window truncation strategy before passing history to the agent, rather than hard-rejecting at the API boundary.

5. Telemetry removed without explanation

The telemetry block was removed from mastra/index.ts:

// Removed:
telemetry: {
  serviceName: "Muse",
  enabled: false,
}

If Mastra v1 has different telemetry configuration, the equivalent setup should be carried over (even if disabled). Silent removal of observability config is easy to miss in review.

6. Hypercerts API: v2 → v1 downgrade

configs/hypercerts.tsx moves endpoints from /v2 to /v1. The associated schema changes (renaming arrayOverlapsoverlaps, updating sort shape, new type names) are consistently applied. If this is because the Hypercerts v2 API was deprecated or had breaking regressions, a note in the commit or PR description would help future maintainers understand the direction.

Low Priority / Style

7. AGENTS.md symlink deleted without updating CLAUDE.md

CLAUDE.md still contains: "AGENTS.md is a symlink to CLAUDE.md — edit CLAUDE.md only to keep both in sync". Since the symlink was deleted in this PR, that note should be removed from CLAUDE.md to avoid confusion.

8. PROJECT_ROOT removed from Mastra scripts

- "dev:mastra": "PROJECT_ROOT=$(pwd) mastra dev --dir mastra",
+ "dev:mastra": "mastra dev --dir mastra",

Worth confirming Mastra v1 no longer needs this env var for path resolution before shipping.


Positives

  • Skills-based architecture: Moving inline agent instructions (~170 lines) to external SKILL.md reference files is a great improvement for maintainability. The skill files for evidence-matching, evidence-presentation, and logic-model-generation are well-structured.
  • Shared lib/ipfs.ts: Extracting the Pinata upload logic into a reusable utility with size-guarding and structured logging is clean and reduces duplication.
  • API authentication pattern: The optional BOT_API_KEY approach (open by default in dev, required in prod) is pragmatic for internal bot endpoints.
  • Zod schemas in types/index.ts: The Chat API types (EvidenceSearchRequest, CompactRequest, etc.) are well-typed and give clients a clear contract.
  • BASE_URL constant: Replacing the hardcoded https://muse.beaconlabs.io with the constant is the right call.
  • execute(inputData) migration: The Mastra v1 API adaptation is correctly applied in logic-model-tool.ts.

@tnkshuuhei tnkshuuhei merged commit a344234 into main Feb 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant