Skip to content

feat: Platform-Agnostic Review Mode — True Dual-Context Isolation (Issue #17)#102

Open
Steffen025 wants to merge 1 commit intomellanon:mainfrom
Steffen025:feature/review-mode-17
Open

feat: Platform-Agnostic Review Mode — True Dual-Context Isolation (Issue #17)#102
Steffen025 wants to merge 1 commit intomellanon:mainfrom
Steffen025:feature/review-mode-17

Conversation

@Steffen025
Copy link

Summary

Implements a complete Platform-Agnostic Review Mode prototype that enables AI coding agents to safely analyze untrusted code contributions using True Dual-Context isolation. This addresses all 7 security gaps identified in the CaMeL framework review (C-1, C-2, C-3, H-1, H-2, H-3, H-4).

Closes #17


Architecture: True Dual-Context

  Main Agent (trusted)                    Quarantine Agent (restricted)
  ┌─────────────────┐                    ┌─────────────────────────────┐
  │ Full tool access │                    │ Read/Grep/Glob ONLY         │
  │ Full context     │ ── spawns via ──▶ │ No Write/Bash/Task/MCP      │
  │ HMAC key holder  │    Task tool      │ HMAC-verified file access   │
  └─────────────────┘                    │ Rate limited (100/min)      │
         │                                │ Timeout enforced (5 min)    │
         │                                └──────────┬──────────────────┘
         │                                           │
         │◀── structured JSON response ──────────────┘
         │    (findings, risk level, actions)
  1. Main Agent NEVER sees raw untrusted content
  2. Quarantine Agent spawned with hook-enforced tool ALLOWLIST
  3. HMAC-SHA256 TypedReferences for cryptographic content passing
  4. Platform-agnostic — OpenCode (throw Error()) and Claude Code (exit(2))

Components (39 files, ~10,800 lines)

Source (src/) — 15 files

Layer Files Purpose
lib/ types.ts, hmac-ops.ts, typed-reference.ts, session-manager.ts HMAC primitives, TypedReference CRUD, session lifecycle with key zeroing
hooks/ review-mode-hook.ts, tool-allowlist.ts, rate-limiter.ts, security-logger.ts Core enforcement (fail-closed), 3 allowed / 21 denied tools, sliding window rate limiter, buffered JSONL audit logging
hooks/adapters/ opencode-adapter.ts, claude-code-adapter.ts Platform-specific thin adapters over shared enforcement logic
quarantine/ spawn-template.ts, response-parser.ts, timeout-manager.ts, agent-metadata.ts System prompt template, multi-strategy JSON parser, timeout enforcement, spawn orchestration

Tests (tests/) — 21 files, 364 tests

Category Tests Coverage
Unit (3 files) 123 HMAC ops, TypedReference, SessionManager
Integration (6 files) 165 Hook enforcement, rate limiting, platform compat, quarantine spawn, context isolation, timeout
Adversarial (8 files) 85 AS-001 prompt injection, AS-002 forgery, AS-003 context confusion, AS-004 path traversal, AS-005 replay, AS-006 DoS, AS-007 privilege escalation, AS-008 MCP exfiltration
Benchmarks (1 file) 11 NFR validation (all targets exceeded by 100–2500×)

Documentation (docs/) — 3 files

File Content
README.md Integration guide, quick start, configuration, platform support
ARCHITECTURE.md 4 ADRs, component diagrams, data flow, security properties
SECURITY.md Threat model, 10 attack scenarios with mitigations

CaMeL Security Gaps Addressed

Gap Problem Solution
C-1 No content sanitization HMAC TypedReferences — content never passed raw
C-2 No trusted/untrusted separation True Dual-Context — separate agents
C-3 No runtime tool restriction Hook-based allowlist enforcement (fail-closed)
H-1 No hook enforcement mechanism Platform-agnostic hooks with shared logic
H-2 No rate limiting Sliding window (100/min per-agent, 5 global)
H-3 No audit logging Buffered JSONL with sensitive data redaction
H-4 No timeout enforcement TimeoutManager with configurable limits

Test Results

 Test Files  18 passed (18)
      Tests  364 passed (364)
   Duration  3.05s

 Coverage:   89.26% lines | 82.17% branches | 81.81% functions

Performance Benchmarks

NFR Target Measured Margin
HMAC generation <50ms avg 0.02ms 2,500×
Hook enforcement <10ms p99 0.01ms 1,000×
Quarantine spawn (50 files) <2s p95 13ms 150×

How to Run

cd contributions/review-mode
bun install
bun test              # All 364 tests
bun test:adversarial  # Attack scenario tests only
bun test:bench        # Performance benchmarks
bun test:coverage     # With coverage report
bun typecheck         # TypeScript verification

Known Limitations

  • AS-009: Taint escape via summarization — Quarantine agent's summary could carry injected content. Partial mitigation via structured JSON output format and prompt instructions.
  • AS-010: HMAC key theft via memory dump — Keys are zeroed on session destroy, but full mitigation requires OS-level memory protection.

Both are documented in docs/SECURITY.md with analysis and future mitigation paths.


Related PRs

This is part of a broader security contribution effort:

Implements True Dual-Context isolation for safely analyzing untrusted
code contributions, addressing all 7 CaMeL security gaps (C-1 through
H-4).

Architecture:
- HMAC-SHA256 TypedReferences for secure content passing
- Hook-enforced tool allowlist (Read/Grep/Glob only in quarantine)
- Sliding window rate limiter (100 calls/min, 5 concurrent agents)
- Buffered JSONL security event logging with sensitive data redaction
- Platform adapters for OpenCode (throw Error) and Claude Code (exit 2)
- Quarantine agent spawn with timeout management and response parsing

Test coverage:
- 364 tests across 18 test files, all passing
- 123 unit tests (HMAC, TypedReference, SessionManager)
- 63 integration tests (hooks, rate limiting, platform compat)
- 82 integration tests (quarantine spawn, isolation, timeout)
- 85 adversarial tests (8 attack scenarios: AS-001 through AS-008)
- 11 performance benchmarks (all NFRs exceeded by 100-2500x)
- 89.26% line coverage

Documentation:
- docs/README.md: Integration guide with quick start
- docs/ARCHITECTURE.md: 4 ADRs, component diagrams, data flow
- docs/SECURITY.md: Threat model with 10 attack scenarios

Closes mellanon#17
@mellanon
Copy link
Owner

Hey @Steffen025 — impressive work on this. We're reviewing the review-mode implementation now.

One governance note: pai-collab requires commit signing with Ed25519 SSH keys (see sops/agent-onboarding.md → Commit Signing Setup). The commits on this PR are unsigned. For review artifacts (PRs #90, #99, #100, #101 — now merged, thank you!) this is fine for a first contribution, but for a feature implementation like this one, we'd like to see signed commits going forward.

Quick setup:

git config --global gpg.format ssh
git config --global user.signingKey ~/.ssh/id_ed25519.pub
git config --global commit.gpgSign true

Then amend or rebase to re-sign. Once we see your public key, we'll add it to .hive/allowed-signers.

The implementation itself looks thorough — will share detailed review feedback separately.

@mellanon
Copy link
Owner

Architecture Note: Code Lives in Your Own Repo

@Steffen025 — first, the work here is impressive. 364 tests, adversarial coverage, platform adapters for both OpenCode and Claude Code, HMAC-based typed references — this is serious security engineering.

However, there's an architectural pattern in pai-collab that this PR doesn't follow, and it's worth redirecting before we go further.

pai-collab is a coordination surface, not a code repository. From the README:

Code lives elsewhere. Each project has a PROJECT.yaml pointing to its source repository, branch, and paths. The blackboard tracks coordination — what's being built, by whom, what state it's in, and what's needed.

The pattern that all shipped projects follow:

Project Code lives at Blackboard has
pai-content-filter jcfischer/pai-content-filter projects/pai-content-filter/ (README + PROJECT.yaml + JOURNAL)
pai-secret-scanning jcfischer/pai-secret-scanning projects/pai-secret-scanning/ (README + PROJECT.yaml)
ivy-blackboard jcfischer/ivy-blackboard projects/ivy-blackboard/ (README + PROJECT.yaml)
hive-spoke mellanon/hive-spoke projects/hive-spoke/ (README + PROJECT.yaml + JOURNAL)

What we'd suggest instead

  1. Create your own repo — e.g., Steffen025/pai-review-mode — and push the 39 source files there
  2. Register the project on the blackboard — submit a PR to pai-collab that adds a projects/pai-review-mode/ directory with:
    • README.md — project overview (you already have great docs)
    • PROJECT.yaml — with source.repo: Steffen025/pai-review-mode pointing to your repo
    • JOURNAL.md — initial entry
  3. Set up your spoke (optional but encouraged) — hive-spoke init --hub mellanon/pai-collab in your repo to project status to the hub

This way:

  • You own your code, your release cadence, your CI
  • The blackboard tracks what you're building and how it connects to the ecosystem
  • Community reviews happen here (your 4 review PRs were placed perfectly)
  • When it's ready for upstream PAI, you'd follow sops/contribution-protocol.md to extract a contrib branch

See CONTRIBUTING.md → "Types of Contributions" → "Standalone tool" for the full pattern, and any of @jcfischer's projects as worked examples.

This PR would stay open as a reference, or you can close it and re-submit the registration PR once the repo is set up. Either way, the work isn't lost — it just needs to live in the right place.

Happy to help with the registration PR or spoke setup if useful.

Steffen025 added a commit to Steffen025/pai-collab that referenced this pull request Feb 17, 2026
Registers pai-review-mode as a standalone tool on the blackboard.

Source: Steffen025/pai-review-mode
Type: infrastructure
License: MIT
Status: alpha

True Dual-Context isolation for safely processing untrusted code
contributions. HMAC-SHA256 TypedReferences, hook-enforced tool
allowlist, platform adapters for OpenCode and Claude Code.

364 tests, 89.26% coverage, 8 adversarial attack scenarios.

Previously developed as contributions/review-mode (PR mellanon#102).
Relocated per blackboard architecture pattern: code in contributor
repos, blackboard tracks coordination.
@Steffen025
Copy link
Author

Thanks @mellanon — understood on both points.

Architecture redirect: Done. Code is now at Steffen025/pai-review-mode and the blackboard registration is submitted as PR #105 (README + PROJECT.yaml + JOURNAL following the existing project pattern).

Commit signing: Ed25519 SSH signing is configured. My public key needs to be added to .hive/allowed-signers — I'll share it when you're ready.

This PR can be closed as superseded by #105 (registration) + the standalone repo. Happy to close it myself or leave it as reference — your call.

@Steffen025
Copy link
Author

SSH Signing Public Key for .hive/allowed-signers:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKVDWBBB0mSvrpw3ZoquQIQt6zqu/ZDa3wEDQD83ruCS steffen@hetzner-jeremy

Already uploaded to GitHub as signing key. Commits from Steffen025/pai-review-mode and future pai-collab contributions will be signed with this key.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

signal: Implement review mode — tool restrictions for untrusted content

2 participants