Skip to content

Conversation

@fangchenli
Copy link
Contributor

Summary

This PR adds three major features to Delibera:

LLM Abstraction + Gemini Provider

  • Add llm/ module with stable LLMClient protocol interface
  • Implement GeminiClient with SDK and HTTP fallback
  • Add LLM request/response types with structured tracing
  • Add redaction helpers for trace-safe logging (emails, API keys, etc.)
  • Implement ProposerLLM agent with JSON output
  • Add CLI flags: --use-llm-proposer, --llm-model, --llm-temperature, --llm-max-output-tokens
  • Add optional dependency: pip install delibera[llm]

LLM usage is restricted to WORK steps only (never CLAIM_CHECK). Stubs remain the default; LLM mode is opt-in.

Inspect Subsystem

  • Add inspect/ module for run inspection and reporting
  • Implement text and markdown renderers
  • Add summarize module for run analysis
  • Add CLI commands: delibera inspect, delibera report
  • Support inspection by run-id or path

Refinement Support

  • Add refinement round support in protocol spec
  • Implement convergence criteria checking
  • Add refinement pipeline execution
  • Support multi-round deliberation

Additional Improvements

  • Add CONTRIBUTING.md with contributor guidelines
  • Add docs/README.md documentation index
  • Add DocsSearch and DocsRead tools with capability classification
  • Add smoke tests for CLI and imports
  • Improve tool policy with capability-based access control
  • Add __version__.py for version tracking

Test plan

  • All 373 tests pass
  • mypy type checking passes
  • ruff linting passes
  • Pre-commit hooks pass
  • Manual test: delibera run --question "Test" --use-llm-proposer (requires GEMINI_API_KEY)
  • Manual test: delibera inspect --run-id <id>
  • Manual test: delibera report --run-id <id>

🤖 Generated with Claude Code

This PR adds three major features to Delibera:

## LLM Abstraction + Gemini Provider (PR #17)

- Add `llm/` module with stable LLMClient protocol interface
- Implement GeminiClient with SDK and HTTP fallback
- Add LLM request/response types with structured tracing
- Add redaction helpers for trace-safe logging
- Implement ProposerLLM agent with JSON output
- Add CLI flags: --use-llm-proposer, --llm-model, etc.
- Add optional dependency: delibera[llm]

LLM usage is restricted to WORK steps only (never CLAIM_CHECK).
Stubs remain the default; LLM mode is opt-in.

## Inspect Subsystem (PR #15)

- Add `inspect/` module for run inspection and reporting
- Implement text and markdown renderers
- Add summarize module for run analysis
- Add CLI commands: inspect, report
- Support inspection by run-id or path

## Refinement Support (PR #14)

- Add refinement round support in protocol spec
- Implement convergence criteria checking
- Add refinement pipeline execution
- Support multi-round deliberation

## Additional Improvements

- Add CONTRIBUTING.md with contributor guidelines
- Add docs/README.md documentation index
- Add DocsSearch and DocsRead tools with capability classification
- Add smoke tests for CLI and imports
- Improve tool policy with capability-based access control
- Add __version__.py for version tracking

Tests: 373 passing
Type check: mypy passes with no errors
Lint: ruff passes with no errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds three major features to Delibera:

  1. LLM Abstraction + Gemini Provider: Adds a stable LLM client interface with Gemini implementation, including request/response types, redaction helpers, and an LLM-backed proposer agent
  2. Inspect Subsystem: Adds run inspection and reporting capabilities with text and markdown renderers
  3. Refinement Support: Adds bounded refinement loops with convergence checking

Changes:

  • New llm/ module with LLM client protocol and Gemini implementation
  • New inspect/ module for run analysis and reporting
  • New DocsSearchTool for evidence discovery
  • Protocol spec extensions for refinement rounds
  • CLI commands: delibera inspect and delibera report

Reviewed changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
uv.lock Added dependencies for LLM support (google-generativeai, pydantic, etc.)
tests/llm/test_llm_proposer.py Tests for LLM proposer with fake client
tests/inspect/test_inspect.py Tests for inspect/report functionality
tests/test_refinement.py Tests for refinement loop
tests/test_smoke.py Smoke tests for CLI and imports
tests/tools_docs/test_docs_search.py Tests for DocsSearchTool
src/delibera/llm/ LLM abstraction layer with base types, Gemini client, prompts, redaction
src/delibera/inspect/ Run inspection with summarize, text renderer, markdown renderer
src/delibera/agents/llm_proposer.py LLM-backed proposer agent
src/delibera/tools/builtin/docs.py DocsSearchTool implementation
src/delibera/protocol/spec.py ConvergenceSpec with refinement parameters
src/delibera/engine/orchestrator.py Refinement loop execution
src/delibera/cli.py New CLI commands and LLM flags

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Returns:
Dict with refined artifact and refinement notes.
"""
node_id = context.get("node_id", "") # noqa: F841
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable node_id is not used.

Copilot uses AI. Check for mistakes.
@fangchenli fangchenli merged commit 7316a95 into main Jan 26, 2026
9 checks passed
@fangchenli fangchenli deleted the feat/llm-inspect-refinement branch January 26, 2026 07:31
fangchenli added a commit that referenced this pull request Jan 28, 2026
- Pass Gemini API key via x-goog-api-key header instead of URL query
  parameter to prevent key leakage in logs and tracebacks
- Use Gemini SDK's native system_instruction parameter instead of
  concatenating system messages into the user prompt
- Deduplicate _build_sdk_contents and _build_http_contents into a
  single _build_contents method
- Redact error messages in llm_call_failed trace events using
  redact_text() to prevent sensitive data in traces
- Fix _extract_protocol_info reading wrong key ("protocol" vs
  "protocol_name") causing empty protocol name in inspect output
- Fix CLI catch-all error message referencing ANTHROPIC_API_KEY
  instead of being provider-agnostic
- Wire check_llm_allowed_in_step runtime guard into the engine's
  PROPOSE step to enforce the LLM-only-in-WORK invariant

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
fangchenli added a commit that referenced this pull request Jan 28, 2026
- Pass Gemini API key via x-goog-api-key header instead of URL query
  parameter to prevent key leakage in logs and tracebacks
- Use Gemini SDK's native system_instruction parameter instead of
  concatenating system messages into the user prompt
- Deduplicate _build_sdk_contents and _build_http_contents into a
  single _build_contents method
- Redact error messages in llm_call_failed trace events using
  redact_text() to prevent sensitive data in traces
- Fix _extract_protocol_info reading wrong key ("protocol" vs
  "protocol_name") causing empty protocol name in inspect output
- Fix CLI catch-all error message referencing ANTHROPIC_API_KEY
  instead of being provider-agnostic
- Wire check_llm_allowed_in_step runtime guard into the engine's
  PROPOSE step to enforce the LLM-only-in-WORK invariant

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant