-
Notifications
You must be signed in to change notification settings - Fork 0
Feat: add LLM abstraction, inspect subsystem, and refinement support #12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This PR adds three major features to Delibera: ## LLM Abstraction + Gemini Provider (PR #17) - Add `llm/` module with stable LLMClient protocol interface - Implement GeminiClient with SDK and HTTP fallback - Add LLM request/response types with structured tracing - Add redaction helpers for trace-safe logging - Implement ProposerLLM agent with JSON output - Add CLI flags: --use-llm-proposer, --llm-model, etc. - Add optional dependency: delibera[llm] LLM usage is restricted to WORK steps only (never CLAIM_CHECK). Stubs remain the default; LLM mode is opt-in. ## Inspect Subsystem (PR #15) - Add `inspect/` module for run inspection and reporting - Implement text and markdown renderers - Add summarize module for run analysis - Add CLI commands: inspect, report - Support inspection by run-id or path ## Refinement Support (PR #14) - Add refinement round support in protocol spec - Implement convergence criteria checking - Add refinement pipeline execution - Support multi-round deliberation ## Additional Improvements - Add CONTRIBUTING.md with contributor guidelines - Add docs/README.md documentation index - Add DocsSearch and DocsRead tools with capability classification - Add smoke tests for CLI and imports - Improve tool policy with capability-based access control - Add __version__.py for version tracking Tests: 373 passing Type check: mypy passes with no errors Lint: ruff passes with no errors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds three major features to Delibera:
- LLM Abstraction + Gemini Provider: Adds a stable LLM client interface with Gemini implementation, including request/response types, redaction helpers, and an LLM-backed proposer agent
- Inspect Subsystem: Adds run inspection and reporting capabilities with text and markdown renderers
- Refinement Support: Adds bounded refinement loops with convergence checking
Changes:
- New
llm/module with LLM client protocol and Gemini implementation - New
inspect/module for run analysis and reporting - New
DocsSearchToolfor evidence discovery - Protocol spec extensions for refinement rounds
- CLI commands:
delibera inspectanddelibera report
Reviewed changes
Copilot reviewed 40 out of 41 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Added dependencies for LLM support (google-generativeai, pydantic, etc.) |
| tests/llm/test_llm_proposer.py | Tests for LLM proposer with fake client |
| tests/inspect/test_inspect.py | Tests for inspect/report functionality |
| tests/test_refinement.py | Tests for refinement loop |
| tests/test_smoke.py | Smoke tests for CLI and imports |
| tests/tools_docs/test_docs_search.py | Tests for DocsSearchTool |
| src/delibera/llm/ | LLM abstraction layer with base types, Gemini client, prompts, redaction |
| src/delibera/inspect/ | Run inspection with summarize, text renderer, markdown renderer |
| src/delibera/agents/llm_proposer.py | LLM-backed proposer agent |
| src/delibera/tools/builtin/docs.py | DocsSearchTool implementation |
| src/delibera/protocol/spec.py | ConvergenceSpec with refinement parameters |
| src/delibera/engine/orchestrator.py | Refinement loop execution |
| src/delibera/cli.py | New CLI commands and LLM flags |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Returns: | ||
| Dict with refined artifact and refinement notes. | ||
| """ | ||
| node_id = context.get("node_id", "") # noqa: F841 |
Copilot
AI
Jan 14, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Variable node_id is not used.
- Pass Gemini API key via x-goog-api-key header instead of URL query
parameter to prevent key leakage in logs and tracebacks
- Use Gemini SDK's native system_instruction parameter instead of
concatenating system messages into the user prompt
- Deduplicate _build_sdk_contents and _build_http_contents into a
single _build_contents method
- Redact error messages in llm_call_failed trace events using
redact_text() to prevent sensitive data in traces
- Fix _extract_protocol_info reading wrong key ("protocol" vs
"protocol_name") causing empty protocol name in inspect output
- Fix CLI catch-all error message referencing ANTHROPIC_API_KEY
instead of being provider-agnostic
- Wire check_llm_allowed_in_step runtime guard into the engine's
PROPOSE step to enforce the LLM-only-in-WORK invariant
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Pass Gemini API key via x-goog-api-key header instead of URL query
parameter to prevent key leakage in logs and tracebacks
- Use Gemini SDK's native system_instruction parameter instead of
concatenating system messages into the user prompt
- Deduplicate _build_sdk_contents and _build_http_contents into a
single _build_contents method
- Redact error messages in llm_call_failed trace events using
redact_text() to prevent sensitive data in traces
- Fix _extract_protocol_info reading wrong key ("protocol" vs
"protocol_name") causing empty protocol name in inspect output
- Fix CLI catch-all error message referencing ANTHROPIC_API_KEY
instead of being provider-agnostic
- Wire check_llm_allowed_in_step runtime guard into the engine's
PROPOSE step to enforce the LLM-only-in-WORK invariant
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Summary
This PR adds three major features to Delibera:
LLM Abstraction + Gemini Provider
llm/module with stableLLMClientprotocol interfaceGeminiClientwith SDK and HTTP fallbackProposerLLMagent with JSON output--use-llm-proposer,--llm-model,--llm-temperature,--llm-max-output-tokenspip install delibera[llm]LLM usage is restricted to WORK steps only (never CLAIM_CHECK). Stubs remain the default; LLM mode is opt-in.
Inspect Subsystem
inspect/module for run inspection and reportingdelibera inspect,delibera reportRefinement Support
Additional Improvements
__version__.pyfor version trackingTest plan
delibera run --question "Test" --use-llm-proposer(requires GEMINI_API_KEY)delibera inspect --run-id <id>delibera report --run-id <id>🤖 Generated with Claude Code