Update docs to match current CLI and runtime details #12
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Summary of ChangesHello @Zochory, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly updates the Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates fleet-rlm’s documentation and bundled Claude Code scaffold assets to reflect new runtime capabilities (built-in llm_query / llm_query_batched, Final convention, stdout summarization) and expanded scaffold inventory (teams + hooks), while also extending the CLI and test suite to cover these additions.
Changes:
- Add built-in sandbox tools (
llm_query,llm_query_batched),Finalvariable convention, and metadata-only stdout summarization support in the Modal interpreter/driver path. - Expand scaffold support beyond skills/agents to also include team templates and hook templates; update
fleet-rlm initCLI accordingly. - Refresh docs and add/adjust tests for scaffold listing/installation and RLM runtime behavior.
Reviewed changes
Copilot reviewed 68 out of 69 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Updates locked dependency versions and editable package version metadata. |
| pyproject.toml | Bumps package version and includes *.json scaffold assets in package data. |
| src/fleet_rlm/interpreter.py | Adds sub-LLM querying tools, call limiting, and stdout summarization behavior. |
| src/fleet_rlm/driver.py | Adds built-in sandbox tool injection and Final variable convention handling. |
| src/fleet_rlm/scaffold.py | Expands scaffold listing/installation to include nested agents, teams, and hooks. |
| src/fleet_rlm/cli.py | Extends init command with teams/hooks options and listing output. |
| tests/test_llm_query_mock.py | Adds unit tests for llm_query/llm_query_batched behavior with mocks. |
| tests/test_driver_protocol.py | Adds regression tests for Final variable behavior in the driver protocol. |
| tests/test_scaffold.py | Updates scaffold tests for new teams/hooks inventory and nested agent paths. |
| tests/test_scaffold_scripts.py | Adds regression coverage for scaffolded scripts and semantic chunking ranges. |
| tests/test_cli_smoke.py | Adds CLI smoke tests for new init modes (teams/hooks + include/exclude flags). |
| test_full_integration.py | Adds a manual integration script for live API testing of llm_query features. |
| test_llm_query_features.py | Adds a manual workflow script testing chunking + batched querying + synthesis. |
| test_v2_volume.py | Adds a manual integration script for verifying V2 volume behavior with RLM. |
| docs/index.md | Updates docs index wording to reflect broader scaffold assets (teams/hooks). |
| docs/getting-started.md | Updates onboarding text from “skills/agents” to “scaffold assets” and inventory. |
| docs/concepts.md | Updates conceptual docs to reflect built-in sub-LLM tools and helper defaults. |
| docs/tutorials/doc-analysis.md | Updates tutorial descriptions to match actual sandbox/signature behaviors. |
| docs/guides/cli-reference.md | Updates init command docs and helper defaults to match runtime. |
| docs/guides/skills-and-agents.md | Updates guide to include teams/hooks and revised install tree. |
| docs/guides/skills-usage.md | Updates references from skills/agents to scaffold assets + updated inventory. |
| README.md | Refreshes README to reflect new runtime + scaffold categories and CLI flags. |
| CLAUDE.md | Adds repo guidance for commands/architecture/testing aligned with latest behavior. |
| AGENTS.md | Updates quick commands to include scaffold install/list and teams env flag note. |
| src/fleet_rlm/_scaffold/skills/rlm/SKILL.md | Documents new tools/config/output conventions in the bundled RLM skill. |
| src/fleet_rlm/_scaffold/skills/rlm-long-context/scripts/orchestrate.py | Improves script importability when executed directly vs as a module. |
| src/fleet_rlm/_scaffold/skills/rlm-long-context/scripts/rank_chunks.py | Moves pickle import into function and clarifies pickle format in docstring. |
| src/fleet_rlm/_scaffold/skills/rlm-long-context/scripts/semantic_chunk.py | Improves JSON chunk span recovery with fallback to size-based chunking. |
| src/fleet_rlm/_scaffold/skills/rlm-debug/scripts/diagnose.py | Tweaks secret-check output formatting and failure reporting. |
| src/fleet_rlm/_scaffold/hooks/README.md | Adds documentation for hook templates and naming conventions. |
| src/fleet_rlm/_scaffold/hooks/hookify.fleet-rlm-document-process.local.md | Adds prompt hook guidance for document-processing prompts. |
| src/fleet_rlm/_scaffold/hooks/hookify.fleet-rlm-large-file.local.md | Adds prompt hook guidance for large-file prompts. |
| src/fleet_rlm/_scaffold/hooks/hookify.fleet-rlm-llm-query-error.local.md | Adds prompt hook guidance for llm_query-related errors. |
| src/fleet_rlm/_scaffold/hooks/hookify.fleet-rlm-modal-error.local.md | Adds prompt hook guidance for Modal/sandbox-related errors. |
| src/fleet_rlm/_scaffold/agents/rlm-orchestrator.md | Updates agent docs for new tools/output conventions/summarization behavior. |
| src/fleet_rlm/_scaffold/agents/rlm-specialist.md | Updates agent docs for debugging/cost/limits related to new tools. |
| src/fleet_rlm/_scaffold/agents/modal-interpreter-agent.md | Updates agent docs for new tools/config/output conventions. |
| src/fleet_rlm/_scaffold/agents/teams/agent-designer.md | Adds team-support agent definition for scaffolded teams. |
| src/fleet_rlm/_scaffold/agents/teams/architect-explorer.md | Adds team-support agent definition for scaffolded teams. |
| src/fleet_rlm/_scaffold/agents/teams/fleet-rlm-explorer-team.md | Adds multi-agent “explorer team” definition. |
| src/fleet_rlm/_scaffold/agents/teams/testing-analyst.md | Adds team-support agent definition for scaffolded teams. |
| src/fleet_rlm/_scaffold/agents/teams/ux-reviewer.md | Adds team-support agent definition for scaffolded teams. |
| src/fleet_rlm/_scaffold/teams/fleet-rlm/config.json | Adds scaffolded team template config for Claude Code agent teams. |
| src/fleet_rlm/_scaffold/teams/fleet-rlm/inboxes/modal-interpreter-agent.json | Adds scaffolded initial inbox seed for modal-interpreter-agent. |
| src/fleet_rlm/_scaffold/teams/fleet-rlm/inboxes/rlm-orchestrator.json | Adds scaffolded initial inbox seed for rlm-orchestrator. |
| src/fleet_rlm/_scaffold/teams/fleet-rlm/inboxes/rlm-specialist.json | Adds scaffolded initial inbox seed for rlm-specialist. |
| src/fleet_rlm/_scaffold/teams/fleet-rlm/inboxes/rlm-subcall.json | Adds scaffolded initial inbox seed for rlm-subcall. |
| src/fleet_rlm/_scaffold/teams/fleet-rlm/inboxes/team-lead.json | Adds scaffolded initial inbox seed for team-lead. |
| .claude/skills/rlm/SKILL.md | Mirrors updated RLM skill content in repo-local Claude scaffolding. |
| .claude/hooks/README.md | Mirrors hook README in repo-local Claude scaffolding. |
| .claude/hooks/hookify.fleet-rlm-document-process.local.md | Mirrors hook template in repo-local Claude scaffolding. |
| .claude/hooks/hookify.fleet-rlm-large-file.local.md | Mirrors hook template in repo-local Claude scaffolding. |
| .claude/hooks/hookify.fleet-rlm-llm-query-error.local.md | Mirrors hook template in repo-local Claude scaffolding. |
| .claude/hooks/hookify.fleet-rlm-modal-error.local.md | Mirrors hook template in repo-local Claude scaffolding. |
| .claude/agents/rlm-orchestrator.md | Mirrors updated agent doc in repo-local Claude scaffolding. |
| .claude/agents/rlm-specialist.md | Mirrors updated agent doc in repo-local Claude scaffolding. |
| .claude/agents/modal-interpreter-agent.md | Mirrors updated agent doc in repo-local Claude scaffolding. |
| .claude/agents/teams/agent-designer.md | Mirrors team-support agent definition in repo-local Claude scaffolding. |
| .claude/agents/teams/architect-explorer.md | Mirrors team-support agent definition in repo-local Claude scaffolding. |
| .claude/agents/teams/fleet-rlm-explorer-team.md | Mirrors explorer team definition in repo-local Claude scaffolding. |
| .claude/agents/teams/testing-analyst.md | Mirrors team-support agent definition in repo-local Claude scaffolding. |
| .claude/agents/teams/ux-reviewer.md | Mirrors team-support agent definition in repo-local Claude scaffolding. |
| .claude/teams/fleet-rlm/config.json | Adds a repo-local Claude teams config (appears to include machine/session-specific state). |
| .claude/teams/fleet-rlm/inboxes/modal-interpreter-agent.json | Adds repo-local team inbox state. |
| .claude/teams/fleet-rlm/inboxes/rlm-orchestrator.json | Adds repo-local team inbox state (contains sensitive credential-like text). |
| .claude/teams/fleet-rlm/inboxes/rlm-specialist.json | Adds repo-local team inbox state. |
| .claude/teams/fleet-rlm/inboxes/rlm-subcall.json | Adds repo-local team inbox state. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| }, | ||
| { | ||
| "from": "team-lead", | ||
| "text": "The API credentials are in the `.env` file at the project root. Re-run the integration test with the environment properly loaded.\n\nThe `.env` file contains:\n- DSPY_LM_MODEL=openai/gemini-3-flash-preview\n- DSPY_LM_API_BASE=https://litellm-proxy-gojcb5mtua-uc.a.run.app \n- DSPY_LLM_API_KEY=sk-bd2c13c7...\n\nMake sure the test loads the `.env` file before running, or run with:\n`uv run --env-file .env python test_llm_query_features.py`\n\nRe-run the integration test now.", |
There was a problem hiding this comment.
This inbox log includes what appears to be an API key value (DSPY_LLM_API_KEY=sk-...). Secrets (even partially redacted) and credential hints should not be committed; please remove this content from the repo and rotate/revoke the exposed key if it was real.
Suggested change
| "text": "The API credentials are in the `.env` file at the project root. Re-run the integration test with the environment properly loaded.\n\nThe `.env` file contains:\n- DSPY_LM_MODEL=openai/gemini-3-flash-preview\n- DSPY_LM_API_BASE=https://litellm-proxy-gojcb5mtua-uc.a.run.app \n- DSPY_LLM_API_KEY=sk-bd2c13c7...\n\nMake sure the test loads the `.env` file before running, or run with:\n`uv run --env-file .env python test_llm_query_features.py`\n\nRe-run the integration test now.", | |
| "text": "The API credentials are in the `.env` file at the project root. Re-run the integration test with the environment properly loaded.\n\nThe `.env` file contains:\n- DSPY_LM_MODEL=openai/gemini-3-flash-preview\n- DSPY_LM_API_BASE=https://litellm-proxy-gojcb5mtua-uc.a.run.app \n- DSPY_LLM_API_KEY=<redacted>\n\nMake sure the test loads the `.env` file before running, or run with:\n`uv run --env-file .env python test_llm_query_features.py`\n\nRe-run the integration test now.", |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the documentation to align with new CLI commands and runtime features, particularly around sub-LLM calls (llm_query, llm_query_batched) and agent teams. While the changes significantly improve clarity and provide excellent examples, with robust implementation in interpreter.py and driver.py, a medium-severity security vulnerability was identified related to insecure deserialization using the pickle module in the rank_chunks.py script. Additionally, there are minor documentation formatting issues and the inclusion of user-specific absolute paths in some configuration and log files, which impacts portability.
| "model": "kimi-for-coding", | ||
| "joinedAt": 1770672919989, | ||
| "tmuxPaneId": "", | ||
| "cwd": "/Volumes/Samsung-SSD-T7/Workspaces/Github/qredence/agent-framework/v0.5/_WORLD/_RLM/fleet-rlm-dspy", |
Contributor
There was a problem hiding this comment.
The cwd property is set to a user-specific absolute path ("/Volumes/Samsung-SSD-T7/..."). This is also present for other members in this file. Configuration files in version control should not contain absolute paths to ensure portability across different developer machines. This should be replaced with a relative path, an environment variable, or a placeholder that can be resolved at runtime.
Comment on lines
17
to
18
| with open(state_path, "rb") as f: | ||
| state = pickle.load(f) |
Contributor
There was a problem hiding this comment.
The load_context function uses the pickle module to deserialize data from a file specified by state_path. The pickle module is known to be insecure and can lead to arbitrary code execution if the input data is malicious. Since this script is part of the scaffolded skills and can be executed with user-supplied paths, an attacker could potentially achieve remote code execution by providing a crafted pickle file. It is highly recommended to replace pickle with a secure serialization format such as json.
Comment on lines
+53
to
+60
| ```markdown | ||
| ## 1. Design Philosophy | ||
| [Core principles observed] | ||
|
|
||
| ## 2. Coordination Patterns | ||
| ``` | ||
| [Diagram or description of delegation flows] | ||
| ``` |
Contributor
There was a problem hiding this comment.
There's an inconsistency in how placeholders are documented in the Output Format section. [Core principles observed] is inside the main markdown code block, while [Diagram or description of delegation flows] is wrapped in its own separate code block. For consistency, placeholders should either all be inside or all be outside the example code blocks. Placing them outside as descriptive text is generally clearer for documentation.
| }, | ||
| { | ||
| "from": "team-lead", | ||
| "text": "{\"type\":\"permission_response\",\"request_id\":\"perm-1770699937657-v2y1cew\",\"subtype\":\"success\",\"response\":{\"updated_input\":{\"command\":\"for dir in dspy-knowledge rlm-knowledge output memory cache knowledge; do\\n echo \\\"=== Backing up $dir ===\\\"\\n mkdir -p /tmp/rlm-volume-backup/$dir\\n modal volume get rlm-volume-dspy $dir /tmp/rlm-volume-backup/ 2>&1 || echo \\\"Failed to backup $dir\\\"\\ndone\",\"timeout\":120000,\"description\":\"Back up all directories from the volume\"},\"permission_updates\":[]}}", |
Contributor
There was a problem hiding this comment.
This file appears to be a log or message history from a specific run, and it contains user-specific absolute paths (e.g., in the command field of permission responses). Checking in runtime-specific files with absolute paths is not recommended as they are not portable and can expose environment details. These files should either be sanitized to act as generic templates or be excluded from version control via .gitignore.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
llm_query/llm_query_batchedusage while keeping doc tone.driver.pyandchunking.py.Testing