docs: update claude skill and related docs

[schuellerf]

Update claude skill and related docs

I tried to align README, code and claude skill
Please check thoroughly if I understood the config correctly

Summary by CodeRabbit

• Documentation

  • Updated MCP agent configuration documentation with new fields and command-line flags.
  • Updated evaluation command guidance and file format references.

• Configuration

  • Added useVirtualHome and allowedToolsJoinSeparator configuration options for agents.
  • Added --strict-mcp-config and --allowedTools command-line flags.

[coderabbitai]

Walkthrough

Documentation updates across three files: skill documentation for eval file handling and gevals command, MCP agent configuration example in README with new fields (useVirtualHome, allowedToolsJoinSeparator) and flags (--strict-mcp-config, --allowedTools), and a config struct comment reflecting the template variable naming update.

Changes

Cohort / File(s)	Summary
Skill and eval documentation .claude/skills/create-eval/SKILL.md	Updated eval file reference from eval.yaml to eval.md in from-scratch creation flow; changed gevals command invocation from gevals run to gevals eval subcommand.
Configuration documentation README.md, pkg/agent/config.go	Added new MCP agent configuration fields in README example: useVirtualHome: false and allowedToolsJoinSeparator: ",". Updated runPrompt command with flags --strict-mcp-config and --allowedTools "{{ .AllowedToolArgs }}". Updated documentation comment in AgentCommands struct to reflect template variable name change from {{ .AllowedTools }} to {{ .AllowedToolArgs }}.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Verify consistency between README example and config.go comment template variable naming
• Confirm eval file reference change (eval.yaml → eval.md) and gevals subcommand rename (run → eval) are correct and documented consistently
• Check that new MCP agent configuration fields align with actual implementation expectations

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'docs: update claude skill and related docs' accurately reflects the main changes: documentation updates to the claude skill in SKILL.md, README.md, and a comment in pkg/agent/config.go.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 912c90c and cf8e6e1.

📒 Files selected for processing (3)

• .claude/skills/create-eval/SKILL.md (2 hunks)
• README.md (1 hunks)
• pkg/agent/config.go (1 hunks)

🔇 Additional comments (3)

.claude/skills/create-eval/SKILL.md (1)

30-30: LGTM! Command rename is consistent.

The command has been correctly updated from gevals run to gevals eval, which aligns with the usage shown in README.md line 24.

pkg/agent/config.go (1)

48-48: LGTM! Documentation aligned with README.

The comment has been correctly updated to reference {{ .AllowedToolArgs }}, which matches the template variable used in the README.md example (line 77). This improves consistency across the documentation.

README.md (1)

72-77: LGTM! Agent configuration example updated correctly.

The agent.yaml example has been enhanced with:

• useVirtualHome: false - matches the struct field in config.go line 30
• allowedToolsJoinSeparator: "," - matches the struct field in config.go line 43
• Updated runPrompt with new flags --strict-mcp-config and --allowedTools "{{ .AllowedToolArgs }}" - aligns with the comment in config.go line 48

These changes properly document the new configuration options and demonstrate their usage.

[Cali0707]

Thanks for updating this @schuellerf !

[Cali0707]

Nice catch, forgot to update this when we added #19

On that note, would you mind adding a line here explaining the -run flag? I'm assuming a lot of the time, claude will want to run only a subset of an eval suite