Skip to content

feat: test chaining and response assertions#92

Merged
mikolajbadyl merged 2 commits intomainfrom
feat/test-chaining-assertions
Feb 28, 2026
Merged

feat: test chaining and response assertions#92
mikolajbadyl merged 2 commits intomainfrom
feat/test-chaining-assertions

Conversation

@mikolajbadyl
Copy link
Member

  • Extract values from responses and reuse them in later tests via `{{var_name}}`
  • Assert response fields with operators: `eq`, `neq`, `exists`, `contains`, `gt`, `gte`, `lt`, `lte`
  • Variables persist across tests in a group and are reset between groups
  • Assertion failures shown with `⚠` in TUI, counted as test failures

@kilo-code-bot
Copy link

kilo-code-bot bot commented Feb 28, 2026

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
internal/cli/update_tests.go 191-196 Unintentional behavioral change in test completion handling

Description: The logic for sending messages after test completion was modified. Previously:

  • When triggered by LLM tool_use (hadToolID = true): sent empty message m.sendChatMessage("")
  • When triggered by UI (hadToolID = false): sent summary message

Now the code always sends the summary message regardless of how the tests were triggered:

if hadToolID {
    return m, m.sendChatMessage("")
}
summary := fmt.Sprintf("Tests completed. %d tests executed...")
return m, m.sendChatMessage(summary)

The original code had explicit comments explaining the different behaviors:

  • "MUST send it to backend now! tool_result must be the next message after tool_use"
  • "Tests were triggered by UI - send summary message to Claude"

This change may affect the LLM conversation flow. Please verify this is intentional.

Other Observations (not in diff)

No issues found in unchanged code.

Files Reviewed (6 files)
  • internal/agents/chat.go - Added extract/assertions parameters to tool definitions
  • internal/agents/types.go - Added Extract and Assertion types
  • internal/cli/tui.go - Added testVars field to model
  • internal/cli/update.go - Added extract/assertions handling
  • internal/cli/update_tests.go - Added variable substitution and assertion execution
  • internal/core/tester/asserter.go - New file with ResolvePath and RunAssertions

Positive observations:

  • The new extract/assertions feature is well implemented
  • Variable substitution works correctly with proper nil checks
  • The asserter.go file is well-structured with good error handling
  • Code follows project conventions

@mikolajbadyl mikolajbadyl merged commit 251b824 into main Feb 28, 2026
1 check passed
@mikolajbadyl mikolajbadyl deleted the feat/test-chaining-assertions branch February 28, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant