Skip to content

feat: response schema validation against OpenAPI spec#89

Merged
mikolajbadyl merged 3 commits intomainfrom
feat/schema-validation
Feb 28, 2026
Merged

feat: response schema validation against OpenAPI spec#89
mikolajbadyl merged 3 commits intomainfrom
feat/schema-validation

Conversation

@mikolajbadyl
Copy link
Member

@mikolajbadyl mikolajbadyl commented Feb 28, 2026

Validates response bodies against the OpenAPI schema after each test — not just the HTTP status code. A test can now pass on status but fail on schema mismatch (wrong types, missing required fields, etc.).

  • parser.go — extracts response schemas from OpenAPI spec, resolves $ref references
  • schema_validator.go — new validator (type, required, nullable, arrays), no external deps
  • handlers.go — runs validation after each test, path template matching (/users/{id}), fixed nil panic on existing project load
  • update_tests.go — shows with schema errors in TUI, sets exit code 1 on mismatch
  • chat.go — agent reports schema errors, PDF report only on explicit user request

@kilo-code-bot
Copy link

kilo-code-bot bot commented Feb 28, 2026

Code Review Summary

Status: 3 Suggestions Found | Recommendation: Merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 3
Issue Details (click to expand)

SUGGESTION

File Line Issue
internal/core/tester/schema_validator.go 26-82 No defensive nil check in validateValue, validateObject, validateArray - If called directly with a nil schema (bypassing ValidateSchema), these functions would panic. Consider adding defensive nil checks for robustness.
internal/core/tester/schema_validator.go - No support for additionalProperties keyword - The schema validator doesn't handle additionalProperties, which means it can't validate whether extra properties in a response are allowed. This could lead to false positives.
internal/core/parser/parser.go 242 External $ref not supported - The resolveRefs function only handles local references (#/components/schemas/X). External file references are not handled.
Resolved Items

The following issues from the previous review have been addressed:

Status File Issue
✅ Resolved internal/cli/handlers.go Tests added for validateResponseSchema and matchPath functions in handlers_test.go
✅ Resolved internal/core/parser/parser.go Tests added for extractResponseSchema and resolveRefs functions in parser_test.go
Files Reviewed (8 files)
  • internal/agents/chat.go - Tool description updates
  • internal/cli/handlers.go - Schema validation integration
  • internal/cli/handlers_test.go - Tests for handlers (NEW)
  • internal/cli/update_tests.go - Tests for update
  • internal/core/parser/parser.go - Schema extraction
  • internal/core/parser/parser_test.go - Tests for parser (NEW)
  • internal/core/tester/schema_validator.go - New file with tests
  • internal/core/tester/schema_validator_test.go - New test file

Code Quality: The implementation is solid and follows existing codebase patterns. Schema validation logic is correctly integrated into the test execution flow. The new code adds valuable functionality for validating API responses against OpenAPI schemas. All new functions now have adequate test coverage.

Recommendation: Ready to merge. The three suggestions are minor enhancements that don't block the merge.

@mikolajbadyl mikolajbadyl merged commit 698fc82 into main Feb 28, 2026
1 check passed
@mikolajbadyl mikolajbadyl deleted the feat/schema-validation branch February 28, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant