Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/cli-tool-version-updates.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions .changeset/patch-add-noop-safe-output.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

100 changes: 84 additions & 16 deletions .github/agents/copilot-add-safe-output-type.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,73 @@ Safe output types are structured data formats that AI agents can emit to perform
};
```

### Step 3: Update Collection JavaScript
### Step 3: Update Safe Outputs Tools JSON

**File**: `pkg/workflow/js/safe_outputs_tools.json`

Add a tool signature for your new safe output type to expose it to AI agents through the MCP server. This file defines the tools that AI agents can call.

Add a new tool definition to the array:

```json
{
"name": "your_new_type",
"description": "Brief description of what this tool does (use underscores in name, not hyphens)",
"inputSchema": {
"type": "object",
"required": ["required_field"],
"properties": {
"required_field": {
"type": "string",
"description": "Description of the required field"
},
"optional_field": {
"type": "string",
"description": "Description of the optional field"
},
"numeric_field": {
"type": ["number", "string"],
"description": "Numeric field that accepts both number and string types"
}
},
"additionalProperties": false
}
}
```

**Tool Signature Guidelines**:
- Tool `name` must use underscores (e.g., `your_new_type`), matching the type field in the JSONL output
- The `name` field should match the safe output type name with underscores instead of hyphens
- Include a clear `description` explaining what the tool does
- Define an `inputSchema` with all fields the AI agent will provide
- Use `required` array for mandatory fields
- Set `additionalProperties: false` to prevent extra fields
- For numeric fields, use `"type": ["number", "string"]` to allow both types (agents sometimes send strings)
- Use descriptive field names and descriptions to guide AI agents

**Examples from existing tools**:
```json
{
"name": "create_issue",
"description": "Create a new GitHub issue",
"inputSchema": {
"type": "object",
"required": ["title", "body"],
"properties": {
"title": { "type": "string", "description": "Issue title" },
"body": { "type": "string", "description": "Issue body/description" },
"labels": {
"type": "array",
"items": { "type": "string" },
"description": "Issue labels"
}
},
"additionalProperties": false
}
}
```

### Step 4: Update Collection JavaScript

**File**: `pkg/workflow/js/collect_ndjson_output.ts`

Expand Down Expand Up @@ -187,7 +253,7 @@ case "your-new-type":
- Use `validateIssueOrPRNumber()` for GitHub issue/PR number fields
- Continue the loop on validation errors to process remaining items

### Step 4: Create JavaScript Implementation
### Step 5: Create JavaScript Implementation

**File**: `pkg/workflow/js/your_new_type.cjs`

Expand Down Expand Up @@ -289,7 +355,7 @@ await main();
- Use GitHub Actions context variables for repo information
- Follow the existing pattern for environment variable handling

### Step 5: Create Test File
### Step 6: Create Test File

**File**: `pkg/workflow/js/your_new_type.test.cjs`

Expand All @@ -303,7 +369,7 @@ Create comprehensive tests following existing patterns in the codebase:
- Use vitest framework with proper mocking
- Follow existing test patterns in `.test.cjs` files

### Step 6: Update Collection Tests
### Step 7: Update Collection Tests

**File**: `pkg/workflow/js/collect_ndjson_output.test.cjs`

Expand All @@ -313,7 +379,7 @@ Add test cases for your new type following existing patterns:
- Test field type validation
- Follow existing test structure in the file

### Step 7: Create Test Agentic Workflows
### Step 8: Create Test Agentic Workflows

Create test workflows for each supported engine to validate the new safe output type:

Expand Down Expand Up @@ -349,7 +415,7 @@ Create a your-new-type output with:
Output as JSONL format.
```

### Step 8: Create Go Job Builder
### Step 9: Create Go Job Builder

**File**: `pkg/workflow/your_new_type.go`

Expand Down Expand Up @@ -551,7 +617,7 @@ func NewPermissionsContentsReadYourPermissions() *Permissions {
}
```

### Step 9: Build and Test
### Step 10: Build and Test

1. **Compile TypeScript**: `make js`
2. **Format code**: `make fmt-cjs`
Expand All @@ -560,7 +626,7 @@ func NewPermissionsContentsReadYourPermissions() *Permissions {
5. **Compile workflows**: `make recompile`
6. **Full validation**: `make agent-finish`

### Step 10: Manual Validation
### Step 11: Manual Validation

1. Create a simple test workflow using your new safe output type
2. Test both staged and non-staged modes
Expand All @@ -572,6 +638,7 @@ func NewPermissionsContentsReadYourPermissions() *Permissions {

- [ ] JSON schema validates your new type correctly
- [ ] TypeScript types compile without errors
- [ ] Safe outputs tools JSON includes the new tool signature
- [ ] Collection logic validates fields properly
- [ ] JavaScript implementation handles all cases
- [ ] Tests achieve good coverage
Expand All @@ -581,14 +648,15 @@ func NewPermissionsContentsReadYourPermissions() *Permissions {

## Common Pitfalls to Avoid

1. **Inconsistent naming**: Ensure type names match exactly across all files (kebab-case in JSON, camelCase in TypeScript)
2. **Missing validation**: Always validate required fields and sanitize string content
3. **Incorrect union types**: Add your new type to all relevant union types
4. **Missing exports**: Export all new interfaces and types
5. **Test coverage gaps**: Test both success and failure scenarios
6. **Schema violations**: Follow JSON Schema draft-07 syntax strictly
7. **GitHub API misuse**: Use proper error handling for API calls
8. **Staged mode**: Always implement preview functionality for staged mode
1. **Inconsistent naming**: Ensure type names match exactly across all files (kebab-case in JSON, camelCase in TypeScript, underscores in tools.json)
2. **Missing tools.json update**: Don't forget to add the tool signature in `safe_outputs_tools.json` - AI agents won't be able to call your new type without it
3. **Missing validation**: Always validate required fields and sanitize string content
4. **Incorrect union types**: Add your new type to all relevant union types
5. **Missing exports**: Export all new interfaces and types
6. **Test coverage gaps**: Test both success and failure scenarios
7. **Schema violations**: Follow JSON Schema draft-07 syntax strictly
8. **GitHub API misuse**: Use proper error handling for API calls
9. **Staged mode**: Always implement preview functionality for staged mode

## Resources and References

Expand Down
Loading
Loading