From fc38f4b108f82369349935c58498262816249928 Mon Sep 17 00:00:00 2001 From: Andrei Gasparian Date: Thu, 19 Mar 2026 20:27:50 +0100 Subject: [PATCH] Upgrade make-yt-issue skill with resolve-or-create flow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the create-only behavior with a unified resolve-or-create pattern: validate existing tickets, draft new ones from conversation context, and handle explicit creation requests — all under the same /make-yt-issue handle. 7 eval cases cover the full surface. Co-Authored-By: Claude Opus 4.6 --- .claude/skills/make-yt-issue/SKILL.md | 89 +++++++++++++------ .claude/skills/make-yt-issue/evals/evals.json | 68 +++++++++++--- CLAUDE.md | 6 +- 3 files changed, 122 insertions(+), 41 deletions(-) diff --git a/.claude/skills/make-yt-issue/SKILL.md b/.claude/skills/make-yt-issue/SKILL.md index ac479f41..de6005f5 100644 --- a/.claude/skills/make-yt-issue/SKILL.md +++ b/.claude/skills/make-yt-issue/SKILL.md @@ -1,52 +1,89 @@ --- name: make-yt-issue -description: Create a YouTrack issue for planned work using MCP tools. Use when no ticket exists for the work, when the user asks to create a ticket, or before starting untracked work. +description: Ensure a YouTrack issue exists before starting work. Use at the start of any task when the user has not provided a ticket ID, when you need to verify a ticket exists, when the user asks to create a ticket, or before starting untracked work. --- -# Make YouTrack Issue +# Ensure YouTrack Issue -Create a YouTrack issue in the DBA project using the YouTrack MCP tools. +Ensure a YouTrack issue exists for the current work. Depending on context, this +skill resolves an existing ticket, or drafts and creates a new one after user +approval. ## Steps -### 1. Gather issue details +### 1. Determine intent -Ask the user for (or propose from conversation context): +Read the conversation to decide which path to take: -- **Summary**: concise one-line title -- **Description**: what the work involves and why -- **Type**: Bug, Task, Feature, or other applicable type +| Signal | Path | +|--------|------| +| User provides a ticket ID (`DBA-123`, `123`, or a YouTrack URL) | Go to **step 2** (validate) | +| User explicitly asks to create a ticket (e.g., "create a ticket for …") | Go to **step 3** (draft) | +| No ticket mentioned | Ask: *"Do you have a YouTrack ticket for this work?"* — then route based on answer | -### 2. Show the proposed issue +Accept `DBA-XXX`, a bare number (expand to `DBA-XXX`), or a full YouTrack URL. -Present the summary, description, and type to the user. -**Wait for explicit approval before creating.** +### 2. Validate the ticket -### 3. Create the issue +Use the `get_issue` MCP tool to fetch the issue. -Use the `create_issue` MCP tool to create the issue in the DBA project. +- **Found** — display the issue summary and confirm with the user that it + matches the intended work. If confirmed, proceed to step 5. +- **Not found / error** — inform the user the ticket could not be loaded and + offer to create a new one (continue to step 3). -### 4. Report +### 3. Draft the issue from context -Output the issue ID (e.g., `DBA-123`) so the user can reference it. +Generate a proposed issue using details from the conversation: -### 5. Optionally transition to Develop +- **Summary**: concise one-line title in imperative mood. +- **Description**: what the work involves and why it matters (2-4 sentences). +- **Type**: Bug, Task, or Feature — infer from context, default to Task. -If the user is starting work immediately, move the issue to **Develop** -state using `update_issue` (set the `State` field). +Present the draft clearly: + +``` +Summary: +Type: + +Description: + +``` + +Ask the user to **approve, edit, or reject** the draft. +If the user rejects and does not want a ticket, respect that and stop. + +### 4. Create the issue + +After the user approves (or edits and then approves): + +- Use the `create_issue` MCP tool targeting the **DBA** project. +- Report the created issue ID (e.g., `DBA-456`). + +### 5. Transition to Develop + +Move the issue to **Develop** state using `update_issue` (set the `State` +custom field) so the board reflects active work. + +Report the final state: issue ID and current status. ## Guardrails - Never create an issue without explicit user approval of the summary, description, and type. -- If the YouTrack MCP server is unavailable, inform the user and refer - them to `DEVELOPMENT.md` for setup instructions. -- Default to the `DBA` project unless the user specifies otherwise. +- Never skip the validation step when the user provides an existing ticket ID — + always confirm the ticket matches the intended work. +- If the YouTrack MCP server is unavailable, inform the user and refer them to + `DEVELOPMENT.md` for setup instructions. +- Default to the **DBA** project unless the user specifies otherwise. +- Accept flexible input: `DBA-123`, `123`, or a YouTrack URL should all resolve + correctly. +- If the user declines to create a ticket, respect that and do not push back. ## What this skill does NOT do -- It does not manage issue state transitions beyond the initial Develop - move — ongoing state management (Develop → Review) is handled by the - development workflow in CLAUDE.md. -- It does not search or update existing issues — use `get_issue` and - `update_issue` MCP tools directly for that. +- It does not manage state transitions beyond the initial move to Develop — + ongoing state management (Develop → Review) is handled by the workflow in + CLAUDE.md. +- It does not assign the issue or set priority — use `update_issue` or + `change_issue_assignee` MCP tools directly for that. diff --git a/.claude/skills/make-yt-issue/evals/evals.json b/.claude/skills/make-yt-issue/evals/evals.json index 244b0b36..5540fd46 100644 --- a/.claude/skills/make-yt-issue/evals/evals.json +++ b/.claude/skills/make-yt-issue/evals/evals.json @@ -3,16 +3,70 @@ "evals": [ { "id": 1, + "prompt": "I want to add a new 'export' CLI command for exporting project data.", + "expected_output": "Agent asks whether the user has a YouTrack ticket, and when told no, drafts an issue from the description, waits for approval, creates it, and moves it to Develop.", + "assertions": [ + "Agent asks whether the user has an existing YouTrack ticket", + "Agent drafts a summary, description, and type from the conversation context", + "Agent waits for explicit user approval before calling create_issue", + "Agent reports the created issue ID (e.g., DBA-XXX) after creation", + "Agent moves the issue to Develop state using update_issue" + ] + }, + { + "id": 2, + "prompt": "I need to fix a bug in the status command. The ticket is DBA-200.", + "expected_output": "Agent fetches DBA-200 with get_issue, shows the summary, confirms it matches the work, and moves it to Develop.", + "assertions": [ + "Agent calls get_issue to validate that DBA-200 exists", + "Agent displays the issue summary for user confirmation", + "Agent does NOT create a new issue when the ticket already exists", + "Agent moves the issue to Develop state after confirmation" + ] + }, + { + "id": 3, + "prompt": "Let's work on ticket 175.", + "expected_output": "Agent expands bare number to DBA-175, fetches the issue, confirms with the user, and transitions to Develop.", + "assertions": [ + "Agent expands bare number 175 to DBA-175", + "Agent calls get_issue to validate the expanded ticket ID", + "Agent shows the issue summary and asks for confirmation before proceeding" + ] + }, + { + "id": 4, "prompt": "Create a YouTrack ticket for adding a new 'config' CLI command that lets users manage project settings.", - "expected_output": "Agent proposes an issue with summary, description, and type, waits for approval, then creates it via MCP.", + "expected_output": "Agent recognizes explicit creation request, drafts issue details, waits for approval, then creates via MCP.", "assertions": [ + "Agent does NOT ask whether a ticket exists — user explicitly wants to create", "Agent proposes a summary, description, and type before creating", "Agent waits for explicit user approval before calling create_issue", "Agent reports the created issue ID (e.g., DBA-XXX) after creation" ] }, { - "id": 2, + "id": 5, + "prompt": "I want to refactor the MCP tool registry but I don't have a ticket. Don't create one, I'll do it later.", + "expected_output": "Agent respects the user's decision and does not create an issue.", + "assertions": [ + "Agent does NOT call create_issue when the user declines", + "Agent acknowledges the user's decision without pushing back" + ] + }, + { + "id": 6, + "prompt": "Start work on DBA-999. I think that ticket was deleted.", + "expected_output": "Agent tries to fetch DBA-999, reports it cannot be found, and offers to create a new one.", + "assertions": [ + "Agent calls get_issue for DBA-999", + "Agent informs the user the ticket could not be loaded", + "Agent offers to create a new issue as a fallback", + "Agent does NOT silently proceed without a valid ticket" + ] + }, + { + "id": 7, "prompt": "I want to start working on something but there's no ticket yet. Can you make one?", "expected_output": "Agent asks for details to populate the issue, then proposes and creates after approval.", "assertions": [ @@ -20,16 +74,6 @@ "Agent does NOT create an issue without gathering sufficient information", "Agent presents the proposed issue for approval before creating" ] - }, - { - "id": 3, - "prompt": "Create a YouTrack issue and start working on it right away.", - "expected_output": "Agent creates the issue after approval and moves it to Develop state.", - "assertions": [ - "Agent waits for approval before creating the issue", - "After creation, agent moves the issue to Develop state using update_issue", - "Agent reports both the issue ID and the state transition" - ] } ] } diff --git a/CLAUDE.md b/CLAUDE.md index 0d967150..b418843f 100755 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -88,13 +88,13 @@ style issues; do not manually fix formatting. ## YouTrack Ticket Workflow -- Before starting work, ask for a YouTrack ticket ID (e.g., `DBA-123` or just - `123` — expand bare numbers to `DBA-123`). +- Before starting work, use the `make-yt-issue` skill to verify or create a + YouTrack ticket. It handles asking for the ID, validating it exists, and + creating one if needed. - If the YouTrack MCP server is unavailable, refer the user to `DEVELOPMENT.md` for setup instructions. - If a ticket is provided, read it with the `get_issue` tool to understand the full scope before writing any code. -- If no ticket exists for the work, use the `make-yt-issue` skill to create one. - When starting work on a ticket, move it to **Develop** state using `update_issue` (set `State` field). - After creating a PR, move the ticket to **Review** state and add the