chore(ci): add Claude coded architecture doc validation workflow - Add automated validation of SOFTWARE_ARCHITECTURE.md on PRs to ensure - skip:test:long_running #216
Conversation
refactor(uv): define required uv version in pyproject.toml, for use across GHA chore(deps): bump various github actions versions
chore(heta): further adaptation to changed output file sizes
chore(deps): bump sec(uv): use uv > 0.8.6 in pre-commit hook fix(bucket): in GUI use static version of download operation offered by service chore(pytest): show recent notifications if asserted one not found chore(test): don't provide log as job artifact chore(release): announce release on internal Slack (experimental)
…ame host with no isolation
chore(bucket): better logging for flaky test
…on analysis and exe building
…app even in non-chrome mode
…_for_test (#213) Add missing expires_seconds argument to _get_three_spots_payload_for_test() call in test_platform_test_app_submit_and_wait test. The function requires 1 positional argument but was being called without any. Fixes #211 Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Helmut Hoffer von Ankershoffen né Oertel <helmut-hoffer-von-ankershoffen@users.noreply.github.com>
…ribe_and_cancel_and_download_and_delete as this causes internal server errors for some runs
feat(ketryx): integrate Ketryx compliance framework with requirements traceability
…d automated validation of SOFTWARE_ARCHITECTURE.md on PRs to ensure - skip:test:long_running documentation stays in sync with code changes. Workflow checks for: - New modules not documented - Dependency changes in pyproject.toml - CI/CD workflow modifications - Architecture pattern changes - Module structure consistency
|
Claude finished @muhabalwan-aginx's task —— View job Reviewing PR #216: Claude Code Architecture Validation WorkflowReview Complete ✅I've conducted a comprehensive review of the new architecture validation workflow. Here are my findings: 🎯 OverviewThis PR adds a new automated workflow (
✅ STRENGTHS1. Excellent Design Pattern
2. Comprehensive Validation Logic
3. Security & Permissions
4. Documentation Quality
5. Commit Convention
❌ BLOCKING ISSUES1. Workflow Not Listed in .github/CLAUDE.mdIssue: New workflow file added but Location: Current State: The workflow inventory lists 8 entry points but doesn't include this new one Required Action: Add entry to the "Entry Point Workflows" table Suggested Content: | **claude-code-architecture-validation.yml** | PR opened/sync, workflow_dispatch | Architecture doc validation | _claude-code (automation) |Reproduce: grep -n "claude-code-architecture-validation" .github/CLAUDE.md
# Returns: (empty - workflow not documented)Verify Fix: grep "claude-code-architecture-validation" .github/CLAUDE.md
# Should return: the new table entryReference:
|
| Aspect | PR Review Workflow | Architecture Validation (This PR) |
|---|---|---|
| Triggers | ✅ Same (pull_request, workflow_dispatch) |
✅ Same |
| Mode | ✅ automation |
✅ automation |
| Track Progress | ✅ Same logic | ✅ Same logic |
| Allowed Tools | Write,Edit,MultiEdit,WebFetch,WebSearch,make |
❌ More restrictive (read-only) ✅ Appropriate |
| Secrets | ✅ All 6 secrets | ✅ All 6 secrets |
| Purpose | Code review | Documentation validation |
Assessment: Tool restrictions are appropriate - validation workflow should be read-only.
🔍 VALIDATION CHECKS PERFORMED
✅ YAML Syntax: Valid (workflow uses standard GitHub Actions syntax)
✅ Reusable Workflow Call: Correctly uses _claude-code.yml with proper inputs
✅ Secret Handling: All required secrets passed through (ANTHROPIC_API_KEY + platform credentials)
✅ Trigger Configuration: Appropriate triggers with correct event types
✅ Bash Script Logic: Commands are sound, though could use better error handling
✅ Documentation Quality: Excellent inline documentation in prompt
✅ Security: Read-only tools appropriate for validation workflow
✅ Conventional Commits: ✅ Commit message follows chore(ci): pattern
🎯 ACTION REQUIRED
Before merging this PR:
- ✅ BLOCKING: Update
.github/CLAUDE.mdto list the new workflow in the "Entry Point Workflows" table (lines 69-80)
Optional improvements (can be done in follow-up PRs):
2. Consider adding error handling for critical bash commands
3. Consider consolidating PR_NUMBER extraction
4. Consider adding workflow description header comment
5. Consider whether validation should be a required CI check
✨ FINAL ASSESSMENT
Overall: Excellent workflow design that follows repository patterns and adds valuable automation.
Recommendation: Approve with one blocking change - update .github/CLAUDE.md to document the new workflow in the workflow inventory.
Code Quality: ⭐⭐⭐⭐⭐ (5/5)
- Well-structured validation logic
- Comprehensive checks
- Appropriate security restrictions
- Excellent documentation
Integration: ⭐⭐⭐⭐ (4/5)
- Follows established patterns
- Missing CI/CD documentation update (blocking)
📝 COMMANDS TO VERIFY
# Check workflow syntax is valid
cat .github/workflows/claude-code-architecture-validation.yml | head -20
# Verify SOFTWARE_ARCHITECTURE.md exists
ls -lh SOFTWARE_ARCHITECTURE.md
# Check that the workflow can be triggered
gh workflow list | grep -i architecture
# Verify .github/CLAUDE.md needs update
grep -c "claude-code-architecture" .github/CLAUDE.md
# Should return: 0 (needs to be added)Excellent work @muhabalwan-aginx! This is a high-quality addition that will help maintain documentation accuracy. Just need to update the CI/CD documentation to complete the integration. 🚀
|
Claude finished @muhabalwan-aginx's task —— View job Reviewing PR #216: Claude Code Architecture Validation WorkflowTodo List
🎯 Review SummaryThis PR adds a valuable automated validation workflow for Overall Assessment: ❌ BLOCKING ISSUES (Must Fix Before Merge)1. CRITICAL: Document Doesn't Exist in Main BranchIssue: The workflow validates # Verification:
git ls-tree origin/main SOFTWARE_ARCHITECTURE.md
# Returns: nothing (file doesn't exist)Impact: This workflow will fail on every PR until Fix Options:
Action Required: Choose option 1 or 2 and implement before merging. 2. Workflow Trigger InefficiencyIssue: Workflow triggers on all PR types ( Current: on:
pull_request:
types: [opened, synchronize] # Runs on EVERY commit pushProblem: Architecture validation only needs to run when:
Recommended Fix: on:
pull_request:
types: [opened, synchronize]
paths:
- 'src/aignostics/**' # Source code changes
- 'pyproject.toml' # Dependency changes
- '.github/workflows/**' # CI/CD changes
- 'SOFTWARE_ARCHITECTURE.md' # Doc itselfBenefit: Saves CI minutes by only running when architecturally relevant files change (e.g., skips on README-only changes). 3. Missing Skip ConditionsIssue: No skip conditions for draft PRs or skip labels, unlike other workflows in this repo. Current behavior: Runs on all PRs including drafts and those marked for skipping. Repository pattern (from jobs:
architecture-validation:
if: |
github.event.pull_request.draft == false &&
(!contains(github.event.pull_request.labels.*.name, 'skip:ci'))Recommended Fix: Add skip condition to job: jobs:
architecture-validation:
if: |
github.event.pull_request.draft == false &&
(!contains(github.event.pull_request.labels.*.name, 'skip:ci'))
uses: ./.github/workflows/_claude-code.yml
# ... rest of configRationale:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Claude finished @muhabalwan-aginx's task —— View job Reviewing PR #216: Claude Code Architecture Validation WorkflowReview Complete ✅I've conducted a comprehensive review of this new architecture validation workflow against the repository's CI/CD standards and best practices. Here are my findings: 🎯 OverviewThis PR adds Good news: The ✅ STRENGTHS1. Excellent Workflow Design
2. Comprehensive Validation Logic
3. Security & Permissions
4. Documentation Quality
❌ BLOCKING ISSUES1. Missing Workflow Documentation in .github/CLAUDE.md 🚫Issue: New workflow file added but Location: Current State: The workflow inventory lists 8 entry points but doesn't include this new workflow Verification: grep -c "claude-code-architecture-validation" .github/CLAUDE.md
# Returns: 0 (workflow not documented)Required Action: Add entry to the "Entry Point Workflows" table at line 76-77: | **claude-code-architecture-validation.yml** | PR opened/sync, workflow_dispatch | Architecture doc validation | _claude-code (automation) |Also add to "Workflow Files Summary" (around line 1072): | `claude-code-architecture-validation.yml` | Entry | Architecture doc sync check | ~5-10 min |Reference: 2. CodeQL Security Warning: Missing Explicit Permissions
|
| Aspect | PR Review Workflow | Architecture Validation (This PR) |
|---|---|---|
| Triggers | ✅ pull_request, workflow_dispatch |
✅ Same |
| Path Filters | ❌ No path filters | ✅ Only arch-relevant files |
| Skip Conditions | ✅ Draft check, bot exclusion | ❌ No skip conditions |
| Concurrency | ✅ Has concurrency control | ❌ Missing |
| Allowed Tools | Write, Edit, MultiEdit, WebFetch | ✅ More restrictive (read-only focused) |
| Secrets | All 6 secrets | ✅ Same (may be excessive) |
🎯 ACTION ITEMS SUMMARY
Critical (Blocking Merge)
- ✅ Update
.github/CLAUDE.md- Add workflow to Entry Point Workflows table (lines 76-77) and Workflow Files Summary (line 1072) ⚠️ Review CodeQL Warning - Add explanatory comment to permissions block or investigate if more constraints needed
Recommended (High Priority)
⚠️ Add skip conditions for draft PRs andskip:cilabel⚠️ Add concurrency control to prevent duplicate runs⚠️ Review tool permissions - Clarify ifWrite/Edittools are intentional⚠️ Review platform credentials - Remove if API access not needed
Nice-to-Have (Consider)
- 💡 Simplify prompt template - Replace 120-line example with concise format
- 💡 Enable progress tracking for manual runs
- 💡 Improve error handling in bash commands
- 💡 Consolidate variable assignments
✅ EXCELLENT WORK
What's Great About This Workflow
- ✅ Solves real problem: Architecture drift is a legitimate concern for medical device software
- ✅ Well-structured validation logic: 5 comprehensive checks
- ✅ Proper path filtering: Only runs when relevant files change (saves CI minutes)
- ✅ Clear output format: BLOCKING vs SUGGESTIONS distinction helps prioritize fixes
- ✅ Follows repo patterns: Uses
_claude-code.ymlreusable workflow correctly - ✅ Good security baseline: Read-only tool access, explicit permissions
🔧 QUICK FIX COMMANDS
1. Update .github/CLAUDE.md:
# Add to line 76-77 in Entry Point Workflows table:
| **claude-code-architecture-validation.yml** | PR opened/sync, workflow_dispatch | Architecture doc validation | _claude-code (automation) |
# Add to line 1072+ in Workflow Files Summary:
| `claude-code-architecture-validation.yml` | Entry | Architecture doc sync check | ~5-10 min |2. Verify workflow will work:
# Check SOFTWARE_ARCHITECTURE.md exists
ls -la SOFTWARE_ARCHITECTURE.md
# Test workflow trigger filters
gh pr view 216 --json files
# Verify current modules
find src/aignostics -maxdepth 1 -type d -not -name "__pycache__" -not -name "aignostics"📚 REFERENCES
- Repository CI/CD Guide:
.github/CLAUDE.md(1098 lines of comprehensive workflow documentation) - Workflow patterns:
.github/workflows/ci-cd.yml(skip conditions, concurrency) - Claude workflow patterns:
.github/workflows/claude-code-automation-pr-review.yml(tool allowlist, prompt structure) - Security best practices: Root
CLAUDE.mdsection on security and secrets management
🎉 FINAL ASSESSMENT
Overall: Excellent initiative that adds valuable automation for maintaining architecture documentation accuracy. The workflow is well-designed and follows most repository patterns.
Recommendation: Approve with one critical fix - update .github/CLAUDE.md to document the new workflow in both workflow tables (required by repository standards).
Code Quality: ⭐⭐⭐⭐ (4/5)
- Well-structured validation logic
- Comprehensive checks
- Good security baseline
- Missing some standard repo patterns (skip conditions, concurrency)
Integration: ⭐⭐⭐⭐ (4/5)
- Follows reusable workflow pattern
- Missing CI/CD documentation update (blocking)
- Could align better with other workflow patterns
Great work @muhabalwan-aginx! 🚀 This is a high-quality addition that will help maintain documentation accuracy. Once the .github/CLAUDE.md documentation is updated, this will be ready to merge.
muhabalwan-aginx
force-pushed
the
chore/pysdk-21-claude-review-docs
branch
from
b402d0c to
a34d956
Compare
|
Claude finished @muhabalwan-aginx's task —— View job 🏗️ Software Architecture ValidationPR: #216 📊 Validation Summary
✅ VALIDATED SUCCESSFULLYCode Changes That Don't Require Doc Updates✅ New Workflow File Only - This PR adds a single CI/CD workflow file (
✅ No Source Code Changes - All SDK modules remain unchanged:
✅ Architecture Doc Accuracy - Current
❌ BLOCKING ISSUESIssue #1: Claude Code Automation Not DocumentedProblem: The architecture document doesn't mention Claude Code automation workflows, which are a significant part of the CI/CD pipeline (multiple workflows exist). Evidence:
Architectural Significance:
Required Action: Add "AI-Assisted Code Review & Validation" subsection to Section 1.4 Suggested Location: After "Code Quality & Analysis" subsection (after line 336), before "Testing & Coverage" Suggested Content: **AI-Assisted Code Review & Validation:**
- **Claude Code PR Reviews**: Automated comprehensive code review on pull requests with architectural guidance
- **Architecture Validation**: Automated verification that SOFTWARE_ARCHITECTURE.md stays in sync with code changes
- **Operational Excellence Monitoring**: Weekly automated checks for code quality, security, and best practices
- **Interactive Sessions**: On-demand Claude Code assistance for development tasks via @claude mentions
- **Automation Mode**: Runs unattended on CI/CD events (PR opened, scheduled triggers)Why This is Blocking:
Verification Command: # Check Claude Code workflows
ls -la .github/workflows/claude-code*.yml
# Should return 4 files:
# - claude-code-automation-pr-review.yml
# - claude-code-architecture-validation.yml (NEW)
# - claude-code-automation-operational-excellence-weekly.yml
# - claude-code-interactive.yml
|
|
Claude finished @muhabalwan-aginx's task —— View job Reviewing PR #216: Claude Code Architecture Validation WorkflowTodo List
🎯 Review CompleteI've conducted a thorough review of this new architecture validation workflow against the repository's CI/CD standards, comparing it with existing workflows and checking previous review feedback. ✅ EXCELLENT WORK - What's Right1. Solid Workflow Design
2. Comprehensive Validation Logic
3. Security & Permissions
4. Good Documentation in Prompt
❌ BLOCKING ISSUE (Must Fix Before Merge)1. Missing Workflow Documentation in .github/CLAUDE.md 🚫Issue: The repository requires ALL workflows to be documented in Location: a) Entry Point Workflows Table (line 69-80)Current: Lists 8 entry point workflows but missing this one Required: Add this line after line 80 (after | **claude-code-architecture-validation.yml** | PR opened/sync, workflow_dispatch | Architecture doc validation | _claude-code (automation) |b) Workflow Files Summary Table (line 1069-1092)Current: Lists all 19+ workflows but missing this one Required: Add this line after line 1080 (after | `claude-code-architecture-validation.yml` | Entry | Architecture doc sync check | ~5-10 min |Verification: # Before fix - returns 0:
grep -c "claude-code-architecture-validation" .github/CLAUDE.md
# After fix - should return 2:
grep -c "claude-code-architecture-validation" .github/CLAUDE.mdReference:
|
| Aspect | PR Review Workflow | Architecture Validation (This PR) |
|---|---|---|
| Triggers | ✅ pull_request, workflow_dispatch |
✅ Same |
| Path Filters | ❌ None (runs on all changes) | ✅ Only arch-relevant files |
| Skip Conditions | ❌ None | ❌ None (suggestion #4) |
| Concurrency | ❌ None | ❌ None (suggestion #3) |
| Allowed Tools | Write, Edit, MultiEdit, WebFetch, WebSearch | ✅ More restrictive |
| Explicit Permissions | ❌ None | ✅ Has explicit permissions |
| Track Progress | ✅ Same logic | ✅ Same logic (suggestion #5) |
Assessment: This workflow is actually MORE secure and better-designed than the PR review workflow in some aspects (path filters, explicit permissions).
🎯 ACTION ITEMS SUMMARY
Critical (Blocking Merge)
- ✅ Update
.github/CLAUDE.md- Add workflow to both tables (lines ~80 and ~1080)
Recommended (High Priority)
⚠️ Add concurrency control - Saves CI minutes, faster feedback⚠️ Add skip conditions - Respect draft PRs andskip:cilabel⚠️ Document permissions - Add comments explaining why each permission needed⚠️ Clarify tool access - Document if Write/Edit are intentional or remove them
Nice-to-Have (Consider)
- 💡 Enable progress tracking for manual runs
- 💡 Review platform credentials - Remove if validation doesn't need API access
- 💡 Simplify prompt template - Replace 120-line example with concise format
🔧 QUICK FIX COMMANDS
Fix #1: Update .github/CLAUDE.md (BLOCKING)
Step 1: Add to Entry Point Workflows table (after line 80):
# Open file at line 80
# Add this line after the codeql-scheduled.yml entry:
| **claude-code-architecture-validation.yml** | PR opened/sync, workflow_dispatch | Architecture doc validation | _claude-code (automation) |Step 2: Add to Workflow Files Summary table (after line 1080):
# Open file at line 1080
# Add this line after claude-code-automation-pr-review.yml entry:
| `claude-code-architecture-validation.yml` | Entry | Architecture doc sync check | ~5-10 min |Verify:
grep -n "claude-code-architecture-validation" .github/CLAUDE.md
# Should show 2 matchesOptional Fixes (Recommended):
Add concurrency control (after line 24):
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: trueAdd skip conditions (before line 27):
jobs:
architecture-validation:
if: |
github.event.pull_request.draft == false &&
(!contains(github.event.pull_request.labels.*.name, 'skip:ci'))Document permissions (add comments at lines 28-33):
permissions:
actions: read # Read workflow run status
contents: write # Read files + push fixes (if auto-fix enabled)
issues: write # Post comments (if triggered on issues)
pull-requests: write # Post validation results on PRs
id-token: write # OIDC token for cloud auth📚 REFERENCES
- Repository CI/CD Guide:
.github/CLAUDE.md(1098 lines of comprehensive workflow documentation) - Workflow Documentation Standards:
.github/CLAUDE.mdlines 67-97, 1069-1092 - Workflow Patterns:
ci-cd.yml(concurrency control, skip conditions)claude-code-automation-pr-review.yml(Claude automation patterns)claude-code-automation-operational-excellence-weekly.yml(prompt structure)
- Security Best Practices: Root
CLAUDE.mdon permissions and secrets
✨ FINAL ASSESSMENT
Overall: Excellent workflow that adds valuable automation for maintaining architecture documentation accuracy. Well-designed with good security practices.
Recommendation: Approve with ONE blocking change - update .github/CLAUDE.md to document the workflow (required by repository standards for medical device software traceability).
Code Quality: ⭐⭐⭐⭐⭐ (5/5)
- Comprehensive validation logic
- Smart path filtering (better than similar workflows)
- Good security baseline with explicit permissions
- Well-structured bash commands
Integration: ⭐⭐⭐⭐ (4/5)
- Follows reusable workflow pattern ✅
- Missing standard patterns (concurrency, skip conditions)
⚠️ - Missing CI/CD documentation (BLOCKING) ❌
Security: ⭐⭐⭐⭐⭐ (5/5)
- Explicit permissions block ✅
- Least privilege principle ✅
- Path filtering reduces attack surface ✅
- All secrets properly managed ✅
🎉 SUMMARY
What You Did Right:
- ✅ Solved a real problem (architecture documentation drift)
- ✅ Smart path filtering (saves CI minutes, reduces noise)
- ✅ Comprehensive validation checks (5 different aspects)
- ✅ Explicit security permissions (addresses CodeQL concern)
- ✅ Follows repository reusable workflow pattern
What Needs Fixing:
- ❌ BLOCKING: Add workflow to
.github/CLAUDE.mdtables (lines ~80 and ~1080)
What Would Make It Even Better:
⚠️ Add concurrency control (saves costs, faster feedback)⚠️ Add skip conditions (respects draft PRs and skip:ci)⚠️ Document permissions (explain why each is needed)- 💡 Simplify prompt template (more efficient)
Great work @muhabalwan-aginx! 🚀 This is a thoughtful addition to the CI/CD pipeline. Once the .github/CLAUDE.md documentation is updated, this will be ready to merge and will provide valuable automation for keeping architecture documentation synchronized with code changes.
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
…on sdk modules equally
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
| pull_request: | ||
| types: [opened, synchronize] | ||
| paths: | ||
| - 'src/**' # Any source code changes |
There was a problem hiding this comment.
update to incldue only /*py
| ```bash | ||
| PR_NUMBER="${{ github.event.pull_request.number || inputs.pr_number }}" | ||
|
|
||
| if [ -n "$PR_NUMBER" ]; then |
There was a problem hiding this comment.
PR_NUMBER not needed
|
helmut-hoffer-von-ankershoffen
force-pushed
the
main
branch
2 times, most recently
from
9905ef4 to
997dfe8
Compare
helmut-hoffer-von-ankershoffen
force-pushed
the
main
branch
from
c10d0b2 to
b88bccd
Compare
…
documentation stays in sync with code changes. Workflow checks for: