[CI] (ceefcf5) flask/flask3-social-media

[edwinyjlim]

Automated wizard CI run

Source: local
Trigger ID: ceefcf5
App: flask/flask3-social-media
App directory: apps/flask/flask3-social-media
Workbench branch: wizard-ci-ceefcf5-flask-flask3-social-media
Wizard branch: main
Examples branch: main
PostHog (MCP) branch: master
Timestamp: 2026-01-20T22:47:58.108Z
Duration: 472.8s

[edwinyjlim]

Now I have a complete picture. Let me provide the PR evaluation report:

---

PR Evaluation Report

Summary

This PR integrates PostHog analytics into a Flask 3 social media (microblog) application. It adds comprehensive server-side event tracking across authentication flows, user engagement actions, and API endpoints using the PostHog Python SDK v3+ context-based API. The integration includes exception capture via a global error handler.

Files changed	Lines added	Lines removed
8	+239	-5

Confidence score: 2/5 ❌

• Global error handler breaks existing error pages: The new @app.errorhandler(Exception) handler catches ALL exceptions and returns JSON responses only, completely overriding the existing HTML error pages (404.html, 500.html) that the app already has. This breaks the web UI error handling. [CRITICAL]
• Missing page view tracking: No `` event tracking is implemented for this server-rendered Flask application. For a server-rendered app, page views should be captured on each route hit. [MEDIUM]
• No reverse proxy configured: The integration does not include a reverse proxy configuration to circumvent ad blockers when sending events to PostHog. [MEDIUM]

---

File changes

Filename	Score	Description
app/__init__.py	2/5	Adds PostHog initialization and global exception handler. The exception handler is problematic - it returns JSON for all errors, breaking existing HTML error pages.
app/api/tokens.py	4/5	Adds api_token_generated event capture with proper context-based API usage. Missing event for token revocation.
app/api/users.py	4/5	Adds api_user_created event with user properties. Correct implementation using tag() for user identification. Missing API user update event.
app/auth/routes.py	4/5	Comprehensive auth event tracking: login, logout, signup, password reset. Well-structured with proper user identification.
app/main/routes.py	4/5	Good coverage of user engagement events: posts, follows, messages, search, exports. Missing explore page and notifications tracking.
config.py	5/5	Clean PostHog configuration with environment variables and sensible defaults.
requirements.txt	5/5	Correctly adds posthog>=3.0.0 dependency.
posthog-setup-report.md	4/5	Useful documentation but contains hardcoded API key which should not be committed.

---

App sanity check: 2/5 ❌

Criteria	Result	Description
App builds and runs	Partial	App will run but error handling is broken for web users
Preserves existing env vars & configs	Yes	Existing configuration preserved, new PostHog vars added cleanly
No syntax or type errors	Yes	Code is syntactically correct
Correct imports/exports	Yes	All imports are valid and properly structured
Minimal, focused changes	No	Global error handler is overly broad and changes app behavior

Issues

• Global error handler overrides existing error pages: The new @app.errorhandler(Exception) in app/__init__.py catches all exceptions and returns JSON responses only. The app already has proper error handlers in app/errors/handlers.py that render HTML templates (404.html, 500.html) for web requests and JSON for API requests using wants_json_response(). The new handler completely bypasses this logic, breaking the web UI's error experience. Users will see raw JSON instead of styled error pages. [CRITICAL]
• Database session not rolled back on 500 errors: The existing internal_error handler calls db.session.rollback() to handle transaction cleanup. The new global handler doesn't do this, potentially leaving the database session in an inconsistent state after errors. [CRITICAL]
• Hardcoded API key in documentation: The posthog-setup-report.md file contains what appears to be a real API key (sTMFPsFhdP1Ssg). This should not be committed to version control. [MEDIUM]

Other completed criteria

• Preserves existing blueprint registrations
• Preserves existing Flask extensions initialization order
• PostHog checks are properly guarded with config flags
• No unnecessary file modifications

---

PostHog implementation: 3/5 ⚠️

Criteria	Result	Description
PostHog SDKs installed	Yes	posthog>=3.0.0 added to requirements.txt
PostHog client initialized	Yes	Initialized in create_app() with api_key, host, and debug settings
capture()	Yes	14 distinct events captured across auth, main routes, and API
identify()	Yes	Uses identify_context() within new_context() blocks correctly for Python SDK v7+
Error tracking	Partial	posthog.capture_exception(e) implemented but handler approach is flawed
Reverse proxy	No	No reverse proxy configuration to avoid ad blockers

Issues

• Exception capture breaks existing error handling flow: While capture_exception() is used correctly, the surrounding error handler completely changes app behavior. A better approach would be to integrate PostHog exception capture into the existing error handlers in app/errors/handlers.py rather than creating a new global handler. [CRITICAL]
• No page view tracking: For a server-rendered Flask app, page views should be tracked. This could be done via a @app.before_request hook that captures page views, or by adding capture calls to individual routes. [MEDIUM]
• No reverse proxy setup: Production deployments should use a reverse proxy to avoid ad blockers. Consider documenting nginx/server configuration for proxying PostHog requests. [MEDIUM]
• Missing token revocation tracking: revoke_token() in tokens.py doesn't capture an event. [LOW]

Other completed criteria

• API key loaded from environment variable, not hardcoded in code
• Correct API host configuration (https://us.i.posthog.com)
• POSTHOG_DISABLED flag allows disabling tracking
• Debug mode tied to Flask app debug setting
• Proper use of new_context() and identify_context() pattern for Python SDK v7+
• User properties tagged appropriately (email, username)
• No PII in event properties (IDs are stringified)

---

PostHog insights and events: 4/5 ✅

Filename	PostHog events	Description
app/__init__.py	capturedException	Global exception capture for error tracking
app/auth/routes.py	user_signed_up, user_logged_in, user_logged_out, password_reset_requested, password_reset_completed	Complete authentication lifecycle tracking
app/main/routes.py	post_created, profile_updated, user_followed, user_unfollowed, search_performed, message_sent, posts_exported	Core user engagement events with useful properties
app/api/users.py	api_user_created	API signup tracking with signup_method: api property
app/api/tokens.py	api_token_generated	API authentication token generation tracking

Issues

• Missing failed login tracking: No event for failed login attempts, which is valuable for security monitoring and understanding friction. [MEDIUM]
• Missing explore/notifications tracking: The explore() and notifications() routes don't capture events, missing out on understanding content discovery patterns. [LOW]

Other completed criteria

• Events represent real user actions (signups, logins, content creation, social interactions)
• Events can build useful funnels (signup → first post)
• Events enriched with relevant properties (post_length, language, results_count, recipient_id)
• Event naming is consistent and follows verb_noun pattern
• API vs form signup differentiated via signup_method property
• Search events include query metadata without exposing query content (privacy-conscious)

---

Reviewed by wizard workbench PR evaluator