feat: support SQL_DSN and REDIS_CONN_STRING connection strings #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add compatibility with NewAPI's connection string format: - SQL_DSN: postgresql://user:pass@host:port/db or user:pass@tcp(host:port)/db - REDIS_CONN_STRING: redis://[:password@]host[:port][/db] Connection strings take priority over separate environment variables.
- Redis: add username support for ACL authentication - PostgreSQL/MySQL: URL encode password to handle special characters (@, %, etc.) - MySQL: make port optional in DSN, default to 3306
- Add REFUNDED status enum and refund API endpoint POST /api/top-ups/{id}/refund
- Auto-deduct user quota on refund with atomic transaction to prevent double refunds
- Use GREATEST(0, quota - amount) to prevent negative quota
- Add refunded statistics card and refund confirmation modal in frontend
- Import shared TOKENS_PER_USD constant instead of hardcoding
- Lazy load GeoIP databases on first query instead of startup (saves 60-100MB) - Add capacity limit (1000 entries) and periodic cleanup to SimpleCache - Reduce database connection pool size (pool_size: 3→1, max_overflow: 5→2) - Use context manager for SQLite initialization to prevent connection leaks
- Replace GeoLite2-City with GeoLite2-Country for significant memory savings - City/region info no longer available, only country-level geolocation - Update all related code paths and API response keys
Complete rewrite of Python/FastAPI backend for improved performance: - Core infrastructure: config, MySQL/SQLite, Redis cache, Zap logger - Auth middleware: JWT + API Key dual-mode authentication - 12 business modules: Dashboard, TopUp, Redemption, User, Risk, IP Monitoring, AI Auto Ban, Analytics, Model Status, System, Storage - GeoIP service with lazy loading for IP geolocation - Docker multi-stage build (~20MB production image) - Expected 3-5x performance improvement, 50%+ memory reduction
- Dockerfile: Replace Python/FastAPI with Go backend build - Multi-stage build: Node (frontend) + Go (backend) + Alpine (runtime) - Final image based on Alpine Linux for smaller size - docker-compose.yml: Update environment variables for Go backend - DATABASE_* prefix for database config - AUTH_* prefix for authentication config - REDIS_CONN_STRING support
- Add GetHourlyTrends for hourly usage statistics - Add GetIPDistribution for IP geographic distribution - Add background task framework with TaskManager - Add cache warmup task (Dashboard, IP distribution, leaderboard) - Add log sync task for analytics processing - Add AI auto-ban scan task - Add GeoIP database update task - Add IP recording enforcement task - Add background index creation task - Integrate GeoIP initialization in main.go - Add GeoIP license key config support Go backend feature completeness: ~40-50% → ~90%+
- Add SystemScaleService with automatic scale detection - Detect system scale based on users, logs, and RPM - Support small/medium/large/xlarge scale levels - Provide recommended cache TTL settings per scale - Add three-layer CacheManager (Redis + SQLite) - L1: Redis for high-performance hot cache - L2: SQLite for local persistent cache - Support cache restoration from SQLite to Redis - Add incremental slot cache support - Time slot caching for 3d/7d/14d periods - Slot-based data aggregation - Missing slot detection and warmup - Update warmup status handler to use tasks package
数据库中 logs.created_at 字段为 bigint (Unix 时间戳)， 但代码使用 time.Time 和字符串日期格式比较，导致 PostgreSQL 报错： - date_trunc(unknown, bigint) does not exist - invalid input syntax for type bigint - function date(bigint) does not exist 修复内容： - 将 models.Log.CreatedAt 从 time.Time 改为 int64 - 所有 service 层查询改用 Unix 时间戳比较 - 日期函数改用 TO_TIMESTAMP/FROM_UNIXTIME 转换 - 移除 channels 表不存在的 deleted_at 条件
- 修复 warmup-status 接口返回格式，添加 status/progress/message/steps 字段 - 补全 Storage 模块缺失接口：config/:key、cache/info、cache/stats 等 8 个 - 补全 Analytics 模块缺失接口：batch、sync-status、check-consistency - 补全 TopUp 模块缺失接口：/:id 获取单个充值记录 - 添加 Model Status embed 公开接口（无需认证）共 6 个 - 添加 /api/ip/* 别名路由兼容 Python 版本路径 - 完善 Dashboard GetSystemInfo 使用 runtime 包获取真实系统信息 - 完善 AI Ban 服务添加审计日志、白名单搜索、连接测试等方法
- 将 model-status/embed 路由移到 authenticated 组外部，确保公开访问 - 为空实现方法添加 TODO 注释标记待完善功能
- 批量删除用户接口：改用 activity_level + dry_run 参数（与 Python 版本一致） - 用户风险分析接口：修复路由参数名 id → user_id - AI 封禁评估接口：从 URL 参数改为 request body 获取 user_id - 用户列表响应格式：records → items，添加 total_pages 字段 - 封禁记录响应格式：records → items，添加 page_size 和 total_pages - 白名单搜索参数：支持 q 和 keyword 两种参数名 - 修复 page_size=0 时的除零 panic 问题
- TopUpListResult: records → items, 添加 total_pages - RedemptionListResult: records → items, 添加 total_pages - 与前端期望的响应格式保持一致
- 将 status 字段从 int 改为 string 类型，与数据库一致 - 将 created_at/updated_at 改为 create_time/complete_time (Unix 时间戳) - 添加 money 字段，修正 payment_method 列名映射 - 更新 TopUp 状态常量为字符串类型 - 修复 IsSuccess/IsRefunded 方法支持多种状态值格式 - 更新 topup service 查询逻辑适配新字段类型
- formatNumber 函数添加 null/undefined 检查 - getMaxValue 函数处理数组中的空值 - 修复模型请求数百分比计算的空值问题 - 修复 KingCard 组件中 request_count 和 quota_used 的可选链访问 - StatCard value 添加类型检查避免 toLocaleString 调用失败
- 新增 frontend/embed.go 使用 go:embed 嵌入前端构建产物 - 实现 ServeFrontend 函数提供静态文件服务和 SPA 路由支持 - 智能区分 API 路由和前端路由，API 404 返回 JSON 格式 - main.go 添加版本信息变量（构建时注入） - 启动日志输出版本、构建时间和 Git commit 信息 - 路由末尾注册前端服务作为 fallback
- Dockerfile 从 Nginx+Supervisor 多进程改为单 Go 二进制 - 使用 Bun 替代 npm 加速前端构建（快 10-100 倍） - 统一服务端口从 8000/1145 改为 3000 - docker-compose.yml 添加资源限制和健康检查优化 - 移除 FRONTEND_PORT 配置，使用统一的 PORT 变量 - 镜像体积大幅减小（移除 nginx/supervisor/curl）
- 新增根目录 Makefile 统一管理前后端构建流程 - 新增 backend-go/Makefile 支持热重载开发和版本注入 - 新增 VERSION 文件 (v0.1.0) 用于版本管理 - 新增 .dockerignore 优化 Docker 构建上下文 - 新增 .air.toml 配置 Go 热重载开发环境 - go.mod 将 geoip2-golang 从 indirect 改为 direct 依赖 - 添加 frontend/pnpm-lock.yaml 锁定前端依赖版本
修复 Docker 构建失败问题，统一使用 bun 作为前端包管理器
- User 模型新增 linux_do_id、aff_count、aff_quota、aff_history 字段 - GetUsers 支持通用搜索（用户名、邮箱、linux_do_id、aff_code） - GetInvitedUsers 返回邀请人信息和统计数据，与 Python 版本一致 - UserRecord 新增 linux_do_id 字段
- 新增 8 阶段渐进式缓存预热（restore/check/leaderboard/dashboard/user_activity/ip_monitoring/ip_distribution/model_status） - 新增 CacheCleanupTask（每 1 小时清理过期缓存）和 ModelStatusRefreshTask（每 30 分钟刷新模型状态） - 重构 WarmupStatus 结构，支持更细粒度的进度追踪 - 重构 GetWarmupStatusHandler，直接使用 tasks.WarmupStatus 状态 - 修复预热任务重启时状态未完全重置的问题
- 修复后端 /api/analytics/summary 返回数据结构与前端期望不一致 - 新增 GetFullSummary() 返回完整的 state/ranking/statistics 数据 - 修复 analytics_state 表无限增长问题（改用 upsert 模式） - 新增 Legacy Analytics 方法支持前端同步状态显示 - 新增 analytics_meta 表存储初始化截止点配置
将 redeemed_user_id 改为 used_user_id，与数据库实际 schema 保持一致
processLogsWithCutoff 函数中的 GORM 查询缺少 Model() 调用， 导致查询无法正确执行，日志处理始终返回 0 条记录
Python 版本使用 key-value 结构 (key TEXT PRIMARY KEY, value INTEGER)， Go 版本之前使用的是行结构 (id, last_processed_id, ...)，导致 SQL 错误。 修改内容： - database.go: 表结构改为 key-value 格式 - analytics.go: getStateFromDB/updateStateInDB 适配 key-value 结构 - background.go: GetState/ProcessNewLogs 适配 key-value 结构
- Change time fields from time.Time to int64 Unix timestamps (created_time, expired_time, redeemed_time) - Rename redeemed_by to used_user_id to match upstream schema - Add count field for redemption usage tracking - Remove deleted_at field not present in upstream
…SQLite - Add UpsertSQL() and UpsertWithIncrement() helper functions - Support MySQL ON DUPLICATE KEY UPDATE syntax - Support PostgreSQL/SQLite ON CONFLICT DO UPDATE syntax - Export GetLocalDBEngine() for engine detection - Change default local_db_path to ./data/local.db
Replace hardcoded SQLite ON CONFLICT syntax with UpsertSQL helper
…ility Replace hardcoded INSERT OR REPLACE syntax with UpsertSQL/UpsertWithIncrement helpers
- Add github.com/joho/godotenv dependency - Auto-load .env file on config initialization
- Add dry_run mode for safe testing - Add API key, base_url, model configuration - Add custom_prompt support - Add excluded_models/excluded_groups filtering - Add whitelist_ips/blacklist_ips for AI context - Add API health status tracking
- Add hardDelete option to BatchDeleteUsersByActivity - Add inactive activity level support - Add GetSoftDeletedUsersCount endpoint - Add PurgeSoftDeletedUsers endpoint for physical deletion - Register new routes: GET/POST /users/soft-deleted/*
- Comment out separated config vars, recommend connection string - Document recent database and config changes
Add new API endpoints compatible with uptime-kuma format for model status
monitoring, implemented in both Python and Go backends:
- /api/uptime-kuma/monitors - list all model monitors
- /api/uptime-kuma/monitors/{model_name} - get monitor with heartbeats
- /api/uptime-kuma/heartbeats/{model_name} - get heartbeat history
- /api/uptime-kuma/status-page - status page data
- /api/uptime-kuma/status-page/batch - batch status page data
- /api/uptime-kuma/overall - overall status summary
- /api/uptime-kuma/push/{push_token} - push monitor compatibility
Status mapping based on success_rate:
- UP(1): >= 95% or no requests
- PENDING(2): 80-95%
- DOWN(0): < 80%
All endpoints are public (no auth required) for embedding in status pages.
Uses stable MD5-based monitorID and UTC timestamps with timezone info.
Change API path from /api/uptime-kuma/* to /api/status-page/* to match the official uptime-kuma API format: - GET /api/status-page/:slug - status page config and monitor list - GET /api/status-page/heartbeat/:slug - heartbeat data - GET /api/status-page/:slug/badge - badge data (custom) - GET /api/status-page/:slug/summary - summary data (custom) This allows direct integration with services that expect uptime-kuma compatible endpoints. The slug parameter can be any value (e.g., "uptime-kuma").
- Fix field name mismatch (requests/quota vs request_count/quota_used) - Display Top 5 users instead of only the first place - Add ranking numbers and compact list layout
The /api/risk/users/{id}/analysis endpoint was missing fields expected
by the frontend, causing blank page due to JS errors when accessing
undefined properties.
Added:
- risk object with requests_per_minute, avg_quota_per_request, risk_flags, ip_switch_analysis
- recent_logs array with last 10 log entries
- avg_ip_duration and switch_details fields to IPSwitchAnalysis struct
The logs table has channel_id field, not channel. The incorrect field reference caused the query to fail silently, returning empty results.
The frontend expects activity_level field to display user activity badges. Added calculateActivityLevel function to compute activity based on last request time: - active: within 7 days - inactive: 7-30 days - very_inactive: over 30 days - never: no requests
Convert numeric status (1=unused, 3=used) to string status (unused/used/expired) in API response to match frontend expectations. Previously, status=3 was incorrectly displayed as "已过期" instead of "已使用" because frontend expected string comparison.
Change field names and types to match frontend interface: - created_at (string) -> created_time (int64) - redeemed_at (string) -> redeemed_time (int64) - Add expired_time field Frontend expects Unix timestamps with _time suffix, not formatted date strings with _at suffix.
- Add Tooltip on "已使用" badge showing redeemer name - Click badge opens user behavior analysis dialog - Display user quota/used_quota/remaining in analysis dialog - Add quota and used_quota fields to user analysis API response - Create reusable Tooltip UI component
P0 fixes: - Make redeemed_by/redeemer_name optional in RedemptionCode interface - Tooltip: merge child event handlers instead of overwriting - Tooltip: add keyboard focus/blur support and aria attributes P1 fixes: - Clear selectedUser/analysis state when closing dialog - Check response.ok before parsing JSON in fetchUserAnalysis - Extract QUOTA_PER_USD constant (500000) for quota conversion
- Add shared UserAnalysisDialog component for consistent user behavior analysis - Add shared types/user.ts for UserAnalysis interface and constants - TopUps page: make username clickable to open analysis dialog - Dashboard page: make ranking usernames clickable (Request King & Top Spender) - Redemptions page: refactor to use shared component, reducing ~250 lines of duplicate code - Support time window switching (1h/3h/6h/12h/24h/3d/7d) - Display account quota, risk flags, request stats, model preferences, IP sources, and recent traces - Fix RISK_FLAG_LABELS key naming to match existing UI code (uppercase format) - Update CHANGELOG.md with feature description
修复用户额度显示逻辑错误： - 数据库 quota 字段存储的是剩余额度，而非总额度 - 总额度 = quota + used_quota 变更： 1. UserAnalysisDialog: 修复账户额度计算（quota + used_quota） 2. UserAnalysisDialog: 修复剩余额度显示（直接使用 quota） 3. UserManagement: 优化表头标签（"额度" → "剩余额度"） 问题：用户分析页面显示剩余额度为负数（-$1.00） 根因：错误计算 quota - used_quota，实际 quota 已是剩余额度
Major dependency upgrades (Plan B - aggressive upgrade): ## Core Framework Updates - Vite: 5.0.8 → 7.3.1 (2 major versions) - @vitejs/plugin-react: 4.2.1 → 5.1.2 - Tailwind CSS: 3.4.0 → 4.1.18 (complete rewrite) - ESLint: 8.55.0 → 9.39.2 (Flat Config) - TypeScript ESLint: 6.14.0 → 8.53.0 - eslint-plugin-react-hooks: 4.6.0 → 7.0.1 ## Configuration Migrations - Migrate Tailwind config from JS to CSS (@theme directive) - Add @plugin "tailwindcss-animate" for animation utilities - Create eslint.config.js (ESLint 9 Flat Config format) - Add @tailwindcss/vite plugin integration - Remove postcss.config.js and tailwind.config.js ## Breaking Changes Addressed - Fix @apply directive compatibility in src/index.css - Map CSS variables to Tailwind theme tokens - Add Node.js >= 20 engine requirement - Update .gitignore to exclude .backup files ## Code Fixes - Fix unused variable errors (catch _e pattern) - Fix expression statement errors - Adjust ESLint rules for React 19 compatibility Performance improvements: - Tailwind incremental builds 100x faster - Vite cold start and HMR optimization
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
SQL_DSN和REDIS_CONN_STRING优先于分离的环境变量支持的格式
数据库 (SQL_DSN)
postgresql://user:pass@host:port/dbnameuser:pass@tcp(host:port)/dbnameRedis (REDIS_CONN_STRING)
redis://host或redis://:password@host:6379/0Test plan