diff --git a/README.ja.md b/README.ja.md index 67d425260..0d01ba175 100644 --- a/README.ja.md +++ b/README.ja.md @@ -178,7 +178,19 @@ bash start.sh --- **注意**: アプリケーションを実行する前に、すべての前提条件がインストールされ、環境変数が適切に設定されていることを確認してください。 -長期間更新がない場合は、プロジェクト内のデータベースファイル(`lancedb/`、`valuecell.db`、`.knowledgebase/`)を削除してから再起動できます。 +長期間更新がない場合は、ローカルデータを削除して再起動できます: +- LanceDB ディレクトリ(システムアプリディレクトリに保存。`.env` と同じ場所): + - macOS: `~/Library/Application Support/ValueCell/lancedb` + - Linux: `~/.config/valuecell/lancedb` + - Windows: `%APPDATA%\\ValueCell\\lancedb` +- Knowledge ディレクトリ(システムアプリディレクトリに保存。`.env` と同じ場所): + - macOS: `~/Library/Application Support/ValueCell/.knowledge` + - Linux: `~/.config/valuecell/.knowledge` + - Windows: `%APPDATA%\\ValueCell\\.knowledge` +- SQLite データベースファイル(システムアプリディレクトリに保存。`.env` と同じ場所): + - macOS: `~/Library/Application Support/ValueCell/valuecell.db` + - Linux: `~/.config/valuecell/valuecell.db` + - Windows: `%APPDATA%\\ValueCell\\valuecell.db` # 開発者 diff --git a/README.md b/README.md index 33394e5bc..381fa92af 100644 --- a/README.md +++ b/README.md @@ -181,7 +181,19 @@ Once the application is running, you can explore the web interface to interact w - To ensure your account safety, you need to reset your API keys regularly. --- -**Note**: Before running the application, ensure all prerequisites are installed and environment variables are properly configured. If it has been a long time since the last update, you can delete the database files in the project directories: `lancedb/`, `valuecell.db`, `.knowledgebase/` and start fresh +**Note**: Before running the application, ensure all prerequisites are installed and environment variables are properly configured. If it has been a long time since the last update, you can delete local data stores and start fresh: +- LanceDB directory (stored in your system application directory): + - macOS: `~/Library/Application Support/ValueCell/lancedb` + - Linux: `~/.config/valuecell/lancedb` + - Windows: `%APPDATA%\ValueCell\lancedb` +- Knowledge directory (stored in your system application directory): + - macOS: `~/Library/Application Support/ValueCell/.knowledge` + - Linux: `~/.config/valuecell/.knowledge` + - Windows: `%APPDATA%\ValueCell\.knowledge` +- SQLite database file (stored in your system application directory): + - macOS: `~/Library/Application Support/ValueCell/valuecell.db` + - Linux: `~/.config/valuecell/valuecell.db` + - Windows: `%APPDATA%\ValueCell\valuecell.db` # Developers diff --git a/README.zh.md b/README.zh.md index 640c3e7ef..ea56ba504 100644 --- a/README.zh.md +++ b/README.zh.md @@ -182,7 +182,19 @@ bash start.sh --- **注意**:运行应用程序前,请确保所有前提条件已安装且环境变量已正确配置 -如长时间没有更新可以删除项目中数据库文件`lancedb/`,`valuecell.db`, `.knowledgebase/`再进行启动 +如长时间没有更新,可以删除本地数据并重新启动: +- LanceDB 目录(位于系统应用目录,与 `.env` 同路径): + - macOS: `~/Library/Application Support/ValueCell/lancedb` + - Linux: `~/.config/valuecell/lancedb` + - Windows: `%APPDATA%\\ValueCell\\lancedb` +- 知识目录(位于系统应用目录,与 `.env` 同路径): + - macOS: `~/Library/Application Support/ValueCell/.knowledge` + - Linux: `~/.config/valuecell/.knowledge` + - Windows: `%APPDATA%\\ValueCell\\.knowledge` +- SQLite 数据库文件(位于系统应用目录,与 `.env` 同路径): + - macOS: `~/Library/Application Support/ValueCell/valuecell.db` + - Linux: `~/.config/valuecell/valuecell.db` + - Windows: `%APPDATA%\\ValueCell\\valuecell.db` # 开发者 diff --git a/README.zh_Hant.md b/README.zh_Hant.md index 5444231fe..812dde29c 100644 --- a/README.zh_Hant.md +++ b/README.zh_Hant.md @@ -179,7 +179,19 @@ bash start.sh --- **注意**:啟動應用程式前,請確認所有前置條件已安裝且環境變數正確設定。 -若長時間未有更新,可以刪除專案中的資料庫檔案 `lancedb/`、`valuecell.db`、`.knowledgebase/` 後重新啟動。 +若長時間未有更新,可以刪除本地資料並重新啟動: +- LanceDB 目錄(位於系統應用程式目錄,與 `.env` 同路徑): + - macOS: `~/Library/Application Support/ValueCell/lancedb` + - Linux: `~/.config/valuecell/lancedb` + - Windows: `%APPDATA%\\ValueCell\\lancedb` +- 知識目錄(位於系統應用程式目錄,與 `.env` 同路徑): + - macOS: `~/Library/Application Support/ValueCell/.knowledge` + - Linux: `~/.config/valuecell/.knowledge` + - Windows: `%APPDATA%\\ValueCell\\.knowledge` +- SQLite 資料庫檔案(位於系統應用程式目錄,與 `.env` 同路徑): + - macOS: `~/Library/Application Support/ValueCell/valuecell.db` + - Linux: `~/.config/valuecell/valuecell.db` + - Windows: `%APPDATA%\\ValueCell\\valuecell.db` # 開發者 diff --git a/docs/CONFIGURATION_GUIDE.md b/docs/CONFIGURATION_GUIDE.md index 321d943d8..fdb0b6e51 100644 --- a/docs/CONFIGURATION_GUIDE.md +++ b/docs/CONFIGURATION_GUIDE.md @@ -58,10 +58,19 @@ bash start.sh The system will auto-detect available providers based on configured API keys. -> **Note**: If you get database compatibility errors, delete these directories: -> - `lancedb/` -> - `valuecell.db` -> - `.knowledgebase` +> **Note**: If you get database compatibility errors, delete these locations: +> - LanceDB directory (system application directory, same as `.env`): +> - macOS: `~/Library/Application Support/ValueCell/lancedb` +> - Linux: `~/.config/valuecell/lancedb` +> - Windows: `%APPDATA%\\ValueCell\\lancedb` +> - Knowledge directory (system application directory, same as `.env`): +> - macOS: `~/Library/Application Support/ValueCell/.knowledge` +> - Linux: `~/.config/valuecell/.knowledge` +> - Windows: `%APPDATA%\\ValueCell\\.knowledge` +> - SQLite database file (system application directory, same as `.env`): +> - macOS: `~/Library/Application Support/ValueCell/valuecell.db` +> - Linux: `~/.config/valuecell/valuecell.db` +> - Windows: `%APPDATA%\\ValueCell\\valuecell.db` --- diff --git a/python/valuecell/core/coordinate/tests/test_e2e_persistence.py b/python/valuecell/core/coordinate/tests/test_e2e_persistence.py index 231a93056..6ecc5e8a9 100644 --- a/python/valuecell/core/coordinate/tests/test_e2e_persistence.py +++ b/python/valuecell/core/coordinate/tests/test_e2e_persistence.py @@ -11,7 +11,8 @@ @pytest.mark.asyncio async def test_orchestrator_buffer_store_e2e(tmp_path, monkeypatch): db_path = tmp_path / "e2e_valuecell.db" - monkeypatch.setenv("VALUECELL_SQLITE_DB", str(db_path)) + # Use DATABASE_URL for tests; utils resolve_db_path will honor sqlite DSN + monkeypatch.setenv("VALUECELL_DATABASE_URL", f"sqlite:///{db_path}") monkeypatch.setattr( factory_mod, "create_model", lambda *args, **kwargs: "stub-model" diff --git a/python/valuecell/server/config/settings.py b/python/valuecell/server/config/settings.py index 2e4805c89..74d1aa0b1 100644 --- a/python/valuecell/server/config/settings.py +++ b/python/valuecell/server/config/settings.py @@ -4,6 +4,8 @@ from functools import lru_cache from pathlib import Path +from valuecell.utils.env import get_system_env_dir + def _get_project_root() -> str: """Get project root directory path. @@ -17,9 +19,15 @@ def _get_project_root() -> str: def _default_db_path() -> str: - """Get default database path in project root.""" - repo_root = _get_project_root() - return f"sqlite:///{os.path.join(repo_root, 'valuecell.db')}" + """Get default database DSN under the system application directory. + + Mirrors `.env` location so the SQLite file lives alongside user-level config: + - macOS: `~/Library/Application Support/ValueCell/valuecell.db` + - Linux: `~/.config/valuecell/valuecell.db` + - Windows: `%APPDATA%\ValueCell\valuecell.db` + """ + system_dir = get_system_env_dir() + return f"sqlite:///{os.path.join(str(system_dir), 'valuecell.db')}" class Settings: @@ -42,7 +50,13 @@ def __init__(self): self.CORS_ORIGINS = cors_origins.split(",") if cors_origins != "*" else ["*"] # Database Configuration - self.DATABASE_URL = os.getenv("VALUECELL_SQLITE_DB", _default_db_path()) + # Prefer `VALUECELL_DATABASE_URL` if provided; otherwise use system application directory default. + env_db = os.getenv("VALUECELL_DATABASE_URL") + if env_db: + # If it's already a full DSN (sqlite or other), use as-is + self.DATABASE_URL = env_db + else: + self.DATABASE_URL = _default_db_path() # File Paths self.BASE_DIR = Path(__file__).parent.parent.parent diff --git a/python/valuecell/server/db/README.md b/python/valuecell/server/db/README.md index 526609a6a..db280c61f 100644 --- a/python/valuecell/server/db/README.md +++ b/python/valuecell/server/db/README.md @@ -21,8 +21,11 @@ db/ Database configuration is defined in `valuecell/server/config/settings.py`: -- **DATABASE_URL**: Database connection URL, defaults to `sqlite:///./valuecell.db` -- **DB_ECHO**: Whether to output SQL logs, defaults to `false` +- `VALUECELL_DATABASE_URL`: Database connection URL. Defaults to a SQLite file under the system application directory (same place as `.env`): + - macOS: `sqlite:///~/Library/Application Support/ValueCell/valuecell.db` + - Linux: `sqlite:///~/.config/valuecell/valuecell.db` + - Windows: `sqlite:///%APPDATA%/ValueCell/valuecell.db` +- `DB_ECHO`: Whether to output SQL logs, defaults to `false` ## Database Models @@ -224,7 +227,7 @@ else: 1. **Password Security**: Default admin user password is a placeholder and should be replaced with proper hashed password in production environment 2. **Database Backup**: SQLite database file should be backed up regularly 3. **Permission Management**: Ensure database file has appropriate filesystem permissions -4. **Environment Variables**: Database connection can be customized through `DATABASE_URL` environment variable +4. **Environment Variables**: Database connection can be customized through `VALUECELL_DATABASE_URL` environment variable ## Troubleshooting @@ -239,8 +242,13 @@ else: If you need to completely reset the database: ```bash -# Delete existing database file -rm valuecell.db +# Delete existing database file (choose your OS path) +# macOS +rm -f "$HOME/Library/Application Support/ValueCell/valuecell.db" +# Linux +rm -f "$HOME/.config/valuecell/valuecell.db" +# Windows (PowerShell) +Remove-Item "$env:APPDATA\ValueCell\valuecell.db" -ErrorAction SilentlyContinue # Re-initialize python3 -m valuecell.server.db.init_db diff --git a/python/valuecell/server/db/init_db.py b/python/valuecell/server/db/init_db.py index b0a9f9f95..28e3a9c3b 100644 --- a/python/valuecell/server/db/init_db.py +++ b/python/valuecell/server/db/init_db.py @@ -2,6 +2,7 @@ import json import logging +import shutil import sys from pathlib import Path from typing import Optional @@ -112,9 +113,25 @@ def create_database_file(self) -> bool: # Create parent directories if they don't exist db_path.parent.mkdir(parents=True, exist_ok=True) - # Create empty database file - db_path.touch() - logger.info(f"Created database file: {db_path}") + # If old repo-root DB exists and new system-path DB is missing, migrate it + try: + repo_root = ( + Path(__file__).resolve().parent.parent.parent.parent.parent + ) + old_repo_db = repo_root / "valuecell.db" + except Exception: + old_repo_db = None + + if old_repo_db and old_repo_db.exists() and not db_path.exists(): + shutil.copy2(old_repo_db, db_path) + logger.info( + f"Migrated existing database file from repo root to system directory: {db_path}" + ) + + # Ensure database file exists + if not db_path.exists(): + db_path.touch() + logger.info(f"Created database file: {db_path}") return True except Exception as e: diff --git a/python/valuecell/utils/db.py b/python/valuecell/utils/db.py index 16c2138e9..edba1be81 100644 --- a/python/valuecell/utils/db.py +++ b/python/valuecell/utils/db.py @@ -1,15 +1,68 @@ import os +import shutil +from pathlib import Path +from .env import get_system_env_dir from .path import get_repo_root_path +def _strip_sqlite_prefix(url_or_path: str) -> str: + """Normalize a potential SQLite DSN to a filesystem path. + + - If `url_or_path` starts with `sqlite:///`, return the stripped path portion. + - Otherwise, return it unchanged. + """ + if url_or_path.startswith("sqlite:///"): + return url_or_path.replace("sqlite:///", "", 1) + return url_or_path + + def resolve_db_path() -> str: - return os.environ.get("VALUECELL_SQLITE_DB") or os.path.join( - get_repo_root_path(), "valuecell.db" - ) + """Resolve the SQLite database file path used by conversation stores. + + Resolution order: + 1) `DATABASE_URL` env var (if starts with `sqlite:///`, strip to path; otherwise ignore) + 2) Default to system application directory (e.g., `~/Library/Application Support/ValueCell/valuecell.db` on macOS) + + Note: This function returns a filesystem path, not a SQLAlchemy DSN. + """ + # Prefer generic VALUECELL_DATABASE_URL if it points to SQLite + db_url = os.environ.get("VALUECELL_DATABASE_URL") + if db_url and db_url.startswith("sqlite:///"): + return _strip_sqlite_prefix(db_url) + + # Default: store under system application directory alongside `.env` + return os.path.join(get_system_env_dir(), "valuecell.db") def resolve_lancedb_uri() -> str: - return os.environ.get("VALUECELL_LANCEDB_URI") or os.path.join( - get_repo_root_path(), "lancedb" - ) + """Resolve LanceDB directory path. + + Resolution order: + 1) Default to system application directory: `/lancedb` + + Additionally, if an old repo-root `lancedb` directory exists and the new + system directory does not, migrate the contents once for continuity. + """ + # Default: system directory + system_dir = get_system_env_dir() + new_path = Path(system_dir) / "lancedb" + new_path.mkdir(parents=True, exist_ok=True) + + # Migrate from old repo-root location if needed + old_path = Path(get_repo_root_path()) / "lancedb" + try: + if old_path.exists() and not any(new_path.iterdir()): + # Copy contents only if destination is empty + for item in old_path.iterdir(): + src = item + dst = new_path / item.name + if item.is_dir(): + shutil.copytree(src, dst, dirs_exist_ok=True) + else: + shutil.copy2(src, dst) + except Exception: + # Non-fatal: if migration fails, just proceed with new_path + pass + + return str(new_path) diff --git a/python/valuecell/utils/path.py b/python/valuecell/utils/path.py index 61d2da619..26a6cb045 100644 --- a/python/valuecell/utils/path.py +++ b/python/valuecell/utils/path.py @@ -1,6 +1,16 @@ +"""Utilities for resolving Python package and repository root paths. + +Note: Database default location logic moved to `valuecell.utils.db.resolve_db_path`, +which defaults to storing `valuecell.db` in the system application directory +(same path as `.env`). +""" + import os +import shutil from pathlib import Path +from .env import get_system_env_dir + def get_python_root_path() -> str: """ @@ -28,10 +38,11 @@ def get_python_root_path() -> str: def get_repo_root_path() -> str: - """Resolve repository root and return default DB path valuecell.db. + """ + Resolve repository root directory path. - Layout assumption: this file is at repo_root/python/valuecell/utils/db.py - We walk up 3 levels to reach repo_root. + Assumes this file is at `repo_root/python/valuecell/utils/path.py`. + Walk up three levels to reach `repo_root`. """ here = os.path.dirname(__file__) repo_root = os.path.abspath(os.path.join(here, "..", "..", "..")) @@ -52,11 +63,28 @@ def get_agent_card_path() -> str: def get_knowledge_path() -> str: """ - Returns the path to the knowledge directory located in the project root. + Resolve the Knowledge directory path under the system application directory. - Returns: - str: Absolute path of the knowledge directory + Behavior: + - Default location: `/.knowledge` (same base dir as `.env`) + - One-time migration: if old repo-root `.knowledge` exists and the system dir + is empty, copy the contents for continuity. """ - root_path = get_repo_root_path() - knowledge_path = Path(root_path) / ".knowledge" - return str(knowledge_path) + new_path = Path(get_system_env_dir()) / ".knowledge" + new_path.mkdir(parents=True, exist_ok=True) + + old_path = Path(get_repo_root_path()) / ".knowledge" + try: + if old_path.exists() and not any(new_path.iterdir()): + for item in old_path.iterdir(): + src = item + dst = new_path / item.name + if item.is_dir(): + shutil.copytree(src, dst, dirs_exist_ok=True) + else: + shutil.copy2(src, dst) + except Exception: + # Non-fatal: proceed with new_path even if migration fails + pass + + return str(new_path)