Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,19 @@ bash start.sh

---
**注意**: アプリケーションを実行する前に、すべての前提条件がインストールされ、環境変数が適切に設定されていることを確認してください。
長期間更新がない場合は、プロジェクト内のデータベースファイル(`lancedb/`、`valuecell.db`、`.knowledgebase/`)を削除してから再起動できます。
長期間更新がない場合は、ローカルデータを削除して再起動できます:
- LanceDB ディレクトリ(システムアプリディレクトリに保存。`.env` と同じ場所):
- macOS: `~/Library/Application Support/ValueCell/lancedb`
- Linux: `~/.config/valuecell/lancedb`
- Windows: `%APPDATA%\\ValueCell\\lancedb`
- Knowledge ディレクトリ(システムアプリディレクトリに保存。`.env` と同じ場所):
- macOS: `~/Library/Application Support/ValueCell/.knowledge`
- Linux: `~/.config/valuecell/.knowledge`
- Windows: `%APPDATA%\\ValueCell\\.knowledge`
- SQLite データベースファイル(システムアプリディレクトリに保存。`.env` と同じ場所):
- macOS: `~/Library/Application Support/ValueCell/valuecell.db`
- Linux: `~/.config/valuecell/valuecell.db`
- Windows: `%APPDATA%\\ValueCell\\valuecell.db`


# 開発者
Expand Down
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,19 @@ Once the application is running, you can explore the web interface to interact w
- To ensure your account safety, you need to reset your API keys regularly.

---
**Note**: Before running the application, ensure all prerequisites are installed and environment variables are properly configured. If it has been a long time since the last update, you can delete the database files in the project directories: `lancedb/`, `valuecell.db`, `.knowledgebase/` and start fresh
**Note**: Before running the application, ensure all prerequisites are installed and environment variables are properly configured. If it has been a long time since the last update, you can delete local data stores and start fresh:
- LanceDB directory (stored in your system application directory):
- macOS: `~/Library/Application Support/ValueCell/lancedb`
- Linux: `~/.config/valuecell/lancedb`
- Windows: `%APPDATA%\ValueCell\lancedb`
- Knowledge directory (stored in your system application directory):
- macOS: `~/Library/Application Support/ValueCell/.knowledge`
- Linux: `~/.config/valuecell/.knowledge`
- Windows: `%APPDATA%\ValueCell\.knowledge`
- SQLite database file (stored in your system application directory):
- macOS: `~/Library/Application Support/ValueCell/valuecell.db`
- Linux: `~/.config/valuecell/valuecell.db`
- Windows: `%APPDATA%\ValueCell\valuecell.db`

# Developers

Expand Down
14 changes: 13 additions & 1 deletion README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,19 @@ bash start.sh
---

**注意**:运行应用程序前,请确保所有前提条件已安装且环境变量已正确配置
如长时间没有更新可以删除项目中数据库文件`lancedb/`,`valuecell.db`, `.knowledgebase/`再进行启动
如长时间没有更新,可以删除本地数据并重新启动:
- LanceDB 目录(位于系统应用目录,与 `.env` 同路径):
- macOS: `~/Library/Application Support/ValueCell/lancedb`
- Linux: `~/.config/valuecell/lancedb`
- Windows: `%APPDATA%\\ValueCell\\lancedb`
- 知识目录(位于系统应用目录,与 `.env` 同路径):
- macOS: `~/Library/Application Support/ValueCell/.knowledge`
- Linux: `~/.config/valuecell/.knowledge`
- Windows: `%APPDATA%\\ValueCell\\.knowledge`
- SQLite 数据库文件(位于系统应用目录,与 `.env` 同路径):
- macOS: `~/Library/Application Support/ValueCell/valuecell.db`
- Linux: `~/.config/valuecell/valuecell.db`
- Windows: `%APPDATA%\\ValueCell\\valuecell.db`


# 开发者
Expand Down
14 changes: 13 additions & 1 deletion README.zh_Hant.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,19 @@ bash start.sh
---

**注意**:啟動應用程式前,請確認所有前置條件已安裝且環境變數正確設定。
若長時間未有更新,可以刪除專案中的資料庫檔案 `lancedb/`、`valuecell.db`、`.knowledgebase/` 後重新啟動。
若長時間未有更新,可以刪除本地資料並重新啟動:
- LanceDB 目錄(位於系統應用程式目錄,與 `.env` 同路徑):
- macOS: `~/Library/Application Support/ValueCell/lancedb`
- Linux: `~/.config/valuecell/lancedb`
- Windows: `%APPDATA%\\ValueCell\\lancedb`
- 知識目錄(位於系統應用程式目錄,與 `.env` 同路徑):
- macOS: `~/Library/Application Support/ValueCell/.knowledge`
- Linux: `~/.config/valuecell/.knowledge`
- Windows: `%APPDATA%\\ValueCell\\.knowledge`
- SQLite 資料庫檔案(位於系統應用程式目錄,與 `.env` 同路徑):
- macOS: `~/Library/Application Support/ValueCell/valuecell.db`
- Linux: `~/.config/valuecell/valuecell.db`
- Windows: `%APPDATA%\\ValueCell\\valuecell.db`


# 開發者
Expand Down
17 changes: 13 additions & 4 deletions docs/CONFIGURATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,19 @@ bash start.sh

The system will auto-detect available providers based on configured API keys.

> **Note**: If you get database compatibility errors, delete these directories:
> - `lancedb/`
> - `valuecell.db`
> - `.knowledgebase`
> **Note**: If you get database compatibility errors, delete these locations:
> - LanceDB directory (system application directory, same as `.env`):
> - macOS: `~/Library/Application Support/ValueCell/lancedb`
> - Linux: `~/.config/valuecell/lancedb`
> - Windows: `%APPDATA%\\ValueCell\\lancedb`
> - Knowledge directory (system application directory, same as `.env`):
> - macOS: `~/Library/Application Support/ValueCell/.knowledge`
> - Linux: `~/.config/valuecell/.knowledge`
> - Windows: `%APPDATA%\\ValueCell\\.knowledge`
> - SQLite database file (system application directory, same as `.env`):
> - macOS: `~/Library/Application Support/ValueCell/valuecell.db`
> - Linux: `~/.config/valuecell/valuecell.db`
> - Windows: `%APPDATA%\\ValueCell\\valuecell.db`

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
@pytest.mark.asyncio
async def test_orchestrator_buffer_store_e2e(tmp_path, monkeypatch):
db_path = tmp_path / "e2e_valuecell.db"
monkeypatch.setenv("VALUECELL_SQLITE_DB", str(db_path))
# Use DATABASE_URL for tests; utils resolve_db_path will honor sqlite DSN
monkeypatch.setenv("VALUECELL_DATABASE_URL", f"sqlite:///{db_path}")

monkeypatch.setattr(
factory_mod, "create_model", lambda *args, **kwargs: "stub-model"
Expand Down
22 changes: 18 additions & 4 deletions python/valuecell/server/config/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
from functools import lru_cache
from pathlib import Path

from valuecell.utils.env import get_system_env_dir


def _get_project_root() -> str:
"""Get project root directory path.
Expand All @@ -17,9 +19,15 @@ def _get_project_root() -> str:


def _default_db_path() -> str:
"""Get default database path in project root."""
repo_root = _get_project_root()
return f"sqlite:///{os.path.join(repo_root, 'valuecell.db')}"
"""Get default database DSN under the system application directory.

Mirrors `.env` location so the SQLite file lives alongside user-level config:
- macOS: `~/Library/Application Support/ValueCell/valuecell.db`
- Linux: `~/.config/valuecell/valuecell.db`
- Windows: `%APPDATA%\ValueCell\valuecell.db`
"""
system_dir = get_system_env_dir()
return f"sqlite:///{os.path.join(str(system_dir), 'valuecell.db')}"


class Settings:
Expand All @@ -42,7 +50,13 @@ def __init__(self):
self.CORS_ORIGINS = cors_origins.split(",") if cors_origins != "*" else ["*"]

# Database Configuration
self.DATABASE_URL = os.getenv("VALUECELL_SQLITE_DB", _default_db_path())
# Prefer `VALUECELL_DATABASE_URL` if provided; otherwise use system application directory default.
env_db = os.getenv("VALUECELL_DATABASE_URL")
if env_db:
# If it's already a full DSN (sqlite or other), use as-is
self.DATABASE_URL = env_db
else:
self.DATABASE_URL = _default_db_path()

# File Paths
self.BASE_DIR = Path(__file__).parent.parent.parent
Expand Down
18 changes: 13 additions & 5 deletions python/valuecell/server/db/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,11 @@ db/

Database configuration is defined in `valuecell/server/config/settings.py`:

- **DATABASE_URL**: Database connection URL, defaults to `sqlite:///./valuecell.db`
- **DB_ECHO**: Whether to output SQL logs, defaults to `false`
- `VALUECELL_DATABASE_URL`: Database connection URL. Defaults to a SQLite file under the system application directory (same place as `.env`):
- macOS: `sqlite:///~/Library/Application Support/ValueCell/valuecell.db`
- Linux: `sqlite:///~/.config/valuecell/valuecell.db`
- Windows: `sqlite:///%APPDATA%/ValueCell/valuecell.db`
- `DB_ECHO`: Whether to output SQL logs, defaults to `false`

## Database Models

Expand Down Expand Up @@ -224,7 +227,7 @@ else:
1. **Password Security**: Default admin user password is a placeholder and should be replaced with proper hashed password in production environment
2. **Database Backup**: SQLite database file should be backed up regularly
3. **Permission Management**: Ensure database file has appropriate filesystem permissions
4. **Environment Variables**: Database connection can be customized through `DATABASE_URL` environment variable
4. **Environment Variables**: Database connection can be customized through `VALUECELL_DATABASE_URL` environment variable

## Troubleshooting

Expand All @@ -239,8 +242,13 @@ else:
If you need to completely reset the database:

```bash
# Delete existing database file
rm valuecell.db
# Delete existing database file (choose your OS path)
# macOS
rm -f "$HOME/Library/Application Support/ValueCell/valuecell.db"
# Linux
rm -f "$HOME/.config/valuecell/valuecell.db"
# Windows (PowerShell)
Remove-Item "$env:APPDATA\ValueCell\valuecell.db" -ErrorAction SilentlyContinue

# Re-initialize
python3 -m valuecell.server.db.init_db
Expand Down
23 changes: 20 additions & 3 deletions python/valuecell/server/db/init_db.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import json
import logging
import shutil
import sys
from pathlib import Path
from typing import Optional
Expand Down Expand Up @@ -112,9 +113,25 @@ def create_database_file(self) -> bool:
# Create parent directories if they don't exist
db_path.parent.mkdir(parents=True, exist_ok=True)

# Create empty database file
db_path.touch()
logger.info(f"Created database file: {db_path}")
# If old repo-root DB exists and new system-path DB is missing, migrate it
try:
repo_root = (
Path(__file__).resolve().parent.parent.parent.parent.parent
)
old_repo_db = repo_root / "valuecell.db"
except Exception:
old_repo_db = None

if old_repo_db and old_repo_db.exists() and not db_path.exists():
shutil.copy2(old_repo_db, db_path)
logger.info(
f"Migrated existing database file from repo root to system directory: {db_path}"
)

# Ensure database file exists
if not db_path.exists():
db_path.touch()
logger.info(f"Created database file: {db_path}")
return True

except Exception as e:
Expand Down
65 changes: 59 additions & 6 deletions python/valuecell/utils/db.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,68 @@
import os
import shutil
from pathlib import Path

from .env import get_system_env_dir
from .path import get_repo_root_path


def _strip_sqlite_prefix(url_or_path: str) -> str:
"""Normalize a potential SQLite DSN to a filesystem path.

- If `url_or_path` starts with `sqlite:///`, return the stripped path portion.
- Otherwise, return it unchanged.
"""
if url_or_path.startswith("sqlite:///"):
return url_or_path.replace("sqlite:///", "", 1)
return url_or_path


def resolve_db_path() -> str:
return os.environ.get("VALUECELL_SQLITE_DB") or os.path.join(
get_repo_root_path(), "valuecell.db"
)
"""Resolve the SQLite database file path used by conversation stores.

Resolution order:
1) `DATABASE_URL` env var (if starts with `sqlite:///`, strip to path; otherwise ignore)
2) Default to system application directory (e.g., `~/Library/Application Support/ValueCell/valuecell.db` on macOS)

Note: This function returns a filesystem path, not a SQLAlchemy DSN.
"""
# Prefer generic VALUECELL_DATABASE_URL if it points to SQLite
db_url = os.environ.get("VALUECELL_DATABASE_URL")
if db_url and db_url.startswith("sqlite:///"):
return _strip_sqlite_prefix(db_url)

# Default: store under system application directory alongside `.env`
return os.path.join(get_system_env_dir(), "valuecell.db")


def resolve_lancedb_uri() -> str:
return os.environ.get("VALUECELL_LANCEDB_URI") or os.path.join(
get_repo_root_path(), "lancedb"
)
"""Resolve LanceDB directory path.

Resolution order:
1) Default to system application directory: `<system_env_dir>/lancedb`

Additionally, if an old repo-root `lancedb` directory exists and the new
system directory does not, migrate the contents once for continuity.
"""
# Default: system directory
system_dir = get_system_env_dir()
new_path = Path(system_dir) / "lancedb"
new_path.mkdir(parents=True, exist_ok=True)

# Migrate from old repo-root location if needed
old_path = Path(get_repo_root_path()) / "lancedb"
try:
if old_path.exists() and not any(new_path.iterdir()):
# Copy contents only if destination is empty
for item in old_path.iterdir():
src = item
dst = new_path / item.name
if item.is_dir():
shutil.copytree(src, dst, dirs_exist_ok=True)
else:
shutil.copy2(src, dst)
except Exception:
# Non-fatal: if migration fails, just proceed with new_path
pass

return str(new_path)
46 changes: 37 additions & 9 deletions python/valuecell/utils/path.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
"""Utilities for resolving Python package and repository root paths.

Note: Database default location logic moved to `valuecell.utils.db.resolve_db_path`,
which defaults to storing `valuecell.db` in the system application directory
(same path as `.env`).
"""

import os
import shutil
from pathlib import Path

from .env import get_system_env_dir


def get_python_root_path() -> str:
"""
Expand Down Expand Up @@ -28,10 +38,11 @@ def get_python_root_path() -> str:


def get_repo_root_path() -> str:
"""Resolve repository root and return default DB path valuecell.db.
"""
Resolve repository root directory path.

Layout assumption: this file is at repo_root/python/valuecell/utils/db.py
We walk up 3 levels to reach repo_root.
Assumes this file is at `repo_root/python/valuecell/utils/path.py`.
Walk up three levels to reach `repo_root`.
"""
here = os.path.dirname(__file__)
repo_root = os.path.abspath(os.path.join(here, "..", "..", ".."))
Expand All @@ -52,11 +63,28 @@ def get_agent_card_path() -> str:

def get_knowledge_path() -> str:
"""
Returns the path to the knowledge directory located in the project root.
Resolve the Knowledge directory path under the system application directory.

Returns:
str: Absolute path of the knowledge directory
Behavior:
- Default location: `<system_env_dir>/.knowledge` (same base dir as `.env`)
- One-time migration: if old repo-root `.knowledge` exists and the system dir
is empty, copy the contents for continuity.
"""
root_path = get_repo_root_path()
knowledge_path = Path(root_path) / ".knowledge"
return str(knowledge_path)
new_path = Path(get_system_env_dir()) / ".knowledge"
new_path.mkdir(parents=True, exist_ok=True)

old_path = Path(get_repo_root_path()) / ".knowledge"
try:
if old_path.exists() and not any(new_path.iterdir()):
for item in old_path.iterdir():
src = item
dst = new_path / item.name
if item.is_dir():
shutil.copytree(src, dst, dirs_exist_ok=True)
else:
shutil.copy2(src, dst)
except Exception:
# Non-fatal: proceed with new_path even if migration fails
pass

return str(new_path)