Skip to content

Commit 56cf047

Browse files
Copilotfaizanazim11
Copilot
and
faizanazim11
committed
Fix linting issues and add copilot instructions
Co-authored-by: faizanazim11 <20454506+faizanazim11@users.noreply.github.com>
1 parent 1c24dc8 commit 56cf047

File tree

3 files changed

+240
-2
lines changed

3 files changed

+240
-2
lines changed

‎.github/copilot-instructions.md‎

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
# sql-db-utils - GitHub Copilot Instructions
2+
3+
**SQL database utilities package for Python developers that provides declarative model generation and session management.**
4+
5+
Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
6+
7+
## Working Effectively
8+
9+
### Bootstrap, Build, and Test Repository:
10+
- `pip install -e .` -- installs the package in development mode. NEVER CANCEL: Takes 10-60 seconds, may timeout due to network issues. Set timeout to 120+ seconds (extra margin for slow mirrors or network issues).
11+
- `pip install coverage pre-commit pytest pytest-cov ruff` -- installs development dependencies. NEVER CANCEL: Takes 30-120 seconds. Set timeout to 180+ seconds (extra margin for slow mirrors or network issues).
12+
- `python -m pytest tests/ -v` -- runs unit tests (when tests are available)
13+
- `ruff check .` -- runs linting (takes ~0.01 seconds)
14+
- `ruff format --check .` -- checks code formatting (takes ~0.01 seconds)
15+
16+
### Environment Configuration:
17+
- Package requires Python >= 3.13
18+
- Core dependencies: SQLAlchemy >= 2.0.38, sqlalchemy-utils, psycopg, python-dateutil, whenever
19+
- Optional dependencies available: polars, pandas, async, binary, codegen
20+
- Database support: PostgreSQL (primary), with pooling and connection management
21+
- Configuration via environment variables and PostgreSQL connection strings
22+
23+
### Run Integration Tests with Real Database:
24+
- Start PostgreSQL: `docker run -d --name test-postgres -p 5432:5432 -e POSTGRES_PASSWORD=test postgres:15-alpine` (NEVER CANCEL: Takes 30-60 seconds for first download)
25+
- Wait for startup: `sleep 10`
26+
- Run integration tests: `POSTGRES_URI=postgresql://postgres:test@localhost:5432/test python -c "from sql_db_utils import SQLSessionManager; print('SQL integration test passed')"` (basic session management test)
27+
- Clean up: `docker stop test-postgres && docker rm test-postgres`
28+
29+
## Validation Scenarios
30+
31+
### Always Test After Making Changes:
32+
1. **Import Test**: `python -c "from sql_db_utils import SQLSessionManager; from sql_db_utils.asyncio import SQLSessionManager as AsyncSQLSessionManager; print('Import successful')"`
33+
2. **Basic Session Management Test** (requires PostgreSQL running):
34+
```bash
35+
POSTGRES_URI=postgresql://postgres:test@localhost:5432/test python -c "
36+
from sql_db_utils import SQLSessionManager
37+
manager = SQLSessionManager()
38+
print('Session Manager created successfully')
39+
"
40+
```
41+
3. **Precreate/Postcreate Functionality Test**:
42+
```bash
43+
python -c "
44+
from sql_db_utils import SQLSessionManager
45+
manager = SQLSessionManager()
46+
47+
@manager.register_precreate('test_db')
48+
def test_precreate(tenant_id):
49+
return 'SELECT 1;'
50+
51+
@manager.register_postcreate('test_db')
52+
def test_postcreate(tenant_id):
53+
return 'SELECT 2;'
54+
55+
print('Precreate/Postcreate registration successful')
56+
"
57+
```
58+
4. **Run Full Test Suite**: `python -m pytest tests/ -v --cov=sql_db_utils --cov-report=term-missing` (when tests are available)
59+
5. **Linting**: `ruff check . && ruff format --check .`
60+
61+
### Manual Testing Requirements:
62+
- ALWAYS test session management functionality after code changes
63+
- Test both synchronous and asynchronous implementations
64+
- Verify precreate and postcreate hooks work correctly
65+
- Test with different PostgreSQL configurations and connection parameters
66+
- Test database creation, connection pooling, and engine management
67+
68+
## Common Tasks
69+
70+
### Repository Structure:
71+
```
72+
sql-db-utils/
73+
├── .github/workflows/ # CI/CD pipelines
74+
├── sql_db_utils/ # Main package source
75+
│ ├── __init__.py # Main exports
76+
│ ├── config.py # Configuration settings
77+
│ ├── constants.py # Package constants
78+
│ ├── datetime_utils.py # Date/time utilities
79+
│ ├── session_management.py # Core session management (sync)
80+
│ ├── sql_creations.py # SQL table creation utilities
81+
│ ├── sql_extras.py # Additional SQL utilities
82+
│ ├── sql_retry_handler.py # Query retry mechanisms
83+
│ ├── sql_utils.py # General SQL utilities
84+
│ ├── declarative_utils.py # Declarative model utilities
85+
│ ├── declaratives.py # Base declarative classes
86+
│ ├── codegen.py # Code generation utilities
87+
│ ├── aggrid/ # AG Grid integration utilities
88+
│ │ ├── date_filters.py
89+
│ │ ├── number_filters.py
90+
│ │ └── text_filters.py
91+
│ └── asyncio/ # Asynchronous implementations
92+
│ ├── __init__.py
93+
│ ├── session_management.py # Async session management
94+
│ ├── sql_creations.py # Async SQL creation utilities
95+
│ ├── sql_creation_helper.py # Async creation helpers
96+
│ ├── sql_retry_handler.py # Async retry mechanisms
97+
│ ├── sql_utils.py # Async SQL utilities
98+
│ ├── declarative_utils.py # Async declarative utilities
99+
│ ├── declaratives.py # Async base classes
100+
│ ├── codegen.py # Async code generation
101+
│ └── inspector_utils.py # Database inspection utilities
102+
├── tests/ # Test files (when available)
103+
├── pyproject.toml # Project configuration
104+
└── README.md # Documentation
105+
```
106+
107+
### Key Files to Check After Changes:
108+
- Always verify `sql_db_utils/__init__.py` after changing main exports
109+
- Check `sql_db_utils/config.py` after modifying configuration handling
110+
- Verify sync/async parity between `sql_db_utils/session_management.py` and `sql_db_utils/asyncio/session_management.py`
111+
- Update `sql_db_utils/sql_creations.py` and `sql_db_utils/asyncio/sql_creations.py` for SQL creation changes
112+
- Test declarative utilities in both sync and async versions
113+
- Verify codegen functionality if making changes to code generation features
114+
- Update tests when adding new functionality
115+
- Run integration tests with real PostgreSQL database
116+
117+
### Development Dependencies:
118+
- **Testing**: pytest, pytest-cov, coverage
119+
- **Linting**: ruff (replaces black, flake8, isort)
120+
- **Git hooks**: pre-commit
121+
- **Type checking**: Built into package development
122+
- **Core Dependencies**: SQLAlchemy, sqlalchemy-utils, psycopg, python-dateutil, whenever
123+
124+
### Build and Package:
125+
- `python -m build` -- builds distribution packages. NEVER CANCEL: May fail due to network timeouts depending on the configured build backend and network environment. Package requires Python >= 3.13.
126+
- Package metadata in `pyproject.toml`
127+
- Uses hatchling as build backend
128+
- **Note**: Package requires specific Python version (>=3.13) which may not be available in all environments
129+
130+
### Session Management Features:
131+
- **Precreate Hooks**: Execute before database/table creation for setup tasks
132+
- **Postcreate Hooks**: Execute after database/table creation for initialization
133+
- **Auto Hooks**: Return SQL statements to be executed automatically
134+
- **Manual Hooks**: Receive session objects for custom operations
135+
- **Multi-database Support**: Register hooks for single or multiple databases
136+
- **Tenant Support**: All hooks receive tenant_id parameter for multi-tenant applications
137+
138+
## Database Features and Testing
139+
140+
### Supported Database Features:
141+
- **PostgreSQL**: Primary database with full feature support
142+
- **Connection Pooling**: Configurable via PostgresConfig.PG_ENABLE_POOLING
143+
- **Connection Retry**: Built-in retry mechanisms with PostgresConfig.PG_MAX_RETRY
144+
- **Database Creation**: Automatic database creation if not exists
145+
- **Schema Support**: Multi-schema support with declarative utilities
146+
- **Transaction Management**: Automatic transaction handling in hooks
147+
148+
### Session Management Testing:
149+
- **Engine Creation**: Test `_get_engine()` method with various configurations
150+
- **Hook Execution**: Verify precreate/postcreate hooks execute in correct order
151+
- **Connection Pooling**: Test with/without pooling enabled
152+
- **Multi-tenant**: Test with different tenant_id values
153+
- **Error Handling**: Test connection failures and retry mechanisms
154+
155+
### Setting up Test Database with Docker:
156+
- PostgreSQL: `docker run -d --name test-postgres -p 5432:5432 -e POSTGRES_PASSWORD=test postgres:15-alpine`
157+
- Create test database: `docker exec -it test-postgres psql -U postgres -c "CREATE DATABASE test;"`
158+
159+
## CI/CD Pipeline (.github/workflows)
160+
161+
### Linter Pipeline:
162+
- Runs on pull requests and pushes
163+
- Uses ruff for linting and formatting
164+
- ALWAYS run `ruff check .` and `ruff format --check .` before committing
165+
- Pre-commit hooks should handle formatting automatically
166+
167+
### Package Publishing:
168+
- Triggers on git tags
169+
- Builds with hatchling backend
170+
- Publishes to PyPI
171+
- Requires Python >= 3.13 environment
172+
173+
## Critical Notes
174+
175+
### Session Management Execution Order:
176+
1. **Engine Creation**: Database connection and engine setup
177+
2. **Precreate Hooks**: Execute custom setup before table creation
178+
3. **Table Creation**: `create_default_psql_dependencies()` creates tables/metadata
179+
4. **Postcreate Hooks**: Execute initialization after table creation
180+
181+
### Hook Implementation Patterns:
182+
- **Auto Hooks**: Return SQL strings or lists of SQL strings
183+
- **Manual Hooks**: Receive (session, tenant_id) parameters for custom logic
184+
- **Registration**: Use `@manager.register_precreate()` or `@manager.register_precreate_manual()`
185+
- **Multi-Database**: Pass list of database names to register for multiple databases
186+
187+
### Async/Sync Parity:
188+
- Both implementations must have identical API and functionality
189+
- Async version uses `async`/`await` patterns appropriately
190+
- Session types differ: `Session` vs `AsyncSession`
191+
- Engine types differ: `Engine` vs `AsyncEngine`
192+
193+
### Configuration and Environment:
194+
- PostgresConfig class manages all database configuration
195+
- Environment variables control connection parameters
196+
- ModuleConfig handles application-level settings
197+
- Connection pooling and retry settings are configurable
198+
199+
## Troubleshooting
200+
201+
### Common Issues:
202+
1. **Import Error**: Check Python version (>=3.13 required)
203+
2. **Connection Failures**: Verify PostgreSQL is running and accessible
204+
3. **Linting Failures**: Run `ruff format .` to auto-fix formatting issues
205+
4. **Missing Dependencies**: Run `pip install -e .` to reinstall package
206+
5. **Hook Execution**: Verify hooks are registered before engine creation
207+
208+
### Session Management Issues:
209+
- **Engine Not Created**: Check database URI and connection parameters
210+
- **Hooks Not Executing**: Ensure registration occurs before `get_session()` calls
211+
- **Transaction Errors**: Verify database permissions and connection state
212+
- **Pooling Issues**: Check PostgresConfig.PG_ENABLE_POOLING setting
213+
214+
### Development Environment:
215+
- Python 3.13+ required (check with `python --version`)
216+
- PostgreSQL server required for integration testing
217+
- Docker recommended for consistent database testing
218+
- Pre-commit hooks help maintain code quality
219+
220+
### Async/Sync Coordination:
221+
- Changes to sync version should be mirrored in async version
222+
- Test both implementations when making session management changes
223+
- Verify async patterns use proper `await` keywords
224+
- Check that both versions handle errors consistently

‎sql_db_utils/asyncio/session_management.py‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,14 @@
1313

1414

1515
class SQLSessionManager:
16-
__slots__ = ("_db_engines", "database_uri", "_postcreate_auto", "_postcreate_manual", "_precreate_auto", "_precreate_manual")
16+
__slots__ = (
17+
"_db_engines",
18+
"database_uri",
19+
"_postcreate_auto",
20+
"_postcreate_manual",
21+
"_precreate_auto",
22+
"_precreate_manual",
23+
)
1724

1825
def __init__(self, database_uri: Union[str, None] = None) -> None:
1926
self._db_engines = {}

‎sql_db_utils/session_management.py‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,14 @@
1313

1414

1515
class SQLSessionManager:
16-
__slots__ = ("_db_engines", "database_uri", "_postcreate_auto", "_postcreate_manual", "_precreate_auto", "_precreate_manual")
16+
__slots__ = (
17+
"_db_engines",
18+
"database_uri",
19+
"_postcreate_auto",
20+
"_postcreate_manual",
21+
"_precreate_auto",
22+
"_precreate_manual",
23+
)
1724

1825
def __init__(self, database_uri: Union[str, None] = None) -> None:
1926
self._db_engines = {}

0 commit comments

Comments
 (0)