Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,14 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
- **Apply**: Execute the migration with safety features like concurrent change detection, transaction-adaptive execution, and lock timeout control

The tool is written in Go 1.24+ (toolchain go1.24.7) and uses:

- Cobra for CLI commands
- embedded-postgres v1.29.0 for plan command (temporary instances) and testing (no Docker required)
- pgx/v5 v5.7.5 for database connections
- Supports PostgreSQL versions 14-17

Key differentiators:

- Comprehensive Postgres support for virtually all schema-level objects
- State-based Terraform-like workflow (no migration history table)
- Schema-level focus for single-schema apps to multi-tenant architectures
Expand Down Expand Up @@ -66,6 +68,7 @@ go test -v ./...
For interactive database validation, see the **Validate with Database** skill (`.claude/skills/validate_db/SKILL.md`).

Connection details are in `.env`:

```
PGHOST=localhost
PGDATABASE=employee
Expand All @@ -78,13 +81,15 @@ PGPASSWORD=testpwd1
### Core Components

**CLI Commands** (`cmd/`):

- `dump/` - Schema extraction from live database
- `plan/` - Migration planning by comparing schemas
- `apply/` - Migration execution with safety checks
- `util/` - Shared utilities (connection, env, ignore file processing)
- `root.go` - Main CLI setup with Cobra

**Core Packages**:

- `ir/` - Intermediate Representation (IR) package
- Schema objects (tables, indexes, functions, procedures, triggers, policies, etc.)
- Database inspector using pgx (queries pg_catalog for schema extraction)
Expand All @@ -93,6 +98,7 @@ PGPASSWORD=testpwd1
- Note: Parser removed in favor of embedded-postgres approach

**Internal Packages** (`internal/`):

- `diff/` - Schema comparison and migration DDL generation
- `plan/` - Migration plan structures and execution
- `dump/` - Schema dump formatting and output
Expand All @@ -119,7 +125,8 @@ PGPASSWORD=testpwd1
**Inspector-Only Approach**: Both desired state (from user SQL files) and current state (from target database) are obtained through database inspection. The plan command spins up an embedded PostgreSQL instance, applies user SQL files, then inspects it to get the desired state IR. This eliminates the need for SQL parsing and ensures consistency.

**External Database for Plan Generation**: As an alternative to embedded postgres, users can provide an external PostgreSQL database using `--plan-host` flags or `PGSCHEMA_PLAN_*` environment variables. The external database approach:
- Creates temporary schemas with timestamp suffixes (e.g., `pgschema_plan_20251030_154501_123456789`)

- Creates temporary schemas with timestamp suffixes (e.g., `pgschema_tmp_20251030_154501_123456789`)
- Validates major version compatibility with target database (exact match required)
- Cleans up temporary schemas after use (best effort)
- Useful for environments where embedded postgres has limitations (ARM architectures, containerized environments)
Expand Down Expand Up @@ -182,48 +189,57 @@ The tool supports comprehensive PostgreSQL schema objects (see `ir/ir.go` for co
## Important Implementation Notes

**Trigger Features**:

- Full support for WHEN conditions using `pg_get_expr(t.tgqual, t.tgrelid, false)` from `pg_catalog.pg_trigger`
- Constraint triggers with deferrable options
- REFERENCING OLD TABLE / NEW TABLE for statement-level triggers

**Online Migration Support**:

- CREATE INDEX CONCURRENTLY for non-blocking index creation
- ALTER TABLE ... ADD CONSTRAINT ... NOT VALID for online constraint addition
- Proper transaction handling - some operations must run outside transactions

**pgschema Directives**:

- Special SQL comments control behavior: `--pgschema-lock-timeout`, `--pgschema-no-transaction`
- Handled in `cmd/apply/directive.go`

**Reference Implementations**:

- PostgreSQL's pg_dump serves as reference for system catalog queries (see **pg_dump Reference** skill)
- PostgreSQL's gram.y defines canonical SQL syntax (see **PostgreSQL Syntax Reference** skill)

## Key Files Reference

**Entry Point & CLI**:

- `main.go` - Entry point, loads .env and calls cmd.Execute()
- `cmd/root.go` - Root CLI with global flags

**IR Package** (`./ir`):

- `ir/ir.go` - Core IR data structures for all schema objects
- `ir/inspector.go` - Database introspection using pgx (queries pg_catalog)
- `ir/normalize.go` - Schema normalization (version-specific differences, type mappings)
- `ir/quote.go` - Identifier quoting utilities
- Note: `ir/parser.go` removed - now using embedded-postgres for desired state

**Diff Package** (`internal/diff/`):

- `diff.go` - Main diff logic, topological sorting
- `table.go`, `index.go`, `trigger.go`, `view.go`, `function.go`, `procedure.go`, `sequence.go`, `type.go`, `policy.go`, `aggregate.go` - Object-specific diff operations

**Testing**:

- `cmd/migrate_integration_test.go` - Main integration test suite (TestPlanAndApply)
- `testdata/diff/` - 100+ test cases covering all schema object types
- See **Run Tests** skill for complete testing workflows

## Test Data Structure

Tests are organized in `testdata/diff/` by object type:

- `comment/` (8 tests), `create_domain/` (3), `create_function/` (4), `create_index/` (1)
- `create_materialized_view/` (3), `create_policy/` (8), `create_procedure/` (3), `create_sequence/` (3)
- `create_table/` (40 tests), `create_trigger/` (7), `create_type/` (3), `create_view/` (6)
Expand Down
36 changes: 36 additions & 0 deletions cmd/apply/apply.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@ var (
applyNoColor bool
applyLockTimeout string
applyApplicationName string

// Plan database connection flags (optional - for using external database instead of embedded postgres)
applyPlanDBHost string
applyPlanDBPort int
applyPlanDBDatabase string
applyPlanDBUser string
applyPlanDBPassword string
)

var ApplyCmd = &cobra.Command{
Expand Down Expand Up @@ -63,6 +70,13 @@ func init() {
ApplyCmd.Flags().StringVar(&applyLockTimeout, "lock-timeout", "", "Maximum time to wait for database locks (e.g., 30s, 5m, 1h)")
ApplyCmd.Flags().StringVar(&applyApplicationName, "application-name", "pgschema", "Application name for database connection (visible in pg_stat_activity) (env: PGAPPNAME)")

// Plan database connection flags (optional - for using external database instead of embedded postgres when using --file)
ApplyCmd.Flags().StringVar(&applyPlanDBHost, "plan-host", "", "Plan database host (env: PGSCHEMA_PLAN_HOST). If provided, uses external database instead of embedded postgres for validating desired state schema")
ApplyCmd.Flags().IntVar(&applyPlanDBPort, "plan-port", 5432, "Plan database port (env: PGSCHEMA_PLAN_PORT)")
ApplyCmd.Flags().StringVar(&applyPlanDBDatabase, "plan-db", "", "Plan database name (env: PGSCHEMA_PLAN_DB)")
ApplyCmd.Flags().StringVar(&applyPlanDBUser, "plan-user", "", "Plan database user (env: PGSCHEMA_PLAN_USER)")
ApplyCmd.Flags().StringVar(&applyPlanDBPassword, "plan-password", "", "Plan database password (env: PGSCHEMA_PLAN_PASSWORD)")

// Mark file and plan as mutually exclusive
ApplyCmd.MarkFlagsMutuallyExclusive("file", "plan")
}
Expand Down Expand Up @@ -286,6 +300,22 @@ func RunApply(cmd *cobra.Command, args []string) error {
// Using --file flag, will need desired state provider
config.File = applyFile

// Apply environment variables to plan database flags (only needed for File Mode)
util.ApplyPlanDBEnvVars(cmd, &applyPlanDBHost, &applyPlanDBDatabase, &applyPlanDBUser, &applyPlanDBPassword, &applyPlanDBPort)

// Validate plan database flags if plan-host is provided
if err := util.ValidatePlanDBFlags(applyPlanDBHost, applyPlanDBDatabase, applyPlanDBUser); err != nil {
return err
}

// Derive final plan database password
finalPlanPassword := applyPlanDBPassword
if finalPlanPassword == "" {
if envPassword := os.Getenv("PGSCHEMA_PLAN_PASSWORD"); envPassword != "" {
finalPlanPassword = envPassword
}
}

// Create desired state provider (embedded postgres or external database)
planConfig := &planCmd.PlanConfig{
Host: applyHost,
Expand All @@ -296,6 +326,12 @@ func RunApply(cmd *cobra.Command, args []string) error {
Schema: applySchema,
File: applyFile,
ApplicationName: applyApplicationName,
// Plan database configuration
PlanDBHost: applyPlanDBHost,
PlanDBPort: applyPlanDBPort,
PlanDBDatabase: applyPlanDBDatabase,
PlanDBUser: applyPlanDBUser,
PlanDBPassword: finalPlanPassword,
}
provider, err = planCmd.CreateDesiredStateProvider(planConfig)
if err != nil {
Expand Down
Loading