Skip to content

Conversation

@gfyrag
Copy link
Contributor

@gfyrag gfyrag commented Jan 5, 2026

No description provided.

@gfyrag gfyrag requested a review from a team as a code owner January 5, 2026 16:42
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 5, 2026

Caution

Review failed

Failed to post review comments

Walkthrough

This PR introduces comprehensive changes across multiple domains: adding idempotency-hit tracking to controller methods with HTTP header propagation, implementing a new exporter update API, refactoring Pulumi image configuration, updating storage queries with scoped selects, modifying transaction query parameters (timestamp instead of date), adding balance filter support for big integers, updating mock generation with -typed flag, and expanding test coverage for new features.

Changes

Cohort / File(s) Summary
Idempotency Hit Tracking
internal/controller/ledger/controller.go, controller_default.go, controller_with_events.go, controller_with_traces.go, controller_with_too_many_client_handling.go, log_process.go, internal/controller/system/state_tracker.go, internal/api/v1/controllers_*.go, internal/api/v2/controllers_*.go
Added boolean return value to 6 controller methods (CreateTransaction, RevertTransaction, SaveTransactionMetadata, SaveAccountMetadata, DeleteTransactionMetadata, DeleteAccountMetadata) to indicate idempotency hits; handlers now set Idempotency-Hit response header when true; forgeLog returns additional bool indicating whether result came from idempotency key fetch
Mock Generation Overhaul
internal/api/bulking/mocks.go, internal/api/v1/mocks.go, internal/api/v2/mocks.go, internal/controller/ledger/controller.go, *_generated_test.go, internal/replication/exporters.go
Added -typed flag to mockgen directives enabling strongly-typed call wrappers; affected files regenerate with per-method Call types (e.g., LedgerControllerCreateTransactionCall) providing fluent Return/Do/DoAndReturn methods instead of generic gomock.Call
Exporter Update API
internal/api/v2/controllers_exporters_update.go, internal/api/v2/routes.go, internal/replication/grpc/replication_service.proto, internal/replication/controller_grpc_server.go, internal/replication/controller_grpc_client.go, internal/replication/manager.go, internal/replication/store.go, internal/controller/system/controller.go, internal/storage/system/store.go, pkg/client/v2.go, pkg/client/models/operations/v2updateexporter.go
Adds new PUT /exporters/{exporterID} endpoint with UpdateExporter handler; includes gRPC service definition, client/server implementations, manager-level update logic with driver validation and pipeline synchronization, storage integration, and auto-generated client SDK support
Pulumi Image Configuration
deployments/pulumi/pkg/config/config.go, deployments/pulumi/pkg/common/common.go, deployments/pulumi/pkg/utils/convert.go, deployments/pulumi/pkg/api/deployment.go, deployments/pulumi/pkg/worker/deployment.go, deployments/pulumi/pkg/storage/component_postgres.go, deployments/pulumi/pkg/storage/migrate.go, deployments/pulumi/pkg/generator/component.go, deployments/pulumi/pkg/provision/component.go
Introduced ImageConfiguration struct with Registry, Repository, Tag fields; refactored GetMainImage and GetImage functions to accept ImageConfiguration; added methods WithFallbackTag and SetDefaults; updated component deployments to use ImageConfiguration instead of direct Tag; common args now embed ImageConfiguration
Storage Query Scoping
internal/storage/ledger/store.go, internal/storage/ledger/resource_accounts.go, internal/storage/ledger/resource_aggregated_balances.go, internal/storage/ledger/resource_logs.go, internal/storage/ledger/resource_transactions.go, internal/storage/ledger/resource_volumes.go
Added newScopedSelect() method to Store for ledger-scoped queries; replaced h.store.db.NewSelect() with h.store.newScopedSelect() across resource handlers, removing explicit ledger filtering; scoped queries improve performance by avoiding ledger-name filters through strategic WHERE clause optimization
Transaction Query Parameters
internal/api/v1/controllers_transactions.go, internal/api/v1/controllers_transactions_count_test.go, internal/api/v1/controllers_transactions_list_test.go, internal/api/v1/utils.go
Changed query parameter keys from snake_case (start_time, end_time) to camelCase (startTime, endTime); updated query builder from date-based filters to timestamp-based (Gte/Lt on timestamp field); added PIT (Point in Time) and OOT (Out of Time) parameter parsing with fallback to deprecated parameters for backward compatibility
Balance Filter Big Integer Support
internal/storage/common/schema.go, internal/api/v2/controllers_accounts_count_test.go, internal/api/v2/controllers_accounts_list_test.go, internal/api/v2/controllers_bulk_test.go, internal/api/v2/controllers_volumes_test.go
Added big.Int support to TypeNumeric.ValidateValue; updated test fixtures to use big.NewInt() instead of float64 for balance filter comparisons
Database Migrations
internal/storage/bucket/migrations/11-make-stateless/up.sql through 42-fix-missing-inserted-at-in-log-data/up.sql
Refactored migration logic including: revised trigger creation with ledger-specific naming, added temporary table patterns with row_number-based pagination replacing OFFSET/LIMIT, introduced safety assertions for balance computation, updated condition guards to check SYNC features on ledgers, changed error handling to use postgres.ErrNotFound
Storage Error Handling
internal/storage/system/store.go, internal/replication/manager.go, internal/replication/drivers/registry.go, internal/storage/common/errors.go
Replaced sql.ErrNoRows references with postgres.ErrNotFound; removed common.ErrNotFound global variable; adjusted error returns from DeleteExporter and GetExporter to use postgres error types; added postgres import throughout replication/storage layers
HTTP Response Headers
pkg/client/v1.go, pkg/client/v2.go, pkg/client/models/operations/*.go, pkg/client/docs/models/operations/*.go
Added Headers field (map[string][]string) to response types (AddMetadataToAccount, CreateTransaction, RevertTransaction, etc.); HTTP handlers now populate res.Headers with httpRes.Header on successful responses; generated getter methods with nil-safety
Asset Pattern Validation
pkg/assets/asset.go, pkg/assets/asset_test.go
Updated asset regex pattern to allow optional underscore-separated uppercase segment before numeric path; Pattern now: [A-Z][A-Z0-9]{0,16}(_[A-Z]{1,16})?(\/\d{1,6})? enabling codes like EUR_COL, EUR_COL/12
VM & Logic Changes
internal/machine/vm/machine.go, internal/machine/vm/machine_test.go
Added world-address validation in ResolveBalances rejecting @world as account source; guarded OP_SAVE balance modifications to only apply when account/asset entries exist; added test cases for world variable handling
Replication & Pipeline Management
internal/replication/pipeline.go, internal/replication/manager_test.go, internal/replication/driver_facade.go, internal/replication/drivers/clickhouse/driver.go
Reworked pipeline Run loop with asynchronous exporter.Accept via goroutine and context cancellation; added lifecycle logging; refactored manager tests from exporterFactory to driverFactory semantics; updated cancellation source from ctx to c.startContext
Command & Configuration
cmd/root.go, cmd/serve.go, README.md, tools/provisioner/Dockerfile, tools/provisioner/justfile
Removed experimental-numscript-interpreter flags from root command; changed NumscriptInterpreterFlagsToPass from String to StringSlice flag in serve; added FX supply for connectionOptions; updated community link from Slack to GitHub Discussions; bumped Go version in Dockerfile to 1.25; added configurable registry parameter to provisioner build
Test Infrastructure & E2E
test/e2e/suite_test.go, test/e2e/api_exporters_update_test.go, test/e2e/api_idempotency_hit_header_test.go, test/e2e/api_*.go
Added comprehensive E2E test suite for exporter update API with pipeline routing verification; added idempotency-hit header validation tests across multiple operations; added test cases for big int balance filters, PIT with $or filters, and transaction revert responses
Type & Generic Refactoring
internal/storage/common/cursor.go, internal/storage/common/resource.go
Renamed generic type parameter OF to Options in cursor unmarshalling; changed filters map from map[string]any to map[string][]any to support multiple values per key; updated UseFilter logic to handle slice-based values

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Handler as V2 Handler
    participant Controller as Ledger Controller
    participant Store as Storage Layer
    participant Processor as Log Processor

    Client->>Handler: CreateTransaction Request<br/>(with Idempotency-Key)
    Handler->>Controller: CreateTransaction(ctx, params)
    Controller->>Processor: forgeLog(ctx, store, params, fn)
    
    alt Idempotency Key Found
        Processor->>Store: GetLogByIdempotencyKey(key)
        Store-->>Processor: (log, output, nil)
        Processor-->>Controller: (log, output, true, nil)
    else First Request
        Processor->>Processor: Execute transaction processing
        Processor->>Store: PersistLog(...)
        Store-->>Processor: (success)
        Processor-->>Controller: (log, output, false, nil)
    end
    
    Controller->>Handler: Return (log, output, idempotencyHit, error)
    
    alt idempotencyHit == true
        Handler->>Handler: Set Header "Idempotency-Hit: true"
    else idempotencyHit == false
        Handler->>Handler: (No Idempotency-Hit header)
    end
    
    Handler-->>Client: Response with Headers
Loading
sequenceDiagram
    participant Admin
    participant API as V2 UpdateExporter Endpoint
    participant Manager as Replication Manager
    participant Storage
    participant Pipeline
    participant Driver

    Admin->>API: PUT /exporters/{id} (new config)
    API->>Manager: UpdateExporter(ctx, id, config)
    
    Manager->>Manager: ValidateDriver(config.Driver)
    alt Invalid Driver
        Manager-->>API: ErrInvalidDriverConfiguration
        API-->>Admin: 400 Bad Request
    else Not Found
        Manager-->>API: ErrExporterNotFound
        API-->>Admin: 404 Not Found
    else Valid
        Manager->>Manager: stopExporter(id)
        Manager->>Pipeline: Stop pipelines using exporter
        Pipeline->>Driver: Stop()
        
        Manager->>Storage: UpdateExporter(exporter)
        Storage-->>Manager: Success
        
        Manager->>Manager: synchronizePipelines()
        Manager-->>API: nil
        API-->>Admin: 204 No Content
    end
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • PR #1156: Implements idempotency-hit support with identical controller method signature changes and response header logic
  • PR #1112: Shares cursor.go generic type parameter rename from OF to Options
  • PR #732: Adds UpdateExporter API surface with matching exporter update handler and RPC definitions
  • PR #1183: Contains overlapping changes including idempotency tracking, image configuration refactoring, controller mock updates, and exporter UpdateExporter additions

Suggested labels

feature, api, refactor, database-migration, testing

Suggested reviewers

  • flemzord
  • Azorlogh
  • paul-nicolas

Poem

🐰 Hop along the idempotent way,
Where headers hit and transactions stay,
Exporters update with grace and style,
Images scoped in Pulumi's file,
With big ints balanced and queries bright,
This PR shines with might! ✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 2.08% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
Description check ❓ Inconclusive No description was provided by the author; this makes it impossible to verify relation to the changeset or assess clarity. Add a description explaining what migration number drift was fixed and why the changes were necessary.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix: migrations number drift' is concise and directly related to the primary changeset focus on migration numbering issues.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/migrations-numbers-drift

Comment @coderabbitai help to get the list of available commands and usage tips.

@gfyrag gfyrag changed the base branch from main to release/v2.3 January 5, 2026 16:44
@gfyrag gfyrag merged commit a0e648a into release/v2.3 Jan 6, 2026
1 check passed
@gfyrag gfyrag deleted the fix/migrations-numbers-drift branch January 6, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants