Postgres -> Clickhouse error Code: 47. DB::Exception: Missing columns: '_airbyte_unique_key' while processing query

[FedorAredakov]

I have two very similar postgres tables:

CREATE TABLE "beta-0".projects (
	id uuid NOT NULL,
	"createdAt" timestamptz NOT NULL,
	"updatedAt" timestamptz NOT NULL,
	"deletedAt" timestamptz NULL,
	deleted bool NOT NULL DEFAULT false,
	"ownerAccountId" uuid NOT NULL,
	"statusId" varchar(16) NOT NULL DEFAULT 'active'::character varying,
	"name" varchar(100) NOT NULL,
	code varchar(32) NOT NULL,
	"creatorId" uuid NOT NULL,
	"lastEditorId" uuid NULL,
	"externalId" uuid NULL,
	slug varchar(110) NOT NULL,
	"structure" jsonb NOT NULL DEFAULT '{}'::jsonb,
	preview text NOT NULL DEFAULT ''::text,
	"isInitializing" bool NOT NULL DEFAULT false,
	"externalHubId" uuid NULL,
	"deletedById" uuid NULL,
	"copyFromId" uuid NULL,
	"timeZone" varchar(100) NOT NULL DEFAULT 'Europe/Moscow'::character varying,
	"isDemo" bool NOT NULL DEFAULT false,
	"modelApiType" "beta-0"."enum_projects_modelApiType" NOT NULL DEFAULT 'FORGE'::"beta-0"."enum_projects_modelApiType",
	CONSTRAINT projects_pkey PRIMARY KEY (id)
);

CREATE TABLE "beta-0".models (
	id uuid NOT NULL,
	code serial4 NOT NULL,
	"projectId" uuid NOT NULL,
	"creatorId" uuid NOT NULL,
	"createdAt" timestamptz NOT NULL,
	"updatedAt" timestamptz NOT NULL,
	deleted bool NOT NULL DEFAULT false,
	"deletedAt" timestamptz NULL,
	"deletedById" uuid NULL,
	"isModelled" bool NOT NULL DEFAULT true,
	CONSTRAINT models_pkey PRIMARY KEY (id)
);

Source replication method - STANDART
Sync mode - Deduped + History

Both tables have the same settings

Cursor field
updatedAt

Primary key
id

The problem is the project table sync ok, but the modes table sync return the error

09:24:14.692630 [error] [MainThread]: Database Error in model models (models/generated/airbyte_incremental/beta_0/models.sql)
09:24:14.692889 [error] [MainThread]:   :HTTPDriver for http://my-clickhouse-cluster returned response code 404)
09:24:14.693177 [error] [MainThread]:    Code: 47. DB::Exception: Missing columns: '_airbyte_unique_key' while processing query: 'SELECT createdAt, deletedAt, code, deleted, creatorId, deletedById, id, projectId, isModelled, updatedAt, _airbyte_ab_id, _airbyte_emitted_at, _airbyte
09:24:14.446007 [error] [Thread-1 (]: 5 of 6 ERROR creating sql incremental model beta_0.models .............................................................. [ERROR in 0.18s]

and indeed projects table in clickhouse has the "_airbyte_unique_key" column, but models table has not.

What interesting - system tables models_scd, models_dbt_tmp and models_dbt_new_data has this column "_airbyte_unique_key"

I will be very appreciate for any help

[FedorAredakov]

Interesting update -
the problem happened only if I add table(s) to the existing connection, run reset (succeeded) and run sync (failed)
If I add all these tables to the brand new connection sync works fine....
I have about 2000 tables ..... I don't want to recreate connection it it's needed to add one new table :)))))