Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ID Unification example #411

Merged
merged 1 commit into from
Nov 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions tool-box/id-unification-samples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# ID Unification Workflow Samples

ID Unification is the process of stitching together multiple tables using various identifiers to assign a unique customer ID (canonical_id or persistent_id) to each user. In simpler terms, it consolidates identifiers like cookie_id and email addresses from various user data sources to identify and group "the same person."

Since customer data often contains different identifiers across different data sources, simply aggregating this data doesn't link these sources together. This necessitates the ID Unification process to make the data usable.

These workflow samples are used for the following API documentation

- [API Documentation](https://api-docs.treasuredata.com/)
5 changes: 5 additions & 0 deletions tool-box/id-unification-samples/conifg/params.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
td:
tbl_aaa: site_aaa
tbl_xxx: site_xxx
tbl_yyy: site_yyy
tbl_zzz: site_zzz
103 changes: 103 additions & 0 deletions tool-box/id-unification-samples/queries/generate_samples_ex1.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
DROP TABLE IF EXISTS ${td.database}.${td.tbl_aaa};
CREATE TABLE ${td.database}.${td.tbl_aaa} AS
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'), 'aaa.jp','aaa_001','3rd_001')
,(TD_TIME_PARSE('2023/01/15'), 'aaa.jp','aaa_001','3rd_002')
,(TD_TIME_PARSE('2023/01/25'), 'aaa.jp','aaa_001','3rd_003')
,(TD_TIME_PARSE('2023/02/05'), 'aaa.jp','aaa_001','3rd_004')
,(TD_TIME_PARSE('2023/02/15'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/25'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/05'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/15'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/25'), 'aaa.jp','aaa_002','3rd_009')
,(TD_TIME_PARSE('2023/04/05'), 'aaa.jp','aaa_002','3rd_010')
,(TD_TIME_PARSE('2023/04/15'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/25'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/05'), 'aaa.jp','aaa_003','3rd_013')
,(TD_TIME_PARSE('2023/05/15'), 'aaa.jp','aaa_003','3rd_014')
,(TD_TIME_PARSE('2023/05/25'), 'aaa.jp','aaa_003','3rd_015')
,(TD_TIME_PARSE('2023/06/05'), 'aaa.jp','aaa_003','3rd_016')
,(TD_TIME_PARSE('2023/06/15'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/25'), 'aaa.jp',NULL,NULL)
) AS t(time, site, td_client_id, td_global_id);

DROP TABLE IF EXISTS ${td.database}.${td.tbl_xxx};
CREATE TABLE ${td.database}.${td.tbl_xxx} AS
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/15'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/25'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/05'),'xxx.jp','xxx_001','3rd_004')
,(TD_TIME_PARSE('2023/02/15'),'xxx.jp','xxx_001','3rd_005')
,(TD_TIME_PARSE('2023/02/25'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/05'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/15'),'xxx.jp','xxx_002','3rd_008')
,(TD_TIME_PARSE('2023/03/25'),'xxx.jp','xxx_002','3rd_009')
,(TD_TIME_PARSE('2023/04/05'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/15'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/25'),'xxx.jp','xxx_003','3rd_012')
,(TD_TIME_PARSE('2023/05/05'),'xxx.jp','xxx_003','3rd_013')
,(TD_TIME_PARSE('2023/05/15'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/25'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/05'),'xxx.jp','xxx_004','3rd_016')
,(TD_TIME_PARSE('2023/06/15'),'xxx.jp','xxx_004','3rd_017')
,(TD_TIME_PARSE('2023/06/25'),'xxx.jp',NULL,NULL)
) AS t(time, site, td_client_id, td_global_id);

DROP TABLE IF EXISTS ${td.database}.${td.tbl_yyy};
CREATE TABLE ${td.database}.${td.tbl_yyy} AS
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'),'yyy.jp','yyy_001','3rd_001')
,(TD_TIME_PARSE('2023/01/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/25'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/15'),'yyy.jp','yyy_002','3rd_005')
,(TD_TIME_PARSE('2023/02/25'),'yyy.jp','yyy_002','3rd_006')
,(TD_TIME_PARSE('2023/03/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/25'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/05'),'yyy.jp','yyy_003','3rd_010')
,(TD_TIME_PARSE('2023/04/15'),'yyy.jp','yyy_003','3rd_011')
,(TD_TIME_PARSE('2023/04/25'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/25'),'yyy.jp','yyy_004','3rd_015')
,(TD_TIME_PARSE('2023/06/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/25'),'yyy.jp','yyy_005','3rd_018')
) AS t(time, site, td_client_id, td_global_id);

DROP TABLE IF EXISTS ${td.database}.${td.tbl_zzz};
CREATE TABLE ${td.database}.${td.tbl_zzz} AS
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/15'),'zzz.jp','zzz_001','3rd_002')
,(TD_TIME_PARSE('2023/01/25'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/15'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/25'),'zzz.jp','zzz_003','3rd_006')
,(TD_TIME_PARSE('2023/03/05'),'zzz.jp','zzz_003','3rd_007')
,(TD_TIME_PARSE('2023/03/15'),'zzz.jp','zzz_003','3rd_008')
,(TD_TIME_PARSE('2023/03/25'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/15'),'zzz.jp','zzz_004','3rd_011')
,(TD_TIME_PARSE('2023/04/25'),'zzz.jp','zzz_004','3rd_012')
,(TD_TIME_PARSE('2023/05/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/15'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/25'),'zzz.jp','zzz_005','3rd_015')
,(TD_TIME_PARSE('2023/06/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/15'),'zzz.jp','zzz_005','3rd_017')
,(TD_TIME_PARSE('2023/06/25'),'zzz.jp','zzz_005','3rd_018')
) AS t(time, site, td_client_id, td_global_id);
105 changes: 105 additions & 0 deletions tool-box/id-unification-samples/queries/generate_samples_ex2.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
DROP TABLE IF EXISTS ${td.database}.${td.tbl_aaa};
CREATE TABLE ${td.database}.${td.tbl_aaa} AS
SELECT
time, site, td_client_id, td_global_id, td_ssc_id
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'),'aaa.jp','aaa_001','3rd_001','ssc_001')
,(TD_TIME_PARSE('2023/01/15'),'aaa.jp','aaa_001','3rd_002','ssc_001')
,(TD_TIME_PARSE('2023/01/25'),'aaa.jp','aaa_001','3rd_003','ssc_001')
,(TD_TIME_PARSE('2023/02/05'),'aaa.jp','aaa_001','3rd_004','ssc_001')
,(TD_TIME_PARSE('2023/02/15'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/02/25'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/03/05'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/03/15'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/03/25'),'aaa.jp','aaa_002','3rd_009','ssc_002')
,(TD_TIME_PARSE('2023/04/05'),'aaa.jp','aaa_002','3rd_010','ssc_002')
,(TD_TIME_PARSE('2023/04/15'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/04/25'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/05/05'),'aaa.jp','aaa_003','3rd_013','ssc_003')
,(TD_TIME_PARSE('2023/05/15'),'aaa.jp','aaa_003','3rd_014','ssc_003')
,(TD_TIME_PARSE('2023/05/25'),'aaa.jp','aaa_003','3rd_015','ssc_004')
,(TD_TIME_PARSE('2023/06/05'),'aaa.jp','aaa_003','3rd_016','ssc_004')
,(TD_TIME_PARSE('2023/06/15'),'aaa.jp',NULL,NULL,NULL)
,(TD_TIME_PARSE('2023/06/25'),'aaa.jp',NULL,NULL,NULL)
) AS t(time, site, td_client_id, td_global_id, td_ssc_id);

DROP TABLE IF EXISTS ${td.database}.${td.tbl_xxx};
CREATE TABLE ${td.database}.${td.tbl_xxx} AS

SELECT
time, site, td_ssc_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/15'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/25'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/05'),'xxx.jp','ssc_001','3rd_004')
,(TD_TIME_PARSE('2023/02/15'),'xxx.jp','ssc_001','3rd_005')
,(TD_TIME_PARSE('2023/02/25'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/05'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/15'),'xxx.jp','ssc_001','3rd_008')
,(TD_TIME_PARSE('2023/03/25'),'xxx.jp','ssc_001','3rd_009')
,(TD_TIME_PARSE('2023/04/05'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/15'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/25'),'xxx.jp','ssc_002','3rd_010')
,(TD_TIME_PARSE('2023/05/05'),'xxx.jp','ssc_002','3rd_013')
,(TD_TIME_PARSE('2023/05/15'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/25'),'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/05'),'xxx.jp','ssc_003','3rd_016')
,(TD_TIME_PARSE('2023/06/15'),'xxx.jp','ssc_003','3rd_017')
,(TD_TIME_PARSE('2023/06/25'),'xxx.jp',NULL,NULL)
) AS t(time, site, td_ssc_id, td_global_id);

DROP TABLE IF EXISTS ${td.database}.${td.tbl_yyy};
CREATE TABLE ${td.database}.${td.tbl_yyy} AS
SELECT
time, site, email, td_ssc_id
FROM
( VALUES

(TD_TIME_PARSE('2023/01/05'),'yyy.jp','a@ex.com','ssc_001')
,(TD_TIME_PARSE('2023/01/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/25'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/15'),'yyy.jp','a@ex.com','ssc_001')
,(TD_TIME_PARSE('2023/02/25'),'yyy.jp','a@ex.com','ssc_001')
,(TD_TIME_PARSE('2023/03/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/03/25'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/05'),'yyy.jp','b@ex.com','ssc_003')
,(TD_TIME_PARSE('2023/04/15'),'yyy.jp','b@ex.com','ssc_003')
,(TD_TIME_PARSE('2023/04/25'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/25'),'yyy.jp','c@ex.com','ssc_003')
,(TD_TIME_PARSE('2023/06/05'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/15'),'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/25'),'yyy.jp','c@ex.com','ssc_004')
) AS t(time, site, email, td_ssc_id);

DROP TABLE IF EXISTS ${td.database}.${td.tbl_zzz};
CREATE TABLE ${td.database}.${td.tbl_zzz} AS
SELECT
time, site, td_client_id, email
FROM
( VALUES
(TD_TIME_PARSE('2023/01/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/01/15'),'zzz.jp','zzz_001','a@ex.com')
,(TD_TIME_PARSE('2023/01/25'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/15'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/02/25'),'zzz.jp','zzz_003','a@ex.com')
,(TD_TIME_PARSE('2023/03/05'),'zzz.jp','zzz_003','a@ex.com')
,(TD_TIME_PARSE('2023/03/15'),'zzz.jp','zzz_003','a@ex.com')
,(TD_TIME_PARSE('2023/03/25'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/04/15'),'zzz.jp','zzz_004','b@ex.com')
,(TD_TIME_PARSE('2023/04/25'),'zzz.jp','zzz_004','c@ex.com')
,(TD_TIME_PARSE('2023/05/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/15'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/05/25'),'zzz.jp','zzz_005','c@ex.com')
,(TD_TIME_PARSE('2023/06/05'),'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/15'),'zzz.jp','zzz_005','c@ex.com')
,(TD_TIME_PARSE('2023/06/25'),'zzz.jp','zzz_005','c@ex.com')
) AS t(time, site, td_client_id, email);
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
DROP TABLE IF EXISTS ${td.database}.${td.tbl_aaa};
CREATE TABLE ${td.database}.${td.tbl_aaa} AS
SELECT
member_id, email, tel, name
FROM
( VALUES
('1', 'a@ex.com',1111,'Taka')
,('2', 'a@ex.com',2222,'Tatsuo')
,('3', 'b@ex.com',3333,'Naruse')
,('3', 'b@ex.com',4444,'Yuichiro')
,(NULL,'c@ex.com',5555,'Minero')
,(NULL,'c@ex.com',6666,'Kaz')
) AS t(member_id, email, tel, name);
63 changes: 63 additions & 0 deletions tool-box/id-unification-samples/queries/insert_samples_ex5.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
INSERT INTO ${td.database}.${td.tbl_aaa}
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/06/05'), 'aaa.jp','aaa_003','3rd_016')
,(TD_TIME_PARSE('2023/06/15'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/25'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/07/05'), 'aaa.jp','aaa_004','3rd_017')
,(TD_TIME_PARSE('2023/07/15'), 'aaa.jp','aaa_004','3rd_018')
,(TD_TIME_PARSE('2023/07/25'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/05'), 'aaa.jp','aaa_005','3rd_018')
,(TD_TIME_PARSE('2023/08/15'), 'aaa.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/25'), 'aaa.jp','aaa_005','3rd_019')
) AS t(time, site, td_client_id, td_global_id);

INSERT INTO ${td.database}.${td.tbl_xxx}
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/06/05'), 'xxx.jp','xxx_004','3rd_016')
,(TD_TIME_PARSE('2023/06/15'), 'xxx.jp','xxx_004','3rd_017')
,(TD_TIME_PARSE('2023/06/25'), 'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/07/05'), 'xxx.jp','xxx_005','3rd_018')
,(TD_TIME_PARSE('2023/07/15'), 'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/07/25'), 'xxx.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/05'), 'xxx.jp','xxx_005','3rd_018')
,(TD_TIME_PARSE('2023/08/15'), 'xxx.jp','xxx_006','3rd_019')
,(TD_TIME_PARSE('2023/08/25'), 'xxx.jp',NULL,NULL)
) AS t(time, site, td_client_id, td_global_id);

INSERT INTO ${td.database}.${td.tbl_yyy}
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/06/05'), 'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/15'), 'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/25'), 'yyy.jp','yyy_005','3rd_018')
,(TD_TIME_PARSE('2023/07/05'), 'yyy.jp','yyy_006','3rd_019')
,(TD_TIME_PARSE('2023/07/15'), 'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/07/25'), 'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/05'), 'yyy.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/15'), 'yyy.jp','yyy_006','3rd_019')
,(TD_TIME_PARSE('2023/08/25'), 'yyy.jp','yyy_007','3rd_019')
) AS t(time, site, td_client_id, td_global_id);

INSERT INTO ${td.database}.${td.tbl_zzz}
SELECT
time, site, td_client_id, td_global_id
FROM
( VALUES
(TD_TIME_PARSE('2023/06/05'), 'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/06/15'), 'zzz.jp','zzz_005','3rd_017')
,(TD_TIME_PARSE('2023/06/25'), 'zzz.jp','zzz_005','3rd_018')
,(TD_TIME_PARSE('2023/07/05'), 'zzz.jp','zzz_006','3rd_018')
,(TD_TIME_PARSE('2023/07/15'), 'zzz.jp','zzz_007','3rd_018')
,(TD_TIME_PARSE('2023/07/25'), 'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/05'), 'zzz.jp',NULL,NULL)
,(TD_TIME_PARSE('2023/08/15'), 'zzz.jp','zzz_008','3rd_019')
,(TD_TIME_PARSE('2023/08/25'), 'zzz.jp',NULL,NULL)
) AS t(time, site, td_client_id, td_global_id);
33 changes: 33 additions & 0 deletions tool-box/id-unification-samples/test_id_unification_ex1.dig
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
_export:
td:
database: test_id_unification_ex1
unif_name: test_id_unification_ex1
ms_name: master_table_ex1
!include : config/params.yml

##### Comment out all but the first execution.
+prepare_dbs:
td_ddl>:
create_databases: ["${td.database}"]

+generate_samples:
td>: queries/generate_samples_ex1.sql
#####

+call_unification:
http_call>: https://api-cdp.treasuredata.com/unifications/workflow_call
headers:
- authorization: ${secret:td.apikey}
method: POST
retry: true
content_format: json
content:
run_canonical_ids: true
run_enrichments: true
run_master_tables: true

full_refresh: true
keep_debug_tables: true

unification:
!include : unification_ex1.yml
34 changes: 34 additions & 0 deletions tool-box/id-unification-samples/test_id_unification_ex2.dig
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
_export:
td:
database: test_id_unification_ex2
unif_name: test_id_unification_ex2
ms_name: master_table_ex2
!include : config/params.yml

##### Comment out all but the first execution.
+prepare_dbs:
td_ddl>:
create_databases: ["${td.database}"]

+generate_samples:
td>: queries/generate_samples_ex2.sql
#####


+call_unification:
http_call>: https://api-cdp.treasuredata.com/unifications/workflow_call
headers:
- authorization: ${secret:td.apikey}
method: POST
retry: true
content_format: json
content:
run_canonical_ids: true
run_enrichments: true
run_master_tables: true

full_refresh: true
keep_debug_tables: true

unification:
!include : unification_ex2.yml
Loading