Skip to content

Commit 79c7bd0

Browse files
authored
add docs on dynamic iceberg tables (#432)
1 parent d9eaa27 commit 79c7bd0

File tree

1 file changed

+121
-0
lines changed

1 file changed

+121
-0
lines changed

src/content/docs/snowflake/features/dynamic-tables.md

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,3 +88,124 @@ The output should be:
8888
| 2 | bar |
8989
+----+------+
9090
```
91+
92+
## Dynamic Iceberg Tables
93+
94+
Dynamic Iceberg Tables combine the auto-refresh capabilities of Dynamic Tables with the Apache Iceberg open table format. This allows you to create materialized views that automatically update from source queries while storing data in Iceberg format on external object storage (such as S3).
95+
96+
A Dynamic Iceberg Table consists of three key concepts:
97+
- **Dynamic table**: A materialized view that auto-refreshes from a source query on a schedule (controlled by `TARGET_LAG` and `WAREHOUSE`).
98+
- **Iceberg table**: A table stored in Apache Iceberg format on external object storage (configured via `EXTERNAL_VOLUME`, `CATALOG`, and `BASE_LOCATION`).
99+
- **External volume**: A Snowflake object that references an S3 bucket as the backing storage for Iceberg data files.
100+
101+
### Create an S3 bucket
102+
103+
Create a local S3 bucket using the `mb` command with the `awslocal` CLI:
104+
105+
```bash
106+
awslocal s3 mb s3://test-bucket
107+
```
108+
109+
### Create an external volume
110+
111+
Create an external volume to define the storage location for the Iceberg data files:
112+
113+
```sql showLineNumbers
114+
CREATE OR REPLACE EXTERNAL VOLUME test_volume
115+
STORAGE_LOCATIONS = (
116+
(
117+
NAME = 'aws-s3-test'
118+
STORAGE_PROVIDER = 'S3'
119+
STORAGE_BASE_URL = 's3://test-bucket/'
120+
STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/s3-role'
121+
)
122+
)
123+
```
124+
125+
### Create a source table
126+
127+
Create a source table that the Dynamic Iceberg Table will refresh from:
128+
129+
```sql
130+
CREATE TABLE source_table (id INT, name TEXT);
131+
INSERT INTO source_table(id, name) VALUES (1, 'foo'), (2, 'bar');
132+
```
133+
134+
### Create a Dynamic Iceberg Table
135+
136+
Create a Dynamic Iceberg Table using the `CREATE DYNAMIC ICEBERG TABLE` statement:
137+
138+
```sql showLineNumbers
139+
CREATE DYNAMIC ICEBERG TABLE my_dynamic_iceberg_table
140+
TARGET_LAG = '2 minutes'
141+
WAREHOUSE = test
142+
REFRESH_MODE = INCREMENTAL
143+
INITIALIZE = on_create
144+
EXTERNAL_VOLUME = 'test_volume'
145+
CATALOG = 'SNOWFLAKE'
146+
BASE_LOCATION = 'my_table_data'
147+
AS SELECT id, name FROM source_table;
148+
```
149+
150+
The output should be:
151+
152+
```sql
153+
+----------------------------------------------------------+
154+
| result |
155+
|----------------------------------------------------------+
156+
| Dynamic table MY_DYNAMIC_ICEBERG_TABLE successfully created. |
157+
+----------------------------------------------------------+
158+
```
159+
160+
### Query the Dynamic Iceberg Table
161+
162+
You can query the Dynamic Iceberg Table using a standard `SELECT` statement:
163+
164+
```sql
165+
SELECT * FROM my_dynamic_iceberg_table ORDER BY id;
166+
```
167+
168+
The output should be:
169+
170+
```sql
171+
+----+------+
172+
| ID | NAME |
173+
|----+------+
174+
| 1 | foo |
175+
| 2 | bar |
176+
+----+------+
177+
```
178+
179+
### Show Dynamic Tables
180+
181+
You can view the metadata of your Dynamic Iceberg Table using the `SHOW DYNAMIC TABLES` command. The `is_iceberg` column indicates whether the table is a Dynamic Iceberg Table:
182+
183+
```sql
184+
SHOW DYNAMIC TABLES LIKE 'my_dynamic_iceberg_table';
185+
```
186+
187+
### Rename a Dynamic Iceberg Table
188+
189+
You can rename a Dynamic Iceberg Table using the `ALTER DYNAMIC TABLE` statement:
190+
191+
```sql
192+
ALTER DYNAMIC TABLE my_dynamic_iceberg_table RENAME TO my_renamed_table;
193+
```
194+
195+
### Drop a Dynamic Iceberg Table
196+
197+
You can drop a Dynamic Iceberg Table using the `DROP DYNAMIC TABLE` statement:
198+
199+
```sql
200+
DROP DYNAMIC TABLE my_renamed_table;
201+
```
202+
203+
The output should be:
204+
205+
```sql
206+
+------------------------------------------+
207+
| status |
208+
|------------------------------------------+
209+
| MY_RENAMED_TABLE successfully dropped. |
210+
+------------------------------------------+
211+
```

0 commit comments

Comments
 (0)