Updated docs (#4)

* Updated docs
Co-authored-by: Alex Ilyichov <alexander.ilyichov@fivetran.com>

---------

Co-authored-by: Adalbert Makarovych <amakarovych0ua@singlestore.com>
Co-authored-by: Alex Ilyichov <alexander.ilyichov@fivetran.com>

docs/overview.md

@@ -7,10 +7,11 @@ hidden: false
 
 # SingleStore {% typeBadge connector="singlestore" /%} {% availabilityBadge connector="singlestore" /%}
 
-[SingleStore](https://www.singlestore.com/) is a distributed, cloud-native database that can handle
-transactional and analytical workloads with a unified engine. It provides real-time analytics,
-transactions, and streaming capabilities, enabling users to handle diverse workloads on a single
-platform.
+[SingleStore](https://www.singlestore.com/) is a distributed, cloud-native database that can handle transactional and analytical workloads with a unified engine. It provides real-time analytics, transactions, and streaming capabilities, enabling users to handle diverse workloads on a single platform.
+
+> NOTE: this connector utilizes SingleStore's [OBSERVE](https://docs.singlestore.com/cloud/reference/sql-reference/data-manipulation-language-dml/observe/) queries that are currently in a preview state. As such, they are intended for experimental use only. queries which are currently in a preview state. As such, they are intended for experimental use only.
+
+> WARNING: This connector currently doesn't work with [Unlimited Storage Databases](https://docs.singlestore.com/db/v8.7/manage-data/local-and-unlimited-database-storage-concepts/).
 
 ------------------
 
@@ -27,49 +28,37 @@ Private networking: AWS Private Link, GCP Private Service Connect
 
 ## Setup guide
 
-Follow our [step-by-step SingleStore setup guide](/docs/{path}/setup-guide) to connect
-SingleStore with your destination using Fivetran connectors.
+Follow our [step-by-step SingleStore setup guide](/docs/{path}/setup-guide) to connect SingleStore with your destination using Fivetran connectors.
 
 ------------------
 
 ## Sync overview
 
-Once Fivetran is connected to your SingleStore deployment, the connector fetches an initial
-consistent snapshot of all data from your SingleStore table. Once the initial sync is complete, the
-connector stream UPDATE/DELETE/INSERT events made to your SingleStore table.
+Once Fivetran is connected to your SingleStore deployment, the connector fetches an initial consistent snapshot of all data from your SingleStore table. Once the initial sync is complete, the connector stream UPDATE/DELETE/INSERT events made to your SingleStore table.
 
-SingleStore connector
-uses [OBSERVE](https://docs.singlestore.com/cloud/reference/sql-reference/data-manipulation-language-dml/observe/)
-to capture change events.
+SingleStore connector uses [OBSERVE](https://docs.singlestore.com/cloud/reference/sql-reference/data-manipulation-language-dml/observe/) to capture change events.
 
-Connector does not support handling schema changes. You cannot run `ALTER` and `DROP` queries
-while the `OBSERVE` query is running.
+Connector does not support handling schema changes. You cannot run `ALTER` and `DROP` queries while the `OBSERVE` query is running.
 
 ------------------
 
 ## Schema information
 
-Fivetran replicates a single table of the SingleStore database. Selected SingleStore database is
-mapped to Fivetran schema.
+Fivetran replicates a single table of the SingleStore database. Selected SingleStore database is mapped to Fivetran schema.
 
 ### Fivetran-generated columns
 
 Fivetran adds the following columns to table in your destination:
 
 - `_fivetran_deleted` (BOOLEAN) marks deleted rows in the source database.
 - `_fivetran_synced` (UTC TIMESTAMP) indicates when Fivetran last successfully synced the row.
-- `_fivetran_index` (INTEGER) shows the order of updates for tables that do not have a primary key.
-- `_fivetran_id` (STRING) is the hash of the non-Fivetran values of each row. It's a unique ID that
-  Fivetran uses to avoid duplicate rows in tables that do not have a primary key.
+- `InternalId` (BINARY) is a unique ID that connector adds to distinguish rows in tables that do not have a primary key.
 
 ### Type transformations and mapping
 
-As we extract your data, we match SingleStore data types in your SingleStore database to types that
-Fivetran supports. If we don't support a specific data type, we automatically change that type to
-the closest supported type.
+As we extract your data, we match SingleStore data types in your SingleStore database to types that Fivetran supports. If we don't support a specific data type, we automatically change that type to the closest supported type.
 
-The following table illustrates how we transform your SingleStore data types into Fivetran supported
-types:
+The following table illustrates how we transform your SingleStore data types into Fivetran supported types:
 
 | SingleStore Data Type | Fivetran Data Type | Notes                                                                                                                              |
 |-----------------------|--------------------|------------------------------------------------------------------------------------------------------------------------------------|

docs/setup-guide.md

@@ -45,16 +45,21 @@ To authorize Fivetran to connect to your SinlgeStore database, follow these inst
 
 ### <span class="step-item">Configure SingleStore</span>
 
-1. Configure your firewall and/or other access control systems to allow incoming connections to your
-   SingleStore instance from [Fivetran's IPs](https://fivetran.com/docs/using-fivetran/ips) for your
-   region.
+1. Configure your firewall and/or other access control systems to allow incoming connections to your SingleStore instance from [Fivetran's IPs](https://fivetran.com/docs/using-fivetran/ips) for your region.
 2. Ensure that the SingleStore database user has the `SELECT` permission.
 3. Enable support of OBSERVE queries
 
 ```
 SET enable_observe_queries=1
 ```
 
+4. (Optional) Configure `snapshots_to_keep` and `snapshot_trigger_size` [engine variables](https://docs.singlestore.com/cloud/reference/configuration-reference/engine-variables/list-of-engine-variables/). At some point, `offsets` will be considered stale, meaning the connector will no longer be able to retrieve data associated with that logical point in the WAL (Write-Ahead Log). In practical terms, `offsets` become stale once they are older than the oldest snapshot in the system. The `snapshots_to_keep` and `snapshot_trigger_size` variables control the number and size of snapshots providing you with some control over the data retention window. If you increase `snapshots_to_keep` or `snapshot_trigger_size`, offsets will become stale later. If offsets become stale, the connector will be unable to continue streaming change events, and the only way to resolve this is to re-sync all data.
+
+```
+SET GLOBAL snapshot_trigger_size=10737418240
+SET GLOBAL snapshots_to_keep=4
+```
+
 ### <span class="step-item">Finish Fivetran configuration </span>
 
 1. Log in to your Fivetran account.
@@ -82,6 +87,13 @@ Fivetran performs the following SingleStore connection tests:
   provided in the setup form
 - The Table test checks if specified table exists
 
+### <span class="step-item">Post setup changes (Optional)</span>
+
+We recommend to adjust the sync frequency in Fivetran so that syncs run more often. This will reduce the likelihood of `offsets` becoming stale and help avoid the need for a full re-sync.
+
+1. Click on `Setup`
+2. Change `Sync frequency` to `15 minutes`
+
 ---
 
 ## Related articles