ZDM-535, ZDM-403, ZDM-545

modules/ROOT/pages/change-read-routing.adoc

@@ -26,46 +26,13 @@ This is harmless but unnecessary.
 [#change-the-read-routing-configuration]
 == Change the read routing configuration
 
-Read routing is controlled by a mutable configuration variable.
-For more information, see  xref:manage-proxy-instances.adoc#change-mutable-config-variable[Change a mutable configuration variable].
+Read routing is controlled by a xref:manage-proxy-instances.adoc#change-mutable-config-variable[mutable configuration variable].
 
-. Connect to your Ansible Control Host container.
-+
-For example, `ssh` into the jumphost:
-+
-[source,bash]
-----
-ssh -F ~/.ssh/zdm_ssh_config jumphost
-----
-+
-Then, connect to the Ansible Control Host container:
-+
-[source,bash]
-----
-docker exec -it zdm-ansible-container bash
-----
-+
-.Result
-[%collapsible]
-====
-[source,bash]
-----
-ubuntu@52772568517c:~$
-----
-====
-
-. Edit the {product-proxy} core configuration file: `vars/zdm_proxy_core_config.yml`.
+. Edit the {product-proxy} core configuration file `vars/zdm_proxy_core_config.yml`.
 
 . Change the `primary_cluster` variable to `TARGET`.
 
-. Run the rolling restart playbook to apply the configuration change to your entire {product-proxy} deployment:
-+
-[source,bash]
-----
-ansible-playbook rolling_update_zdm_proxy.yml -i zdm_ansible_inventory
-----
-
-. Wait while Ansible restarts the {product-proxy} instances, one by one.
+. xref:ROOT:manage-proxy-instances.adoc#perform-a-rolling-restart-of-the-proxies[Perform a rolling restart] to apply the configuration change to your entire {product-proxy} deployment.
 
 Once the instances are restarted, all reads are routed to the target cluster instead of the origin cluster.
 

modules/ROOT/pages/components.adoc

@@ -41,7 +41,7 @@ While {product-proxy} is active, write requests are sent to both clusters to ens
 
 ==== Writes (dual-write logic)
 
-{product-proxy} sends every write operation (`INSERT`, `UPDATE`, `DELETE`) synchronously to both clusters at the requested consistency level:
+{product-proxy} sends every write operation (`INSERT`, `UPDATE`, `DELETE`) synchronously to both clusters at the client application's requested consistency level:
 
 * If the write is acknowledged in both clusters at the requested consistency level, then the operation returns a successful write acknowledgement to the client that issued the request.
 * If the write fails in either cluster, then {product-proxy} passes a write failure, originating from the primary cluster, back to the client.
@@ -64,6 +64,36 @@ The results of asynchronous reads aren't returned to the client because asynchro
 
 For more information, see xref:ROOT:enable-async-dual-reads.adoc[].
 
+==== Consistency levels
+
+{product-proxy} doesn't directly manage or track consistency levels.
+Instead, it passes the requested consistency level from the client application to each cluster (origin and target) when routing requests.
+
+For reads, the consistency level is always passed to the primary cluster, which always receives read requests.
+The request is then executed within the primary cluster at the requested consistency level.
+
+If asynchronous dual reads are enabled, the consistency level is passed to both clusters, and each cluster executes the read request at the requested consistency level independently.
+If the request fails to attain the required quorum on the primary cluster, the failure is returned to the client application as normal.
+However, failure of asynchronous reads on the secondary cluster are logged but not returned to the client application.
+
+For writes, the consistency level is passed to both clusters, and each cluster executes the write request at the requested consistency level independently.
+If either request fails to attain the required quorum, the failure is returned to the client application as normal.
+
+If either cluster is an {astra-db} database, be aware that `CL.ONE` isn't supported by {astra}.
+Requests sent with `CL.ONE` to {astra-db} databases always fail.
+{product-proxy} doesn't mute these failures because you need to be aware of them.
+You must adapt your client application to use a consistency level that is supported by both clusters to ensure that the migration is seamless and error-free.
+
+==== Timeouts and connection failures
+
+When requests are routed through {product-proxy}, there is a proxy-side timeout and application-side timeout.
+
+If a response isn't received within the timeout period (`xref:ROOT:manage-proxy-instances.adoc#zdm_proxy_request_timeout_ms[zdm_proxy_request_timeout_ms]`), nothing is returned to the request handling thread, and, by extension, no response is sent to the client.
+This inevitably results in a client-side timeout, which is an accurate representation of the fact that at least one cluster failed to respond to the request.
+The clusters that are required to respond depend on the type of request and whether asynchronous dual reads are enabled.
+
+See also xref:ROOT:feasibility-checklists.adoc#driver-retry-policy-and-query-idempotence[Driver retry policy and query idempotence] and `xref:manage-proxy-instances.adoc#zdm_proxy_max_stream_ids[zdm_proxy_max_stream_ids]`.
+
 === High availability and multiple {product-proxy} instances
 
 {product-proxy} is designed to be highly available and run a clustered fashion to avoid a single point of failure.
@@ -77,7 +107,7 @@ Throughout the {product-short} documentation, the term _{product-proxy} deployme
 ====
 
 You can scale {product-proxy} instances horizontally and vertically.
-To avoid downtime when applying configuration changes, you can perform rolling restarts on your {product-proxy} instances.
+To avoid downtime when applying configuration changes, you can xref:ROOT:manage-proxy-instances.adoc#perform-a-rolling-restart-of-the-proxies[perform a rolling restart] of your {product-proxy} instances.
 
 For simplicity, you can use {product-utility} and {product-automation} to set up and run Ansible playbooks that deploy and manage {product-proxy} and its monitoring stack.
 

modules/ROOT/pages/deploy-proxy-monitoring.adoc

@@ -170,11 +170,12 @@ For instructions, see xref:ROOT:tls.adoc[].
 
 There are additional configuration variables in `vars/zdm_proxy_advanced_config.yml` that you might want to change _at deployment time_ in specific cases.
 
-All advanced configuration variables not listed here are considered mutable and can be changed later if needed (changes can be easily applied to existing deployments in a rolling fashion using the relevant Ansible playbook, as explained later, see xref:manage-proxy-instances.adoc#change-mutable-config-variable[Change a mutable configuration variable]).
+All advanced configuration variables that aren't listed here are mutable and can be changed later without recreating the entire deployment.
+For more information, see xref:manage-proxy-instances.adoc[].
 
 ==== Multi-datacenter clusters
 
-For multi-datacenter origin clusters, specify the name of the datacenter that {product-proxy} should consider local.
+For xref:ROOT:feasibility-checklists.adoc[multi-datacenter origin clusters], specify the name of the datacenter that {product-proxy} should consider local.
 To do this, set the `origin_local_datacenter` property to the local datacenter name.
 Similarly, for multi-datacenter target clusters, set the `target_local_datacenter` property to the local datacenter name.
 These two variables are stored in `vars/zdm_proxy_advanced_config.yml`.
@@ -320,14 +321,11 @@ CONTAINER ID  IMAGE                     COMMAND  CREATED      STATUS     PORTS
 
 If the {product-proxy} instances fail to start up due to mistakes in the configuration, you can simply rectify the incorrect configuration values and run the deployment playbook again.
 
-[NOTE]
+[IMPORTANT]
 ====
-With the exception of the origin credentials, target credentials, and the `primary_cluster` variable, which can all be changed for existing deployments in a rolling fashion, all cluster connection configuration variables are considered immutable and can only be changed by recreating the deployment.
-
-If you wish to change any of the cluster connection configuration variables (other than credentials and `primary_cluster`) on an existing deployment, you will need to re-run the `deploy_zdm_proxy.yml` playbook.
-This playbook can be run as many times as necessary.
-
-Be aware that running the `deploy_zdm_proxy.yml` playbook results in a brief window of unavailability of the whole {product-proxy} deployment while all the {product-proxy} instances are torn down and recreated.
+The origin credentials, target credentials, and the `primary_cluster` variable are mutable variables that you can change after deploying {product-proxy}.
+All other cluster connection configuration variables are immutable; the only way to change these values is by completely recreating the {product-proxy} deployment.
+For more information, see xref:ROOT:manage-proxy-instances.adoc[].
 ====
 
 [[_setting_up_the_monitoring_stack]]

modules/ROOT/pages/deployment-infrastructure.adoc

@@ -14,12 +14,19 @@ The {product-proxy} process is lightweight, requiring only a small amount of res
 {product-proxy} should be deployed close to your client application instances.
 This can be on any cloud provider as well as on-premise, depending on your existing infrastructure.
 
-If you have a multi-DC cluster with multiple set of client application instances deployed to geographically distributed data centers, you should plan for a separate {product-proxy} deployment for each data center.
-
-Here's a typical deployment showing connectivity between client applications, {product-proxy} instances, and clusters:
+The following diagram shows a typical deployment with connectivity between client applications, {product-proxy} instances, and clusters:
 
 image::zdm-during-migration3.png[Connectivity between client applications, proxy instances, and clusters.]
 
+=== Multiple datacenter clusters
+
+If you have a multi-datacenter cluster with multiple set of client application instances deployed to geographically distributed datacenters, you must plan a separate {product-proxy} deployment for each datacenter.
+
+In the configuration for each {product-proxy} deployment, specify only the contact points that belong to that datacenter, and set the `xref:ROOT:deploy-proxy-monitoring.adoc#_advanced_configuration_optional[origin_local_datacenter]` and `xref:ROOT:deploy-proxy-monitoring.adoc#_advanced_configuration_optional[target_local_datacenter]` properties as needed.
+
+If your origin and target clusters are both multi-datacenter clusters, this configuration will be more complicated to correctly orchestrate traffic routing through {product-proxy}.
+{company} recommends contacting {support-url}[{company} Support] for assistance with complex multi-region and multi-datacenter migrations.
+
 === Don't deploy {product-proxy} as a sidecar
 
 Don't deploy {product-proxy} as a sidecar because it was designed to mimic communication with a {cass-short}-based cluster.

modules/ROOT/pages/enable-async-dual-reads.adoc

@@ -34,9 +34,9 @@ To avoid unnecessary failures due to missing unmigrated data, don't enable async
 == Configure asynchronous dual reads
 
 Use the `read_mode` variable to enable or disable asynchronous dual reads.
-Then, perform rolling restarts of your {product-proxy} instances to apply the configuration change.
+Then, perform a rolling restart of your {product-proxy} instances to apply the configuration change.
 
-. In `vars/zdm_proxy_core_config.yml`, edit the `read_mode` variable:
+. Edit `vars/zdm_proxy_core_config.yml`, and then set the `read_mode` variable:
 +
 [tabs]
 ======
@@ -59,31 +59,7 @@ read_mode: PRIMARY_ONLY
 --
 ======
 
-. Perform rolling restarts to apply the configuration change to your {product-proxy} instances.
-+
-[tabs]
-======
-With {product-automation}::
-+
---
-If you use {product-automation} to manage your {product-proxy} deployment, run the following command:
-
-[source,bash]
-----
-ansible-playbook rolling_update_zdm_proxy.yml -i zdm_ansible_inventory
-----
---
-
-Without {product-automation}::
-+
---
-If you don't use {product-automation}, you must manually restart each instance.
-
-To avoid downtime, wait for each instance to fully restart and begin receiving traffic before restarting the next instance.
---
-======
-+
-For more information about rolling restarts and changing {product-proxy} configuration variables, see xref:manage-proxy-instances.adoc[].
+. xref:ROOT:manage-proxy-instances.adoc#perform-a-rolling-restart-of-the-proxies[Perform a rolling restart] to apply the configuration change to your {product-proxy} instances.
 
 == Monitor the target cluster's performance
 

modules/ROOT/pages/feasibility-checklists.adoc

@@ -62,6 +62,10 @@ For example, if a table has 10 columns but your client application only uses 5 o
 You can also change the primary key in some cases.
 For example, if your compound primary key is `PRIMARY KEY (A, B)` and you always provide parameters for the `A` and `B` columns in your CQL statements then you could change the key to `PRIMARY KEY (B, A)` when creating the schema on the target because your CQL statements will still run successfully.
 
+== Request and error handling expectations with {product-proxy}
+
+See xref:ROOT:components.adoc#how-zdm-proxy-handles-reads-and-writes[How {product-proxy} handles reads and writes].
+
 == Considerations for {astra-db} migrations
 
 {astra-db} implements guardrails and sets limits to ensure good practices, foster availability, and promote optimal configurations for your databases.
@@ -71,6 +75,26 @@ If you need to make changes to the application or data model to ensure that your
 
 It is also highly recommended to perform tests and benchmarks when connected directly to {astra-db} prior to the migration, so that you don't find unexpected issues during the migration process.
 
+=== {astra} doesn't support CL.ONE
+
+`CL.ONE` isn't supported by {astra}, and read and write requests sent through {product-proxy} with `CL.ONE` to {astra-db} databases always fail.
+
+{product-proxy} doesn't mute these failures because you need to be aware of them.
+You must adapt your client application to use a consistency level that is supported by both clusters to ensure that the migration is seamless and error-free.
+
+=== {astra} doesn't support the Stargate APIs
+
+The Stargate APIs (Document, REST, GraphQL, gRPC) are deprecated for {astra}.
+
+If you are migrating to {astra} from an origin cluster that uses any of these APIs, your client applications won't work with {astra}.
+Before you migrate, you must change your applications to use other programmatic access, such as {cass-short} drivers or the {data-api}.
+For more information, see xref:astra-db-serverless:api-reference:compare-dataapi-to-stargate.adoc[].
+
+=== Multi-region and multi-node {astra} migrations
+
+If your migration involves multi-region or multi-node clusters, plan your strategy for migrating and replicating data to the different regions.
+For more information, see the xref:sideloader:prepare-sideloader.adoc#additional-preparation-for-specific-migration-scenarios[{sstable-sideloader} preparations for specific migration scenarios] and the xref:ROOT:deployment-infrastructure.adoc[{product-proxy} infrastructure guidelines for multi-datacenter clusters].
+
 [[_read_only_applications]]
 === Read-only applications
 
@@ -79,11 +103,8 @@ The default interval is 30,000 milliseconds, and it can be configured with the `
 
 In {product-proxy} versions earlier than 2.1.0, read-only applications require special handling to avoid connection termination due to inactivity.
 
-{company} recommends that you upgrade to version 2.1.0 or later to benefit from the heartbeat feature.
-If you have an existing {product-proxy} deployment, you can xref:ROOT:troubleshooting-tips.adoc#check-version[check your {product-proxy} version].
-For upgrade instructions, see xref:ROOT:manage-proxy-instances.adoc#_upgrade_the_proxy_version[Upgrade the proxy version].
-
-If you cannot upgrade to version 2.1.0 or later, see the alternatives described in xref:ROOT:troubleshooting-tips.adoc#client-application-closed-connection-errors-every-10-minutes-when-migrating-to-astra-db[Client application closed connection errors every 10 minutes when migrating to {astra-db}].
+{company} recommends that you use {product-proxy} version 2.1.0 or later to benefit from the heartbeat feature.
+If you cannot use version 2.1.0 or later, see the alternatives described in xref:ROOT:troubleshooting-tips.adoc#client-application-closed-connection-errors-every-10-minutes-when-migrating-to-astra-db[Client application closed connection errors every 10 minutes when migrating to {astra-db}].
 
 [[non-idempotent-operations]]
 == Lightweight Transactions and other non-idempotent operations
@@ -172,6 +193,7 @@ If the performance impact is unacceptable for your application, or you are using
 Most drivers have utility methods that help you compute these values locally.
 For more information, see your driver's documentation and xref:datastax-drivers:developing:query-timestamps.adoc[Query timestamps in {cass-short} drivers].
 
+[#driver-retry-policy-and-query-idempotence]
 == Driver retry policy and query idempotence
 
 [IMPORTANT]