Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change keys of the discovery flags such that they conform to the convention #17430

Merged
merged 2 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions changelog/22.0/22.0.0/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- **[RPC Changes](#rpc-changes)**
- **[Prefer not promoting a replica that is currently taking a backup](#reparents-prefer-not-backing-up)**
- **[VTOrc Config File Changes](#vtorc-config-file-changes)**
- **[VTGate Config File Changes](#vtgate-config-file-changes)**
- **[Support for More Efficient JSON Replication](#efficient-json-replication)**
- **[Minor Changes](#minor-changes)**
- **[VTTablet Flags](#flags-vttablet)**
Expand Down Expand Up @@ -60,6 +61,19 @@ The following fields can be dynamically changed -

To upgrade to the newer version of the configuration file, first switch to using the flags in your current deployment before upgrading. Then you can switch to using the configuration file in the newer release.

### <a id="vtgate-config-file-changes"/>VTGate Config File Changes</a>

The Viper configuration keys for the following flags has been changed to match their flag names. Previously they had a discovery prefix instead of it being part of the name.

| Flag Name | Old Configuration Key | New Configuration Key |
|--------------------------------------------------|--------------------------------------------------|--------------------------------------------------|
| `discovery_low_replication_lag` | `discovery.low_replication_lag` | `discovery_low_replication_lag` |
| `discovery_high_replication_lag_minimum_serving` | `discovery.high_replication_lag_minimum_serving` | `discovery_high_replication_lag_minimum_serving` |
| `discovery_min_number_serving_vttablets` | `discovery.min_number_serving_vttablets` | `discovery_min_number_serving_vttablets` |
| `discovery_legacy_replication_lag_algorithm` | `discovery.legacy_replication_lag_algorithm` | `discovery_legacy_replication_lag_algorithm` |

To upgrade to the newer version of the configuration keys, first switch to using the flags in your current deployment before upgrading. Then you can switch to using the new configuration keys in the newer release.

### <a id="efficient-json-replication"/>Support for More Efficient JSON Replication</a>

In [#7345](https://github.com/vitessio/vitess/pull/17345) we added support for [`--binlog-row-value-options=PARTIAL_JSON`](https://dev.mysql.com/doc/refman/en/replication-options-binary-log.html#sysvar_binlog_row_value_options). You can read more about [this feature added to MySQL 8.0 here](https://dev.mysql.com/blog-archive/efficient-json-replication-in-mysql-8-0/).
Expand Down
6 changes: 5 additions & 1 deletion go/test/endtoend/cluster/vtgate_process.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,11 @@ type VtgateProcess struct {
}

type VTGateConfiguration struct {
TransactionMode string `json:"transaction_mode,omitempty"`
TransactionMode string `json:"transaction_mode,omitempty"`
DiscoveryLowReplicationLag string `json:"discovery_low_replication_lag,omitempty"`
DiscoveryHighReplicationLag string `json:"discovery_high_replication_lag,omitempty"`
DiscoveryMinServingVttablets string `json:"discovery_min_number_serving_vttablets,omitempty"`
DiscoveryLegacyReplicationLagAlgo string `json:"discovery_legacy_replication_lag_algorithm"`
}

// ToJSONString will marshal this configuration as JSON
Expand Down
75 changes: 75 additions & 0 deletions go/test/endtoend/vtgate/misc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -814,6 +814,81 @@ func TestDDLTargeted(t *testing.T) {
utils.AssertMatches(t, conn, `select id from ddl_targeted`, `[[INT64(1)]]`)
}

// TestDynamicConfig tests the dynamic configurations.
func TestDynamicConfig(t *testing.T) {
t.Run("DiscoveryLowReplicationLag", func(t *testing.T) {
// Test initial config value
err := clusterInstance.VtgateProcess.WaitForConfig(`"discovery_low_replication_lag":30000000000`)
require.NoError(t, err)
defer func() {
// Restore default back.
clusterInstance.VtgateProcess.Config.DiscoveryLowReplicationLag = "30s"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
}()
clusterInstance.VtgateProcess.Config.DiscoveryLowReplicationLag = "15s"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
// Test final config value.
err = clusterInstance.VtgateProcess.WaitForConfig(`"discovery_low_replication_lag":"15s"`)
require.NoError(t, err)
})

t.Run("DiscoveryHighReplicationLag", func(t *testing.T) {
// Test initial config value
err := clusterInstance.VtgateProcess.WaitForConfig(`"discovery_high_replication_lag":7200000000000`)
require.NoError(t, err)
defer func() {
// Restore default back.
clusterInstance.VtgateProcess.Config.DiscoveryHighReplicationLag = "2h"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
}()
clusterInstance.VtgateProcess.Config.DiscoveryHighReplicationLag = "1h"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
// Test final config value.
err = clusterInstance.VtgateProcess.WaitForConfig(`"discovery_high_replication_lag":"1h"`)
require.NoError(t, err)
})

t.Run("DiscoveryMinServingVttablets", func(t *testing.T) {
// Test initial config value
err := clusterInstance.VtgateProcess.WaitForConfig(`"discovery_min_number_serving_vttablets":2`)
require.NoError(t, err)
defer func() {
// Restore default back.
clusterInstance.VtgateProcess.Config.DiscoveryMinServingVttablets = "2"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
}()
clusterInstance.VtgateProcess.Config.DiscoveryMinServingVttablets = "1"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
// Test final config value.
err = clusterInstance.VtgateProcess.WaitForConfig(`"discovery_min_number_serving_vttablets":"1"`)
require.NoError(t, err)
})

t.Run("DiscoveryLegacyReplicationLagAlgo", func(t *testing.T) {
// Test initial config value
err := clusterInstance.VtgateProcess.WaitForConfig(`"discovery_legacy_replication_lag_algorithm":""`)
require.NoError(t, err)
defer func() {
// Restore default back.
clusterInstance.VtgateProcess.Config.DiscoveryLegacyReplicationLagAlgo = "true"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
}()
clusterInstance.VtgateProcess.Config.DiscoveryLegacyReplicationLagAlgo = "false"
err = clusterInstance.VtgateProcess.RewriteConfiguration()
require.NoError(t, err)
// Test final config value.
err = clusterInstance.VtgateProcess.WaitForConfig(`"discovery_legacy_replication_lag_algorithm":"false"`)
require.NoError(t, err)
})
}

func TestLookupErrorMetric(t *testing.T) {
conn, closer := start(t)
defer closer()
Expand Down
9 changes: 4 additions & 5 deletions go/vt/discovery/replicationlag.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,34 +28,33 @@ import (
)

var (
configKey = viperutil.KeyPrefixFunc("discovery")
// lowReplicationLag defines the duration that replication lag is low enough that the VTTablet is considered healthy.
lowReplicationLag = viperutil.Configure(
configKey("low_replication_lag"),
"discovery_low_replication_lag",
viperutil.Options[time.Duration]{
FlagName: "discovery_low_replication_lag",
Default: 30 * time.Second,
Dynamic: true,
},
)
highReplicationLagMinServing = viperutil.Configure(
configKey("high_replication_lag"),
"discovery_high_replication_lag",
viperutil.Options[time.Duration]{
FlagName: "discovery_high_replication_lag_minimum_serving",
Default: 2 * time.Hour,
Dynamic: true,
},
)
minNumTablets = viperutil.Configure(
configKey("min_number_serving_vttablets"),
"discovery_min_number_serving_vttablets",
viperutil.Options[int]{
FlagName: "min_number_serving_vttablets",
Default: 2,
Dynamic: true,
},
)
legacyReplicationLagAlgorithm = viperutil.Configure(
configKey("legacy_replication_lag_algorithm"),
"discovery_legacy_replication_lag_algorithm",
viperutil.Options[bool]{
FlagName: "legacy_replication_lag_algorithm",
Default: true,
Expand Down
Loading