Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vtorc: add context.Context to more logic, general cleanup #17903

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

timvaillancourt
Copy link
Contributor

@timvaillancourt timvaillancourt commented Mar 4, 2025

Description

This PR:

  • Adds context.Context to more go/vt/vtorc/inst and logic packages and passes a cancelable context from cli.go down
  • Reduces UnseenInstanceForgetHours from 240 -> 12 - 12 hours is a long time to be unseen
  • Replaces calls to inst.GetKeyspaceShardName with inst.ReadTablet (deletes former)
  • Renames inst.ReadOutdatedInstanceKeys -> inst.ReadOutdatedInstances and accept func vs return slice
  • Moves code that is not backend DAO code out of *_dao.go -> new/existing files

Related Issue(s)

#17330

Checklist

  • "Backport to:" labels have been added if this change should be back-ported to release branches
  • If this change is to be back-ported to previous releases, a justification is included in the PR description
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on CI?
  • Documentation was added or is not required

Deployment Notes

@timvaillancourt timvaillancourt added Type: Internal Cleanup Component: VTorc Vitess Orchestrator integration labels Mar 4, 2025
@timvaillancourt timvaillancourt self-assigned this Mar 4, 2025
Copy link
Contributor

vitess-bot bot commented Mar 4, 2025

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

  • Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

  • Apply the release notes (needs details) label if users need to know about this change.
  • New features should be documented.
  • There should be some code comments as to why things are implemented the way they are.
  • There should be a comment at the top of each new or modified test to explain what the test does.

New flags

  • Is this flag really necessary?
  • Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
  • vtctl command output order should be stable and awk-able.

@vitess-bot vitess-bot bot added NeedsBackportReason If backport labels have been applied to a PR, a justification is required NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Mar 4, 2025
@github-actions github-actions bot added this to the v22.0.0 milestone Mar 4, 2025
@timvaillancourt timvaillancourt removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says NeedsIssue A linked issue is missing for this Pull Request NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Mar 5, 2025
ts = topo.Open()
tmc = inst.InitializeTMC()
// Clear existing cache and perform a new refresh.
if _, err := db.ExecVTOrc("DELETE FROM vitess_tablet"); err != nil {
if err := inst.DeleteAllTablets(); err != nil {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DELETE FROM should be in inst DAO code

)

// ErrTabletAliasNil is a fixed error message.
var ErrTabletAliasNil = errors.New("tablet alias is nil")
var tmc tmclient.TabletManagerClient
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TMClient stuff isn't really DAO code

@@ -711,38 +692,17 @@ func ReadInstancesWithErrantGTIds(keyspace string, shard string) ([]*Instance, e
return readInstancesByCondition(condition, args, "")
}

// GetKeyspaceShardName gets the keyspace shard name for the given instance key
func GetKeyspaceShardName(tabletAlias string) (keyspace string, shard string, err error) {
Copy link
Contributor Author

@timvaillancourt timvaillancourt Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inst.ReadTablet (vs .GetKeyspaceShardName) will get you keyspace/shard from the same table, so let's just use that 🤷

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
@timvaillancourt timvaillancourt force-pushed the vtorc-more-code-cleanup branch from 5c86191 to 750567b Compare March 5, 2025 00:20
@@ -36,7 +36,7 @@ const (
DiscoveryQueueCapacity = 100000
DiscoveryQueueMaxStatisticsSize = 120
DiscoveryCollectionRetentionSeconds = 120
UnseenInstanceForgetHours = 240 // Number of hours after which an unseen instance is forgotten
UnseenInstanceForgetHours = 12 // Number of hours after which an unseen instance is forgotten
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1/2 a day unseen feels like long enough

@timvaillancourt timvaillancourt marked this pull request as ready for review March 5, 2025 00:35
defer func() {
if r := recover(); r != nil {
err = logReadTopologyInstanceError(tabletAlias, "Unexpected, aborting", tb.Errorf("%+v", r))
}
}()

var waitGroup sync.WaitGroup
Copy link
Contributor Author

@timvaillancourt timvaillancourt Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This waitGroup is never used aside from a waitGroup.Wait() call

There is never a waitGroup.Add(n) or .Done() 🤷

Copy link

codecov bot commented Mar 5, 2025

Codecov Report

Attention: Patch coverage is 17.97753% with 73 lines in your changes missing coverage. Please review.

Project coverage is 67.51%. Comparing base (2c10c95) to head (750567b).
Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
go/vt/vtorc/logic/vtorc.go 0.00% 16 Missing ⚠️
go/vt/vtorc/inst/instance_utils.go 16.66% 10 Missing ⚠️
go/vt/vtorc/logic/tablet_discovery.go 0.00% 10 Missing ⚠️
go/vt/vtorc/logic/keyspace_shard_discovery.go 0.00% 9 Missing ⚠️
go/vt/vtorc/logic/topology_recovery.go 0.00% 8 Missing ⚠️
go/vt/vtorc/inst/tablet.go 0.00% 7 Missing ⚠️
go/cmd/vtorc/cli/cli.go 0.00% 4 Missing ⚠️
go/vt/vtorc/inst/instance_dao.go 63.63% 4 Missing ⚠️
go/vt/vtorc/inst/tablet_dao.go 0.00% 3 Missing ⚠️
go/vt/vtorc/server/discovery.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #17903      +/-   ##
==========================================
- Coverage   67.52%   67.51%   -0.01%     
==========================================
  Files        1594     1596       +2     
  Lines      259564   259660      +96     
==========================================
+ Hits       175266   175317      +51     
- Misses      84298    84343      +45     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@timvaillancourt timvaillancourt marked this pull request as draft March 5, 2025 00:48
@timvaillancourt
Copy link
Contributor Author

Will 👀 into the local_example/region_example failures

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: VTorc Vitess Orchestrator integration Type: Internal Cleanup
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant