Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VStream: When doing a full copy, first send table schemas as DDL statement events #17150

Closed
wants to merge 10,000 commits into from
Closed

VStream: When doing a full copy, first send table schemas as DDL statement events #17150

wants to merge 10,000 commits into from

Conversation

mattlord
Copy link
Contributor

@mattlord mattlord commented Nov 5, 2024

Description

WiP PoC for #17143

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported to release branches
  • If this change is to be back-ported to previous releases, a justification is included in the PR description
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on CI?
  • Documentation was added or is not required

frouioui and others added 30 commits July 30, 2024 18:20
@frouioui
Improve the queries upgrade/downgrade CI workflow by using same test …
1676ebd 
…code version as binary (vitessio#16494)

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@harshit-gangal
Modify distributed transaction commit flow (vitessio#16468)
258be53 
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@shlomi-noach
shcemadiff: support INSTANT DDL for changing column visibility (v…
5d2a604 
…itessio#16503)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@mcrauwel @shlomi-noach
clarify collations are also supported for db_charset (vitessio#16423)
d0927e7 
Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
schemadiff: Online DDL support, declarative based (vitessio#16462)
0933269 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Throttler: CheckThrottlerResponseCode to replace HTTP status codes (v…
bbb7a3f 
…itessio#16491)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Throttler: SelfMetric interface, simplify adding new throttler metr…
c834df8 
…ics (vitessio#16469)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Deprecate UpdateThrottlerConfig's --check-as-check-self and `--ch…
da99102 
…eck-as-check-shard` flags (vitessio#16507)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@rohit-nayak-ps
Multi-tenant Movetables: add e2e test for vdiff (vitessio#16309)
69e6fce 
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
@mattlord
VTAdmin: Upgrade websockets js package (vitessio#16504)
598faf3
@systay
bugfix: Allow cross-keyspace joins (vitessio#16520)
31de9fd 
Signed-off-by: Andres Taylor <andres@planetscale.com>
@vitess-bot @frouioui
Upgrade the Golang Dependencies (vitessio#16514)
f083e22 
Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: frouioui <frouioui@users.noreply.github.com>
@harshit-gangal
fix: show tables to use any keyspace on system schema (vitessio#16521)
1021543 
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@harshit-gangal
fix: sequence table next value acl permission to writer role (vitessi…
8ff4b9a 
…o#16509)

Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@systay
simplify merging logic (vitessio#16525)
660a097 
Signed-off-by: Andres Taylor <andres@planetscale.com>
@harshit-gangal
vindex function: error when keyspace not selected (vitessio#16534)
93d12eb 
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@harshit-gangal
vttablet api distributed transaction changes (vitessio#16506)
2714453 
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@beingnoble03
evalengine: Implement PERIOD_ADD (vitessio#16492)
8940eb5
@GuptaManan100
Fix: Offset planning in hash joins (vitessio#16540)
5629525 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@GrahamCampbell
Remove unused formatRelativeTime import (vitessio#16549)
b5c7cca 
Signed-off-by: Graham Campbell <GrahamCampbell@users.noreply.github.com>
@mattlord
VReplication: Properly ignore errors from trying to drop tables that …
c160f46 
…don't exist (vitessio#16505)

Signed-off-by: Matt Lord <mattalord@gmail.com>
@arthurschreiber
Fix RemoveTablet during TabletExternallyReparented causing connec…
342a4bd 
…tion issues (vitessio#16371)

Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
@shlomi-noach
Throttler/vreplication: fix app name used by VPlayer (vitessio#16578)
4a91e63 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Tablet throttler: remove cached metric associated with removed tablet (…
76434ad 
…vitessio#16555)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@GuptaManan100
Atomic Transaction bug fix with PRS disruption (vitessio#16576)
68c19b4 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@beingnoble03
evalengine: Implement PERIOD_DIFF (vitessio#16557)
32d1e60 
Signed-off-by: Noble Mittal <noblemittal@outlook.com>
@harshit-gangal
allow innodb_lock_wait_timeout as system variable (vitessio#16574)
942f97f 
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@mattlord
VReplication: Improve table plan builder errors (vitessio#16588)
d3dd2fa 
Signed-off-by: Matt Lord <mattalord@gmail.com>
@mattlord
VReplication: Return lock error everywhere that LockName fails (vites…
90b6bc5 
…sio#16560)

Signed-off-by: Matt Lord <mattalord@gmail.com>
@GuptaManan100
Add command to see the unresolved transactions (vitessio#16589)
6f345a9 
Signed-off-by: Manan Gupta <manan@planetscale.com>
vitess-bot and others added 12 commits October 29, 2024 06:46
@vitess-bot @rohit-nayak-ps
[main] Copy v21.0.0 release notes (vitessio#17097)
5b3074b 
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
@harshit-gangal
java package updates for grpc and protobuf and release plugins (vites…
a7f1eb0 
…sio#17100)

Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@frouioui @mattlord
Fix unreachable errors when taking a backup (vitessio#17062)
7713f9a 
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <35779988+frouioui@users.noreply.github.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
@frouioui
Add release-21.0 to the Golang Upgrade workflow (vitessio#16916)
b6fe439 
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@GuptaManan100
Fix flaky test TestShardSync (vitessio#17095)
f76b359 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@frouioui
Fix panic in vttablet when closing topo server twice (vitessio#17094)
c1f1dde
@mattlord
CI: Correct etcd and ncurses package names for Ubuntu 24.04 (vitessio…
d20460b 
…#16966)

Signed-off-by: Matt Lord <mattalord@gmail.com>
@beingnoble03
refac: Remove duplicate forAllShards methods from `vt/vtctl/workflo…
2191360 
…w` (vitessio#17025)

Signed-off-by: Noble Mittal <noblemittal@outlook.com>
@mattlord
VReplication: Fixes for generated column handling (vitessio#17107)
fbddae3 
Signed-off-by: Matt Lord <mattalord@gmail.com>
@GuptaManan100
Fix flakiness in TestDisruptions for two pc testing (vitessio#17106)
ced2010 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@dbussink @timvaillancourt
Move to native sqlite3 queries (vitessio#17124)
012e601 
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com>
@shlomi-noach
schemadiff: consistent key ordering in table diff (vitessio#17141)
f907205 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@mattlord mattlord added Type: Enhancement Logical improvement (somewhere between a bug and feature) Component: VReplication labels Nov 5, 2024
@vitess-bot vitess-bot
Copy link
Contributor

vitess-bot bot commented Nov 5, 2024

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

  • Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

  • Apply the release notes (needs details) label if users need to know about this change.
  • New features should be documented.
  • There should be some code comments as to why things are implemented the way they are.
  • There should be a comment at the top of each new or modified test to explain what the test does.

New flags

  • Is this flag really necessary?
  • Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
  • vtctl command output order should be stable and awk-able.

@vitess-bot vitess-bot bot added NeedsBackportReason If backport labels have been applied to a PR, a justification is required NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Nov 5, 2024
@mattlord mattlord removed NeedsIssue A linked issue is missing for this Pull Request NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Nov 5, 2024
@mattlord mattlord force-pushed the vstream_copy_schemas branch from b1490c7 to 9fdd09d Compare November 5, 2024 16:34
@github-actions github-actions bot added this to the v22.0.0 milestone Nov 5, 2024
@mattlord mattlord force-pushed the vstream_copy_schemas branch from 9fdd09d to 63e8dfe Compare November 5, 2024 16:44
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 4, 2025

This PR is being marked as stale because it has been open for 30 days with no activity. To rectify, you may do any of the following:

  • Push additional commits to the associated branch.
  • Remove the stale label.
  • Add a comment indicating why it is not stale.

If no action is taken within 7 days, this PR will be closed.

@github-actions github-actions bot added the Stale Marks PRs as stale after a period of inactivity, which are then closed after a grace period. label Jan 4, 2025
@github-actions GitHub Actions
Copy link
Contributor

This PR was closed because it has been stale for 7 days with no activity.

@github-actions github-actions bot closed this Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GuptaManan100 GuptaManan100 Awaiting requested review from GuptaManan100 GuptaManan100 will be requested when the pull request is marked ready for review GuptaManan100 is a code owner

@frouioui frouioui Awaiting requested review from frouioui frouioui will be requested when the pull request is marked ready for review frouioui is a code owner

@rohit-nayak-ps rohit-nayak-ps Awaiting requested review from rohit-nayak-ps rohit-nayak-ps will be requested when the pull request is marked ready for review rohit-nayak-ps is a code owner

@deepthi deepthi Awaiting requested review from deepthi deepthi will be requested when the pull request is marked ready for review deepthi is a code owner

@harshit-gangal harshit-gangal Awaiting requested review from harshit-gangal harshit-gangal will be requested when the pull request is marked ready for review harshit-gangal is a code owner

@systay systay Awaiting requested review from systay systay will be requested when the pull request is marked ready for review systay is a code owner

@shlomi-noach shlomi-noach Awaiting requested review from shlomi-noach shlomi-noach will be requested when the pull request is marked ready for review shlomi-noach is a code owner

@timvaillancourt timvaillancourt Awaiting requested review from timvaillancourt timvaillancourt will be requested when the pull request is marked ready for review timvaillancourt is a code owner

@derekperkins derekperkins Awaiting requested review from derekperkins derekperkins will be requested when the pull request is marked ready for review derekperkins is a code owner

@vmg vmg Awaiting requested review from vmg vmg will be requested when the pull request is marked ready for review vmg is a code owner

@ajm188 ajm188 Awaiting requested review from ajm188 ajm188 will be requested when the pull request is marked ready for review ajm188 is a code owner

@notfelineit notfelineit Awaiting requested review from notfelineit notfelineit will be requested when the pull request is marked ready for review notfelineit is a code owner

Assignees
No one assigned
Labels
Component: VReplication NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says Stale Marks PRs as stale after a period of inactivity, which are then closed after a grace period. Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Milestone
v22.0.0
Development

Successfully merging this pull request may close these issues.

Feature Request: richer tablet/column type info in VStream copy phase output