Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft: table level RAW #524

Draft
wants to merge 41 commits into
base: main
Choose a base branch
from
Draft

Draft: table level RAW #524

wants to merge 41 commits into from

Conversation

terry-xuan-gao
Copy link
Contributor

Related Issue(s) & Descriptions

Checklist

  • Tests were added or are not required
  • Documentation was added or is not required

@terry-xuan-gao
add: latest_gtid_for_table.go
9a69582 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
deal: safe_session.go with LatestGTIDForTable
7e760f4 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
deal: scatter_conn.go with LatestGTIDForTable
db2b0a4 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: e2e test
3c1c947 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: tools version latest->0.24.0
2ffe951 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: tools version latest->0.24.0
8f50f5f 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
feat: update gtid
7241a8e 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
update: query.proto
9417b7c 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
update: .pb.go
d013467 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: go.mod go.sum
4843ef7 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
gerayking
gerayking reviewed Sep 14, 2024
View reviewed changes
go/vt/vtgate/scatter_conn.go Outdated
@@ -296,6 +296,9 @@ func (stc *ScatterConn) ExecuteMultiShard(
if qr.SessionStateChanges != "" {
session.SetReadAfterWriteGTID(qr.SessionStateChanges)
stc.gateway.AddGtid(qr.SessionStateChanges)

tableName := primitive.GetTableName()
session.latestGTIDForTable.UpdateGTID(tableName, qr.SessionStateChanges)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for network issues to cause a GTID rollback here?

For example:

SQL1 with GTID 1 is sent.
SQL2 with GTID 2 is sent.
MySQL server completes SQL1.
MySQL server completes SQL2.

Network error occurs.

Result 2 arrives at WeScale and triggers update_gtid(gtid2).
Result 1 arrives at WeScale and triggers update_gtid(gtid1).

The resulting GTID is 1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since session.GetReadAfterWrite().GetReadAfterWriteGtid() always stores the latest GTID at the session level. So, directly taking session.GetReadAfterWrite().GetReadAfterWriteGtid() to update latestGTIDForTable may avoid the problem of GTID rollback?

But I think what's more important here is that we should judge the sequence of the existing GTID and the GTID to be updated in latestGTIDForTable.
Since there is only one MySQL instance performing write operations, I understand that the returned GTIDs should all have the same UUID. Can I directly take the transaction number in the GTID for comparison?

@terry-xuan-gao
feat: get gtid map
042d799 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
update: vtgate.proto
db1c431 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
add: getReadAfterWriteGtid()
c4a81bf 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
update: init opts.TableReadAfterWriteGtidMap
55ca658 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: gtid rollback issue
f969358 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: init latestGTIDForTable in safeSession
f3fd223 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: add latest_gtid_for_table in ReadAfterWrite
0819854 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: scatter_conn.go
5f4a287 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: vtgate.proto
7269c30 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: vtgate.proto
43f38f6 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: vtgate.proto
ee20ef2 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: query_executor.go
9948c39 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: query_executor.go
c655033 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: query_executor.go
a256a89 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: query_executor.go
337e205 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: query_executor.go
0f8711f 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: query_executor.go
51d7751 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: scattor_conn.go
6d44dfb 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: scattor_conn.go
05363b6 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: scattor_conn.go
01478cb 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: safe_session.go
5b55897 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: safe_session.go
905d205 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: safe_session.go
72432fc 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: getRAWGtid
63916e7 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
feat: support set table_level
f34339d 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: getRAWGtid
a27e2cb 
Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
Copy link
Contributor Author

when session1 has a long transaction on t1, other sessions can achieve blocking when querying t1 and non-blocking when querying t2.
However, set @@table_level ="off"; has not taken effect. After closing the table-level RAW, querying t2 is still not blocked.

d124d30df42308c01bae15f64122285

@terry-xuan-gao
Test Table Level RAW (wesql#541)
1eb066d 
* add: latest_gtid_for_table.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* deal: safe_session.go with LatestGTIDForTable

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* deal: scatter_conn.go with LatestGTIDForTable

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: e2e test

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: tools version latest->0.24.0

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: tools version latest->0.24.0

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* feat: update gtid

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* update: query.proto

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* update: .pb.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: go.mod go.sum

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* feat: get gtid map

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* update: vtgate.proto

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* add: getReadAfterWriteGtid()

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* update: init opts.TableReadAfterWriteGtidMap

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: gtid rollback issue

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: init latestGTIDForTable in safeSession

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: add latest_gtid_for_table in ReadAfterWrite

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: scatter_conn.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: vtgate.proto

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: vtgate.proto

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: vtgate.proto

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: query_executor.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: query_executor.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: query_executor.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: query_executor.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: query_executor.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: query_executor.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: scattor_conn.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: scattor_conn.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: scattor_conn.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: safe_session.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: safe_session.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: safe_session.go

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: getRAWGtid

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* feat: support set table_level

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

* fix: getRAWGtid

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

---------

Signed-off-by: Terry Gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: reuse Parse(sql)
2d40c7f 
Signed-off-by: terry-xuan-gao <gaoxuanhit@qq.com>
@terry-xuan-gao
Merge branch 'feature/table_level_RAW' into ospp-dev
d391498
@terry-xuan-gao
fix: reuse Parse(sql)
fe5575b 
Signed-off-by: terry-xuan-gao <gaoxuanhit@qq.com>
@terry-xuan-gao
fix: function name
d9b4d32 
Signed-off-by: terry-xuan-gao <gaoxuanhit@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gerayking gerayking gerayking left review comments

@earayu earayu Awaiting requested review from earayu

@newborn22 newborn22 Awaiting requested review from newborn22

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@terry-xuan-gao @gerayking