From 0802f79b47aee2a8c8951696bd09c13f6f875035 Mon Sep 17 00:00:00 2001 From: Tyler Treat Date: Mon, 24 Feb 2020 14:26:23 -0700 Subject: [PATCH 1/4] Add MsgType field to envelope protocol --- README.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 025723d..8fadeeb 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ with the following header and be encoded as a `Message` (defined in ├───────────────┴───────────────┴───────────────┴───────────────┤ │ Magic Number │ ├───────────────┬───────────────┬───────────────┬───────────────┤ -│ Version │ HeaderLen │ Flags │ Reserved │ +│ Version │ HeaderLen │ Flags │ MsgType │ ├───────────────┴───────────────┴───────────────┴───────────────┤ │ CRC-32C (optional) │ └───────────────────────────────────────────────────────────────┘ @@ -58,6 +58,16 @@ The flag bits are defined as follows: | --- | --------------- | | 0 | CRC-32C enabled | +#### MsgType [1 byte] + +This is the Liftbridge-specific message type the envelope contains: + +| MsgType | Description | +| ------- | ----------- | +| 0 | Publish | +| 1 | Ack | +| 2 | Replication | + #### CRC-32C [4 bytes, optional] The CRC-32C (Castagnoli) is the checksum of the payload (i.e. from HeaderLen to From 3470204da6fe741d05abcc1f805619750d06af44 Mon Sep 17 00:00:00 2001 From: Tyler Treat Date: Thu, 27 Feb 2020 15:33:56 -0700 Subject: [PATCH 2/4] Update MsgType table --- README.md | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 8fadeeb..6820e71 100644 --- a/README.md +++ b/README.md @@ -55,18 +55,30 @@ safety. The flag bits are defined as follows: | Bit | Description | -| --- | --------------- | +| :-- | :-------------- | | 0 | CRC-32C enabled | #### MsgType [1 byte] This is the Liftbridge-specific message type the envelope contains: -| MsgType | Description | -| ------- | ----------- | -| 0 | Publish | -| 1 | Ack | -| 2 | Replication | +| MsgType | Name | Description | Internal | +| :------ | :------------------------ | :----------------------------------------------------- | :------- | +| 0 | Publish | Client-published message | no | +| 1 | Ack | Server-published ack | no | +| 2 | ReplicationRequest | Request to replicate partition data | yes | +| 3 | ReplicationResponse | Response to ReplicationRequest | yes | +| 4 | RaftJoinRequest | Request to join Raft cluster | yes | +| 5 | RaftJoinResponse | Response to RaftJoinRequest | yes | +| 6 | LeaderEpochOffsetRequest | Request for partition leader's latest offset for epoch | yes | +| 7 | LeaderEpochOffsetResponse | Response to LeaderEpochOffsetRequest | yes | +| 8 | PropagatedRequest | Request forwarded to metadata leader | yes | +| 9 | PropagatedResponse | Response to PropagatedRequest | yes | +| 10 | ServerInfoRequest | Request for cluster information | yes | +| 11 | ServerInfoResponse | Response to ServerInfoRequest | yes | +| 12 | PartitionStatusRequest | Request to get partition status | yes | +| 13 | PartitionStatusResponse | Response to PartitionStatusRequest | yes | +| 14 | PartitionNotification | Signal new data is available for partition | yes | #### CRC-32C [4 bytes, optional] From fe585cf35d9aaca4e44494cae003053b73cfbb37 Mon Sep 17 00:00:00 2001 From: Tyler Treat Date: Thu, 27 Feb 2020 15:50:27 -0700 Subject: [PATCH 3/4] Update README.md --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 6820e71..233f61c 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,10 @@ with the following header and be encoded as a `Message` (defined in ### Liftbridge Envelope Header +All Liftbridge messages and RPCs sent over NATS are prefixed with an envelope +header. This includes client-facing messages, such as publishes and acks, as +well as internal RPCs like replication. + ``` 0 8 16 24 32 ├───────────────┴───────────────┴───────────────┴───────────────┤ From 114e77e59b12cc8506eee35bf2a04c05fb026ffe Mon Sep 17 00:00:00 2001 From: Tyler Treat Date: Thu, 27 Feb 2020 16:33:58 -0700 Subject: [PATCH 4/4] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 233f61c..4733e07 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,8 @@ bumped if the envelope format changes or if the message encoding changes in a non-backwards-compatible way. Adding fields to the messages should not require a version bump. +Currently, the only supported protocol version is v0, i.e. `0x00`. + #### HeaderLen [1 byte] The header length is the offset of the payload. This is included primarily for