diff --git a/sdk-core/src/main/service-protocol/dev/restate/service/protocol.proto b/sdk-core/src/main/service-protocol/dev/restate/service/protocol.proto index 71ca06c7..02fe4280 100644 --- a/sdk-core/src/main/service-protocol/dev/restate/service/protocol.proto +++ b/sdk-core/src/main/service-protocol/dev/restate/service/protocol.proto @@ -75,6 +75,14 @@ message ErrorMessage { string message = 2; // Contains a verbose error description, e.g. the exception stacktrace. string description = 3; + + // Entry that caused the failure. This may be outside the current stored journal size. + // If no specific entry caused the failure, the current replayed/processed entry can be used. + uint32 related_entry_index = 4; + // Name of the entry that caused the failure. Empty if no name was set. + string related_entry_name = 5; + // Entry type. 0 if unknown. + uint32 related_entry_type = 6; } // Type: 0x0000 + 4 @@ -96,7 +104,7 @@ message EndMessage { // * bytes value = 14 for carrying the result value // * Failure failure = 15 for carrying a failure // -// The tag numbers 13, 14 and 15 are reserved and shouldn't be used for other fields. +// The tag numbers 12, 13, 14 and 15 are reserved and shouldn't be used for other fields. // ------ Input and output ------ @@ -107,6 +115,9 @@ message InputEntryMessage { repeated Header headers = 1; bytes value = 14; + + // Entry name + string name = 12; } // Completable: No @@ -117,6 +128,9 @@ message OutputEntryMessage { bytes value = 14; Failure failure = 15; }; + + // Entry name + string name = 12; } // ------ State access ------ @@ -132,6 +146,9 @@ message GetStateEntryMessage { bytes value = 14; Failure failure = 15; }; + + // Entry name + string name = 12; } // Completable: No @@ -140,6 +157,9 @@ message GetStateEntryMessage { message SetStateEntryMessage { bytes key = 1; bytes value = 3; + + // Entry name + string name = 12; } // Completable: No @@ -147,12 +167,17 @@ message SetStateEntryMessage { // Type: 0x0800 + 2 message ClearStateEntryMessage { bytes key = 1; + + // Entry name + string name = 12; } // Completable: No // Fallible: No // Type: 0x0800 + 3 message ClearAllStateEntryMessage { + // Entry name + string name = 12; } // Completable: Yes @@ -167,6 +192,9 @@ message GetStateKeysEntryMessage { StateKeys value = 14; Failure failure = 15; }; + + // Entry name + string name = 12; } // ------ Syscalls ------ @@ -183,6 +211,9 @@ message SleepEntryMessage { Empty empty = 13; Failure failure = 15; } + + // Entry name + string name = 12; } // Completable: Yes @@ -203,6 +234,9 @@ message InvokeEntryMessage { bytes value = 14; Failure failure = 15; }; + + // Entry name + string name = 12; } // Completable: No @@ -224,6 +258,9 @@ message BackgroundInvokeEntryMessage { // If this invocation has a key associated (e.g. for objects and workflows), then this key is filled in. Empty otherwise. string key = 6; + + // Entry name + string name = 12; } // Completable: Yes @@ -235,6 +272,9 @@ message AwakeableEntryMessage { bytes value = 14; Failure failure = 15; }; + + // Entry name + string name = 12; } // Completable: No @@ -248,6 +288,9 @@ message CompleteAwakeableEntryMessage { bytes value = 5; Failure failure = 6; }; + + // Entry name + string name = 12; } // --- Nested messages diff --git a/sdk-core/src/main/service-protocol/service-invocation-protocol.md b/sdk-core/src/main/service-protocol/service-invocation-protocol.md index 7aef70e1..17278c08 100644 --- a/sdk-core/src/main/service-protocol/service-invocation-protocol.md +++ b/sdk-core/src/main/service-protocol/service-invocation-protocol.md @@ -11,7 +11,7 @@ The system is composed of two actors: - SDK, which contains the implementation of the Restate Protocol - User business logic, which interacts with the SDK to access Restate system calls (or syscalls) -Each service method invocation is modeled by the protocol as a state machine, where state transitions can be caused +Each invocation is modeled by the protocol as a state machine, where state transitions can be caused either by user code or by _Runtime events_. Every state transition is logged in the _Invocation journal_, used to implement Restate's durable execution model. The @@ -32,11 +32,12 @@ The state machine is summarized in the following diagram: ```mermaid sequenceDiagram Note over Runtime,SDK: Start - Runtime->>SDK: HTTP Request to /invoke/{service}/{method} + Runtime->>SDK: HTTP Request to /invoke/{service}/{handler} Runtime->>SDK: StartMessage Note over Runtime,SDK: Replaying Runtime->>SDK: [...]EntryMessage(s) Note over Runtime,SDK: Processing + SDK->>Runtime: HTTP Response headers loop SDK->>Runtime: [...]EntryMessage Runtime->>SDK: CompletionMessage and/or EntryAckMessage @@ -102,9 +103,7 @@ protocol mandates the following messages: ### Message stream -In order to execute a service method invocation, service deployment and restate Runtime open a single stream between the -runtime and the service deployment. Given 10 concurrent service method invocations to a service deployment, there are 10 -concurrent streams, each of them mapping to a specific invocation. +In order to execute an invocation, service deployment and restate Runtime open a single stream between the runtime and the service deployment. Given 10 concurrent invocations to a service deployment, there are 10 concurrent streams, each of them mapping to a specific invocation. Every unit of the stream contains a Message serialized using the [Protobuf encoding](https://protobuf.dev/programming-guides/encoding/), using the definitions in @@ -119,10 +118,23 @@ in two modes: runtime. Once the service deployment starts sending messages to the runtime, the runtime cannot send messages anymore back to the service deployment. -When opening the stream, the request method MUST be `POST` and the request path MUST have the following format: +A message stream MUST start with `StartMessage` and MUST end with either: + +- One [`SuspensionMessage`](#suspension) +- One [`ErrorMessage`](#failures) +- One `EndMessage` + +If the message stream does not end with any of these two messages, it will be considered equivalent to sending an +`ErrorMessage` with an [unknown failure](#failures). + +The `EndMessage` marks the end of the invocation lifecycle, that is the end of the journal. + +### Initiating the stream + +When opening the stream, the HTTP request method MUST be `POST` and the request path MUST have the following format: ``` -/invoke/{fullyQualifiedServiceName}/{methodName} +/invoke/{serviceName}/{handlerName} ``` For example: @@ -133,19 +145,29 @@ For example: An arbitrary path MAY prepend the aforementioned path format. -In case the path format is not respected, or `fullyQualifiedServiceName` or `methodName` is unknown, the SDK MUST close -the stream replying back with a `404` status code. +In case the path format is not respected, or `serviceName` or `handlerName` is unknown, the SDK MUST close the stream replying back with a `404` status code. -A message stream MUST start with `StartMessage` and MUST end with either: +In case the invocation is accepted, `200` status code MUST be returned. -- One [`SuspensionMessage`](#suspension) -- One [`ErrorMessage`](#failures) -- One `EndMessage` +Additionally, the header `x-restate-user-agent` MAY be sent back, with the following format: -If the message stream does not end with any of these two messages, it will be considered equivalent to sending an -`ErrorMessage` with an [unknown failure](#failures). +```http request +x-restate-user-agent: / ; +``` -The `EndMessage` marks the end of the invocation lifecycle, that is the end of the journal. +E.g.: + +```http request +x-restate-user-agent: restate-sdk-java/0.8.0 +``` + +Or: + +```http request +x-restate-user-agent: restate-sdk-java/0.8.0; gitHash=0c5917b +``` + +This header is used for observability purposes by the Restate observability tools. ### Message header @@ -275,6 +297,11 @@ index of the corresponding entry. | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +#### Entry names + +Every Journal entry has a field `string name = 12`, which can be set by the SDK when recording the entry. This field is +used for observability purposes by Restate observability tools. + ### Journal entries reference The following tables describe the currently available journal entries. For more details, check the protobuf message @@ -365,6 +392,11 @@ additional features to the users. The protocol allows the SDK to register an arbitrary entry type within the journal. The type MUST be `>= 0xFC00`. The runtime will treat this entry as any other entry, persisting it and sending it during replay in the correct order. +Custom entries MAY have the entry name field `12`, as described in [entry names](#entry-names). + +The field numbers 13, 14 and 15 MUST not be used, as they're reserved for completable journal entries, as described in +[completable journal entries](#completable-journal-entries-and-completionmessage). + **Header** 0 1 2 3 @@ -405,3 +437,15 @@ A possible implementation could be the following. Given a user requests a state In order for the aforementioned algorithm to work, set, clear and clear all state operations must be reflected on the local `state_map` as well. + +### Side effect/Run + +The side effect/run feature allows users to execute arbitrary non-deterministic code within their service and record the +result, such that on re-executions the stored value will be used instead of replaying the given code. + +SDKs MAY implement the side effect feature by storing the result using a custom entry message, as described in +[Custom entry](#custom-entry-messages). By convention, the SDKs SHOULD use the entry type `FC01` for side effects. + +When storing side effects, SDKs MAY need to wait for the +[acknowledgment of the stored entry](#acknowledgment-of-stored-entries) before continuing the execution of the +invocation.