Skip to content

Commit

Permalink
Merge commit '4b172b60ee5ac3667affca647bd13a0cba71ccc2' into named-si…
Browse files Browse the repository at this point in the history 
…de-effects
  • Loading branch information
@slinkydeveloper
slinkydeveloper committed Apr 16, 2024
2 parents 0c5917b + 4b172b6 commit 4868e29
Show file tree
Hide file tree
Showing 2 changed files with 104 additions and 17 deletions.
45 changes: 44 additions & 1 deletion sdk-core/src/main/service-protocol/dev/restate/service/protocol.proto
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ message ErrorMessage {
string message = 2;
// Contains a verbose error description, e.g. the exception stacktrace.
string description = 3;

// Entry that caused the failure. This may be outside the current stored journal size.
// If no specific entry caused the failure, the current replayed/processed entry can be used.
uint32 related_entry_index = 4;
// Name of the entry that caused the failure. Empty if no name was set.
string related_entry_name = 5;
// Entry type. 0 if unknown.
uint32 related_entry_type = 6;
}

// Type: 0x0000 + 4
Expand All @@ -96,7 +104,7 @@ message EndMessage {
// * bytes value = 14 for carrying the result value
// * Failure failure = 15 for carrying a failure
//
// The tag numbers 13, 14 and 15 are reserved and shouldn't be used for other fields.
// The tag numbers 12, 13, 14 and 15 are reserved and shouldn't be used for other fields.

// ------ Input and output ------

Expand All @@ -107,6 +115,9 @@ message InputEntryMessage {
repeated Header headers = 1;

bytes value = 14;

// Entry name
string name = 12;
}

// Completable: No
Expand All @@ -117,6 +128,9 @@ message OutputEntryMessage {
bytes value = 14;
Failure failure = 15;
};

// Entry name
string name = 12;
}

// ------ State access ------
Expand All @@ -132,6 +146,9 @@ message GetStateEntryMessage {
bytes value = 14;
Failure failure = 15;
};

// Entry name
string name = 12;
}

// Completable: No
Expand All @@ -140,19 +157,27 @@ message GetStateEntryMessage {
message SetStateEntryMessage {
bytes key = 1;
bytes value = 3;

// Entry name
string name = 12;
}

// Completable: No
// Fallible: No
// Type: 0x0800 + 2
message ClearStateEntryMessage {
bytes key = 1;

// Entry name
string name = 12;
}

// Completable: No
// Fallible: No
// Type: 0x0800 + 3
message ClearAllStateEntryMessage {
// Entry name
string name = 12;
}

// Completable: Yes
Expand All @@ -167,6 +192,9 @@ message GetStateKeysEntryMessage {
StateKeys value = 14;
Failure failure = 15;
};

// Entry name
string name = 12;
}

// ------ Syscalls ------
Expand All @@ -183,6 +211,9 @@ message SleepEntryMessage {
Empty empty = 13;
Failure failure = 15;
}

// Entry name
string name = 12;
}

// Completable: Yes
Expand All @@ -203,6 +234,9 @@ message InvokeEntryMessage {
bytes value = 14;
Failure failure = 15;
};

// Entry name
string name = 12;
}

// Completable: No
Expand All @@ -224,6 +258,9 @@ message BackgroundInvokeEntryMessage {

// If this invocation has a key associated (e.g. for objects and workflows), then this key is filled in. Empty otherwise.
string key = 6;

// Entry name
string name = 12;
}

// Completable: Yes
Expand All @@ -235,6 +272,9 @@ message AwakeableEntryMessage {
bytes value = 14;
Failure failure = 15;
};

// Entry name
string name = 12;
}

// Completable: No
Expand All @@ -248,6 +288,9 @@ message CompleteAwakeableEntryMessage {
bytes value = 5;
Failure failure = 6;
};

// Entry name
string name = 12;
}

// --- Nested messages
Expand Down
76 changes: 60 additions & 16 deletions sdk-core/src/main/service-protocol/service-invocation-protocol.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The system is composed of two actors:
- SDK, which contains the implementation of the Restate Protocol
- User business logic, which interacts with the SDK to access Restate system calls (or syscalls)

Each service method invocation is modeled by the protocol as a state machine, where state transitions can be caused
Each invocation is modeled by the protocol as a state machine, where state transitions can be caused
either by user code or by _Runtime events_.

Every state transition is logged in the _Invocation journal_, used to implement Restate's durable execution model. The
Expand All @@ -32,11 +32,12 @@ The state machine is summarized in the following diagram:
```mermaid
sequenceDiagram
Note over Runtime,SDK: Start
Runtime->>SDK: HTTP Request to /invoke/{service}/{method}
Runtime->>SDK: HTTP Request to /invoke/{service}/{handler}
Runtime->>SDK: StartMessage
Note over Runtime,SDK: Replaying
Runtime->>SDK: [...]EntryMessage(s)
Note over Runtime,SDK: Processing
SDK->>Runtime: HTTP Response headers
loop
SDK->>Runtime: [...]EntryMessage
Runtime->>SDK: CompletionMessage and/or EntryAckMessage
Expand Down Expand Up @@ -102,9 +103,7 @@ protocol mandates the following messages:

### Message stream

In order to execute a service method invocation, service deployment and restate Runtime open a single stream between the
runtime and the service deployment. Given 10 concurrent service method invocations to a service deployment, there are 10
concurrent streams, each of them mapping to a specific invocation.
In order to execute an invocation, service deployment and restate Runtime open a single stream between the runtime and the service deployment. Given 10 concurrent invocations to a service deployment, there are 10 concurrent streams, each of them mapping to a specific invocation.

Every unit of the stream contains a Message serialized using the
[Protobuf encoding](https://protobuf.dev/programming-guides/encoding/), using the definitions in
Expand All @@ -119,10 +118,23 @@ in two modes:
runtime. Once the service deployment starts sending messages to the runtime, the runtime cannot send messages anymore
back to the service deployment.

When opening the stream, the request method MUST be `POST` and the request path MUST have the following format:
A message stream MUST start with `StartMessage` and MUST end with either:

- One [`SuspensionMessage`](#suspension)
- One [`ErrorMessage`](#failures)
- One `EndMessage`

If the message stream does not end with any of these two messages, it will be considered equivalent to sending an
`ErrorMessage` with an [unknown failure](#failures).

The `EndMessage` marks the end of the invocation lifecycle, that is the end of the journal.

### Initiating the stream

When opening the stream, the HTTP request method MUST be `POST` and the request path MUST have the following format:

```
/invoke/{fullyQualifiedServiceName}/{methodName}
/invoke/{serviceName}/{handlerName}
```

For example:
Expand All @@ -133,19 +145,29 @@ For example:

An arbitrary path MAY prepend the aforementioned path format.

In case the path format is not respected, or `fullyQualifiedServiceName` or `methodName` is unknown, the SDK MUST close
the stream replying back with a `404` status code.
In case the path format is not respected, or `serviceName` or `handlerName` is unknown, the SDK MUST close the stream replying back with a `404` status code.

A message stream MUST start with `StartMessage` and MUST end with either:
In case the invocation is accepted, `200` status code MUST be returned.

- One [`SuspensionMessage`](#suspension)
- One [`ErrorMessage`](#failures)
- One `EndMessage`
Additionally, the header `x-restate-user-agent` MAY be sent back, with the following format:

If the message stream does not end with any of these two messages, it will be considered equivalent to sending an
`ErrorMessage` with an [unknown failure](#failures).
```http request
x-restate-user-agent: <sdk-name> / <sdk-version>; <additional-metadata?>
```

The `EndMessage` marks the end of the invocation lifecycle, that is the end of the journal.
E.g.:

```http request
x-restate-user-agent: restate-sdk-java/0.8.0
```

Or:

```http request
x-restate-user-agent: restate-sdk-java/0.8.0; gitHash=0c5917b
```

This header is used for observability purposes by the Restate observability tools.

### Message header

Expand Down Expand Up @@ -275,6 +297,11 @@ index of the corresponding entry.
| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

#### Entry names

Every Journal entry has a field `string name = 12`, which can be set by the SDK when recording the entry. This field is
used for observability purposes by Restate observability tools.

### Journal entries reference

The following tables describe the currently available journal entries. For more details, check the protobuf message
Expand Down Expand Up @@ -365,6 +392,11 @@ additional features to the users.
The protocol allows the SDK to register an arbitrary entry type within the journal. The type MUST be `>= 0xFC00`. The
runtime will treat this entry as any other entry, persisting it and sending it during replay in the correct order.

Custom entries MAY have the entry name field `12`, as described in [entry names](#entry-names).

The field numbers 13, 14 and 15 MUST not be used, as they're reserved for completable journal entries, as described in
[completable journal entries](#completable-journal-entries-and-completionmessage).

**Header**

0 1 2 3
Expand Down Expand Up @@ -405,3 +437,15 @@ A possible implementation could be the following. Given a user requests a state

In order for the aforementioned algorithm to work, set, clear and clear all state operations must be reflected on the
local `state_map` as well.

### Side effect/Run

The side effect/run feature allows users to execute arbitrary non-deterministic code within their service and record the
result, such that on re-executions the stored value will be used instead of replaying the given code.

SDKs MAY implement the side effect feature by storing the result using a custom entry message, as described in
[Custom entry](#custom-entry-messages). By convention, the SDKs SHOULD use the entry type `FC01` for side effects.

When storing side effects, SDKs MAY need to wait for the
[acknowledgment of the stored entry](#acknowledgment-of-stored-entries) before continuing the execution of the
invocation.

0 comments on commit 4868e29

Please sign in to comment.