Skip to content

Commit

Permalink
Document the batch behaviors of Direct Get
Browse files Browse the repository at this point in the history
Signed-off-by: R.I.Pienaar <rip@devco.net>
  • Loading branch information
ripienaar committed Jan 24, 2024
1 parent 2c170c0 commit 8d74ce3
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 8 deletions.
29 changes: 23 additions & 6 deletions adr/ADR-31.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,20 +70,25 @@ When Allow Direct is true, each of the stream's servers configures a responder a
Clients may make requests with the same payload as the Get message API populating the following server struct:

```text
Seq uint64 `json:"seq,omitempty"`
LastFor string `json:"last_by_subj,omitempty"`
NextFor string `json:"next_by_subj,omitempty"`
Seq uint64 `json:"seq,omitempty"`
LastFor string `json:"last_by_subj,omitempty"`
NextFor string `json:"next_by_subj,omitempty"`
Batch int `json:"batch,omitempty"`
MaxBytes int `json:"max_bytes,omitempty"`
```

Example request payloads:
* `{last_by_subj: string}` - get the last message having the subject

* `{seq: number}` - get a message by sequence
* `{last_by_subj: string}` - get the last message having the subject
* `{next_by_subj: string}` - get the first message (lowest seq) having the specified subject
* `{seq: number, next_by_subj: string}` - get the first message with a seq >= to the input seq that has the specified subject
* `{seq: number, batch: number, next_by_subj: string}` - gets up to batch number of messages >= than seq that has specified subject
* `{seq: number, batch: number, next_by_subj: string, max_bytes: number}` - as above but limited to a maximum size of messages received in bytes

#### Subject-Appended Direct Get API

The purpose of this form is so that environmnents may choose to apply subject-based interest restrictions (user permissions
The purpose of this form is so that environments may choose to apply subject-based interest restrictions (user permissions
within an account and/or cross-account export/import grants) such that only specific
subjects in stream may be read (vs any subject in the stream).

Expand All @@ -93,19 +98,31 @@ where `last_by_subj` is derived by the token (or series of tokens) following the

It is an error (408) if a client calls Subject-Appended Direct Get and includes a request payload.

#### Batched requests

Using the `batch` and `max_bytes` keys one can request multiple messages in a single API call.

The server will send multiple messages without any flow control to the reply subject, it will send up to `max_bytes` messages. When `max_bytes` is unset the server will use the Slow Consumer max configuration setting or the server detault (currently 64MB)

The last message in the batch will have the `Nats-Pending-Messages` header set that clients can use to determine if further batch calls are needed. It would also have the `Status` header set to `204` with the `Description` header being `EOB`.

#### Response Format

Direct Get may return an error code:

- `204` indicates the message is the end of a batch of messages, the description header would have value `EOB`
- `404` if the request is valid but no matching message found in stream
- `408` if the request is empty or invalid

> Error code is returned as a header, e.g. `NATS/1.0 408 Bad Request`. Success returned as `NATS/1.0` with no code.
Otherwise, Direct Get reply contains the message along with the following message headers:

- `Nats-Stream`: stream name
- `Nats-Sequence`: message sequence number
- `Nats-Time-Stamp`: message publish timestamp
- `Nats-Subject`: message subject
- `Nats-Pending-Messages`: the last message in a batch would have this set indicating how many messages are left matching the request

> A _regular_ (not JSON-encoded) NATS message is returned (from the stream store).
Expand Down Expand Up @@ -180,4 +197,4 @@ Reply:
```text
HMSG _INBOX.4NogKOPzKEWTqhf4hFIUJV.yo2tE1ep 1 28 28
NATS/1.0 408 Bad Request
```
```
3 changes: 1 addition & 2 deletions adr/ADR-8.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ type Status interface {
// TTL is how long the bucket keeps values for
TTL() time.Duration

// Keys return a list of all keys in the bucket - not possible now except in caches
// Keys return a list of all keys in the bucket
Keys() ([]string, error)

// IsCompressed indicates if the data is compressed on disk
Expand Down Expand Up @@ -664,4 +664,3 @@ ditto for service registeries and so forth.
On the name `Entry` for the returned result. `Value` seemed a bit generic and I didn't want to confuse matters mainly in the go client
that has the unfortunate design of just shoving everything and the kitchen sink into a single package. `KVValue` is a stutter and so
settled on `Entry`.

0 comments on commit 8d74ce3

Please sign in to comment.