Merge pull request #262 from nats-io/direct_batch

Document the batch behaviors of Direct Get

adr/ADR-31.md

@@ -70,20 +70,25 @@ When Allow Direct is true, each of the stream's servers configures a responder a
 Clients may make requests with the same payload as the Get message API populating the following server struct:
 
  ```text
-Seq     uint64 `json:"seq,omitempty"`
-LastFor string `json:"last_by_subj,omitempty"`
-NextFor string `json:"next_by_subj,omitempty"`
+Seq      uint64 `json:"seq,omitempty"`
+LastFor  string `json:"last_by_subj,omitempty"`
+NextFor  string `json:"next_by_subj,omitempty"`
+Batch    int `json:"batch,omitempty"`
+MaxBytes int `json:"max_bytes,omitempty"`
 ```
 
 Example request payloads:
-* `{last_by_subj: string}` - get the last message having the subject
+
 * `{seq: number}` - get a message by sequence
+* `{last_by_subj: string}` - get the last message having the subject
 * `{next_by_subj: string}` - get the first message (lowest seq) having the specified subject
 * `{seq: number, next_by_subj: string}` - get the first message with a seq >= to the input seq that has the specified subject
+* `{seq: number, batch: number, next_by_subj: string}` - gets up to batch number of messages >= than seq that has specified subject
+* `{seq: number, batch: number, next_by_subj: string, max_bytes: number}` - as above but limited to a maximum size of messages received in bytes
 
 #### Subject-Appended Direct Get API
 
-The purpose of this form is so that environmnents may choose to apply subject-based interest restrictions (user permissions
+The purpose of this form is so that environments may choose to apply subject-based interest restrictions (user permissions
 within an account and/or cross-account export/import grants) such that only specific
 subjects in stream may be read (vs any subject in the stream).
 
@@ -93,19 +98,34 @@ where `last_by_subj` is derived by the token (or series of tokens) following the
 
 It is an error (408) if a client calls Subject-Appended Direct Get and includes a request payload.
 
+#### Batched requests
+
+Using the `batch` and `max_bytes` keys one can request multiple messages in a single API call.
+
+The server will send multiple messages without any flow control to the reply subject, it will send up to `max_bytes` messages.  When `max_bytes` is unset the server will use the `max_pending` configuration setting or the server default (currently 64MB)
+
+After the batch is sent a zero length payload message will be sent with the `Nats-Num-Pending` and `Nats-Last-Sequence` headers set that clients can use to determine if further batch calls are needed. Additionally the `Nats-Last-Sequence` will hold the sequence of the last message sent. It would also have the `Status` header set to `204` with the `Description` header being `EOB`.
+
+When requests are made against servers that do not support `batch` the first response will be received and nothing will follow. Old servers can be detected by the absence of the `Nats-Num-Pending` header in the first reply.
+
 #### Response Format 
 
-Direct Get may return an error code:
+Responses may include these status codes:
+
+- `204` indicates the the end of a batch of messages, the description header would have value `EOB`
 - `404` if the request is valid but no matching message found in stream 
 - `408` if the request is empty or invalid
 
 > Error code is returned as a header, e.g. `NATS/1.0 408 Bad Request`. Success returned as `NATS/1.0` with no code.
 
-Otherwise, Direct Get reply contains the message along with the following message headers:
+Direct Get replies contain the message along with the following message headers:
+
 - `Nats-Stream`: stream name
 - `Nats-Sequence`: message sequence number 
 - `Nats-Time-Stamp`: message publish timestamp
 - `Nats-Subject`: message subject
+- `Nats-Num-Pending`: when batched, the number of messages left in the stream matching the batch parameters
+- `Nats-Last-Sequence`: when batched, the stream sequence of the previous message
 
 > A _regular_ (not JSON-encoded) NATS message is returned (from the stream store).
 
@@ -180,4 +200,4 @@ Reply:
 ```text
 HMSG _INBOX.4NogKOPzKEWTqhf4hFIUJV.yo2tE1ep 1 28 28
 NATS/1.0 408 Bad Request
-```
+```

adr/ADR-8.md

@@ -123,7 +123,7 @@ type Status interface {
 	// TTL is how long the bucket keeps values for
 	TTL() time.Duration
 
-	// Keys return a list of all keys in the bucket - not possible now except in caches
+	// Keys return a list of all keys in the bucket
 	Keys() ([]string, error)
 
 	// IsCompressed indicates if the data is compressed on disk
@@ -664,4 +664,3 @@ ditto for service registeries and so forth.
 On the name `Entry` for the returned result. `Value` seemed a bit generic and I didn't want to confuse matters mainly in the go client
 that has the unfortunate design of just shoving everything and the kitchen sink into a single package. `KVValue` is a stutter and so
 settled on `Entry`.
-