Skip to content

Latest commit

 

History

History
225 lines (168 loc) · 9.82 KB

0006-220809-freighter.md

File metadata and controls

225 lines (168 loc) · 9.82 KB
Raw

6 - Freighter - Modular Transport Abstraction

Feature Name - Freighter - Modular Transport Abstraction
Status - Complete
Start Date - 2022-08-09
Authors - Emiliano Bonilla

0 - Summary

In this RFC I propose a design for a protocol agnostic transport abstraction that can be used for both internal and client communications in a Delta cluster. The new package, named freighter, provides generic unary and streaming interfaces for exchanging messages between a client and server. These interfaces can be implemented by a variety of languages and protocols (gRPC, Websocket, WebRTC, HTTP).

Freighter also defines standards for communicating errors between services, allowing exceptions to be communicated in a meaningful manner.

1 - Motivation

We're currently using the x/transport package to abstract away gRPC implementation details for node internal communications. Both Delta's distribution layer and our Aspen key-value store make use of this package with success. It provides a crucial separation between core application logic and networking details, allowing us to swap out different protocols, and test our application using mock networks (see x/transport/tmock). Unfortunately, the x/transport package defines no standards for cross-language communication, and focuses its implementation on the gRPC protocol.

As we develop client libraries in Python, Rust, and TypeScript, we need to extend these transport standards to support other languages and protocols (such as Websockets for the browser).

These extensions should not only include definitions for how a client and server should behave when exchanging messages, but also how some of these messages should be structured. This is particularly import for exceptions, as it's critical to communicate errors in a meaningful manner (such as displaying help text when a user enters invalid data in a form).

2 - Design

At its core, Freighter defines two behavioral interfaces for exchanging messages a client and a server. The unary interface defines a simple request-response cycle, while the streaming interface enables performant asynchronous communication over long periods of time. While I describe them using go idioms, these interfaces can be implemented in many languages.

1 - Behavioral Interface - Unary

The unary interface describes a single request-response cycle between a client and a server.

1 - Client

A client can send a payload to a server and receive a response using a send method with the following signature:

send(target, requestPayload) (responsePayload, error)

target is the endpoint to send the payload to. An endpoint doesn't have to be a full address. A client library, for example, may only require the caller to pass in an endpoint ("/api/v1/foo"). Cluster communications, on the other hand, may require a host:port pair ("172.16.245.1:8080"). The transport implementation can handle the construction of a full route or address internally (gRPC's service registry for example). Target syntax is defined by the transport implementation.

requestPayload is the payload to send to the server. This payload encapsulates all parameters necessary to execute the request (aside from the target, of course).

responsePayload is the payload returned by the server after executing the request.

The encoding of the payloads is left to the transport implementation.

error is an error that occurs while executing the request. This error can be a client side validation error, a network failure, or an error returned by the server. Returning an error as a value resembles the go pattern for error handling, and some languages typically raise exceptions instead of returning them. Clients and servers may use this error for control flow (such as displaying a message to the user), so I'm encouraging implementations to return errors as values. This allows the caller to parse the error as a data structure, and make a decision whether to raise an exception, retry the request, or do something else. Using errors for control flow becomes particularly important when defining a streaming interface. More details on error handling to follow.

2 - Server

A server can receive a payload from a client by binding a handler to the transport's bindHandler method.

bindHandler(handler(requestPayload) (responsePayload, error))

handler receives a payload, processes it, and returns a response to the client. If the server encounters an error during processing, it can return a non-nil error, which the transport will encode and return to the client.

2 - Behavioral Interface - Streaming

The streaming interface enables non-blocking bidirectional communication between a client and a server. Streaming interface is complex, requiring delicate control flow. Unary communication should be preferred in cases where performance and asynchronicity are not essential.

1 - Client

Streaming starts with a client issuing a stream request to a particular target:

stream(target) (stream, error)

Instead of issuing a request, stream returns a type that can be used to send and receive payloads. stream returns an error when the transport cannot reach the target or fails to assemble the stream networking infrastructure.

The client-side stream provides three methods for communication:

send(requestPayload) error - Sends a payload to the server. Send is non-blocking, so delivery of the payload is not guaranteed, as the stream may close before all payloads have been exchanged. Send returns an error when:

  1. The server closes the stream -> Returns an EOF (end-of-file) error letting the caller know that the server is no longer accepting messages. If the server closes the stream with an abnormal error, send will still return an EOF error. The client can discover the error the server closed the stream with by calling receive.

  2. The client calls closeSend -> Returns a StreamClosed error letting the caller know that the client is no longer accepting messages. This case represents a bug in the application, and listening for a StreamClosed error should not be used for control flow.

  3. The transport implementation fails -> Returns the error that caused the transport to fail.

receive() (responsePayload, error) - Blocks until a payload is received from the server or the stream closes. Receive returns an error when:

  1. The server closes stream nominally (i.e. returns a nil error from the handler) -> Returns an EOF error letting the caller know that the server is no longer exchanging messages.

  2. The server closes the stream abnormally (returns a non-nil error) -> Returns the error that caused the stream to close.

  3. The transport implementation fails -> Returns the error that caused the transport

  4. to fail.

closeSend() error - Closes the sending direction of the stream, letting the server know the client will no issue new payloads. closeSend is idempotent, and will only return an error if the underlying transport implementation fails.

It's important to note that closeSend is purely for flow-control, and has no impact on the receiving direction of the stream. In most cases, clients should wait for receive to return a non-nil error to ensure they have received all relevant payloads.

2 - Server

A server can bind a handler to the streaming transport using a bindHandler function:

bindHandler(handler(stream) error)

handler provides a stream that can communicate with the client using send and receive methods:

receive() (requestPayload, error) - Blocks until a payload is received from the client. Returns an error when:

  1. The client calls closeSend -> Returns an EOF error letting the server know it should process remaining payloads and then close the stream.

  2. The handler returns -> Returns a StreamClosed error letting the server know the receiving direction of the stream is no longer accepting messages. This case typically represent a bug in the application, and listening for a StreamClosed error should not be used for control flow.

  3. The transport implementation fails -> Returns the error that caused the transport to fail.

send(responsePayload) error: Sends a payload to the server. As with client streams, send does not block, so delivery is not guaranteed. send returns an error when:

  1. The handler has returned -> Returns a StreamClosed error letting the server know the sending direction of the stream is no longer accepting messages. As with receive, this case typically represents a bug in the application, and listening for a StreamClosed error should not be used for control flow.

  2. If the transport implementation fails -> Returns the error that caused the transport to fail.

Unlike a client stream, a server stream has no closeSend a method. Instead, the caller can close the stream by returning an error from the handler. If the error is nil, the client will receive an EOF error. If the error is non-nil, the transport implementation encodes the error and returns it to the client.

3 - Typical Lifecycle

To better understand the lifecycle of a streaming request, let's look at a simple example where a client sends a stream of integers to a server, the server squares each integer, and then returns the result.

  1. Client calls stream with the server's address and receives a client stream to exchange payloads with.
  2. Server opens a new handler and provides it with a client stream to exchange payloads with.
  3. Client sends the integers 1, 2, 3, and 4 to the server.
  4. Client calls closeSend to indicate that it has no more integers to send.
  5. Server receives each integer, squares it, and sends the result to the client. After receiving an EOF error, the server exits a loop and returns a nil error to the handler.
  6. Client receives payloads in a loop until it receives an EOF error, indicating the end of communication.

That's it!