Streaming proxy subscriptions over GRPC

[FZambia]

Proposed changes

This is the second iteration of the idea in #680. The differences here:

• we are dropping client connections streams, leaving only subscription streams
• moving proto definititions to existing proxy.proto - so it can be simpler utilized and we could reuse existing features for proxies (like granular proxy mode)

Overview

Proposing an experimental feature called Proxy Subscription Streams.

Proxy subscription streams allow pushing data towards client channel subscription directly and individually from your application backend over the unidirectional or bidirectional GRPC stream.

The stream is established between Centrifugo and your application backend as soon as user subscribes to a channel. The scheme may be useful if you want to generate individual streams and these streams should only work for a time while client is subscribed to a channel.

In this case Centrifugo plays a role of WebSocket-to-GRPC streaming proxy – keeping numerous real-time connections from application's clients and establishing GRPC streams to the backend, multiplexing them using a pool of HTTP/2 (transport used by GRPC) connections.

This may be configured similar to our subscribe proxy. For subscription stream proxy we open unidirectional or bidirectional GRPC stream for the time of subscription liveness (backend can close stream at any point).

The downside of the idea is the number of additional goroutines we need for each such stream. Every stream will cost 3 extra goroutines. So for a WS connection with 5 streams the number of goroutines is 2 + 5*3 = 17. This will result into more CPU and more memory. The approach should scale horizontally (scaling GRPC on the backend is not a straightforward but definitely possible). Not every use case involves millions concurrent GRPC streams. This should fit well for short-lived stream use cases. This is like Websocketd - but over GRPC.