Inquiry on Limits for Streams and Consumers in a NATS JetStream Cluster

[suchen-sci]

Hi Team,

I am seeking clarification on the following for a 3-node or 5-node NATS JetStream cluster:

1. What is the maximum number of streams that can be created?
2. What is the maximum number of consumers allowed per stream for message consumption?

I understand that these limits may be influenced by the underlying hardware specifications. However, I would appreciate a general estimate—whether the limits are around 10, 100, 1,000, 10,000, or more.

Thank you for your assistance.

[derekcollison]

We generally focus on HA Assets, or replicated JetStream assets. In our global clusters we limit servers, at the moment, to 2k HA Assets. We have customers that have higher and are ok.

1. We suggest folks use muxed streams and have consumers filter out what they need, or R1 mirrors that do the same.
2. Consumers are varied. There are heavy, R3/R5 variants, but can also be pretty light. In general if the consumer is simply inheriting the R3 from the parent stream, we expect these to be limited. For others, as you point out it depends on the hardware and network. If You need over 100k feel free to reach out. We have many approaches to these situations and even higher numbers are supported today with custom solutions.

We are continually looking at ways to allow default consumers in higher numbers. A secondary priority is how to support more and more HA Assets. When they are all active its mostly a hardware problem, however if the mix has idle, or low volume streams we can improve there.

[nickchomey]

I have a question about the 100k consumers.

I'm developing a SaaS that will be used to augment clients' existing self hosted social web platforms, in various ways. I was planning to use NATS websockets with browser clients to do live updating of user action, but do it through jetstream or kv.

Are you saying that I could only have 100k nats consumers connected at any given time? Even if the updates being sent are relatively sporadic in either direction?

If so, what is the difference between nats and, say, http or TCP? I was under the impression that websockets or server-sent events could scale to many millions on most servers (largely limited just by cpu and ram capacity).

Would a reasonable alternative be to have an application - perhaps written in go - that opens a single connection to the NATS server and then receives and maintains websockets or sse connections and subscribes to the relevant channels/kv buckets on their behalf and relays the events when they come through?

And, I suppose, even if NATS could handle millions of ws connections directly, would there be an advantage to doing what I've described - particularly if I just use SSE, which is dead-simple and benefits from HTTP protocol features. Or is it just extra complexity that I could avoid with just using nats.ws?

Thanks!

[derekcollison]

There are lots of ways to approach that problem. You can use R1 consumers, possibly even memory based and have the client be able to recreate a lost consumer on demand. We have some customers that opt to have a single muxed KV that stores the sequences for the 100k (in your case) consumers that is an R3 and highly available. The apps can consult that when they need to create the consumer again on the fly, by using that value as the OptStartingSeq to the consumer config.