Proposal for input rate-limiting.

[grzkv]

What is input rate-limiting?

NT will limit the number of input data points per unit of time. When the input rate reaches a threshold, the new data points are not accepted.
This can be done in multiple ways that will be discussed below.

Why do it?

There are several problems this will solve. Different implementations will resolve some of the problems better than others.

1. Rate-limiting may prevent noisy-neighbor problems. This problem manifests itself twofold:
   1. When NT shares resources, it prevents high resources consumption.
   2. When NT is shared among several users, per-use rate limiting will prevent others to be impacted by an overload created by a single user.
2. It helps establish a clear contract between users and NT.
   1. User will know what to expect from NT. E.g. there will be fewer surprises caused by NT overload.
   2. NT will know what to expect from a user.
3. It makes capacity testing much easier and more predictable since the maximum load is known in advance.

How to do it?

There are several approaches we can take when implementing rate limiting. Each of the proposed approaches will have it's pros and cons. We may also consider having more than one mechanism at the same time. Possible implementations are divided by limitation scope.

Approaches by scope:

1. Total input rate-limiting.

Limit the number of input data for all connections globally.

Pros:

• Clear expectations for the input load.
• Allows for simple and clear capacity planning.

Cons:

• Requires global control that may become a contention bottleneck.
• Does not resolve noisy-neighbor problem well.

2. Per-connection input rate-limiting.

Limit the number of input data for each connection. The limiter starts from scratch every time a connection is created.

Pros:

• No global contention points.
• Relatively easy to implement.

Cons:

• No clear mapping to each particular user.
• Single user can create multiple connections and circumvent the limiter.

3. Limiting by-user.

We try to limit the rate of input for each particular user. This comes to the question: "How do we identify the user?"
There are several approaches:

3.1 Identify user by ID.

Pros:

• This is the most correct way and will resolve all the listed problems most efficiently.

Cons:

• We cannot do this at the moment for TCP and UDP protocols.
• Requires user identification framework.
• Introduces partial sync locks in code.

3.2 Identify user by IP.

Pros:

• A compromise to solve the key problems.
• Can be done with current protocols.

Cons:

• IPs may not map to users well.
• Introduces partial contention points in code.

What to do when a limit is reached?

With the TCP protocol and without user identification, it is not possible to send a signal to the user that the limit was reached. In the light of this, we're left with two choices:

1. Throttle the incoming data points.
2. Stop reading the data points from an incoming connection.

The second option seems more sensible as it helps to avoid losing data.

Monitoring

Clearly, the rate-limiting will include metrics to monitor its saturation and/or logging of particular throttling cases. These will let us know when limits are reached and will provide feedback on usage.

Please share your ideas and feedback!