Problem with throttling by delaying less than 1ms

[lwlee2608]

I'm attempting to implement some kind of throttling at high throughput.

For example, In order to throttle to 10,000 transaction per second (TPS), I will need to add a small delay of 0.1ms in between each event. 0.05ms to get 20k TPS and so on.

Delaying with Duration.ofNanos doesn't seem to work.

Multi<Integer> delayed = multi
    .onItem().call(i ->
        // Delay the emission until the returned uni emits its item
        Uni.createFrom().nullItem().onItem().delayIt().by(Duration.ofNanos(100_000))
    );

I suspect this might be why. Any duration smaller than 1ms will simply be converted to 0.

However, changing the scheduler to accept nano second time unit present another problem where the scheduler cannot process it accurately.

cheduledFuture = executor.schedule(dispatch, duration.toNanos(), TimeUnit.NANOSECONDS);

Any idea what I can do?

[cescoffier]

Does Java scheduler accept nanos?
Looking at https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html#schedule(java.lang.Runnable,long,java.util.concurrent.TimeUnit), it does not mention anything.

[Ladicek]

I'm no expert, but I'd generally expect that attempting to wait for a fraction of millisecond will be futile. The actual time waited is likely to be much longer, because of inaccuracies of system clocks.

[jponge]

We might explore throttling in the future.

[lwlee2608]

Thanks for the reply, looking forward to it.

What I do for now is skipping the delay every X amount of events. For example, delay 1 ms every 10 events (each event is 0.1ms delay on average) to achieve 10,000 TPS. It is not perfect, but at least it's close to what we need.

[lwlee2608]

At the mean time, do you think we need to change the scheduler to accept nano?

The problem is UniOnItemDelay.by() accept Duration.

Although I agree with @Ladicek that system clock will not be accurate as we goes down fraction of milliseconds. But at least it's not 0ms if user set anything smaller than 1ms.