(2.11) Internal: Intra-Process Queue fixes/improvements.

[kozlovic]

Go says that when using pool we should use pointer to a slice, not the slice itself to save on copy when getting/putting things back.

That's what I tried to do, but I believe that the gain was defeated by the fact that I was storing it as a []T in the queue object, and more importantly, was returning the address of the local variable when putting back in the pool. The way it was used worked, but is dangerous if queues were to be use differently. For instance with implementation before this change, this in a loop would fail:

var elts []int
for i:=0;i<1000;i++{
    q.push(i+1)
    expected += i+1
    elts = q.pop()
    for _, v := range elts {
        sum += v
    }
    q.recycle(&elts)

    q.push(i+2)
    expected += i+2
    elts = q.pop()

    q.push(i+3)
    expected += i+3

    for _, v := range elts {
        sum += v
    }
    q.recycle(&elts)

    elts = q.pop()
    for _, v := range elts {
        sum += v
    }
    q.recycle(&elts)
}
if sum != expected {
   // ERROR!
}

If we use different variables, such as elts1 := q.pop(), etc.. then it works. And again, the way it was used before did not cause issues.

The changes here use a pointer to a []T all the way.

I have tried dummy queue implementations of simply using []T all the way, including when putting it back to the pool, and the perf and number of allocs etc... does not seem to change regardless of what I was using. However, with the real queue implementation, it somehow changes, so sticking with *[T] for now.

The other changes are to use the "in progress" count (and now size) when checking for the limits, which we were not. So after a pop(), since the queue is empty, the push() side would be able to store up to the limits while the receiver was processing the popped elements.

I have added APIs to indicate progress when processing elements in the "for-loop" that goes over the pop() result. I have modified code that uses a queue with limit (only 2 so far) so that they use the new API.

I have added benchmarks so we can evaluate future changes. Aside from the modifications to queue with limits, running the benchmark from original code to new shows a slight improvement. Of course, updating progress for each element popped() is slower than doing as a bulk, but it allows for fine-grained control on the queue limits. And when using with processing of JetStream messages, it is likely that the effect is not relevant.

Signed-off-by: Ivan Kozlovic ivan@synadia.com

[kozlovic]

@derekcollison @neilalexander I set this as a Draft because I would like to be sure that we want to go this direction, also, there will be a small conflict probably with one of @neilalexander branch/future PR with the jsapi rate limit.

[derekcollison]

Thanks Ivan. In general LGTM, but will let @neilalexander weigh in and do the formal review.

[kozlovic]

@derekcollison In other words, this is ready for review, but I did not want it to be merged until we all agree that this is the way to go and ear from all participants feedback.

[derekcollison]

Understood.