Skip to content

Commit

Permalink
fix(queues): shutdown flag needs to be checked while waiting
Browse files Browse the repository at this point in the history
The previous fix (#11376) to enable batching during shutdown failed to check
the shutdown flag while waiting for more items to batch.  If the coalescing
delay was large enough, this could cause the last batch to be lost.
  • Loading branch information
hanshuebner committed Aug 25, 2023
1 parent dbacabb commit 9d29ac9
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions kong/tools/queue.lua
Original file line number Diff line number Diff line change
Expand Up @@ -234,17 +234,19 @@ function Queue:process_once()

-- We've got our first entry from the queue. Collect more entries until max_coalescing_delay expires or we've collected
-- max_batch_size entries to send
if ngx.worker.exiting() then
-- minimize coalescing delay during shutdown to quickly process remaining entries
self.max_coalescing_delay = COALESCE_MIN_TIME
end
while entry_count < self.max_batch_size
and self.max_coalescing_delay - (now() - data_started) >= COALESCE_MIN_TIME
do
-- Instead of waiting for the coalesce time to expire, we cap the semaphore wait to COALESCE_POLL_TIME
-- so that we can check for worker shutdown periodically.
local wait_time = math_min(self.max_coalescing_delay - (now() - data_started), COALESCE_POLL_TIME)

if ngx.worker.exiting() then
-- minimize coalescing delay during shutdown to quickly process remaining entries
self.max_coalescing_delay = COALESCE_MIN_TIME
wait_time = COALESCE_MIN_TIME
end

ok, err = self.semaphore:wait(wait_time)
if not ok and err ~= "timeout" then
self:log_err("could not wait for semaphore: %s", err)
Expand Down

0 comments on commit 9d29ac9

Please sign in to comment.