Skip to content

Commit e1eb8e9

Browse files
mdellwegpatchback[bot]
authored andcommitted
Fix-up running tasks that are not unblocked
This is supposed to be a very rare corner case up to now only known to happen during a certain upgrade (where unblocked_at was introduced). Since someone decided to run such tasks already, it must be OK to mark them unblocked now. fixes #6225 (cherry picked from commit 901f8f3)
1 parent 416b368 commit e1eb8e9

File tree

2 files changed

+13
-0
lines changed

2 files changed

+13
-0
lines changed

CHANGES/6225.bugfix

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Added an extra check to the task unblocking logic to catch a rare case of stuck tasks in running, but never unblocked.
2+
Tasks in normal operation should never end in this state, but at least with a specific upgrade it can happen.

pulpcore/tasking/worker.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -298,6 +298,17 @@ def unblock_tasks(self):
298298
)
299299
task.unblock()
300300
changed = True
301+
elif task.state == TASK_STATES.RUNNING and task.unblocked_at is None:
302+
# This should not happen in normal operation.
303+
# And it is only an issue if the worker running that task died, because it will
304+
# never be considered for cleanup.
305+
# But during at least one specific upgrade this situation can emerge.
306+
# In this case, we can assume that the old algorithm was employed to identify the
307+
# task as unblocked, and we just rectify the situation here.
308+
_logger.warn(
309+
_("Running task %s was not previously marked unblocked. Fixing.", task.pk)
310+
)
311+
task.unblock()
301312

302313
# Record the resources of the pending task
303314
taken_exclusive_resources.update(exclusive_resources)

0 commit comments

Comments
 (0)