Skip to content

Commit

Permalink
Close DB connection before job completion
Browse files Browse the repository at this point in the history
RQ workers create a fork to run the function. The issue arises
when, after completing the job, the MariaDB logs display the
warning:  `Aborted connection to db. Got an error reading
communication packets`. This change ensures the database
connection is closed before the fork ends, preventing the
warning from appearing.

Signed-off-by: Jose Javier Merchante <jjmerchante@bitergia.com>
  • Loading branch information
jjmerchante committed Aug 30, 2024
1 parent 1332669 commit 4142ab5
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 1 deletion.
12 changes: 12 additions & 0 deletions releases/unreleased/connection-closed-when-job-is-executed.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
title: Connection closed when job is executed
category: fixed
author: Jose Javier Merchante <jjmerchante@bitergia.com>
issue: null
notes: >
RQ workers create a fork to run the jobs. The issue arises
when, after completing the job, the MariaDB logs display the
warning: `Aborted connection to db. Got an error reading
communication packets`. This change ensures the database
connection is closed before the fork ends, preventing the
warning from appearing.
4 changes: 4 additions & 0 deletions sortinghat/config/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,10 @@
}
}

RQ = {
'JOB_CLASS': 'sortinghat.core.jobs.SortingHatJob'
}

#
# SortingHat Multi-tenant
#
Expand Down
18 changes: 17 additions & 1 deletion sortinghat/core/jobs.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@
import django_rq.utils
import rq
import redis.exceptions
from django.db import IntegrityError, transaction
from django.db import IntegrityError, transaction, connection
from rq.job import Job

from .db import find_individual_by_uuid, find_organization
from .api import enroll, merge, update_profile, add_scheduled_task, delete_scheduled_task
Expand Down Expand Up @@ -140,6 +141,21 @@ def job_in_tenant(job, tenant):
return sorted_jobs


class SortingHatJob(Job):
"""Custom RQ Job class for SortingHat jobs.
This class closes the db connection before finishing
the job to avoid the 'aborted connection' error in
the database.
See also https://github.com/rq/django-rq/issues/17
"""
def perform(self):
result = super().perform()
connection.close()
return result


@django_rq.job
@job_using_tenant
def recommend_affiliations(ctx, uuids=None, last_modified=MIN_PERIOD_DATE):
Expand Down

0 comments on commit 4142ab5

Please sign in to comment.