Add global Sequelize hook to make count behavior consistent with find #4320
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In ORM queries incorporating association joins, Sequelize's
count
sometimes reports a larger number than the number of rows that Sequelize returns by the same query. This causes inconsistencies between the actual number of returned results and the number of total results reported, a problem which specifically comes up with some of our reports.This PR adds a global
beforeCount
hook that changes Sequelize's default behavior to makecount
consistent. It also introduces a test which fails in the absence of this hook.In addition to the test case, the misbehavior can be seen in the admin interface at
/reports/users
:Without the hook a current local development instance will show 14 rows of user data:
![Screenshot 2023-12-05 at 2 30 05 PM](https://private-user-images.githubusercontent.com/12150/288185383-c5138f77-3f2f-45da-b820-3298f0bb78e2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwNjYxMDcsIm5iZiI6MTczOTA2NTgwNywicGF0aCI6Ii8xMjE1MC8yODgxODUzODMtYzUxMzhmNzctM2YyZi00NWRhLWI4MjAtMzI5OGYwYmI3OGUyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDAxNTAwN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWM2MDUzYzAyMmQ5Njg0NjE2NjkyYjM1NWI3YzhlOWEyNTAzYTA1ZTNkNDk4NzlkZjU1YWRjNWIzN2Q3ZGQ5NmYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.o70CiI0UZIZI_Ty5zmwL7ZJVy_qYrkAUPv9ha6t66iY)
but an inconsistent "total results: 17":
![image](https://private-user-images.githubusercontent.com/12150/288185793-41cddf44-5eda-489d-a4de-3ea9ada88727.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwNjYxMDcsIm5iZiI6MTczOTA2NTgwNywicGF0aCI6Ii8xMjE1MC8yODgxODU3OTMtNDFjZGRmNDQtNWVkYS00ODlkLWE0ZGUtM2VhOWFkYTg4NzI3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDAxNTAwN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWM3NDI1ZjUxNmNkMjdhZWNmMDk4YzZjNjVmYmZlZDIzNjg4YjhlOThjY2E1ZTA3MGI2Nzk2MDkyNzY2OTNhZWMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.OwR2MlFR4h9JLPpVi0gXdyqtJ3sGV89FBNn0q03YUIM)
With the hook the total is consistent:
![Screenshot 2023-12-05 at 2 30 10 PM](https://private-user-images.githubusercontent.com/12150/288186054-c6492571-a88c-49dd-b7e1-744766a8128d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwNjYxMDcsIm5iZiI6MTczOTA2NTgwNywicGF0aCI6Ii8xMjE1MC8yODgxODYwNTQtYzY0OTI1NzEtYTg4Yy00OWRkLWI3ZTEtNzQ0NzY2YTgxMjhkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDAxNTAwN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTIwYzRjMWQ4YzNmMjNkZGI2NTRiOTQzNmRkZTkyOGQyN2VlNTIyZmFiNDBhYjUwODFlNzgzNDRkN2Q0ZDkxYmEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.K3JMYu_oQ5FQIrFUNMleEKX_9wfnodgZWM1UxVYOe-A)
Background
In working on #4313 it came to my attention that some of our PaginatedQueryPages were displaying inconsistent information. Specifically, the "total results: n" in PaginationBanner differed from the number of rows shown.
@sknep identified this number as coming from the Sequelize's
count
via thepaginate
function inapi/utils/index.js
, and found other people reporting related issues.This led me to sequelize/sequelize#9481 and from there to sequelize/sequelize#10557 where various workarounds are discussed. I experimented with simply adding
distinct: true
, but that caused other things to fail. What did work for me was James M. Greene's beforeCount hook, which I opted to place inapi/models/index.js
.Changes proposed in this pull request:
test/api/unit/models/sequelize-hook.test.js
— is there a better place for this to live?security considerations
None. This change is a bug fix that makes a reported count consistent with the size of the result set it refers to.