Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a define-after-update will cause 2x the query count #195

Open
crisptrutski opened this issue Nov 12, 2024 · 1 comment
Open

Adding a define-after-update will cause 2x the query count #195

crisptrutski opened this issue Nov 12, 2024 · 1 comment

Comments

@crisptrutski
Copy link
Contributor

crisptrutski commented Nov 12, 2024

Found this in Metabase, where I need to add an after-update hook to :model/DashboardCard.

To reproduce, run the following test: https://github.com/metabase/metabase/blob/1f07e88cb40cc74813cdeedad205d11d077b5c84/test/metabase/models/dashboard_card_test.clj#L213

When run without an update hook, there are 10 / 11 queries.

If you then add the following no-op hook:

(t2/define-after-update :model/DashboardCard
  [instance]
  (prn "hook called for: " (:id instance))
  ;; we could also return instance, it does not make a difference.
  nil)

There will now be 16 / 17 queries. The hook will be called exactly 3 times, for the 3 distinct dashcards.

The extra queries are 3 pairs of extra SELECT and UPDATE statements.

On reflection this perhaps makes sense - we would need to query the db again to get default values and function results like NOW, but this would only explain the SELECT and not the extra UPDATE. That defo seems like a buggo.


In my particular case I need to add a new hook to a large number of models, and the query amplification could be a large problem. What I need is a way to perform side effects using the same data that went to the before-hook. In fact, all I need to know are which fields changed (and don't care about the values). I do however need the transaction to have committed already, as my hook will trigger a database read from another thread.

Do you have any ideas on how I can do this without causing a large perf regression?

@crisptrutski
Copy link
Contributor Author

Using toucan monitor:

Before

query params
SELECT * FROM "report_dashboardcard" WHERE "id" = ? [156]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 156]
DELETE FROM "dashboardcard_series" WHERE "dashboardcard_id" IN (?) [156]
INSERT INTO "dashboardcard_series" ("dashboardcard_id", "card_id", "position") VALUES (?, ?, ?) [156 638 0]
SELECT * FROM "report_dashboardcard" WHERE "id" = ? [157]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 157]
DELETE FROM "dashboardcard_series" WHERE "dashboardcard_id" IN (?) [157]
INSERT INTO "dashboardcard_series" ("dashboardcard_id", "card_id", "position") VALUES (?, ?, ?) [157 639 0]
SELECT * FROM "report_dashboardcard" WHERE "id" = ? [158]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 158]

After

query params
SELECT * FROM "report_dashboardcard" WHERE "id" = ? [168]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 168]
SELECT * FROM "report_dashboardcard" WHERE "id" IN (?) [168]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 168]
DELETE FROM "dashboardcard_series" WHERE "dashboardcard_id" IN (?) [168]
INSERT INTO "dashboardcard_series" ("dashboardcard_id", "card_id", "position") VALUES (?, ?, ?) [168 650 0]
SELECT * FROM "report_dashboardcard" WHERE "id" = ? [169]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 169]
SELECT * FROM "report_dashboardcard" WHERE "id" IN (?) [169]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 169]
DELETE FROM "dashboardcard_series" WHERE "dashboardcard_id" IN (?) [169]
INSERT INTO "dashboardcard_series" ("dashboardcard_id", "card_id", "position") VALUES (?, ?, ?) [169 651 0]
SELECT * FROM "report_dashboardcard" WHERE "id" = ? [170]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 170]
SELECT * FROM "report_dashboardcard" WHERE "id" IN (?) [170]
UPDATE "report_dashboardcard" SET "size_x" = ?, "updated_at" = NOW(), "col" = ?, "row" = ? WHERE "id" = ? [3 2 1 170]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant