RfC Search: Apply last_per_user filter after other search terms

[MrSerth]

Without this change, we would first filter for the last 2 RfCs per record and then perform the search. Imagine, a user has three RfCs for exercises 1, 2, 3 (created in the order of the exercises). Now, when searching for open RfCs of the first exercise, the learner's RfC wasn't shown: The last_per_user would only return RfC for exercise 2 and 3, where the search term for exercise 1 would remove those two from the result set further.

With the new version, we first apply the custom filter and then limit the result set by two RfCs per learner.

We further took the opportunity to fix the broken sorting (by exercise name). The resulting code seems pretty complex, but I wasn't able to find a better, more suitable (and performant!) version.

Here are examples of the resulting SQL queries and their performance for a non-administrator:

/request_for_comments

SELECT "request_for_comments".*
FROM (SELECT "request_for_comments"."id",
             "request_for_comments"."user_id",
             "request_for_comments"."exercise_id",
             "request_for_comments"."file_id",
             "request_for_comments"."created_at",
             "request_for_comments"."updated_at",
             "request_for_comments"."user_type",
             "request_for_comments"."question",
             "request_for_comments"."solved",
             "request_for_comments"."submission_id",
             "request_for_comments"."thank_you_note",
             "request_for_comments"."full_score_reached",
             "request_for_comments"."times_featured",
             COALESCE(MAX("comments"."updated_at"), "request_for_comments"."updated_at") AS last_activity
      FROM (SELECT "request_for_comments".*
            FROM "request_for_comments"
            WHERE "request_for_comments"."id" IN (SELECT "request_for_comments"."id"
                                                  FROM (SELECT "request_for_comments".*
                                                        FROM (SELECT "request_for_comments"."id",
                                                                     "request_for_comments"."user_id",
                                                                     "request_for_comments"."exercise_id",
                                                                     "request_for_comments"."file_id",
                                                                     "request_for_comments"."created_at",
                                                                     "request_for_comments"."updated_at",
                                                                     "request_for_comments"."user_type",
                                                                     "request_for_comments"."question",
                                                                     "request_for_comments"."solved",
                                                                     "request_for_comments"."submission_id",
                                                                     "request_for_comments"."thank_you_note",
                                                                     "request_for_comments"."full_score_reached",
                                                                     "request_for_comments"."times_featured"
                                                              FROM (SELECT "request_for_comments".*,
                                                                           row_number()
                                                                           OVER (PARTITION BY "request_for_comments"."user_id", "request_for_comments"."user_type" ORDER BY "request_for_comments"."created_at" DESC) AS row_number
                                                                    FROM (SELECT "request_for_comments".*
                                                                          FROM "request_for_comments"
                                                                                   LEFT OUTER JOIN external_users
                                                                                                   ON request_for_comments.user_type =
                                                                                                      'ExternalUser' AND
                                                                                                      request_for_comments.user_id =
                                                                                                      external_users.id
                                                                                   LEFT OUTER JOIN internal_users
                                                                                                   ON request_for_comments.user_type =
                                                                                                      'InternalUser' AND
                                                                                                      request_for_comments.user_id =
                                                                                                      internal_users.id
                                                                                   INNER JOIN "exercises" ON "exercises"."id" = "request_for_comments"."exercise_id"
                                                                          WHERE ("external_users"."consumer_id" IN
                                                                                 (SELECT "consumers"."id"
                                                                                  FROM "consumers"
                                                                                  WHERE "consumers"."rfc_visibility" = 0) OR
                                                                                 "internal_users"."consumer_id" IN
                                                                                 (SELECT "consumers"."id"
                                                                                  FROM "consumers"
                                                                                  WHERE "consumers"."rfc_visibility" = 0))
                                                                            AND "exercises"."unpublished" = FALSE) request_for_comments) request_for_comments
                                                              WHERE "request_for_comments"."row_number" <= 2
                                                              GROUP BY "request_for_comments"."row_number",
                                                                       "request_for_comments"."id",
                                                                       "request_for_comments"."user_id",
                                                                       "request_for_comments"."exercise_id",
                                                                       "request_for_comments"."file_id",
                                                                       "request_for_comments"."created_at",
                                                                       "request_for_comments"."updated_at",
                                                                       "request_for_comments"."user_type",
                                                                       "request_for_comments"."question",
                                                                       "request_for_comments"."solved",
                                                                       "request_for_comments"."submission_id",
                                                                       "request_for_comments"."thank_you_note",
                                                                       "request_for_comments"."full_score_reached",
                                                                       "request_for_comments"."times_featured") request_for_comments
                                                        ORDER BY "request_for_comments"."created_at" DESC) request_for_comments
                                                  LIMIT 20 OFFSET 0)) request_for_comments
               INNER JOIN "submissions" ON "submissions"."id" = "request_for_comments"."submission_id"
               LEFT OUTER JOIN "files" ON "files"."context_id" = "submissions"."id"
               LEFT OUTER JOIN "comments" ON "comments"."file_id" = "files"."id"
      GROUP BY "request_for_comments"."id", "request_for_comments"."user_id", "request_for_comments"."exercise_id",
               "request_for_comments"."file_id", "request_for_comments"."created_at",
               "request_for_comments"."updated_at", "request_for_comments"."user_type",
               "request_for_comments"."question", "request_for_comments"."solved",
               "request_for_comments"."submission_id", "request_for_comments"."thank_you_note",
               "request_for_comments"."full_score_reached",
               "request_for_comments"."times_featured") request_for_comments
ORDER BY "request_for_comments"."created_at" DESC
LIMIT 20;

RequestForComment Load (101.6ms)

/my_rfc_activity

SELECT "request_for_comments".*
FROM (SELECT "request_for_comments".*
      FROM (SELECT "request_for_comments".*
            FROM (SELECT "request_for_comments"."id",
                         "request_for_comments"."user_id",
                         "request_for_comments"."exercise_id",
                         "request_for_comments"."file_id",
                         "request_for_comments"."created_at",
                         "request_for_comments"."updated_at",
                         "request_for_comments"."user_type",
                         "request_for_comments"."question",
                         "request_for_comments"."solved",
                         "request_for_comments"."submission_id",
                         "request_for_comments"."thank_you_note",
                         "request_for_comments"."full_score_reached",
                         "request_for_comments"."times_featured",
                         COALESCE(MAX("comments"."updated_at"), "request_for_comments"."updated_at") AS last_activity
                  FROM (SELECT "request_for_comments".*
                        FROM "request_for_comments"
                                 LEFT OUTER JOIN external_users ON request_for_comments.user_type = 'ExternalUser' AND
                                                                   request_for_comments.user_id = external_users.id
                                 LEFT OUTER JOIN internal_users ON request_for_comments.user_type = 'InternalUser' AND
                                                                   request_for_comments.user_id = internal_users.id
                                 INNER JOIN "exercises" ON "exercises"."id" = "request_for_comments"."exercise_id"
                                 INNER JOIN "submissions" ON "submissions"."id" = "request_for_comments"."submission_id"
                                 INNER JOIN "files" ON "files"."context_type" = 'Submission' AND
                                                       "files"."context_id" = "submissions"."id"
                                 INNER JOIN "comments" ON "comments"."file_id" = "files"."id"
                        WHERE ("external_users"."consumer_id" IN
                               (SELECT "consumers"."id" FROM "consumers" WHERE "consumers"."rfc_visibility" = 0) OR
                               "internal_users"."consumer_id" IN
                               (SELECT "consumers"."id" FROM "consumers" WHERE "consumers"."rfc_visibility" = 0))
                          AND "comments"."user_type" = 'InternalUser'
                          AND "comments"."user_id" = 90) request_for_comments
                           INNER JOIN "submissions" ON "submissions"."id" = "request_for_comments"."submission_id"
                           LEFT OUTER JOIN "files" ON "files"."context_id" = "submissions"."id"
                           LEFT OUTER JOIN "comments" ON "comments"."file_id" = "files"."id"
                  GROUP BY "request_for_comments"."id", "request_for_comments"."user_id",
                           "request_for_comments"."exercise_id", "request_for_comments"."file_id",
                           "request_for_comments"."created_at", "request_for_comments"."updated_at",
                           "request_for_comments"."user_type", "request_for_comments"."question",
                           "request_for_comments"."solved", "request_for_comments"."submission_id",
                           "request_for_comments"."thank_you_note", "request_for_comments"."full_score_reached",
                           "request_for_comments"."times_featured") request_for_comments
            ORDER BY "last_activity" DESC) request_for_comments
      LIMIT 20 OFFSET 0) request_for_comments
ORDER BY "last_activity" DESC
LIMIT 20;

RequestForComment Load (27.4ms)

/my_request_for_comments

SELECT "request_for_comments".*
FROM (SELECT "request_for_comments"."id",
             "request_for_comments"."user_id",
             "request_for_comments"."exercise_id",
             "request_for_comments"."file_id",
             "request_for_comments"."created_at",
             "request_for_comments"."updated_at",
             "request_for_comments"."user_type",
             "request_for_comments"."question",
             "request_for_comments"."solved",
             "request_for_comments"."submission_id",
             "request_for_comments"."thank_you_note",
             "request_for_comments"."full_score_reached",
             "request_for_comments"."times_featured",
             COALESCE(MAX("comments"."updated_at"), "request_for_comments"."updated_at") AS last_activity
      FROM (SELECT "request_for_comments".*
            FROM "request_for_comments"
            WHERE "request_for_comments"."id" IN (SELECT "request_for_comments"."id"
                                                  FROM (SELECT "request_for_comments".*
                                                        FROM (SELECT "request_for_comments".*
                                                              FROM "request_for_comments"
                                                                       LEFT OUTER JOIN external_users
                                                                                       ON request_for_comments.user_type =
                                                                                          'ExternalUser' AND
                                                                                          request_for_comments.user_id =
                                                                                          external_users.id
                                                                       LEFT OUTER JOIN internal_users
                                                                                       ON request_for_comments.user_type =
                                                                                          'InternalUser' AND
                                                                                          request_for_comments.user_id =
                                                                                          internal_users.id
                                                                       INNER JOIN "exercises" ON "exercises"."id" = "request_for_comments"."exercise_id"
                                                                       INNER JOIN "submissions"
                                                                                  ON "submissions"."id" = "request_for_comments"."submission_id"
                                                              WHERE ("external_users"."consumer_id" IN
                                                                     (SELECT "consumers"."id"
                                                                      FROM "consumers"
                                                                      WHERE "consumers"."rfc_visibility" = 0) OR
                                                                     "internal_users"."consumer_id" IN
                                                                     (SELECT "consumers"."id"
                                                                      FROM "consumers"
                                                                      WHERE "consumers"."rfc_visibility" = 0))
                                                                AND ("request_for_comments"."user_type" =
                                                                     'InternalUser' AND
                                                                     "request_for_comments"."user_id" = 90 OR
                                                                     "submissions"."contributor_id" =
                                                                     6)) request_for_comments
                                                        ORDER BY "request_for_comments"."created_at" DESC) request_for_comments
                                                  LIMIT 20 OFFSET 0)) request_for_comments
               INNER JOIN "submissions" ON "submissions"."id" = "request_for_comments"."submission_id"
               LEFT OUTER JOIN "files" ON "files"."context_id" = "submissions"."id"
               LEFT OUTER JOIN "comments" ON "comments"."file_id" = "files"."id"
      GROUP BY "request_for_comments"."id", "request_for_comments"."user_id", "request_for_comments"."exercise_id",
               "request_for_comments"."file_id", "request_for_comments"."created_at",
               "request_for_comments"."updated_at", "request_for_comments"."user_type",
               "request_for_comments"."question", "request_for_comments"."solved",
               "request_for_comments"."submission_id", "request_for_comments"."thank_you_note",
               "request_for_comments"."full_score_reached",
               "request_for_comments"."times_featured") request_for_comments
ORDER BY "request_for_comments"."created_at" DESC
LIMIT 20;

RequestForComment Load (260.8ms)

/exercises/{id}/request_for_comments

SELECT "request_for_comments".*
FROM (SELECT "request_for_comments".*
      FROM (SELECT "request_for_comments".*
            FROM (SELECT "request_for_comments"."id",
                         "request_for_comments"."user_id",
                         "request_for_comments"."exercise_id",
                         "request_for_comments"."file_id",
                         "request_for_comments"."created_at",
                         "request_for_comments"."updated_at",
                         "request_for_comments"."user_type",
                         "request_for_comments"."question",
                         "request_for_comments"."solved",
                         "request_for_comments"."submission_id",
                         "request_for_comments"."thank_you_note",
                         "request_for_comments"."full_score_reached",
                         "request_for_comments"."times_featured",
                         COALESCE(MAX("comments"."updated_at"), "request_for_comments"."updated_at") AS last_activity
                  FROM (SELECT "request_for_comments".*
                        FROM "request_for_comments"
                                 LEFT OUTER JOIN external_users ON request_for_comments.user_type = 'ExternalUser' AND
                                                                   request_for_comments.user_id = external_users.id
                                 LEFT OUTER JOIN internal_users ON request_for_comments.user_type = 'InternalUser' AND
                                                                   request_for_comments.user_id = internal_users.id
                                 INNER JOIN "exercises" ON "exercises"."id" = "request_for_comments"."exercise_id"
                        WHERE ("external_users"."consumer_id" IN
                               (SELECT "consumers"."id" FROM "consumers" WHERE "consumers"."rfc_visibility" = 0) OR
                               "internal_users"."consumer_id" IN
                               (SELECT "consumers"."id" FROM "consumers" WHERE "consumers"."rfc_visibility" = 0))
                          AND "request_for_comments"."exercise_id" = 218) request_for_comments
                           INNER JOIN "submissions" ON "submissions"."id" = "request_for_comments"."submission_id"
                           LEFT OUTER JOIN "files" ON "files"."context_id" = "submissions"."id"
                           LEFT OUTER JOIN "comments" ON "comments"."file_id" = "files"."id"
                  GROUP BY "request_for_comments"."id", "request_for_comments"."user_id",
                           "request_for_comments"."exercise_id", "request_for_comments"."file_id",
                           "request_for_comments"."created_at", "request_for_comments"."updated_at",
                           "request_for_comments"."user_type", "request_for_comments"."question",
                           "request_for_comments"."solved", "request_for_comments"."submission_id",
                           "request_for_comments"."thank_you_note", "request_for_comments"."full_score_reached",
                           "request_for_comments"."times_featured") request_for_comments
            ORDER BY "last_activity" DESC) request_for_comments
      LIMIT 20 OFFSET 0) request_for_comments
ORDER BY "last_activity" DESC
LIMIT 20;

RequestForComment Load (103.3ms)