Add filtering functionality to user notes page

[kcne]

This PR addresses #832 by adding sorting and filtering functionality to user note pages. This PR is also an alternative, or better put expanding #5239 so that filtering and sorting functionality can be expanded further on if needed.

Key Changes:

• Filter by Status: Users can filter notes by their status (open, closed, or all).
• Filter by Note Type: View notes the user submitted or commented on (including those with anonymous comments).
• Date Range Filtering: Users can filter notes based on a specified from and to date range.
• Sorting Options: Sort notes by created_at or updated_at in ascending or descending order.

Tests for added methods and scopes have been added to ensure that these features work as expected across different scenarios, including handling of date ranges and note types.

Screenshots:

[AntonKhorev]

About the date pickers:

I wanted to add them to our standard pagination because we already have date indexes. Then we wouldn't need them here, specifically in this notes filer. But this plan haven't worked so far because:

• I wanted to first add a simpler thing to the standard pagination, namely links to first/last pages. I made several versions of this feature - Add oldest/newest links to shared pagination #4710 Add oldest/newest links to shared pagination - with gap between "newer" and "older" #4729 Add oldest/newest links to shared pagination - with gaps around "newer" and "older" #4733 Add oldest/newest links to shared pagination - with gaps around "newer" and "older" v2 #4734 - but none of them got much attention from the maintainers. I can still choose to add date pickers, but this will give me one more pull request that's going to be in conflict with the others which I'll have to maintain.
• As you can see this notes list doesn't even use the standard pagination. I tried doing that too Convert note pagination to newer/older note pages #4532. That wasn't convincing enough because it was too complicated. The complications were there because I used note ids as a cursor but I couldn't sort by them, this wouldn't get notes in the last updated order. The suggested solution by one of the maintainers is to use last comment ids instead, but it's currently being undermined by another maintainer who wants to remove opening comments.

[kcne]

Thank you for the detailed background and context, Anton. I agree that using date pickers makes sense if we’re filtering notes by dates, especially since the current setup isn’t relying on standard pagination. Having date pickers included in standard pagination for entries with created_at and updated_at attributes makes sense in a way but I'm not sure how long would it take for something like this to be implemented and merged.

As for the next steps, I’m not sure whether it’s better to get this PR merged as is and refactor later when the notes move to standard pagination, or if we should address the pagination issue on user note pages first and then refactor this PR accordingly.

On a side note, I’ve managed to implement sortable columns on a table UI end for created_at and updated_at locally, but I’ll hold off on submitting that until this PR is merged to keep things cleaner and easier to review.

[AntonKhorev]

As for the next steps, I’m not sure whether it’s better to get this PR merged as is and refactor later when the notes move to standard pagination, or if we should address the pagination issue on user note pages first and then refactor this PR accordingly.

You can make a pull request that just adds the status filter. That doesn't depend on pagination and shouldn't tank the performance because:

• status is checked anyway to see if the note is not hidden
• we're doing a similar thing in the api

[AntonKhorev]

onchange is not going to work with the csp_enforce: true setting.

[kcne]

Locally, this works as expected for me. Could you provide more details on how to reproduce this issue locally, or is this something that only occurs on production servers? Thanks.

[AntonKhorev]

On a side note, I’ve managed to implement sortable columns on a table UI end for created_at and updated_at locally

I looked again at why similar things were rejected in the past like #1656 and there doesn't seem to be any clear reason.

[AntonKhorev]

Besides the double-join db query, there's couple more issues with the interaction type: "blue on blue" colors and whether the submitted/commented classification makes sense at all (what if I closed a note without a comment?). I wanted to try opened/closed/commented instead.

[kcne]

Thank you for review @AntonKhorev

Few updates since last commit:

1. I've removed interaction type and refactored note type to have opened/closed/commented classification now.

2. After digging a bit deeper I've managed to implement the query using NOT EXISTS command which should perform slightly better avoiding joins. Since note comments table already have index on author_id as first entry in composite index -> # index_note_comments_on_author_id_and_created_at (author_id,created_at) this query should be performing okay now.

3. When you mentioned "blue on blue" colors I'm not sure if this is something I've introduced in this PR since when I toggle dark mode on my end everything is looking fine.

[kcne]

As for splitting this into multiple smaller PRs, I would prefer we merge this as a complete feature/functionality. Adding filters one by one would significantly increase the time required and might leave the feature feeling incomplete. This PR covers filtering part for intended functionality cohesively, and it seems better to refine and adjust post-merge if needed.

[AntonKhorev]

3. When you mentioned "blue on blue" colors I'm not sure if this is something I've introduced in this PR

Obviously you didn't introduce it. That comment was made in July.

since when I toggle dark mode on my end everything is looking fine.

Check it when the note is colored as "submitted".

[AntonKhorev]

Adding filters one by one would significantly increase the time required

Time for what? For PR review?

and might leave the feature feeling incomplete.

Are you sure? Is it complete with exactly the filters you added and not some other filters?

[kcne]

Time for what? For PR review?

Exactly, since each PR would need to be reviewed sequentially, it would prolong the overall process.

Are you sure? Is it complete with exactly the filters you added and not some other filters?

Yes, the goal was to implement the filters that were most commonly requested and improve the feature based on that feedback. However, if you have any suggestions for additional filters that could enhance the usability for mappers, I’m definitely open to discussing and considering those ideas too.

[AntonKhorev]

Yes, the goal was to implement the filters that were most commonly requested and improve the feature based on that feedback.

Where are these requests? For example #832 wants status filter but doesn't say anything about date ranges. Why would a status filter be incomplete without a date range filter?

Also #1534 (comment).

[kcne]

I opened new PR #5297 according to suggestion here. Closing this as not planed for now.

You can make a pull request that just adds the status filter. That doesn't depend on pagination and shouldn't tank the performance because:

- status is checked anyway to see if the note is not hidden

- we're doing a similar thing in the api