Inconsistent and unintuitive duplicate filter behavior

[TheCommandoChicken]

Hydrus version

576

Qt major version

Qt 6

Operating system

Linux (specify distro and version in comments)

Install method

Extract

Install and OS comments

Linux Mint Cinnamon 21.2

Bug description and reproduction

The duplicate filter has unusual behavior that doesn't seem to make any sense. The main problems are as follows:

• The filter shows files in a seemingly fixed but arbitrary order, an option to change the sorting would be ideal.
• The filter shows fewer pairs than you have the cap set to, most noticeable with a cap of over 1000 pairs. This wouldn't be an issue if there were fewer potential pairs that all contain unique files than the cap, but I know for certain that some sets of potential pairs won't appear in one batch but then do appear in the following batch, and:
• As you filter files, each decision begins skipping more and more pairs, possibly the result of files that would appear later in the queue being deleted, and thus removed from the queue. Ideally the queue would only allow a file to appear once per batch to avoid this.
• Less of a bug and more of a feature request, but files in a pair get reloaded when swapping between them, which makes comparing files with large file sizes more difficult, especially on a slower drive. Possibly keep both files loaded in memory, or only unload the previous file once the other one has finished loading?

Log output

No response

[Mengmoshu]

I'll take your items in order:

• This sounds good, but might need to be moved to another issue to keep it from being forgotten. I'll leave that up to people who have a better memory for what issues we've got on the tracker already.

• Two and three are very closely related. The behavior you're noticing is mostly because the cap is on prepared pairs, not a measure of total user decisions. The logic of duplicate relationships means that sometimes a decision stops being necessary, and sometimes the need for a decision isn't apparent until another decision has been made. If you want to always make the same number of decisions before a commit but also only allow a file to appear once per batch the client will need to substitute in and aggressively prioritize which pairs can be presented. And then it will likely have to present you with an undersize batch to resolve all the relationships on those files that have multiple potential relationships.

• Three is addressed by the above.

• This is actually a result of relevant cache settings on your client that aren't enough for the size of images you are viewing. Poke around file > options to find the settings and tinker with them until the viewer no longer needs to reload images from disk. Because the feature you ask for exists, but the default settings are pretty low compared to 4k and 8k images.

As an aside, can you elaborate on why repeating an image in the queue causes you trouble? This seems like the sort of thing that old hands don't think about anymore, so I'd like to understand your viewpoint better.

[TheCommandoChicken]

I think fundamentally what my issue is is that the queue doesn't present enough pairs to fill the set cap, even excluding the skipping of pairs as decisions are made, which seems like a bug based on the fact that there are definitely more than enough distinct pairs to present 1000 in a single batch. I don't think repeating images in the queue is a problem in itself, just that in doing so it seems to cause hydrus to ignore other distinct pairs that could be presented (which could also help with the monotony of duplicate filtering by providing a larger variety of images, which goes in hand with the first point about sorting). It also seems to cause the skipping of pairs as you make decisions, which would be avoided by only allowing an image to appear once per batch.
Also I'll be sure to try tweaking my cache settings, thanks.