Skip to content

Commit

Permalink
Dev notes for #340.
Browse files Browse the repository at this point in the history
  • Loading branch information
lemon24 committed Jun 24, 2024
1 parent fc80a49 commit 694a216
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/dev.rst
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,8 @@ Duplicate entries
Duplicate entries are mainly handled by the :mod:`reader.entry_dedupe` plugin.

* Using MinHash to speed up similarity checks (maybe): https://gist.github.com/lemon24/b9af5ade919713406bda9603847d32e5
* Discussion of unifying "on-line" dedupe (after an entry is added/updated),
and "on-demand" dedupe (backfill): :issue:`340`.

However, it is also possible for a feed to have two entries with the same id
– yes, even though in most (if not all) formats,
Expand Down

0 comments on commit 694a216

Please sign in to comment.