feat: hydrater la table du dernier évènement connu pour un email à partir des évènements passés #896
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Comment on lines
+60
to
+65
| .values_list("poster__email", "created", "kind") | ||
| ) | ||
| qs_anonymous = ( | ||
| Post.objects.filter(poster=None) | ||
| .annotate(kind=Value(EmailLastSeenKind.POST)) | ||
| .values_list("username", "created", "kind") |
There was a problem hiding this comment.
Ça se fait en une requête avec https://docs.djangoproject.com/fr/5.1/ref/models/conditional-expressions/#case. 🤷
Comment on lines
+94
to
+125
| last_seen = collect_users_logged_in() | ||
| sys.stdout.write(f"users logged in: collected {len(last_seen)}\n") | ||
|
|
||
| last_seen += collect_event() | ||
| sys.stdout.write(f"events: collected {len(last_seen)}\n") | ||
|
|
||
| last_seen += collect_DSP() | ||
| sys.stdout.write(f"DSP: collected {len(last_seen)}\n") | ||
|
|
||
| last_seen += collect_upvote() | ||
| sys.stdout.write(f"UpVotes: collected {len(last_seen)}\n") | ||
|
|
||
| last_seen += collect_forum_rating() | ||
| sys.stdout.write(f"forum ratings: collected {len(last_seen)}\n") | ||
|
|
||
| last_seen += collect_post() | ||
| sys.stdout.write(f"posts: collected {len(last_seen)}\n") | ||
|
|
||
| last_seen += collect_clicked_notifs() | ||
| sys.stdout.write(f"clicked notifications: collected {len(last_seen)}\n") | ||
|
|
||
| dedup_last_seen_dict = deduplicate(last_seen) | ||
| sys.stdout.write(f"deduplication: {len(dedup_last_seen_dict)}\n") | ||
|
|
||
| dedup_last_seen_dict = remove_known_last_seen(dedup_last_seen_dict) | ||
| sys.stdout.write(f"remove known last seen: {len(dedup_last_seen_dict)}\n") | ||
|
|
||
| res = insert_last_seen(dedup_last_seen_dict) | ||
| sys.stdout.write(f"insert last seen: {len(res)}\n") | ||
|
|
||
| sys.stdout.write("that's all folks!\n") | ||
| sys.stdout.flush() |
There was a problem hiding this comment.
C’est violent niveau mémoire de tout charger, mais vu la taille de la commu 🤷
J’aurais plutôt utilisé l’identifiant utilisateur comme clé d’un dict[username, namedtuple(last_seen, kind)] et itéré sur les éléments petit à petit.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
🎸 Collecter les
User.last_login,Event,DSP,UpVote,ForumRating,Postanonymes et authentifiés,Notificationvisitées (#891)🎸 dédupliquer les emails en gardant l'evènement le plus récent
🎸 ignorer les emails déjà enregistrés dans
EmailLastSeen, en considerant que l'enregistrement dansEmailLastSeenest le plus récent🎸 Insérer l'ensemble dans
EmailLastSeenType de changement
🚧 technique
Points d'attention
🦺
test_collect_clicked_notifscasse par principe en attendant #891🦺 prérequis #892 & #894
simulation sur données du 27 janvier 2025