Skip to content
This repository has been archived by the owner on Jan 19, 2025. It is now read-only.

Commit

Permalink
fix: latest tweets retrieval optimization is unpredictable
Browse files Browse the repository at this point in the history
`getTweets` does not always return results in a predictable order, which
means that the first tweet in not necessarily the latest one. This
breaks the synchronization optimization because it relies on this
assumption.

This fix reverses the implementation for this optimization: instead of
preventing the sync when the latest tweet if cached, we prevent the
sync by default but allow it to run if we find that *any* of the
${LATEST_TWEETS_COUNT} tweets is not cached.
  • Loading branch information
Frazew committed Nov 16, 2024
1 parent b08e7b5 commit 4d2e26f
Showing 1 changed file with 9 additions and 9 deletions.
18 changes: 9 additions & 9 deletions src/services/tweets-getter.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,16 @@ export const tweetsGetterService = async (
}).start();
log.text = "filtering";

let preventPostsSynchronization = false;
let preventPostsSynchronization = true;
const LATEST_TWEETS_COUNT = 5;

/**
* Synchronization optimization: prevent excessive API calls & potential rate-limiting
*
* Pull the ${LATEST_TWEETS_COUNT}, filter eligible ones.
* This optimization prevents the post sync if the latest eligible tweet is cached.
* This optimization prevents the post sync if all latest eligible tweets are cached.
* The order in which the tweets are retrieved in unpredictable, we therefore cannot
* check only the first one and assume it's also the latest one.
*/
const latestTweets = twitterClient.getTweets(
TWITTER_HANDLE,
Expand All @@ -50,18 +52,16 @@ export const tweetsGetterService = async (

for await (const latestTweet of latestTweets) {
log.text = "post: → checking for synchronization needs";
if (!preventPostsSynchronization) {
if (preventPostsSynchronization) {
// Only consider eligible tweets.
const tweet = await getEligibleTweet(tweetFormatter(latestTweet));

if (tweet) {
// If the latest eligible tweet is cached, mark sync as unneeded.
if (isTweetCached(tweet, cachedPosts)) {
preventPostsSynchronization = true;
// If this tweet is not cached, mark sync as needed
if (!isTweetCached(tweet, cachedPosts)) {
preventPostsSynchronization = false;
break;
}
// If the latest tweet is not cached,
// skip the current optimization and go to synchronization step.
break;
}
}
}
Expand Down

0 comments on commit 4d2e26f

Please sign in to comment.