Skip to content

Conversation

@ElianaArjona
Copy link
Contributor

The event pipeline was crashing with UnhandledPromiseRejection errors when hitting Alchemy API rate limits. The application would terminate completely and require manual restart.

Root causes:

No error handling in the schedule function - Errors from the scraper functions would bubble up as unhandled promise rejections, causing Node.js to crash
No retry backoff logic - After rate limit errors, the pipeline would retry immediately at the same rate, continuing to hit rate limits

Solution

  1. Added top-level error handling to schedule function
    Wrapped all async operations in try-catch blocks
    Prevents unhandled promise rejections from crashing the application
    Enables automatic recovery from transient failures

  2. Implemented exponential backoff retry logic
    Tracks consecutive errors and increases delay between retries (2s → 4s → 8s → 16s → 32s)
    Caps maximum backoff at 60 seconds to ensure eventual recovery
    Resets error counter on successful runs
    Reduces pressure on the API during rate limit periods

src/index.ts Outdated
}

const backoffDelay = Math.min(
SECONDS_BETWEEN_RUNS * 1000 * Math.pow(2, consecutiveErrors - 1),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove the -1 and change MAX_CONSECUTIVE_ERRORS = 3

src/index.ts Outdated
process.exit(1);
}

const backoffDelay = Math.min(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An we can also remove the Math.min(x, 12000) as we know how long the max delay is based on SECONDS_BETWEEN_RUNS and MAX_CONSECUTIVE_ERRORS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants