Skip to content

refactor: move rate limit counters to accounts table, remove send_log #85

@genro

Description

@genro

Problem

The current send_log table creates one record per email sent, used only for rate limiting. This causes:

  • Unbounded growth: 1M emails/day = 1M records/day
  • No automatic purge: records accumulate indefinitely
  • Inefficient queries: counting records instead of reading a value

Proposed Solution

In-memory rate limiter - no database at all for rate limiting.

class RateLimiter:
    """Counters in memory, zero DB."""
    
    def __init__(self):
        # {account_id: {minute_ts: count, hour_ts: count, day_ts: count}}
        self._counters: dict[str, dict[str, int]] = {}
    
    def check_and_increment(self, account_id: str, limits: dict) -> bool:
        """Check limits and increment. Returns True if allowed."""
        now = int(time.time())
        # ... rolling counters logic

Benefits

Aspect DB In-Memory
Latency ~1ms query ~1μs dict lookup
I/O Write every email Zero I/O
Complexity Schema, migrations One dict
Crash recovery Keeps counters Reset to zero

Crash Recovery

If service restarts, counters reset to zero. This is:

  • Conservative: can send again after restart
  • Self-healing: limits are per minute/hour/day, realigns quickly
  • Same as Redis: with TTL keys

Tasks

  • Create RateLimiter class with in-memory counters
  • Modify rate limit check to use RateLimiter instead of send_log
  • Remove send_log table and entities/send_log/ directory
  • Update tests

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions