Skip to content

Memory leak: rotate_tls_ciphers creates new CipherSuiteAdapter on every request without closing the old one #328

@MSPanchenko

Description

@MSPanchenko

Bug Description

rotate_tls_ciphers=True (the default) causes a memory leak by creating a new CipherSuiteAdapter on every HTTP request without closing the previous one.

Root Cause

In cloudscraper/__init__.py, the request() method calls _rotate_tls_cipher_suite() on every request when rotate_tls_ciphers=True. This method creates a new CipherSuiteAdapter (which wraps a urllib3.PoolManager + ssl.SSLContext) and mounts it via self.mount('https://', ...).

The problem: requests.Session.mount() simply replaces the dict entry — it does not call .close() on the old adapter. The orphaned PoolManager objects hold references to sockets, threading.Condition, collections.deque, and threading.RLock, which prevent garbage collection.

Impact

  • ~1 leaked PoolManager per HTTP request
  • ~56 MB/min RSS growth under moderate load (~150 req/min)
  • OOM kill within minutes in containerized environments (e.g., 512MB → OOM in ~7 min)

Reproduction

import cloudscraper
import os

def get_rss_mb():
    # Linux
    with open('/proc/self/status') as f:
        for line in f:
            if line.startswith('VmRSS:'):
                return int(line.split()[1]) / 1024
    return 0

scraper = cloudscraper.create_scraper(rotate_tls_ciphers=True)  # default

rss_before = get_rss_mb()
for i in range(500):
    scraper.get('https://httpbin.org/get')
    if i % 100 == 0:
        print(f"Request {i}: RSS = {get_rss_mb():.0f} MB (delta: +{get_rss_mb() - rss_before:.0f} MB)")

RSS will grow linearly and never stabilize.

Suggested Fix

Close the old adapter before mounting the new one in _rotate_tls_cipher_suite():

def _rotate_tls_cipher_suite(self):
    # Close old adapter to release PoolManager resources
    old_adapter = self.get_adapter('https://')
    if old_adapter:
        old_adapter.close()
    
    self.mount(
        'https://',
        CipherSuiteAdapter(...)
    )

Alternatively, reuse the existing adapter and only rotate the cipher list without creating a new PoolManager each time.

Workaround

Pass rotate_tls_ciphers=False and manage session rotation externally:

scraper = cloudscraper.create_scraper(rotate_tls_ciphers=False)

Possibly Related

Issue #323 ("3.0内存溢出" / memory overflow in 3.0) may be caused by this same leak.

Environment

  • cloudscraper 3.0.0
  • Python 3.14
  • Linux (Docker container with jemalloc)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions