Skip to content

Conversation

@terrerox
Copy link
Contributor

@terrerox terrerox commented Jan 29, 2026

Description

This PR is still in progress. The changes are not final and are for testing purposes only

Address memory exhaustion issues when uploading large numbers of files (e.g., 2000+ small files) on low-RAM machines by implementing adaptive concurrency controls and fixing stream reader leaks.

Changes:

  • Fix stream reader memory leak in crypto.ts by properly releasing readers using try-finally block to ensure cleanup even on errors
  • Add adaptive concurrency to UploadManager based on system RAM (2-6 concurrent uploads depending on available heap size)
  • Add adaptive concurrency to UploadFoldersManager for folder operations (2-6 concurrent folder uploads based on system capabilities)
  • Implement smart file batching when memory usage exceeds 80% (splits large file sets into batches of 100 with 200ms delays)

Impact:

  • High-RAM machines (≥4 GB heap): Maintain full performance with 6 concurrent uploads
  • Medium-RAM machines (2-4 GB heap): Use 4 concurrent uploads
  • Low-RAM machines (<2 GB heap): Use 2 concurrent uploads to prevent freezing
  • Memory leak fix prevents ~500 KB retention per file upload

Resolves memory exhaustion and browser freezing on low-spec machines while
maintaining optimal performance on high-end systems.

Related Issues

Related Pull Requests

Checklist

  • Changes have been tested locally.
  • Unit tests have been written or updated as necessary.
  • The code adheres to the repository's coding standards.
  • Relevant documentation has been added or updated.
  • No new warnings or errors have been introduced.
  • SonarCloud issues have been reviewed and addressed.
  • QA Passed

Testing Process

Additional Notes

@terrerox terrerox self-assigned this Jan 29, 2026
@terrerox terrerox requested a review from a team as a code owner January 29, 2026 06:17
@terrerox terrerox marked this pull request as draft January 29, 2026 06:17
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Jan 29, 2026

Deploying drive-web with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7b9b312
Status: ✅  Deploy successful!
Preview URL: https://4ced7ba0.drive-web.pages.dev
Branch Preview URL: https://fix-upload-adaptive-concurre.drive-web.pages.dev

View logs

@terrerox terrerox force-pushed the fix/upload-adaptive-concurrency branch 2 times, most recently from c5130c3 to e315ad8 Compare January 29, 2026 16:46
@terrerox terrerox force-pushed the fix/upload-adaptive-concurrency branch from a88825b to 7b9b312 Compare January 30, 2026 00:08
@sonarqubecloud
Copy link

@terrerox terrerox closed this Jan 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant