Skip to content

fix(bulk2): bypass jsforce response buffering W-21348168#1388

Open
cristiand391 wants to merge 6 commits intomainfrom
cd/bulk-export-string-limit
Open

fix(bulk2): bypass jsforce response buffering W-21348168#1388
cristiand391 wants to merge 6 commits intomainfrom
cd/bulk-export-string-limit

Conversation

@cristiand391
Copy link
Member

@cristiand391 cristiand391 commented Mar 2, 2026

What does this PR do?

Update data export bulk to use undici instead of jsforce's request methods when retrieving bulk batches.

jsforce buffers batches response in memory a few times, causing one of the early steps to throw on big batches, see:
forcedotcom/cli#3507 (comment)

The buffering happens at the low-level request helper in jsforce so all request methods are affected.
This PR updates the bulkRequest helper to use undici to fetch the batches.

Repro:

I couldn't get to repro this, exporting +1M ScratchOrgInfo records from our na40hub but these still don't generate big enough batches to trigger reported error.

I've created prerelease last week and customer verified it worked:
forcedotcom/cli#3507 (comment)

we do have NUT tests for +1M exports for linux/win:

it('should export +1 million records in csv format', async () => {

What issues does this PR fix or reference?

forcedotcom/cli#3507
@W-21348168@

@cristiand391 cristiand391 marked this pull request as ready for review March 2, 2026 17:06
@cristiand391 cristiand391 requested a review from a team as a code owner March 2, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants