Pool request over http when connecting through a proxy

[tylerjlcy]

I'm currently attempting to send concurrent requests through a proxy using Pool. However, when I utilize pool.request, the server responds with a 301 redirect, whereas when I use ProxyAgent, the response is 200.

Using pool client dispatcher

import { Pool } from 'undici'

const proxyUrl = `https://proxy.com`
const pool = new Pool(proxyUrl)

const endpoints = [
  "https://server.com/path?param1=value1&param2=value2",
  "https://server.com/path?param1=value3&param2=value4",
  "https://server.com/path?param1=value5&param2=value6",
  "https://server.com/path?param1=value7&param2=value8",
]

const promises = endpoints.map((endpoint) => {
  return pool.request({
    path: endpoint,
    method: 'GET',
    headers: {
      authorization: `Bearer token`,
    }
  });
});
const responses = await Promise.all(promises);
const statusCodes = responses.map((response) => {
  return response.statusCode;
});
console.log(statusCodes);

result:

[ 301, 301, 301, 301 ]

Using a global agent dispatcher

import { request, setGlobalDispatcher, ProxyAgent } from 'undici'

const proxyUrl = `https://proxy.com`
const proxyAgent = new ProxyAgent(proxyUrl)
setGlobalDispatcher(proxyAgent)

const paths = [
  "/path?param1=value1&param2=value2",
  "/path?param1=value2&param2=value3",
  "/path?param1=value4&param2=value5",
  "/path?param1=value6&param2=value7",
]

const promises = paths.map((path) => {
  return request({
    origin: "https://server.com",
    path: path,
    method: 'GET',
    headers: {
      authorization: `Bearer token`,
    }
  });
});
const responses = await Promise.all(promises);
const statusCodes = responses.map((response) => {
  return response.statusCode;
});
console.log(statusCodes);

result:

[ 200, 200, 200, 200 ]

I noticed that the 301 redirects are taking me back to the same URL that I initially added to the path. Could someone please provide insights into why this discrepancy might exist and suggest if there is a better method for sending concurrent requests through a proxy? Any advice or alternative approaches would be greatly appreciated.

Edit:
My guess is that the server might be redirecting my requests to the HTTPS version of the page, possibly because the proxy I'm using is requesting a non-HTTPS page. Interestingly, this issue doesn't seem to arise when I use ProxyAgent.

Edit2:
When I add 443 port during pool.request

const endpoints = [
  "https://server.com:443/path?param1=value1&param2=value2",
  "https://server.com:443/path?param1=value3&param2=value4",
  "https://server.com:443/path?param1=value5&param2=value6",
  "https://server.com:443/path?param1=value7&param2=value8",
]

The server replied with 400 The plain HTTP request was sent to HTTPS port
Therefore I can confirm the pool is requesting an HTTP page. It isn't an issue if I am not connecting through a proxy in the pool.

[mcollina]

Proxying requests is a bit more convoluted than just sending them to the proxy server.

The ProxyAgent was written for that purpose, use it.

[tylerjlcy]

In my testing, I noticed that using Pool.request seems to be much faster for sending concurrent requests. I'm currently working with ProxyAgent, and I'm wondering how I can leverage it to send concurrent requests efficiently?

[metcoder95]

The ProxyAgent already uses a Pool implementation underneath. Although, you should consider that overhead added on latency per each hop make when having a redirect.

Ref: https://undici.nodejs.org/#/docs/api/ProxyAgent?id=new-proxyagentoptions