Grass logo

Undici proxyAgent with an authenticated HTTP proxy

22nd Nov 2022

Undici is a modern http framework for node, and is used under the hood in many libraries like discord.js.

Sometimes we need to do requests through a proxy to prevent IP-blocks, rate-limits or geographical restriction.

The nice thing about HTTP proxies is that they are standardized. You can find a phletora of different cheap proxy providers, whereas you go with a solution like ScrapingBee, you will be locked into that system.

Undici without proxy

import { request } from "undici"

const { body } = await request("https://httpbin.org/ip")

console.log(await body.json()) // => {origin: "YOUR_IP_ADDRESS"}

Undici with proxy

import { request, ProxyAgent } from "undici"

const proxyAgent = new ProxyAgent("my.proxy.com")

const { body } = await request("https://httpbin.org/ip", {
  dispatcher: proxyAgent,
})

console.log(await body.json()) // => {origin: "PROXY_IP"}

Undici with 🔐 authenticated proxy

Most proxy urls you find will be authenticated. The ones from Apify have this shape: http://<username>:<password>@proxy.apify.com:8000

If you try this with undici, it will give you this error:

Proxy response !== 200 when HTTP Tunneling

Turns out we need to add a Proxy-Authorization header, because Undici does not do this for us. We can add this by setting the token parameter of ProxyAgent.

The resulting code that works with both authenticated proxies and not, is:

import { request, ProxyAgent } from "undici"
import url from "url"

const proxyUrl = "http://myuser:mypwd>@proxy.apify.com:8000"
const proxyUrlParsed = url.parse(proxyUrl)

const proxyAgent = new ProxyAgent({
  uri: proxyUrl,
  token: proxyUrlParsed.auth
    ? `Basic ${Buffer.from(proxyUrlParsed.auth).toString("base64")}`
    : undefined,
})

const { body } = await request("https://httpbin.org/ip", {
  dispatcher: proxyAgent,
})

console.log(await body.json()) // => {origin: "PROXY_IP"}
 

Hi!

I'm Lars. I write about building scalable systems, software architecture, and programming.

I created Turfemon to be my playground. A place where I can dive into technical topics, share quick insights, and document my learning journey, without contaminating my main site's pristine collection of 'profound' insights.

Working on something exciting? Send me an email: show email