aiohttp Proxy Setup: Authentication, Sessions and Timeouts

Start with the protocol aiohttp supports

aiohttp documents ordinary HTTP proxies and HTTPS destinations reached through HTTP CONNECT. Connecting to a gateway whose own URL starts with https:// has separate TLS-in-TLS considerations. A destination using HTTPS does not, by itself, require an HTTPS gateway. Review aiohttp's proxy documentation for the supported path in your runtime.

Begin with the endpoint provided for your account. A SOCKS5 address is not an HTTP endpoint with a different label. The example below uses HTTP proxy routing; it does not add SOCKS support. If your application specifically requires SOCKS, select and separately validate a compatible connector or use the HTTPX configuration guide with its optional SOCKS dependency.

Test a small page you control before adding parsing, authentication to a target site or task scheduling. That keeps a failure in proxy configuration distinguishable from a failure in your collector.

Run the tested Python example

Install aiohttp in your Python environment. Supply PROXY_URL as scheme, host and port only. Keep PROXY_USER and PROXY_PASSWORD in private environment settings. The code encodes those two values separately, so characters such as @, / and ? do not become URL delimiters.

"""One diagnostic GET; proxy credentials never become origin Authorization."""
import asyncio
import os
import ssl
import sys
from urllib.parse import quote, urlsplit, urlunsplit

import aiohttp


async def main():
    endpoint = urlsplit(os.environ["PROXY_URL"])
    if endpoint.scheme not in ("http", "https") or not endpoint.hostname:
        raise ValueError("Use an HTTP or HTTPS proxy endpoint")
    if endpoint.username is not None or endpoint.path not in ("", "/") or endpoint.query or endpoint.fragment:
        raise ValueError("PROXY_URL must contain only the scheme, host and port")
    user = quote(os.environ["PROXY_USER"], safe="")
    password = quote(os.environ["PROXY_PASSWORD"], safe="")
    proxy = urlunsplit((endpoint.scheme, f"{user}:{password}@{endpoint.netloc}", "", "", ""))
    tls = ssl.create_default_context(cafile=os.environ.get("SSL_CERT_FILE"))
    connector = aiohttp.TCPConnector(limit=2, limit_per_host=2, ssl=tls)
    timeout = aiohttp.ClientTimeout(total=30, connect=10, sock_read=20)
    async with aiohttp.ClientSession(
        connector=connector, timeout=timeout, trust_env=False,
        cookie_jar=aiohttp.DummyCookieJar(),
    ) as session:
        async with session.get(
            os.environ.get("TARGET_URL", "https://example.com/"),
            proxy=proxy, allow_redirects=False,
        ) as response:
            print(f"HTTP {response.status}")
            response.raise_for_status()
            body = await response.read()
            print(f"Received {len(body)} bytes")


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except Exception as error:
        print(f"Request failed: {type(error).__name__}", file=sys.stderr)
        sys.exit(1)

Download the aiohttp example. It sends one GET, prints the status, reads a small response and closes both response and session. Use streaming with a byte cap for larger files. The default target is an example page; replace it with your own diagnostic endpoint when checking an integration.

The sample deliberately does not use a session-level auth= argument for proxy credentials. That setting represents origin authentication. In current aiohttp documentation, proxy URL credentials and proxy-specific headers are documented separately. Keeping the credential in the proxy URL avoids accidentally authenticating to the destination.

Choose the session boundary for the workflow

A ClientSession owns connection-pool behavior and normally keeps cookies. The example uses DummyCookieJar because a stateless connectivity probe has no need to retain login state. A signed-in workflow may require a normal cookie jar scoped to that account. The aiohttp client reference describes those controls.

Create the session inside the event loop that uses it and reuse it across the worker's related requests. Do not share it across independent event loops, or create one for every row in a large input file. For account-specific jobs, a shared cookie jar can mix state even if you change the proxy between requests.

When changing mobile IPs, drain active requests first and verify the exit used by the next connection. Do not assume a new session forces carrier rotation. Our rotation explanation covers that distinction.

Explain a route that changes between local and production

The example sets trust_env=False and passes proxy= on the request. An inherited HTTP_PROXY value therefore does not silently choose a different gateway for this request. If your operations team manages egress through environment variables, adopt that mode deliberately and test the relevant bypass entries too.

Check the process that executes the task: a web server, queue worker and browser container can have different environments. Reproduce the request inside the actual worker. Confirm the proxy host and whether credentials exist without printing them.

For an AI retrieval tool, pass the configured session into the tool's HTTP function. A separate headless browser still needs its own browser proxy configuration. The same principle applies when an MCP server launches subprocesses: configuring its Python session cannot configure arbitrary child programs.

Set limits you can interpret

The sample permits two total connector connections and two per host. Its timeout allows 30 seconds total, 10 seconds for acquiring or establishing the connection, and 20 seconds between socket reads. These are example limits for a small diagnostic request, not published capacity figures for a mobile port.

If jobs accumulate, measure how long they wait before being scheduled as well as how long the HTTP request takes. A connector cap does not limit how many tasks your application can allocate in memory. Feed the session through a bounded queue; start with a few concurrent operations and expand only after observing the permitted destination's responses.

A pool wait, DNS failure and remote timeout need different remedies. Increasing every timeout can make the queue slower without making the connection healthier. Compare with the connection quality guide when the route succeeds but completion time varies.

Separate 407 from website authentication

A 407 Proxy Authentication Required response asks for authentication at a proxy. A website 401 asks for a different credential. For an HTTPS CONNECT failure, aiohttp can raise a proxy-connection exception before creating a normal destination response.

Record the exception class, UTC time and gateway hostname. Check for a stale password, a wrong port or an authentication mode that depends on the worker's source IP. Our allowlisting guide explains why registering a laptop address can fail for a cloud worker.

Do not paste an entire exception object, environment dump or full authenticated URL into an AI tool response. Even a diagnostic log that is private today can later appear in a support export. The example reports only the exception class.

Test scope and production handoff

On October 14, 2026, we ran the published script with Python 3.12.14 and aiohttp 3.14.5. A local fixture verified authenticated HTTP, HTTPS CONNECT, reserved characters in the credentials, wrong-password rejection, an untrusted certificate, a redirect that was not followed and a 429 that was not retried. We also checked that proxy authentication did not arrive at the origin.

No paid mobile line, target-site account or provider speed was measured in that fixture. For your handoff, record the exact package versions, confirm the route from the worker and retain one successful request plus one controlled failure. That gives the next operator evidence they can reproduce.

Frequently asked questions