Python requests and SSRF: A Hardened HTTP Client for URLs an LLM Chooses
Smarter sends its plugins’ requests to the open web through safe_http, a small wrapper around Python’s requests library that refuses to contact anything but a public https address. This post describes the attack it exists for, server-side request forgery (SSRF), the checks it makes, the gap it documents in itself, and what it takes to adopt the same pattern in your own Django or Python service.
The attack is old, and LLMs make it newer. A server that fetches a URL on someone else’s behalf can be pointed at addresses that only the server can reach, such as its own admin port on localhost, a database on a private subnet, or, in a cloud, the instance metadata service at 169.254.169.254, which hands out temporary credentials to anyone on the machine who asks. Classic SSRF needed a form field that took a URL. An LLM application has something better, a model that decides which URL to fetch. The web search plugin reads pages that the model chose from search results. A skill can be loaded from a URL in a manifest. An image search returns URLs that the platform then checks. Every one of those is a string that the platform did not write, and every one is an invitation to make a request on the platform’s network.
The rules
safe_http is a single module, safe_http.py, with one function to make a request, fetch(), and one to check a URL, validate_public_url(). Its rules fit in a list.
- https only. Plain http,
file:,gopher:and every other scheme are refused. - Port 443 only. No
https://example.com:8080/. - No credentials in the URL.
https://user:password@host/is refused. - Public addresses only. The host name is resolved, and every address it resolves to must be public. Loopback, private ranges, link-local (which includes the metadata service), shared and reserved addresses are all refused.
- Redirects followed by hand, at most three by default, and every hop is checked by the same rules.
- Bounded responses. The body is streamed and abandoned once it passes 2 MB by default.
- A timeout on every request, 15 seconds by default.
The heart of it is the address check.
if parsed.scheme != "https" or not parsed.hostname:
raise SafeHttpError(f"only https urls are permitted: {url}")
if parsed.username or parsed.password:
raise SafeHttpError("urls must not contain credentials.")
if port not in (None, HTTPS_PORT):
raise SafeHttpError(f"only the standard https port is permitted: {url}")
try:
addresses = socket.getaddrinfo(parsed.hostname, HTTPS_PORT, proto=socket.IPPROTO_TCP)
except (socket.gaierror, UnicodeError) as e:
raise SafeHttpError(f"host could not be resolved: {parsed.hostname}") from e
for address in addresses:
ip = ipaddress.ip_address(address[4][0].split("%", 1)[0])
if not ip.is_global:
raise SafeHttpError(f"host must have a public address: {parsed.hostname}")
Three details in those lines are the ones that hand-written SSRF filters usually get wrong.
It checks addresses, not names. A filter that rejects the strings localhost and 127.0.0.1 is defeated by any DNS name that resolves to 127.0.0.1, by 0177.0.0.1, by [::1], and by a dozen other spellings. Resolving the name first and then asking Python’s ipaddress module about the result sidesteps every one of them, because the operating system has already done the parsing.
It checks every address. A name can resolve to several addresses, one public and one private. Checking only the first one leaves the choice of which one the HTTP library connects to up to chance.
It allows, rather than denies. ip.is_global is true only for addresses that are globally routable. Rather than listing the ranges to block (and forgetting the carrier-grade NAT range, or an IPv6 unique local prefix), it accepts the one category that is safe. The % split removes an IPv6 zone index, such as fe80::1%eth0, which ipaddress would otherwise refuse to parse.
Redirects are the second door
A filter that validates the first URL and then lets the HTTP library follow redirects has validated nothing. The attacker’s server answers the first request with 302 Location: http://169.254.169.254/..., and the library follows it. So fetch() turns automatic redirects off and follows them itself.
redirects: list[str] = []
for _ in range(max_redirects + 1):
# validated before every request, including each redirect, since a host's addresses can change.
validate_public_url(url)
status_code, response_headers, content = get_response(method, url, params, json_body, max_bytes, accept_status)
location = response_headers.get("location")
if status_code in REDIRECT_STATUS_CODES and location:
redirects.append(url)
url = urljoin(url, location)
if status_code in (301, 302, 303):
method, json_body = "GET", None
params = None
if redirect_policy:
redirect_policy(url)
continue
return SafeResponse(...)
raise SafeHttpError(f"{url} redirected more than {max_redirects} times.")
Each hop goes through validate_public_url() again. A caller can also pass a redirect_policy, a function that raises to refuse a redirect. The web search plugin uses it to apply the manifest’s domain allow list, and the site’s robots.txt, to every hop, so a page on a permitted domain can’t redirect the fetch to one that isn’t permitted.
What it doesn’t stop, and says so
The module’s docstring has a warning that is worth copying into any similar code.
Host names are resolved and validated immediately before each request, which leaves a small window for DNS rebinding. Deployments that require stronger guarantees should also route plugin traffic through an egress proxy that enforces the same policy.
The window is real. validate_public_url() resolves the name, and then requests resolves it again to connect. A DNS server under an attacker’s control can answer the first lookup with a public address and the second with a private one, with a time to live of zero. Closing that window completely means connecting to the exact address that was validated, which requests doesn’t make easy, or moving the check out of the application into a proxy that resolves once and connects to what it resolved. The module documents the window rather than hiding it behind a custom connection adapter, and that seems right to me, because the stronger control belongs at the network layer anyway, and an application-level check that claimed to be complete would discourage anyone from adding it.
Where it’s used
These are the features that send their requests through safe_http.
- the web search plugin, for the pages it reads and their
robots.txt; - the skill plugin, for skills loaded from a URL;
- the image search plugin;
- the extension registry, for its index, signatures and archives;
- vector stores, for documents added by URL;
- an LLMClient’s styles, whose logo and background image URLs are checked when the manifest is applied.
The MCP client shows how to apply the same rules to a library that brings its own HTTP client. The MCP SDK accepts a factory for an httpx client, so Smarter hands it one with a request hook that runs validate_public_url() before every request, and with redirects turned off, from connection.py.
async def guard_request(request: httpx2.Request) -> None:
await anyio.to_thread.run_sync(validate_endpoint_url, str(request.url))
def guarded_http_client(
headers: Optional[dict[str, str]] = None,
timeout: Optional[httpx2.Timeout] = None,
auth: Optional[httpx2.Auth] = None,
) -> httpx2.AsyncClient:
return httpx2.AsyncClient(
headers=headers,
timeout=timeout,
auth=auth,
follow_redirects=False,
event_hooks={"request": [guard_request]},
)
The one exception is deliberate and narrow. On a developer’s laptop, and only there, an MCP client may reach a loopback address, so that it can talk to an MCP server running on the same machine. In every deployed environment, loopback is refused like any other private address.
A guard like this protects only the code that calls it. That is the honest limit of any application-level SSRF defense, and it is the reason for both the egress proxy recommendation above and the admin burden below.
Importance, cost and burden
On a scale of one to ten, I rate SSRF protection a 9 for any service that fetches URLs it didn’t write, and higher than that for one that runs in a cloud with a metadata service. The attack is cheap, it is well known, and its usual prize, the credentials of the machine’s role, turns a bug in a chatbot plugin into access to the cloud account. In an LLM application the attack surface is larger than the forms on the page, because the model can be talked into choosing the URL.
Cost. Effectively nothing. One extra DNS lookup per hop, which the resolver usually answers from its cache, and no infrastructure.
User experience. The rules refuse some legitimate URLs. A manifest author who points a plugin at an http:// site, or at an API on port 8443, gets an error that says which rule the URL broke. When the network itself fails, the error names only the host, not the full URL with its query string, and a test checks that.
Platform stability. Better, not worse. The size cap and the timeout mean that a model that asks for a 4 GB file, or a server that drips one byte a minute, can’t tie up a worker process or exhaust its memory.
Admin burden. Low for the module, which has its own tests for every rule, including the metadata address, IPv6 loopback and a unique local address, and for redirects to http, too many redirects and redirects refused by a policy. The ongoing work is elsewhere. Every new feature that fetches a URL has to call fetch() or validate_public_url() rather than requests directly, and code review has to notice when it doesn’t. A search for requests. and httpx. in a pull request’s diff is a cheap check to add to that review.
Takeaways
- Resolve the host, and check every address it resolves to with
ipaddress, using an allow rule (is_global) rather than a list of blocked ranges. - Refuse non-https schemes, non-standard ports and credentials in the URL before you resolve anything.
- Turn off automatic redirects, and validate every hop as if it were the first request.
- Stream responses with a size cap, and put a timeout on every request.
- When a library brings its own HTTP client, give it a client with a request hook that runs the same check.
- Write down the DNS rebinding window, and put an egress proxy, or a network policy that blocks private ranges, behind the application check.
- Make the safe client the easy path, and review new outbound calls for it.
Closing thoughts
SSRF protection is one of the few security controls that costs almost nothing to run and fits in one file. The parts that matter are the ones that are easy to skip, namely resolving before checking, checking every address, re-checking every redirect, and saying plainly what the check can’t do. In an application where a language model picks the URL, I’d adopt it before almost any other control.