I Trusted urljoin for 48 Hours. Every Request Lost the /v1 Prefix.

The first 404 looked like a bad deploy, which is how I talk myself into wasting a weekend. I had a tiny Python client, a green local suite, and a base URL I never printed. Why would urljoin lie when the unit tests already passed against a mocked transport layer? I spent forty-eight hours blaming routing, auth headers, and DNS while the version prefix quietly fell on the floor.

Field notes from a missing /v1

I needed a client for a versioned HTTP API, nothing fancy, just GET /v1/widgets/{id}. A free coding model drafted the session wrapper while I was still arguing with myself about timeouts. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I dropped the snippet into a throwaway repo and ran it on MonkeyCode's free server option because I wanted a shell that was not my laptop.

The generated helper looked reasonable, and that is the dangerous part of this story. It stored base_url, joined a relative path, and returned response.json() like every tutorial ever written. Local tests patched httpx.Client.send and asserted on status codes, never on the final URL the client actually built.

What I actually pasted

# Proposal: generated-style client. Do not ship this as-is.
from urllib.parse import urljoin
import httpx

class WidgetClient:
    def __init__(self, base_url: str) -> None:
        # "hygiene" that removes the only slash urljoin needs
        self.base_url = base_url.rstrip("/")
        self._http = httpx.Client(timeout=10.0)

    def get_widget(self, widget_id: str) -> dict:
        path = f"/widgets/{widget_id}"  # leading slash is the second landmine
        url = urljoin(self.base_url + "/", path) if False else urljoin(self.base_url, path)
        response = self._http.get(url)
        response.raise_for_status()
        return response.json()

See that rstrip("/")? It felt like cleanliness. Combined with a path that starts with /, RFC 3986 treats the relative part as an absolute path and replaces everything after the host. I did not print url for a day and a half, which is an embarrassing sentence to type.

What I tried before I printed the URL

I treated this like an operations mystery because the local mock kept saying 200. Have you ever watched yourself invent infrastructure plots so you can avoid reading one stdlib docstring? Here is the actual sequence, with the dead ends left in on purpose.

1. Restarted the fake API on port 8080 and curled GET /v1/widgets/w-1 by hand. Curl returned 200, so I blamed Python next.
2. Dumped response headers from httpx and hunted for a reverse proxy that might strip /v1. There was no proxy.
3. Compared Host values, IPv4 versus IPv6, and TLS verification, because last month I burned time on address families. This was not that bug.
4. Added a retry wrapper, which only produced faster 404s and a very confident error budget.
5. Finally logged str(request.url) on the way out. The path was /widgets/w-1. The version prefix was already gone.

python -c "from urllib.parse import urljoin; print(urljoin('https://api.example.test/v1', '/widgets/w-1'))"
# https://api.example.test/widgets/w-1

python -c "from urllib.parse import urljoin; print(urljoin('https://api.example.test/v1', 'widgets/w-1'))"
# https://api.example.test/widgets/w-1

python -c "from urllib.parse import urljoin; print(urljoin('https://api.example.test/v1/', 'widgets/w-1'))"
# https://api.example.test/v1/widgets/w-1

Three one-liners beat eighteen hours of log archaeology. The mock never caught it because the mock never saw a URL; it saw a method name I had patched.

The reproducible artifact

I want a test that fails when joining is wrong, even if every HTTP call is fake. The assertion has to be on the built URL, not on a JSON body that a stub can invent. Save this as test_widget_urljoin.py and run it with pytest -q.

# Runnable pytest. No network. Pin the join rules, not the vibe.
from urllib.parse import urljoin, urlparse
import httpx
import pytest


def build_url(base_url: str, path: str) -> str:
    """Join a versioned API base to a relative path without RFC 3986 surprises."""
    if path.startswith("http://") or path.startswith("https://"):
        raise ValueError("path must be relative, not an absolute URL")
    base = base_url if base_url.endswith("/") else base_url + "/"
    rel = path.lstrip("/")
    joined = urljoin(base, rel)
    parsed_base = urlparse(base)
    parsed_joined = urlparse(joined)
    if not parsed_joined.path.startswith(parsed_base.path):
        raise AssertionError(f"join dropped the base path: {joined!r}")
    return joined


@pytest.mark.parametrize(
    "base,path,expected",
    [
        ("https://api.example.test/v1", "widgets/w-1", "https://api.example.test/v1/widgets/w-1"),
        ("https://api.example.test/v1/", "widgets/w-1", "https://api.example.test/v1/widgets/w-1"),
        ("https://api.example.test/v1", "/widgets/w-1", "https://api.example.test/v1/widgets/w-1"),
        ("https://api.example.test/v1/", "/widgets/w-1", "https://api.example.test/v1/widgets/w-1"),
    ],
)
def test_version_prefix_survives_join(base, path, expected):
    assert build_url(base, path) == expected


def test_urljoin_stdlib_is_the_trap():
    # This is the behavior I trusted, not a bug in urllib.
    assert urljoin("https://api.example.test/v1", "widgets/w-1") == "https://api.example.test/widgets/w-1"
    assert urljoin("https://api.example.test/v1/", "/widgets/w-1") == "https://api.example.test/widgets/w-1"


def test_httpx_transport_records_the_final_path():
    def handler(request: httpx.Request) -> httpx.Response:
        assert str(request.url) == "https://api.example.test/v1/widgets/w-1"
        return httpx.Response(200, json={"id": "w-1"})

    transport = httpx.MockTransport(handler)
    with httpx.Client(transport=transport, base_url="") as client:
        url = build_url("https://api.example.test/v1", "/widgets/w-1")
        response = client.get(url)
    assert response.status_code == 200

Run it locally first, then run the same file on any other machine you actually use for glue work. I used the free server option as a second Python so I could not wave away a join bug as "laptop locale" or "my hosts file." The tests do not need root, extra packages beyond httpx and pytest, or a live API.

python -m pytest test_widget_urljoin.py -q
# test_urljoin_stdlib_is_the_trap should pass: it documents the trap.
# the other tests should pass only if build_url keeps /v1.

Decision table I wish I had on hour two

httpx.URL(...).join(...) follows the same RFC 3986 rule, so switching libraries does not save you. String concatenation with f"{base.rstrip('/')}/{path.lstrip('/')}" is ugly, and for this narrow case it is also honest.

What broke, in one paragraph I can reuse

urllib.parse.urljoin is not os.path.join for websites, even though the names rhyme in my head. If the base has no trailing slash, the last path segment is a file name and gets replaced. If the relative URL starts with /, the whole path on the host gets replaced. Generated clients love both mistakes at once, and mocks love hiding them.

What I would repeat

I would still let a free model draft the boring session class, because typing raise_for_status by hand does not make me a better engineer. I would not accept that draft until a test prints the built URL, including the version prefix, on a machine I did not just mock into obedience. Would I skip the second environment next time, just to move faster? Not after watching /v1 disappear while every assertion stayed green.

I would also keep a three-line probe next to the client, not in a wiki I will not open. The probe is the python -c block above. If that printout does not contain /v1/widgets, I do not get to discuss Kubernetes yet.

Limitations, and who should not copy this

This join helper is for boring REST prefixes, not for RFC 3986 edge cases with ../, query strings, or matrix parameters. It refuses absolute http:// paths so you cannot accidentally follow an open-redirect shaped href from a payload. It does not normalize IDN hosts, default ports, or trailing slash policy for document URLs.

Do not use this approach if your client is generated from OpenAPI and already emits full URLs; a second join will corrupt them. Do not paste production tokens, customer payloads, or private OpenAPI documents into any hosted model or shared server. Do not treat a free scratch box as staging, load testing, or proof that TLS pinning works. If your path grammar is a query-language dialect rather than /v1/resource/id, write a real URL builder and stop calling urljoin.

The forty-eight hour lesson was not that models are reckless. The lesson was that I tested the story I wanted, not the string the stdlib actually built. Print the URL. Assert the prefix. Then you can argue about retries.