Running Qwen Code on a Custom OpenAI-Compatible API: modelProviders, a 41,000-Character First Request and Retries That Keep Going for Minutes

Qwen Code is the terminal coding agent from the Qwen team. It started as a fork of Gemini CLI, but unlike Gemini CLI it speaks the OpenAI Chat Completions format, so it can run against any OpenAI-compatible server: vLLM, Ollama's /v1 endpoint, LiteLLM, or a hosted gateway.

Setup is simple. A few defaults are less obvious, so I tested Qwen Code 0.25.2 on Node 20.19.2 against a small local mock server that logged every request and could return 401, 429 with Retry-After, or 503 on demand. Everything below is what went over the wire. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1; replace YOUR_MODEL_ID with an exact ID from your provider's model list.

The quickest setup: three environment variables

npm install -g @qwen-code/qwen-code
export OPENAI_API_KEY="your-key"
export OPENAI_BASE_URL="https://apiclaw.biz/v1"
export OPENAI_MODEL="YOUR_MODEL_ID"
qwen -p "Explain what this repo's Makefile does"

That was enough. With no settings file at all, Qwen Code picked the OpenAI-compatible route and sent POST /v1/chat/completions with Authorization: Bearer ..., stream: true, stream_options: {"include_usage": true} and a tool list.

Two details matter here:

• Keep /v1 in the base URL. Qwen Code uses the official OpenAI Node SDK, which appends /chat/completions to whatever you give it. With OPENAI_BASE_URL=https://host the request went to /chat/completions with no /v1, which most gateways answer with a 404.
• max_tokens is 32000 by default. Every request in my env-only runs carried "max_tokens": 32000. Some servers reject a value above the model's output limit, and others reserve that much against your quota. If your model's output cap is lower, set it yourself (below).

A wrong key failed cleanly: one request, [API Error: 401 Invalid API key], exit code 1, no retries.

The setup I'd actually use: modelProviders in settings.json

Environment variables create what the docs call a "runtime model." For anything you use daily, a provider entry in ~/.qwen/settings.json is easier to reason about:

{
  "modelProviders": {
    "openai": [
      {
        "id": "YOUR_MODEL_ID",
        "name": "Gateway model",
        "envKey": "APICLAW_API_KEY",
        "baseUrl": "https://apiclaw.biz/v1",
        "generationConfig": {
          "timeout": 120000,
          "maxRetries": 1,
          "contextWindowSize": 128000,
          "samplingParams": { "temperature": 0.2, "max_tokens": 4096 }
        }
      }
    ]
  },
  "security": { "auth": { "selectedType": "openai" } },
  "model": { "name": "YOUR_MODEL_ID" }
}

The key under modelProviders must be a real protocol name (openai, anthropic, gemini, vertex-ai). A made-up key like "apiclaw" is skipped with a warning unless you also map it in a top-level providerProtocol block. envKey is the name of an environment variable, not the key itself. I tested three places for it:

• APICLAW_API_KEY=... in a .env file in the project directory worked.
• The same line in ~/.qwen/.env worked.
• With neither, Qwen Code stopped before sending anything: Missing API key for OpenAI-compatible auth. Set the 'APICLAW_API_KEY' environment variable.

If you use a project .env, add it to .gitignore. A coding agent runs inside your repo, and that's exactly where a stray key gets committed.

With this entry selected, the request body was model, temperature: 0.2, max_tokens: 4096, stream and stream_options. The 32000 default was gone.

Settings that silently stop applying

I also had "model": {"generationConfig": {"samplingParams": {"top_p": 0.5}}} in the same file. It was never sent. That matches the documented rule: once a modelProviders entry is selected, its generationConfig replaces everything, and top-level model.generationConfig is ignored for that model. samplingParams, customHeaders and extra_body are replaced as whole objects, not merged. If a setting seems to have no effect, check whether it's sitting in the wrong layer.

What one "Say hi" actually costs

Non-interactive qwen -p "Say hi" sent two requests, not one.

The first was the answer. Its system prompt was 30,553 characters, and it carried a second 10,963-character block listing available skills and memory paths, plus six tool definitions (glob, grep_search, read_file, skill, tool_call, tool_search). That's about 41,500 characters before your prompt, which at a rough four characters per token is on the order of 10,000 input tokens.

The second request came after the answer was printed. Its system prompt began "You are now acting as the managed memory extraction subagent", it replayed the conversation, and it offered write_file and edit tools so the model could update memory files under ~/.qwen/. This is the managed auto-memory feature, and it's on by default.

On a per-token plan, that baseline matters for short prompts. On a plan that counts requests or has a per-minute limit, the memory pass doubles the count for one-shot runs. You can turn it off:

{
  "memory": { "enableManagedAutoMemory": false, "enableManagedAutoDream": false }
}

With that set, qwen -p "Say hi" sent exactly one request.

Retries: two layers, and they multiply

This is the part I'd configure before running Qwen Code in CI or on a slow endpoint.

A 429 that clears quickly is handled well. When the mock returned 429 with Retry-After: 2 twice, Qwen Code waited about 2 seconds each time and then succeeded on the third request.

A server that stays down is a different story. With default settings and a mock that always returned 503:

• The OpenAI SDK retried each attempt three times with short backoff (about 0.4s, 0.9s, 1.8s), so requests arrived in groups of four.
• Qwen Code's own retry loop repeated those groups with growing gaps (roughly 2, 3, 7, 14, 30 and 30 seconds). Seven groups produced 28 requests in about 107 seconds.
• Then the CLI printed Retrying in 60s (attempt 1/10) and started a new cycle.

I stopped it at 200 seconds after 44 requests, still retrying.

Setting "maxRetries": 0 in generationConfig removed the SDK layer. The same dead endpoint then got 7 requests over about 74 seconds, and the CLI exited with [API Error: 503 upstream busy] and code 1. For scripts and CI jobs I'd set maxRetries to 0 or 1 and wrap the call in your own timeout, so a provider outage fails the job in a minute instead of hanging it.

A guard I was glad to see

My mock had a mode that kept asking for the same tool call with the same arguments. Qwen Code stopped after three requests: Loop detection halted the run (global_tool_call_duplicate: the model repeated the same tool call across the turn...). That's an always-on guard, and it's useful with smaller or cheaper models that sometimes get stuck repeating a step.

Checklist

• OPENAI_BASE_URL (or baseUrl) ends in /v1, without /chat/completions.
• Use modelProviders with an envKey, and keep the key in ~/.qwen/.env or a git-ignored project .env.
• Set samplingParams.max_tokens to your model's real output limit instead of the 32000 default.
• Put every setting for a provider model inside its own generationConfig; top-level ones are ignored for it.
• Turn off managed auto-memory if you count requests or run one-shot -p jobs.
• Set maxRetries low and use an outer timeout in CI.

Disclosure: I build APIClaw, the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.