AutoGen's hidden token tax: why a 3-agent chat costs 15× what you expect

Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight →.

AutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production.

This audit shows you exactly where the tokens go, with line numbers.

The setup: a 3-agent RoundRobin chat

The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here's the minimal version from the docs:

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination

planner  = AssistantAgent("planner",  model_client=client, system_message="You plan.")
coder    = AssistantAgent("coder",    model_client=client, system_message="You code.")
reviewer = AssistantAgent("reviewer", model_client=client, system_message="You review.")

team = RoundRobinGroupChat(
    [planner, coder, reviewer],
    termination_condition=MaxMessageTermination(max_messages=10),
)
await team.run(task="Build a web scraper for Hacker News.")

Three agents, 10 turns total (~3–4 turns each). Seems cheap. It isn't.

The default context: unbounded, per-agent

Every AssistantAgent gets its own UnboundedChatCompletionContext by default:

# autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L708)
if model_context is not None:
    self._model_context = model_context
else:
    self._model_context = UnboundedChatCompletionContext()

source

UnboundedChatCompletionContext.get_messages() returns self._messages — the full list, no cap, no truncation:

# autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file)
async def get_messages(self) -> List[LLMMessage]:
    """Get at most `buffer_size` recent messages."""
    return self._messages

source

(The docstring says "at most buffer_size" — that's a copy-paste artifact from BufferedChatCompletionContext. There is no buffer. It returns everything.)

The handoff tax: every agent sees every message

When an agent's turn arrives, on_messages_stream adds all incoming messages to its own context before calling the LLM:

# _assistant_agent.py, in on_messages_stream (STEP 1: "Add new user/handoff messages
# to the model context")
await self._add_messages_to_context(
    model_context=model_context,
    messages=messages,          # ← the full message_thread from the group manager
    ...
)

And _add_messages_to_context appends each one:

# _assistant_agent.py, static method _add_messages_to_context
await model_context.add_message(llm_msg)
...
await model_context.add_message(msg.to_model_message())

source

The group manager (BaseGroupChatManager) maintains a single _message_thread and appends every response to it:

# _base_group_chat_manager.py
self._message_thread: List[BaseAgentEvent | BaseChatMessage] = []
...
await self.update_message_thread(delta)   # called after every agent response

source

So at turn T, the agent receiving the baton gets T-1 messages added to its already-growing context. Its context now contains everything it has ever seen.

The math: O(N × T²) total tokens

Let's be precise. Define:

• T = total turns in the conversation
• N = number of agents
• m = average tokens per message (system prompt + response, ~300 tokens is realistic for a coding task)

Each agent speaks every N turns. When agent i speaks on turn t, its context contains all t-1 prior messages (because it has been accumulating them since turn 1).

Tokens consumed by agent i on turn t:

context_tokens(t) = (t - 1) × m

Total tokens for agent i across all its turns (it speaks at turns N, 2N, 3N, … up to T):

Σ (kN - 1) × m  for k = 1 to T/N
≈ m × N × (T/N)² / 2
= m × T² / (2N)

Total tokens across all N agents:

N × m × T² / (2N) = m × T² / 2

The N cancels. Total cost scales as T² regardless of how many agents you add.

Worked example: 10 turns, 3 agents, 300 tokens/message

Naïve expectation (10 calls × 300 tokens each): 3,000 tokens

Actual: ~16,500 tokens — 5.5× more.

At 20 turns it's ~63,000 tokens vs 6,000 expected — 10.5× more.

At 30 turns: ~139,500 tokens vs 9,000 — 15.5× more.

The multiplier grows linearly with T. This is the same O(T²) shape as ConversationBufferMemory in LangChain — but AutoGen's version is per-agent, so it's easy to miss in per-call logs.

Why per-call logs hide this

If you're watching your LLM provider's per-call token counts, you see something like:

call 1:  300 tokens  ✓ cheap
call 2:  600 tokens  ✓ fine
call 3:  900 tokens  ✓ ok
...
call 10: 3,000 tokens  ← this one looks expensive

Each call looks like a modest increase. The cumulative total — 16,500 — only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives.

The fix: cap the context

AutoGen ships two bounded alternatives, named in the AssistantAgent.__init__ docstring (around L1034 of _assistant_agent.py): BufferedChatCompletionContext (limits message count) and TokenLimitedChatCompletionContext (limits tokens):

Option 1: BufferedChatCompletionContext (sliding window)

from autogen_core.model_context import BufferedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=BufferedChatCompletionContext(buffer_size=5),  # last 5 messages
)

Cost shape becomes O(T × buffer_size) — linear. For buffer_size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. 3.3× cheaper.

Option 2: TokenLimitedChatCompletionContext (token budget)

from autogen_core.model_context import TokenLimitedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=TokenLimitedChatCompletionContext(token_limit=2000),
)

Caps the context at a fixed token budget. More predictable than a message count because message sizes vary.

Which to use?

Detecting this in CI before it hits production

The pattern is detectable statically: any file that instantiates AssistantAgent without a model_context= argument is using the unbounded default.

# Flag unbounded AssistantAgent instantiations
grep -rn "AssistantAgent(" src/ | grep -v "model_context="

For dynamic detection — measuring actual token growth across a run — this is exactly what tokenscope does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR's token delta exceeds a threshold.

The wartzar-bee/ci-guardrail GitHub Action wraps tokenscope into a one-line workflow addition:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    token_threshold: 50000   # block if PR adds >50k tokens/run
    github_token: ${{ secrets.GITHUB_TOKEN }}

Summary

The default UnboundedChatCompletionContext is correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument — but you have to know to add it.

The broader pattern: every major agent framework defaults to unbounded context because it's the safest correctness choice. Cost is a second-class citizen in the default config. That's the gap this series documents.

Next in the series: CrewAI — the delegation overhead. How hierarchical agent trees multiply your token bill.

tokenscope on npm · wartzar-bee/ci-guardrail · @wartzarbee on dev.to