Agentic Systems: From Single Agent to Orchestration

A single agent — one model, a system prompt, a handful of tools, and a loop — gets you surprisingly far. It's also where most people stop, because it works for demos and falls over the moment a task needs more than one kind of reasoning happening at once.

Multi-agent orchestration isn't a hype upgrade over a single agent — it's what you reach for when a task naturally decomposes into sub-tasks that benefit from different context, different tools, or genuinely different roles. This post covers when that shift is warranted, the orchestration patterns actually used in production, and a small working example of a coordinator delegating to worker agents.

Why a single agent stops being enough

A single agent's context window and tool list grow with every capability you bolt on. Three failure modes show up as that happens:

• Context pollution — a research task's intermediate scratch work bleeds into the reasoning for an unrelated coding sub-task, degrading both
• Tool overload — past a certain number of tools in one system prompt, selection accuracy drops; the model spends more of its reasoning deciding which tool than actually using it well
• No separation of concerns — a single agent can't easily hold "be a careful planner" and "be a fast, narrow executor" as two different behavioral modes at once

Orchestration solves this by splitting one large, blurry job into a coordinator plus specialized workers, each with a narrow role, its own context, and only the tools it actually needs.

The patterns that show up in practice

There isn't one "multi-agent architecture" — a few concrete patterns cover most real systems:

• Sequential (pipeline) — agent A's output feeds agent B, feeds agent C. Good for linear workflows like research → draft → review.
• Concurrent (fan-out/fan-in) — the coordinator dispatches the same or related sub-tasks to multiple agents in parallel, then merges results. Good for independent sub-tasks — e.g. summarizing five documents at once.
• Handoff — one agent decides mid-task that another agent is better suited and transfers control (and context) to it. Good for triage-style systems, like a support bot handing off to a specialist agent.
• Hierarchical (coordinator/worker) — a central planner decomposes the task and delegates to specialized workers, then assembles their outputs into a final answer. This is the most common production pattern and the one the example below uses.

Frameworks like LangGraph model this as an explicit state graph, CrewAI models it as role-based "crews," and minimal SDKs like the OpenAI Agents SDK treat sub-agents as callable tools from a parent agent. The underlying pattern is the same regardless of framework — what differs is how much structure and observability you get by default.

A working example: coordinator + worker agents in Rust

Here's a minimal hierarchical setup: a coordinator agent receives a research request, decomposes it into sub-tasks, dispatches them to worker agents concurrently, and merges the results.

use futures::future::join_all;
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
struct SubTask {
    id: String,
    description: String,
    assigned_role: AgentRole,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
enum AgentRole {
    Researcher,
    CodeAnalyst,
    Summarizer,
}

#[derive(Debug, Serialize)]
struct WorkerResult {
    task_id: String,
    output: String,
    tokens_used: u32,
}

struct Coordinator {
    llm_client: LlmClient,
}

impl Coordinator {
    /// Decompose a high-level request into sub-tasks using the coordinator's own LLM call.
    async fn plan(&self, request: &str) -> Vec<SubTask> {
        let planning_prompt = format!(
            "Break this request into 2-4 independent sub-tasks, each assigned \
             to Researcher, CodeAnalyst, or Summarizer. Request: {request}"
        );
        let response = self.llm_client.complete(&planning_prompt).await;
        parse_subtasks(&response) // structured output parsing, omitted for brevity
    }

    /// Dispatch sub-tasks to worker agents concurrently, then merge.
    async fn execute(&self, request: &str) -> String {
        let subtasks = self.plan(request).await;

        let worker_futures = subtasks.into_iter().map(|task| {
            let client = self.llm_client.clone();
            async move { run_worker(client, task).await }
        });

        let results: Vec<WorkerResult> = join_all(worker_futures).await;

        self.merge(results).await
    }

    /// A final LLM call that synthesizes worker outputs into one coherent answer.
    async fn merge(&self, results: Vec<WorkerResult>) -> String {
        let combined = results.iter()
            .map(|r| format!("[{}]: {}", r.task_id, r.output))
            .collect::<Vec<_>>()
            .join("\n\n");

        let merge_prompt = format!(
            "Synthesize these sub-task results into one coherent response:\n\n{combined}"
        );
        self.llm_client.complete(&merge_prompt).await
    }
}

async fn run_worker(client: LlmClient, task: SubTask) -> WorkerResult {
    let role_prompt = match task.assigned_role {
        AgentRole::Researcher => "You are a focused researcher. Be concise and cite sources.",
        AgentRole::CodeAnalyst => "You are a code analyst. Focus only on technical accuracy.",
        AgentRole::Summarizer => "You are a summarizer. Compress without losing key facts.",
    };

    let prompt = format!("{role_prompt}\n\nTask: {}", task.description);
    let output = client.complete(&prompt).await;

    WorkerResult {
        task_id: task.id,
        output,
        tokens_used: estimate_tokens(&output),
    }
}

A few things worth calling out in this shape:

• Each worker gets its own narrow system prompt (role_prompt) instead of sharing the coordinator's full context — this is the actual mechanism that fixes context pollution.
• join_all runs workers concurrently, not sequentially — for independent sub-tasks, this is a straightforward latency win that a single-agent loop can't get you.
• The merge step is itself an LLM call, not string concatenation — worker outputs are often redundant or need reconciling, and skipping this step is a common source of disjointed final answers.

Where this pattern breaks down

Multi-agent orchestration isn't free — it's worth naming the costs explicitly, since a lot of write-ups skip this part:

• More tokens spent overall — planning, dispatch, and merge steps are all additional LLM calls layered on top of the actual work
• Harder to debug — a wrong final answer could originate in planning, in any worker, or in the merge step; this is exactly why the tracing patterns from the observability post in this series matter more, not less, once you add orchestration
• Coordination overhead can exceed the benefit for genuinely simple, linear tasks — a single well-scoped agent with good tools is still the right default until you hit one of the failure modes above

The Gartner-cited pattern worth taking seriously: a meaningful share of agentic projects get cancelled not because agents don't work, but because teams reached for orchestration before the task actually needed it, and the coordination overhead outweighed the gain.

A reasonable decision rule

Reach for a single agent by default. Move to orchestration when at least one of these is true:

• The task has genuinely independent sub-parts that would otherwise share (and pollute) one context window
• Different sub-tasks need meaningfully different tool sets or behavioral instructions
• You need parallelism for latency reasons, not just organizational tidiness

If none of those apply, a single well-designed agent with good tool selection will usually outperform a multi-agent system on both cost and reliability.

What's next

The next post in this series steps back to a cost/control framework: when running open-weight models yourself makes sense versus calling a closed API, and how to reason about that tradeoff instead of defaulting to whichever one is trending.

This is part of a series on AI infrastructure patterns, drawing on production work building multi-provider AI gateways in Rust. Follow along for the rest of the series.