Loop vs Graph vs Harness: How AI Agents Actually Work

If you're starting to build AI agents, you'll hear three words a lot: loop, graph and harness.

They sound complicated. They're not. In this article I'll explain all three with sticky-note diagrams. No code needed yet: that comes in Part 2.

We'll use one example the whole way through:

A customer asks: "Where's my order #123? If it's late, I want a refund."

Part 1: Two ways to control an agent

Every agent has to answer one question at every step:

What happens next, and who decides?

There are two answers: the loop and the graph.

🔁 The Loop: "Here are the tools. Figure it out."

The model goes around a circle:

1. Think: the model reads the request and picks the next step
2. Call a tool: for example getOrder(123) or getTracking(123)
3. Observe: the result goes back to the model
4. Done? If not, go around again. If yes, send the answer.

For our customer, the model might check the order, see it's 5 days late, check tracking, see it arrives today, and decide a refund isn't needed. You never wrote an if/else. The model chose the steps.

That's the strength of a loop, and also the risk: it could have refunded straight away.

Building blocks of a loop:

👉 The model picks the path. You design the environment.

🗺️ The Graph: "Here's the flowchart. Follow it."

Same customer, but this time you draw the path:

Start → Look up order → Is it late?
   No  → LLM writes a tracking reply → End
   Yes → LLM drafts a refund offer → Manager approves → End

The model never decides to refund. Your code decides, and a manager must approve. The model only writes the friendly text inside the boxes.

Building blocks of a graph:

👉 You pick the path. The model works inside the boxes.

Which one should you use?

In real systems you usually use both: a graph for the business process, with loops running inside the steps that need real reasoning.

My rule of thumb:

Build the loop first. Then ask: "Which decision would I be nervous letting the AI make alone?" Turn that decision into a graph edge.

Part 2: So what is a "harness"?

The model by itself can only take text in and give text out. It can't run tools, remember anything or stop itself. Something has to do all of that around it.

That something is the harness.

Think of a very smart new employee on day one:

• The model is the new hire. They can read, think and write very well. But they have no laptop and no logins, and they forget everything after each conversation.
• The harness is everything the company gives them so they can work:
  • Orchestration: decides the next step. This is where the loop or graph from Part 1 lives.
  • Tools: a laptop and system logins, which in code means functions like getOrder() and refund()
  • Memory: the ticket history, so they remember what the customer already said
  • Rules: company policy, such as "a refund over $100 needs approval"
  • Logs: an audit trail of every action

Agent = Model + Harness

Harness = orchestration (loop, graph or both) + tools + memory + rules + logs

A common shortcut is "the harness is the loop". That's too narrow. The loop or graph is the skeleton, but tools, context, memory and guardrails are where most of the real engineering happens. The same model in two different harnesses can behave like two completely different products.

Cheat sheet

• Model: the brain. Rented.
• Harness: everything around the model that turns it into an agent.
• Orchestration: the part of the harness that decides the next step.
  • Loop: the model decides. "Here are the tools, figure it out."
  • Graph: your code decides. "Here's the flowchart, follow it."

Loop = tools + a circle + a stop sign
Graph = state + boxes + arrows
Agent = model + harness

Which approach are you using in your agents right now: loop, graph or both? Let me know in the comments 👇

Next in this series: we'll build this harness in real Java code with Spring AI, then look at who builds it for you on Azure.

Code & Learn