Over the past year, I’ve noticed a pattern in many AI discussions.
Someone describes a business problem. The team starts brainstorming solutions. A few minutes later, somebody says:
“Let’s build an agent.”
At that point, the conversation often jumps straight into frameworks, orchestration layers, memory systems, tool calling, and multi-agent architectures.
What’s interesting is that many projects never needed an agent in the first place.
The excitement around agentic AI has created a situation where teams sometimes begin with the implementation pattern rather than the actual problem they’re trying to solve. As a result, complexity enters the system long before anyone has clearly defined what the AI needs to know, what information it should access, and what actions it is allowed to take.
A more practical approach is to work backwards from the job itself.
Start With Understanding
Many AI applications only need to understand information, summarize documents, classify content, extract insights, or generate responses.
In these cases, an LLM is often enough.
The challenge isn’t building an agent. The challenge is designing prompts, workflows, and validation processes that consistently produce useful results.
Adding more moving parts doesn’t automatically create more value.
Then Ask Where Knowledge Comes From
The next question is whether the model needs information beyond what it already knows.
If answers depend on internal documentation, company policies, product information, or knowledge that changes frequently, retrieval becomes important.
This is where RAG and vector databases can be extremely useful.
However, I’ve seen teams use retrieval systems for problems that are actually database problems.
If a customer asks for an account balance, order status, invoice history, or CRM record, the answer usually shouldn’t come from a vector database.
A direct system query is often simpler, cheaper, and more reliable.
Access Is More Important Than Intelligence
Once an AI system starts interacting with business tools, the conversation changes.
Download the Medium app
At that point, the most important questions are no longer about model quality.
They’re about permissions.
What can the system access?
What actions can it perform?
What should require approval?
What should remain entirely under human control?
In many situations, a straightforward API integration is all that’s needed. More advanced approaches become valuable when organizations want a standardized way to connect multiple tools, systems, and data sources across different applications.
This Is Where Agents Begin To Matter
Agent design becomes valuable when the system must make decisions across multiple steps while handling exceptions, uncertainty, and changing conditions.
A support chatbot that answers questions from documentation may only require an LLM and retrieval.
A system that checks account information, validates policies, updates tickets, requests approvals, communicates with customers, and adapts to different outcomes is solving a very different problem.
That is where orchestration starts becoming necessary.
Not because agents are fashionable.
Because the workflow genuinely requires them.
The Real Engineering Challenge
What I’ve learned is that production AI isn’t defined by whether a model can use tools.
It’s defined by where you allow the model to exercise judgment and where the surrounding system maintains control.
The most successful AI projects rarely start by asking:
“How do we build an agent?”
They start by asking:
“What job are we trying to accomplish?”
Everything else usually becomes much easier to design after that.
