The most common mistake in building LLM-powered systems is reaching for complexity before it is needed. Agents are powerful, but they introduce latency, cost, and unpredictability. The right approach is to start simple and let measured performance gaps justify each step up in architectural complexity.
Start with the Simplest Thing That Works
Before building any agentic system, ask: can a single, well-crafted LLM call solve this problem? A surprising number of tasks — summarization, classification, extraction, translation, straightforward Q&A — are handled effectively by a single prompt paired with good retrieval. Adding orchestration on top of an already-adequate solution does not improve it; it only makes it slower and more expensive.
The progression should look like this:
- Single LLM call. Optimize the prompt, add retrieval if the model needs external context, and use structured output parsing. Measure performance.
- Simple workflow. If one call is not enough, add a second step — validation, refinement, or routing. Measure again.
- Multi-step workflow. If the task genuinely requires decomposition, build a workflow with clear handoffs between stages. Measure again.
- Agent. If the task is open-ended enough that you cannot predetermine the steps, give the model autonomy over its own loop. Measure again.
Each step should be motivated by evidence that the previous level is insufficient, not by a belief that more complexity equals better results.
When Simple Solutions Suffice
Many real-world applications do not need agents at all:
- Retrieval-augmented generation (RAG) handles most knowledge-grounded Q&A tasks effectively.
- Classification and routing can be done with a single LLM call and a well-designed prompt.
- Content generation with specific formatting requirements is typically better served by a prompt chain than an autonomous loop.
- Data extraction from structured or semi-structured documents is a single-call problem with good few-shot examples.
If your task fits neatly into one of these categories, adding agentic behavior is likely to hurt rather than help.
When Workflows Are the Right Choice
Workflows are appropriate when:
- The task can be broken into well-defined subtasks with clear inputs and outputs.
- You need consistency — the system should handle similar inputs in similar ways.
- Latency and cost are tightly constrained and you need predictable resource usage.
- The task benefits from specialized handling at each stage (e.g., different models or prompts for different steps).
The workflow patterns described in this wiki — prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer — cover a wide range of production use cases without requiring the model to make autonomous decisions.
When Agents Are Justified
Agents earn their complexity when:
- The problem is genuinely open-ended. You cannot predict in advance what steps the model will need to take. Examples include coding tasks where the model must explore a codebase, multi-step research across varied sources, or troubleshooting problems with many possible root causes.
- The task requires adaptive tool use. The model needs to choose from a large set of tools based on evolving context, not a fixed decision tree.
- Human-in-the-loop interaction is feasible. Because agents are less predictable, the ability for a human to review, correct, or redirect the agent mid-task significantly reduces risk.
Measure Everything
Whatever level of complexity you choose, benchmark relentlessly. Define clear success metrics for your task. Measure accuracy, latency, cost, and failure rates. Compare your agentic solution against simpler baselines. If the agent does not meaningfully outperform a well-optimized workflow, the workflow is the better choice — it will be easier to maintain, debug, and reason about.
The goal is not to build the most sophisticated system. The goal is to solve the problem reliably, efficiently, and at a cost that makes sense. Let the evidence guide the architecture.