Twelve months ago, an 'AI agent' in most enterprise contexts meant a chatbot with a few tool calls bolted on. Today it means something categorically different: an autonomous system that receives a goal, breaks it down into sub-tasks, executes those tasks across external services, evaluates its own output, recovers from failures, and delivers a finished result — without a human in the loop for every step. We have spent the past year building these systems for clients in financial services, legal technology, supply chain, and healthcare operations. This post is an honest account of what we have learned: the architectural patterns that hold up in production, the failure modes that only appear at scale, and the organisational prerequisites that determine whether an agent programme succeeds or stalls.
The Spectrum of Agency: Not All Agents Are Equal
The biggest source of confusion in agentic AI is that the word 'agent' covers a massive spectrum. At one end: a simple ReAct loop that calls two tools and returns an answer. At the other: a fully autonomous system that plans across hundreds of steps, spawns sub-agents, manages its own memory, and operates for hours without human input. Most of the production value in 2026 sits in the middle of that spectrum — what we call 'supervised autonomy'. The agent handles everything it is confident about autonomously. It escalates edge cases to a human reviewer. It audits its own outputs before committing them. The teams that try to jump straight to full autonomy almost always fail. The teams that start with supervised autonomy, measure confidence scores, and gradually expand the automation envelope are the ones shipping stable production systems.
Architecture: The Four Layers Every Production Agent Needs
After building agents across a dozen production deployments, a consistent architecture has emerged. The planning layer is the agent's 'brain' — an LLM that receives the goal and produces a structured task plan, typically as a DAG of sub-tasks with dependencies, inputs, outputs, and estimated confidence scores. Never let the planning LLM also execute: separation of planning and execution is the single most important architectural decision. The execution layer runs individual tasks: calling APIs, reading databases, browsing pages, writing code, running shell commands. Each tool call is atomic, logged, and reversible where possible. The memory layer manages context: short-term (current task window), episodic (what this agent did last time it ran), and semantic (a vector store of accumulated domain knowledge). Without structured memory, agents lose coherence on any task longer than a single context window. The supervision layer is where most teams under-invest: a confidence-scoring mechanism that flags low-certainty steps for human review, a full audit trail of every decision and tool call, and a rollback mechanism for any action that modified external state.
The Tool Design Problem Nobody Talks About
Agent failures are almost never LLM failures. In our experience, over 70% of production agent failures trace back to poorly designed tools. A well-designed agent tool has four properties: it is idempotent (calling it twice produces the same result as calling it once), it is scoped (it does one thing and has a clear, narrow contract), it fails loudly (it raises structured errors the LLM can understand and recover from, not stack traces), and it is observable (every call logs inputs, outputs, latency, and cost). The hardest tools to get right are the ones that modify external state: sending emails, creating records in a CRM, triggering payments, updating a database. These need explicit confirmation gates, especially early in a deployment when the agent's confidence calibration is still being tuned. One client's agent sent 3,000 draft emails before a confirmation gate was added. The gate caught a systematic prompt formatting bug before any of those drafts were sent to real customers.
Memory and Context: The Hidden Complexity
An agent with a 128k context window is not an agent with unlimited memory. It is an agent with a 128k working memory that needs to decide what to load into that window for every task. The agents that degrade gracefully at scale are the ones with explicit memory management strategies. We build three memory tiers: working memory (the current task prompt — aggressively summarised, never padded), episodic memory (a structured log of past agent runs with outcomes — queried by similarity to the current goal), and a domain knowledge store (a vector index of company policies, product documentation, customer data — retrieved by relevance with reranking). The most common memory failure we see is context pollution: information from an early task step contaminating the agent's reasoning in a later step. The fix is aggressive summarisation checkpoints: every N steps, the agent writes a structured 'state snapshot' that replaces the raw history in the context window.
Evaluation: You Cannot Trust Vibes in Production
Every agent programme needs an evaluation framework before it goes to production, not after. We run four evaluation categories. Task completion rate: did the agent finish the goal? Tracked per task type, per user segment, and over time. Output quality: human raters score a sample of agent outputs on a rubric specific to the use case — not generic 'helpfulness', but domain-specific criteria. Confidence calibration: when the agent says it is 90% confident, is it right 90% of the time? Poor calibration is more dangerous than low confidence — an agent that is wrong but certain will cause more damage than one that correctly flags uncertainty. Cost per task: token spend, tool call count, and wall-clock time per completed task. All four metrics go into a dashboard that runs on every deployment. Regressions in any metric trigger a freeze on the deployment. Agent behaviour is non-deterministic; the only way to know if a change helped or hurt is systematic measurement.
Multi-Agent Systems: When to Use Them and When Not To
Multi-agent architectures — orchestrator agents that spawn specialist sub-agents — are powerful and genuinely necessary for complex, long-horizon tasks. They are also significantly harder to debug, observe, and keep cost-efficient. Our rule: do not introduce a multi-agent architecture until a single-agent approach has provably hit its ceiling. When you do go multi-agent, three principles matter. Explicit contracts: each sub-agent has a documented input schema and output schema. The orchestrator validates both. Isolated failure: a sub-agent failure should be recoverable at the orchestrator level — sub-agents should never have side effects the orchestrator is not aware of. Cost budgets: each sub-agent has a token budget and a time budget. Runaway sub-agents in a multi-agent system can turn a $2 task into a $200 one before any human notices. We have seen this happen in production. Set hard limits.
The Organisational Layer: What Needs to Be True Before You Ship
The technical architecture is the easier half of the challenge. The harder half is organisational. Before any agentic system goes to production, three things need to be true. First, there must be a named human owner for every action category the agent can take: who is accountable when the agent sends a wrong email, creates a wrong record, or triggers an incorrect workflow? Accountability gaps are where agent programmes stall after the first incident. Second, there must be a documented escalation path: which situations should the agent handle autonomously, which should trigger a human review queue, and which should cause the agent to abort and alert? This matrix should be written down and agreed before the agent touches production data. Third, there must be a rollback plan for every external action: not every action is reversible, but you should know which ones are not, and those need the most conservative confidence thresholds. The teams that get agentic AI right in 2026 are not the teams with the most sophisticated LLM prompts. They are the teams that treated agent deployment with the same operational rigour they apply to any other production system: staged rollouts, monitoring, incident response, and a clear owner for when things go wrong.
