%%{init: {"theme": "base", "themeVariables": {"quadrant1Fill": "#FAF7F0", "quadrant2Fill": "#F3EFE5", "quadrant3Fill": "#FAF7F0", "quadrant4Fill": "#F3EFE5", "quadrantPointFill": "#1E2EB8", "quadrantPointTextFill": "#16171B", "quadrantXAxisTextFill": "#16171B", "quadrantYAxisTextFill": "#16171B", "quadrantTitleFill": "#16171B", "quadrantInternalBorderStrokeFill": "#766F65", "quadrantExternalBorderStrokeFill": "#766F65"}}}%%
quadrantChart
title The two knobs of the agent loop
x-axis Little deliberation --> Much deliberation
y-axis Short commitment --> Long commitment
quadrant-1 Open-loop sense-plan-act
quadrant-2 Commit without deliberating
quadrant-3 Pure reaction
quadrant-4 Closed-loop deliberation
Subsumption: [0.16, 0.18]
Shakey: [0.84, 0.86]
ReAct: [0.92, 0.10]
MPC: [0.80, 0.20]
BDI and PRS: [0.66, 0.30]
4 Agent Architectures: Classical and Neural
Agent = Model + Harness, ran the equation Section 3.4 closed on: the foundation model supplies the raw material, and the apparatus around it is what makes that material act. Part II is about the apparatus, and this chapter is about its shape. Before one can build a harness it helps to know what kinds there are, and on that question the field is the opposite of a blank page. How to arrange perception, deliberation, and action into something that behaves sensibly over time is a problem with three decades of answers behind it — worked out, for the most part, by researchers who possessed no substrate worth the name and designed the architectures anyway. They were drawing up the blueprints for a harness while waiting for an engine to arrive. The engine has since arrived; the blueprints, it turns out, were largely sound.
Some vocabulary first, because the field has quietly renamed its own foundations. Russell and Norvig give the tidy formula that an agent is its architecture plus its program: the architecture is the fixed machinery — sensors, actuators, and the structure that routes between them — and the program is what runs on it (2021). Set this beside the equation of the previous chapter and the two line up almost too neatly. The model supplies the program, or most of it: the competence, the judgement about what a sensible next move would be. The harness is the architecture: the standing structure that feeds the model its perceptions, carries its decisions out into the world, and calls it again tomorrow. Agent architecture, a term of art from the 1980s, is therefore the proper and older name for what practitioners now call the harness — and the classical taxonomy of architectures is, read correctly, a catalogue of the harnesses one might build, complete with the trade-offs each entails, drawn up by people who had already paid for those trade-offs.
This chapter walks that catalogue. The field converged, after a good deal of argument, on a small number of architectural families, and they arrange themselves along a single illuminating axis: how much the agent thinks between perceiving and acting. At one end sits the reactive architecture, which thinks as little as possible and is gone before its rivals have finished modelling the problem; at the other, the deliberative architecture, which builds a model of the world and reasons over it before lifting a finger. Between them sit the hybrid architectures, which try to have it both ways, and the Belief–Desire–Intention (BDI) model, the most psychologically ambitious of the lot. We take each in turn, ask what it was good for and what it was hopeless at, and — this is the chapter’s payoff — show that the modern agent loop, the observe–reason–act cycle that practitioners tune as though it were newly minted, is a special case — deliberation run closed-loop — distinguished chiefly by the one thing the classical designers never had: a deliberating core that is actually any good.
There is a deeper reason to take the old architectures seriously than mere genealogy. Recall the two absences the substrate is most embarrassed by (Section 3.3): it has no goals, and no loop. An architecture is precisely the apparatus that supplies both — the standing structure in which a goal can persist across calls and a perception–action cycle can turn. This chapter is where the idle substrate of Chapter 3 acquires the will to keep going. What it does once it is going — how it reasons and plans (Chapter 5), how it acts through tools (Chapter 6), and how it remembers (Chapter 7) — is the business of the rest of Part II; here we build the frame that holds those pieces together.
4.1 What an Architecture Is, and Why a Substrate Needs One
The word architecture is doing specific work here, and it repays slowing down to see what. In the everyday sense an architecture is a structure one could photograph — a building, a chip — but the agents literature means something more particular: an architecture is a methodology for decomposition. It fixes what modules an agent is built from — perception, a world model, a planner, a set of behaviours, the machinery of action — and, more consequentially, how those modules are wired together and which of them gets to decide what the agent does next. Two agents with identical sensors, identical actuators, and the very same underlying reasoning capacity can behave quite differently because their modules are arranged differently: one consults a plan before it moves, the other answers whatever its sensors last reported. The architecture is that arrangement. It is the part of an agent its designer commits to early and revises late, because everything else hangs from it.
Russell and Norvig’s formula repays a second look. The program is the function from what the agent has perceived to what it does next; the architecture is the standing machinery on which that function runs and through which its choices reach the world. The division earns its keep because the two halves have such different lifespans. The program is content — it is what varies from task to task, and in a foundation-model agent much of it is supplied afresh on every call by the model and its prompt. The architecture is structure — fixed for the duration, and a good one survives the replacement of nearly everything inside it. This is the sense in which Agent = Model + Harness is the older equation in newer dress: the model is most of the program, the swappable supplier of competence, and the harness is the architecture. One corollary deserves saying plainly, since it is among the few steadying facts in a field that reinvents itself weekly: a well-designed architecture lets you change the model beneath it — a new release, a cheaper open-weight stand-in, a different vendor altogether — without redesigning the agent. The harness outlives the model it was built around, which is just as well, given how briefly any particular model stays current.
It is worth separating the architecture, too, from the behaviour it happens to produce. Our running coder agent — the single worker we dissect in detail in Section 4.6 — has one architecture: a loop that reads an issue, proposes a change, edits the code, runs the tests, and inspects the result before going round again. That frame does not change when the agent is asked to chase down a null-pointer exception rather than add a configuration flag; the task changes, the prompt changes, the model’s output changes, but the structure stays put. Mistaking the two — taking a clever transcript on one task for a sound architecture — is precisely how a demonstration that dazzles on Tuesday keels over in production on Wednesday. The architecture is what you may rely on across tasks; the behaviour is merely what happened on this one.
Why a substrate should need any of this brings us back to the two absences. The model has no goal and no loop: handed a context it returns a continuation and falls silent, sublimely indifferent to whether the task is finished. An architecture is exactly the apparatus that makes good both deficiencies. It supplies the loop that calls the model again, the state in which a goal can persist from one call to the next, and the circuit by which a decision becomes an action and the action’s result becomes the following perception. Take the architecture away and you are left with the idle substrate of Chapter 3, fluent and inert. The architecture, in short, is where agency actually lives: the model supplies the capacity to judge what a good next move would be, but it is the architecture that strings a sequence of such judgements into something that pursues an end.
That leaves the question every architecture must answer, and the one along which the classical families divide: how much should an agent think between perceiving and acting? At one extreme it should not think at all, but wire perception straight to action and respond reflexively and fast. At the other it should think a great deal — assemble a model of its situation, weigh the consequences of the alternatives, and commit to a plan before it stirs. The first buys speed and robustness at the price of foresight; the second buys foresight at the price of speed, and of an uncomfortable dependence on the world model being correct. The space of agent architectures is, to a first approximation, the space of answers to this single trade-off, and the field’s canonical survey (Wooldridge & Jennings, 1995) lays it out just so: reactive architectures at the think-as-little-as-possible end, deliberative architectures at the other, hybrid architectures seeking a principled compromise between them, and the Belief–Desire–Intention model furnishing that compromise with something like a theory of mind. The next four sections take them in that order. They are best read not as a museum tour of superseded designs but as four settings of a single dial — a first approximation that Section 4.6 will refine into two — a dial that every agent you build, including the one you may have shipped this morning, has been turned to, whether or not its designer noticed turning it.
4.2 Reactive Architectures: Intelligence Without Deliberation
By the mid-1980s the deliberative orthodoxy had a problem its critics found hard to overlook: the robots it produced spent so long maintaining and reasoning over their internal models of the world that the world had generally moved on by the time they had worked out what to do. Into this impasse strode Rodney Brooks with a heresy whose paper title was itself the manifesto — Intelligence without representation (1991). The claim was deliberately provocative: that the central article of faith of symbolic AI, the explicit internal model over which an agent reasons, was not the foundation of intelligent behaviour but a costly detour. Why maintain an elaborate map that is always slightly out of date and expensive to keep current, when the world is sitting right there to be looked at again? “The world”, in the slogan that became the movement’s banner, “is its own best model.” Intelligence, on this view, is not something an agent computes in its head and then enacts; it emerges from the tight, continuous coupling of perception and action in a real environment.
Brooks’s constructive answer was the subsumption architecture (1986). An agent is built not as a pipeline — sense, then model, then plan, then act — but as a stack of behaviours, each a near-direct wiring of perception to action, and each competent on its own at some simple thing: avoid obstacles, wander, head towards the light. The layers run concurrently and continuously; a higher layer does not so much call a lower one as subsume it, suppressing or overriding its outputs when circumstances warrant, so that avoid-obstacle yields to flee-predator without either consulting a central plan. There is no shared world model, no symbolic representation, and nowhere that the agent’s single coherent picture of the situation is kept — because there is no such picture. Brooks’s robots walked, scuttled, and avoided collisions robustly and in real time, with not a symbol in sight.
What this arrangement buys is precisely what the deliberative robots lacked. It is fast: with no deliberation interposed between sensor and motor, a reactive agent responds at the speed of its hardware. It is robust: there is no internal model to drift out of register with reality, and nothing the agent can be catastrophically wrong about, because it commits to no representation it must later defend. It degrades gracefully — lose a layer and the others carry on — and it excels in exactly the dynamic, unforgiving settings where a planner would still be deliberating as the situation changed beneath it. For a large class of tasks, looking again really is cheaper and safer than remembering.
The price of carrying no model, though, is carrying no foresight. A purely reactive agent cannot plan, cannot reason about a state of affairs that does not currently obtain, and cannot pursue a goal that requires a considered sequence of steps no single one of which the present moment triggers. It cannot tell you why it did what it did, because the only available answer is “the sensors said so.” And — the objection that did most to limit the movement’s reach — reactive agents are deceptively hard to engineer for sophisticated behaviour: as competences multiply, arbitration between behaviours becomes a tangle, the global behaviour grows difficult to predict from the local rules, and there is no principled method for composing reflexes into the solution of a genuinely novel problem (Wooldridge, 2009). Reactivity scales beautifully with the speed of a task and badly with its depth.
The modern echo is closer than it first appears. A foundation model handed an input and a set of tools, emitting a single action and then falling silent — no loop, no plan, no state carried to the next call — is a reflex agent in the strict sense: perception in, action out, and nothing in between but a very well-read reflex. It is what Russell and Norvig call a simple reflex agent (2021), the difference being that the reflex is now answered by a substrate which has, in effect, read the manual for almost everything. The router that reads a request and dispatches it to the right handler, the guardrail that classifies an input and blocks it, the one-shot function call that turns a sentence into an API request: these are reactive architectures, and there is nothing primitive about choosing one. Where a task is fast, bounded, and forgiving, a reflex is not an embarrassment to be apologised for but the cheapest tool that works.
There is a nice irony in all this. Brooks built his reputation on dethroning representation; the foundation model is representation’s emphatic return, a compressed model of the world so vast it would have appalled him. Yet wired as a single reflexive call it behaves exactly as his architecture promised — swift and dependable on the routine, and helpless the instant a task demands more than one informed reflex can supply. The substrate does not abolish the distinction between reacting and deliberating; it merely raises the floor, making the reflex itself far cleverer while leaving wholly intact the fact that a reflex keeps nothing and foresees nothing. That is the lesson of the preface in its architectural form: reach for deliberation only when foresight earns its cost. The matching danger is the mirror image of the virtue — a reflex wired straight to an irreversible action, sending the email or merging the branch or spending the money with no deliberative pause between impulse and deed, is a loaded foot-gun, and we return in Chapter 6 to what it means to let an agent do something it cannot take back.
4.3 Deliberative Architectures: The Sense–Plan–Act Pipeline
If the reactive tradition was a revolt against the internal model, the deliberative tradition is the orthodoxy it revolted against, and it is built around precisely the thing Brooks threw out. Its intellectual charter is the physical symbol system hypothesis, Newell and Simon’s claim that a system which manipulates symbols standing for things in the world has the necessary and sufficient means for general intelligent action (1976). On this view intelligence simply is the manipulation of representations: to act well, an agent should maintain an explicit symbolic model of its world and reason over that model to decide what to do. Where the reactive agent looks again, the deliberative agent looks inward, at a picture of the world it carries and curates.
The canonical embodiment is the sense–plan–act cycle. The agent senses, updating its symbolic model from perception; plans, searching for a sequence of actions that would carry the modelled world from its current state to a goal state; and then acts, executing that plan a step at a time. Shakey, the SRI robot that lent its name to a generation of cautionary tales, ran exactly this loop, and the planner written to give it purpose — Fikes and Nilsson’s STRIPS (1971) — set the template the field would use for decades: represent the world as a set of logical predicates, each action as the conditions it requires and the facts it adds and deletes, and reduce “deciding what to do” to a search for a path from start to goal. The SOAR architecture later generalised the instinct into a complete account of cognition, casting every task as search through a problem space and every action as an operator applied to a symbolic state (Laird et al., 1987). The shared commitment is the exact inverse of Brooks’s: think first, in the model, and act only once the thinking is done.
What this buys is everything reactivity could not reach. A deliberative agent has foresight — it can weigh the consequences of an action before committing to it, and string together a sequence that arrives at a goal no single moment would have prompted. It is goal-directed in the full sense, pursuing ends rather than merely answering stimuli. And because its plan is an explicit object, the architecture is transparent in a way no reflex can be: you can inspect the plan, verify it, and ask the agent why it did something, with the gratifying result that there is an actual answer — it did X because X achieves subgoal Y on the way to goal Z. Given a faithful model and the right operators, a single planner solves every problem its domain contains.
The bill, when it arrives, is steep, and it comes in three parts. The first is the notorious frame problem: to reason about change in logic, an agent must specify not only what each action alters but the immense quantity it leaves untouched, and keeping a symbolic model consistent as the world turns proves both computationally crushing and philosophically slippery (McCarthy & Hayes, 1969). The second is the cost of planning itself: search is combinatorial, deliberation is slow, and a plan computed for the world as it stood at the start of deliberation may be obsolete by the time it is ready — foresight is of limited use if the future arrives before you have finished forecasting it. The third, and the most fatal in practice, is the knowledge bottleneck: the whole edifice rests on a complete, correct, hand-authored symbolic model (the symbol-grounding worry the last chapter catalogued), and the instant reality strays beyond what the model anticipated, the agent is not merely mistaken but blind.
This, of course, is the hollow agent of Section 2.4, viewed from the side of its architecture. Nothing about sense–plan–act was wrong — it remains, to this day, a thoroughly sensible way to organise a goal-directed agent. What it lacked was anything to put inside it. The world model had to be built by hand, the operators coded by hand, the symbols anchored to reality by hand, and so the architecture could only ever be as broad, as current, and as robust as the team maintaining its representations, which is to say not very. The deliberative tradition designed the cockpit with great care and then waited fifty years for an engine. It was, in a word, premature.
And it is the architecture the substrate most directly revives. A foundation model is precisely the queryable world model that sense–plan–act always had to be handed (Section 3.2), supplied now in natural language rather than hand-cut logic; and the chain-of-thought reasoning (Wei et al., 2022) we met in Chapter 3 is its planning step, the model laying out an explicit line of deliberation before it commits to an action. The plan-then-execute agents of the present day — decompose the task, draft a plan, carry it out — are the sense–plan–act pipeline reborn, with a foundation model in the planner’s chair and its compressed knowledge standing in for the predicates no one now has to write. The substrate dissolves the knowledge bottleneck and takes much of the sting from the frame problem, since natural-language reasoning bends where formal logic snapped. But the older difficulties return in new clothes: deliberation is still slow, and now it is metered in tokens and latency; the plan can be confidently, fluently wrong, because the world model it rests on is free to hallucinate; and the world still moves while the model thinks. When deliberation is worth its price — how much an agent should plan and how much it should simply act — is the question of Chapter 5. For now it is enough to see that the most symbolic, most representation-hungry architecture the field ever built is the one the new substrate brings back to life.
4.4 Hybrid Architectures: Layering Reflex and Deliberation
Two camps, each convinced the other had mistaken a half-truth for the whole, made for a tiresome stand-off, and the resolution was the one most stand-offs deserve: both sides were right about themselves and wrong about each other. A useful agent needs the reflex’s speed for the routine and the deliberator’s foresight for the considered, and there is no law of nature requiring it to choose. The hybrid, or layered, architecture is the design that refuses to: it builds a reactive layer and a deliberative layer into a single agent, so that the thing can dodge the unexpected at reflex speed while still reasoning about the longer game. Ferguson’s TouringMachines opened with precisely this diagnosis — that neither purely reactive nor purely deliberative control could produce the range of behaviour a real environment demands (1992) — and the same instinct, pairing a fast layer with a slow one, recurs in a great many of the systems you will build or meet, under names their designers rarely trace this far back.
Granting that an agent should have both layers, the real question is how they are arranged and which of them gets to move the actuators. Wooldridge sorts the answers into two styles (2009). In horizontal layering, exemplified by TouringMachines, every layer runs from perception to action in parallel, each proposing what to do, with a mediating control framework left to referee their competing suggestions — flexible, but only as good as the referee, who must somehow reconcile a reflex shouting “swerve” with a planner murmuring “stay the course”. In vertical layering, exemplified by Müller’s InteRRaP (1996), the layers are stacked and control is passed upward only as needed: a behaviour-based layer handles what it can, escalates to a local planning layer when it cannot, and escalates again to a cooperative planning layer when the problem involves other agents at all — a structure whose top floor quietly anticipates the whole of Part III. The same fast-skills / sequencing / planning stratification recurs in the three-layer architectures of mobile robotics (Gat, 1998).
The genuinely hard part of a hybrid is not having two layers but adjudicating between them. When the reflex wants to act now and the plan wants to proceed as written, something must decide, in real time and usually without a clean theory to appeal to, which layer wins and when control should climb from reflex to deliberation. This is why hybrids, for all their practical success, never acquired the conceptual tidiness of either pure design: they work, but they are fiddly, and the elegance of pure reaction or pure deliberation is traded for a compromise stitched together largely by hand and tuned by experience.
Underneath the engineering lies an idea worth stating in the open, because the modern field leans on it heavily: intelligence seems to want at least two processes running at different tempos — a fast one for the routine and a slow one for the deliberate. Kahneman gave the popular vocabulary (2011): System 1, fast, automatic, and intuitive; System 2, slow, effortful, and deliberative. The layered architectures are, in effect, System 1 and System 2 cast in software, and their arbitration problem is exactly the question of when to spend the expensive slow one.
Which is why the present moment is busily rediscovering them. A foundation-model agent that answers an easy query directly but unrolls an extended chain of reasoning for a hard one is a two-layer hybrid: a fast path, a slow path, and a controller choosing between them. The “reasoning” models that decide how long to think before answering, the routers that send a trivial request to a cheap reflexive call and a thorny one to a full plan-then-execute pass, the coder agent that applies a one-line fix on reflex but stops to plan a multi-file refactor — all are layered architectures, re-deriving horizontal and vertical control under newer names like routing and escalation.
The model families now ship those two layers ready-made. Beside each flagship sit smaller, faster siblings — Claude’s Opus, Sonnet, and Haiku; Gemini’s Pro and Flash; the high- and low-effort tiers the others offer — a menu trading capability against speed and cost, the cheaper of them often distilled from the flagship in the technical sense of the word (Hinton et al., 2015) — as Google reports of Gemini’s Flash (2024), in one of the few cases a laboratory has documented. Wire the big, slow model to the deliberative layer and the small, fast one to the reactive layer, and Kahneman’s two systems fall out of two models: a System 2 that reasons and a System 1 that responds — each, naturally, billed accordingly. The substrate makes both layers dramatically better; it does not make the arbitration go away. It merely moves it into prompt logic and control flow, where it is just as decisive and rather less studied.
Deliberation costs latency, costs tokens, and carries the underrated risk of overthinking a problem a reflex would have dispatched cleanly; so the controller’s decision of when to escalate from fast to slow is where a hybrid agent is won or lost. Spend the slow system too freely and the agent is sluggish and expensive; spend it too sparingly and it blunders, confidently, through problems that wanted a moment’s thought. That decision — when an agent should think and when it should simply act — is contemporary enough, and consequential enough, to deserve a chapter of its own, and it gets one next. The lesson of the hybrids stands in the meantime: the best architectures the field produced were neither pure reflex nor pure deliberation but a managed conversation between the two, and the managing was always the hard part.
4.5 The Belief–Desire–Intention Model
The richest of the classical architectures, and the one that lands nearest to the modern agent, began not in a laboratory but in a work of philosophy. Bratman set out to explain how a creature with finite time and finite attention manages to act sensibly at all (1987). We cannot, before every action, reconsider every option from first principles — there are too many, and the world will not wait while we deliberate. His answer was that intentions do the rationing. Once you have settled on an intention, it persists; it filters the options you bother to consider thereafter, and it frames the further reasoning you do, so that you are not obliged to re-derive your whole life each morning before breakfast. An intention, on this account, is not merely a desire with the volume turned up. It is a distinct mental state with a distinct job: commitment.
The architecture that bears its name gives an agent three attitudes. Beliefs are its model of how the world is; desires are the states of affairs it would, other things being equal, like to bring about; and intentions are the desires it has committed to pursuing. It then runs a deliberation cycle that should, by now, look familiar: revise the beliefs in the light of new perception; generate the options that current beliefs and standing intentions permit; deliberate to promote some of those options to intentions; and reason from means to ends to find a plan that serves them — then act, and go round again. This is the machinery behind the agent-oriented programming we met in Section 2.2, drawn at the level of a single deliberating mind.
The load-bearing idea, the one that distinguishes BDI from a mere plan-then-execute loop, is the persistence of intentions — their resistance to idle reconsideration. Cohen and Levesque gave it a logic in a phrase that doubles as a definition: intention is choice with commitment (1990). The reason persistence matters is bounded rationality in its most vivid form. An agent that reconsiders its intentions at every step spends all its resources deliberating and never makes progress; an agent that never reconsiders them charges ahead with a plan the world has long since invalidated — the over-cautious clerk and the fanatic, and neither is who you want flying the aeroplane. What separates them is a commitment strategy: how doggedly an intention is held before the agent will think again — blindly, until it believes the goal achieved; single-mindedly, until it believes it achieved or impossible; or open-mindedly, as long as it is still believed worth pursuing. How tight to set that dial is not a detail; it is the difference between an agent that is decisive and one that is merely stubborn, and the right setting depends entirely on how fast the agent’s world moves.
None of this stayed on the page. BDI is, by some margin, the most fully realised of the classical architectures: a complete road from a theory of mind to a language you can run. Georgeff and Lansky’s Procedural Reasoning System was the first substantial implementation (1987) — a plan library and explicit beliefs, desires, and intentions, put to work on real-time fault monitoring for spacecraft, which is not the sort of thing one entrusts to a metaphor. Formalised by Rao and Georgeff (1991, 1995) and recast as the AgentSpeak programming language (1996), the architecture is still alive today in the Jason interpreter (Bordini et al., 2007). The classical era left behind not just a picture of how a rational agent should be organised but a working toolchain for building one.
Now the payoff. Of every architecture in the catalogue, BDI is the one a foundation-model agent most nearly is — but only once you see which half of it the model supplies and which half it cannot. The model is a superb engine for belief revision, turning a heap of observations into an updated picture of the situation; for option generation, proposing candidate goals and the plans that might serve them; and for the means-ends reasoning that connects them, which is Chapter 5’s business. What the model does not supply, and cannot, is intention. Being stateless and goalless, it holds no commitment that outlives a single call; it conjures a fresh, fleeting set of quasi-beliefs and quasi-desires each time it is invoked and forgets them the instant it returns. The intention has to live somewhere else, and that somewhere is the harness: the task held in state, the goal carried across calls, the loop that keeps returning to the model until the goal is met or abandoned. That standing structure is the agent’s intention, and it is precisely what the equation Agent = Model + Harness was naming. Read through the substrate, BDI delivers its sharpest one-line lesson: the model is the faculty of deliberation, and the harness is the will.
And with that mapping, Cohen and Levesque’s question returns as a decision every agent builder now makes, knowingly or not. How doggedly should the agent hold its plan before reconsidering it in the light of the latest tool result? Replan after every observation and it thrashes, indecisive and expensive; refuse to replan and it pursues a goal the most recent error message has already doomed. The commitment strategy is no longer a clause in a modal logic but a question of when the harness re-invokes the planner and when it presses on — single-minded commitment with a trigger for reconsideration, to put it in the old vocabulary. It is the hybrids’ arbitration problem of Section 4.4, sharpened to a single edge, and Chapter 5 takes it up in earnest. Intention has one more entrance to make, too: when we reach teamwork in Chapter 10 it returns in the plural, as the joint commitments that let several agents hold to a shared plan without anyone in charge — which is, as much as anything in this chapter, the classical era’s most valuable bequest to the multi-agent half of the field.
| Family | Thinking between perceiving and acting | What it buys | What it costs |
|---|---|---|---|
| Reactive | As little as possible — perception wired straight to action | Speed, robustness, graceful degradation, situatedness | No foresight or multi-step goals; cannot explain itself; hard to engineer at scale |
| Deliberative | A great deal — model the world, plan, then act | Foresight; goal-directed, inspectable plans | Frame problem; slow combinatorial search; a hand-authored model as knowledge bottleneck |
| Hybrid (layered) | Both tempos — reflex for the routine, deliberation for the considered | Speed and foresight in one agent | Arbitration between the layers; fiddly, hand-tuned, conceptually untidy |
| BDI | A deliberation cycle rationed by a commitment strategy | A theory of mind for bounded rationality, with a working toolchain | Tuning the commitment dial between the over-cautious clerk and the fanatic |
| Family | Classical embodiment | Modern echo |
|---|---|---|
| Reactive | Brooks’s subsumption architecture and its walking robots | The single tool-calling turn; routers, guardrails, one-shot function calls |
| Deliberative | Shakey; STRIPS; SOAR | Plan-then-execute agents, with chain-of-thought as the planning step |
| Hybrid (layered) | TouringMachines (horizontal); InteRRaP (vertical); three-layer robotics architectures | Routers and escalation; reasoning models; model tiers as System 1 / System 2 |
| BDI | Procedural Reasoning System; AgentSpeak; the Jason interpreter | The model supplies deliberation; the harness supplies the intention — the will |
4.6 The Modern Agent Loop as a Special Case
Strip a contemporary agent down to its skeleton and what remains is a loop. The model observes — it reads the context it has been handed: the task, the history so far, the result of whatever it last did. It reasons — producing a line of thought and, at the end of it, a choice of what to do next. It acts — or rather the machinery around it acts on its behalf, executing the chosen step against the world. And the outcome of that step is folded back into the context for the next turn, whereupon the loop begins again. This interleaving of reasoning and acting has a name in the contemporary literature, ReAct (Yao et al., 2023), and the loop it describes, or a near relative, is the beating heart of essentially every agent a framework will build for you. Lilian Weng’s much-shared anatomy of the LLM agent draws the same skeleton from another angle (2023): a model at the centre, wrapped in planning, memory, and the use of tools.
4.6.1 Locating the Loop
Hold that loop up against the catalogue we have just assembled and its novelty quietly drains away. In kind it is deliberative: the model reasons before each action, so the choice of what to do is the product of deliberation, not a reflex. But it is not deliberation in the classical, open-loop form of Section 4.3 — compute a whole plan, then execute it blindly. ReAct takes one action and re-perceives, folding the result back into the context before it reasons again; it runs closed-loop, deciding each step in the light of the latest observation rather than committing to a course charted in advance. Closing the loop in exactly this way is the reactive tradition’s hard-won lesson (Section 4.2) — look again, and do not trust a stale plan — now absorbed into a deliberative architecture. The modern loop is therefore best read not as a layered hybrid (Section 4.4) but as sense–plan–act run closed-loop: deliberation chastened by reactivity. And even that synthesis is not new — the field had been building closed-loop deliberators since the 1980s, as we shall shortly see — so what is genuinely novel is none of the shape but the filling: for the first time the deliberating core is a general, competent reasoner rather than a planner hand-built over a world model someone had to author by hand, the deliberative architecture supplied at last with something to deliberate with.
4.6.2 The Harness Is the Architecture
Look more closely at what does the acting, and the chapter’s central identity snaps into focus. Everything in that loop other than the model — the code that calls it, parses the action it chose, dispatches the corresponding tool, catches the error when the tool fails, appends the result, and decides whether to iterate or stop — is the harness, and the harness is nothing but this chapter’s “architecture” rendered in Python. It is where the loop lives, the perception–action cycle the substrate cannot supply for itself; where the state lives, the persisting intention the model cannot hold; where the context is assembled afresh each turn, the belief the model is handed rather than a belief it keeps. When practitioners speak of loop engineering and harness engineering, they are doing agent architecture under another name: arranging perception, deliberation, and action around the model — the very decisions Brooks, Ferguson, and Rao made before them, now taken in code, and too often taken without the knowledge that they are old decisions, with names and a literature.
4.6.3 Two Knobs, Not One
Where on the spectrum a given agent sits is, as this chapter has insisted, a design choice — and it is worth being precise that the choice has two knobs, not one, and that they turn independently. The first is how much the agent deliberates before it acts; the second is how far ahead it commits before stopping to perceive again. The classical families we surveyed happen to set both knobs together — pure reaction turns both down, open-loop sense–plan–act turns both up — and it is because they cluster on that diagonal that a single axis served us well enough as a first approximation. But nothing couples the knobs, and the classical field knew it. The high-deliberation, short-commitment corner — reason hard, then act once and look again — is exactly where Georgeff and Lansky’s Procedural Reasoning System sat, its founding paper pointedly titled Reactive Reasoning and Planning (1987); it is where any BDI agent sits whose commitment strategy (Section 4.5) governs when it pauses to reconsider, that strategy being the second knob; and it is the corner ReAct occupies too. Closed-loop deliberation is thus not merely older than ReAct but a place the classical architectures already stood. What ReAct adds is not the closed loop; it is a deliberator at last worth running inside one.
Control theory has a name for that corner and a mature theory of it. Model predictive control optimises a sequence of moves over a finite prediction horizon, using a model of the system, then applies only the first move before re-measuring the state and optimising afresh (Rawlings et al., 2017); its two textbook parameters — the prediction horizon and the control horizon, the number of moves it commits before re-planning — are our two knobs exactly, and the standard recipe of a long prediction horizon with a control horizon of one is closed-loop deliberation to the letter. A ReAct agent is, in this light, model predictive control with a foundation model standing in for both the predictive model and the optimiser, reasoning its way to the next move rather than computing it; and the same receding-horizon idea is what model-based reinforcement learning reached for when it wrapped planning around a learned model.
None of this makes the modern loop any less useful — but it locates it precisely, in a region of the design space that control theorists, planning researchers, and BDI architects had all arrived at independently. The genuinely new territory lies elsewhere: a system that routes easy requests to a reflex while escalating hard ones to an extended reasoning loop is the layered hybrid of Section 4.4 in modern dress, a reactive path and a deliberative one with an arbiter between them. There is no universally correct setting of any of these dials, only one appropriate to a task’s tempo, its reversibility, and its appetite for foresight; and choosing it deliberately, rather than inheriting whatever a framework’s default loop happens to be, is the architectural judgement this chapter exists to sharpen.
4.6.4 The Coder Agent, Anatomised
The running example makes it concrete. Our coder agent — a single worker in the software-engineering team — is a ReAct loop. Its beliefs are the context assembled for it each turn: the text of the issue, the contents of the files it has opened, the output of the last command it ran. Its deliberation is the model’s reasoning about what to change and why. Its action is a single tool call — read a file, apply an edit, run the test suite. And its intention is the task held in the harness’s state together with the loop’s standing instruction to keep going until the tests pass. It reasons before each edit, as any deliberative agent does, but commits only a single edit at a time, re-checking the world by running the tests before it chooses the next — closed-loop by design, not a fix drafted in full and executed on faith. Its commitment strategy (Section 4.5) shows up in one concrete choice — when a test fails, does it reconsider its approach or retry the same edit? — and setting that dial badly reproduces both of BDI’s failure modes exactly: the agent that abandons a sound plan at the first red test, and the agent that retries a broken edit until the budget is exhausted.
4.6.5 Knowing When to Stop
The loop that confers agency is also the part most apt to misbehave, and in ways the classical architectures could largely ignore because their steps were cheap and finite. A foundation-model loop spends money and time on every pass, and a loop that does not know when to halt is not persistent but pathological — the agent that edits, tests, fails, and edits again into the small hours, accumulating a bill and a measure of ill-will in whoever is paying it. Termination is therefore not a tidying-up detail but a load-bearing part of the architecture: a goal test, a step budget, a ceiling on cost. Getting it right is a thread that runs through the engineering chapters — building the loop honestly from scratch in Chapter 20, and making it dependable, with budgets, retries, idempotency, and interrupts, in Chapter 23. Architecture settles what an agent can do; these settle whether it does so without alarming the finance department.
The modern agent, then, is not a break with the architectural tradition but its most fortunate instance: deliberation run closed-loop, its reasoning core a general one at last, rather than a planner competent only over a world its designers had to author by hand. The shape was settled decades ago — in robots that walked, planners that proved theorems, and logics of intention that never had an engine to drive — and the substrate is merely what made the shape work. With the frame now in place, the remainder of Part II fills in the faculties the loop coordinates: how an agent reasons and plans (Chapter 5), how it acts upon the world through tools (Chapter 6), and how it remembers across the turns of the loop (Chapter 7).
4.7 Summary
- An agent architecture is the structure that turns competence into conduct. It fixes how perception, deliberation, and action are arranged around a substrate. In Russell and Norvig’s terms an agent is its architecture plus its program — which is the older equation Agent = Model + Harness in different dress: the model supplies the swappable program, the harness is the durable architecture, and a good architecture outlives the model it was built around.
- The classical architectures arrange themselves along a single axis: how much an agent should think between perceiving and acting. The four families are four settings of one dial, not four unrelated designs.
- Reactive architectures think as little as possible. Brooks’s subsumption architecture wires perception straight to action with no world model — fast, robust, and situated, but blind to anything that needs foresight. Its modern echo is the single tool-calling turn: a simple reflex agent with a very well-read reflex.
- Deliberative architectures think first and act later. The sense–plan–act pipeline reasons over an explicit symbolic model, buying foresight, goal-directedness, and transparency at the price of the frame problem, combinatorial search, and a hand-authored model that must be complete and correct. This is the hollow agent seen from the architecture side, and the architecture the substrate most directly revives.
- Hybrid architectures layer the two, running a fast reactive layer beneath a slow deliberative one; their genuinely hard problem is arbitration — which layer acts, and when to escalate from one to the other. It is the System 1 / System 2 split cast in software, and the pattern today’s routers and reasoning models are busily rediscovering.
- The Belief–Desire–Intention model is the richest of the four, and the nearest to a modern agent. Its load-bearing idea is intention as persistent commitment — precisely the standing goal that a stateless, goalless substrate cannot hold. The model supplies belief revision and option generation; the harness supplies the intention. Read through the substrate: the model is the faculty of deliberation, and the harness is the will.
- The modern agent loop is a special case, not a novelty. ReAct’s observe–reason–act cycle is sense–plan–act run closed-loop — deliberation that re-perceives after every action, its reasoning core now a genuinely competent one — and the harness is that architecture rendered in code; the genuine layered hybrids are the routers that arbitrate between a reflex and a reasoning loop. Choosing where on the spectrum to sit, and how far to commit before looking again, is the architectural judgement every agent builder makes, whether or not they know its name.
- A loop that cannot stop is not persistent but pathological. Termination — a goal test, a step budget, a ceiling on cost — is a load-bearing part of the architecture rather than a tidying-up detail; the classical architectures could ignore it because their steps were cheap and finite, and a foundation-model loop is neither.
4.8 Exercises
Exercise 1. Place each of the following designs on the two knobs of Section 4.6 — how much it deliberates before each action, and how far it commits before perceiving again — then name the classical family, or the corner of Figure 4.1, that it instantiates, and one failure mode it inherits from that position. (a) A guardrail that reads each outbound message an agent drafts and, in a single model call carrying no memory of earlier calls, passes or blocks it. (b) A release agent that reasons at length to draft a forty-step migration script, then executes all forty steps without a further model call. (c) A support agent that reasons at length before composing each reply, sends it, and does nothing more until the customer answers. (d) A nightly maintenance job that runs a fixed shell script and consults no model at all. (e) One of the four sits in the corner of Figure 4.1 that the classical field left unoccupied: say which, and explain why a design may live there only while its world stands still.
Exercise 2. A team resolves to give its coder agent a hand-authored symbolic model of the repository: one boolean fluent per observable fact — this file compiles, that test passes, the working tree is clean — with F = 60 fluents and A = 25 STRIPS-style operators whose add and delete lists mention exactly four distinct fluents each. In a naive logical axiomatisation of change, every operator requires one frame axiom for each fluent it leaves untouched. (a) How many frame axioms must the team author, and what is the growth law in A and F? (b) Next quarter’s enriched model has F = 200 and A = 80, each operator still touching four fluents; how many now? (c) STRIPS dispenses with the axioms by fiat: any fluent absent from an operator’s delete list is assumed to persist. Exhibit one operator–fluent pair in this repository world for which the assumption is false, and state what the agent would then believe, wrongly, after acting. (d) In one sentence: which of the three costs of deliberation catalogued in Section 4.3 does this arithmetic make concrete, and why does a foundation model lighten it without abolishing it?
Exercise 3. A coder agent’s harness assembles the model’s context afresh each turn: a fixed preamble giving the agent’s role and tools — with no mention of the current task — followed by the most recent w = 6 transcript entries. The transcript opens with a single entry, the user’s instruction “Fix the flaky retry test, and do not touch the public API”, and each completed turn appends exactly two entries: the action the model chose and the observation the tool returned. (a) On which model call is the agent first invoked with no trace of its instruction anywhere in its context? Derive the general formula for a window of w entries. (b) Say, in the chapter’s vocabulary, what the architecture has silently become from that call onward, and predict the fate of the constraint on the public API. (c) Give the minimal repair, stating exactly where the instruction should now live, and name the classical concept of Section 4.5 that your repair implements.
Exercise 4. A coder agent’s edit–test loop costs c = 2{,}000 tokens per iteration, and each iteration makes the failing test pass with probability p = 0.4, independently of every other. The harness stops at the first success or after B iterations, whichever comes first. (a) Compute the probability that the loop succeeds within budget for B = 4. (b) Show that the expected number of iterations consumed is \mathbb{E}[N] = \bigl(1 - (1 - p)^{B}\bigr)/p, and evaluate the expected token spend for B = 4. (c) Find the smallest B for which the loop succeeds with probability at least 0.95. (d) A passing fix is worth V = 10{,}000 tokens of engineer time. Show that, under the model exactly as stated, a rational harness would never stop at all — and identify the assumption whose failure in practice restores the budget’s good sense, taking as your witness the agent of Section 4.6 that retries a broken edit until the budget is exhausted.
Exercise 5. An orchestrator dispatches each incoming issue down one of two paths. The reflex path is a single call to a small model, costing 1 unit; it resolves an easy issue with probability 0.9 and a hard one with probability 0.15. The deliberative path is a full plan-then-execute pass, costing 8 units and resolving either kind with probability 0.95. A fraction 0.25 of issues are hard. The test suite catches every reflex failure at no cost, and a caught failure is escalated to the deliberative path; deliberative failures are handed to a human under every policy alike, and may be ignored in the comparison. (a) Compute the expected cost per issue of three policies: deliberate-always; reflex-first, in which every issue tries the reflex path and failures escalate; and route, in which a classifier call costing 0.2 labels the issue — with accuracy 0.8 on either kind — and labelled-easy issues go reflex-first while labelled-hard issues go straight to deliberation. (b) Show that at these numbers not even a perfect router — accuracy 1, fee waived — beats reflex-first, and isolate the one-line inequality responsible. (c) Now let a quarter of reflex failures slip past the tests, each costing 40 units to repair downstream; recompute reflex-first, route, and the perfect router. Which policy wins? (d) In at most two sentences: when does a router earn its keep, and how does your answer connect to the warning of Section 4.2 about reflexes wired to actions that cannot be taken back?
Exercise 6. A coder agent executes its plan one step at a time, at a cost of one unit per step. Immediately before each step, independently with probability q, the world shifts — a teammate merges a conflicting change, a dependency releases — and the plan, if still valid, becomes invalid; steps executed while the plan is invalid accomplish nothing. The agent reconsiders after every k-th step: reconsideration costs r units, detects any invalidation, and replans, so each block of k steps begins with a valid plan. (a) Show that the expected number of useful steps in a block is \sum_{i=1}^{k} (1 - q)^{i}, and find the sum’s closed form. (b) Justify that the long-run expected cost per useful step is
c(k) \;=\; \frac{k + r}{\sum_{i=1}^{k} (1 - q)^{i}}.
- For r = 2 and q = 0.25, evaluate c(k) for k \in \{1, 2, 3, 6, 12\} and pick the best interval on the menu. (d) Repeat for q = 0.02. (e) Show that c(k) \to \infty as k \to \infty whenever q > 0, and read your results as settings of the commitment dial of Section 4.5 — the interior optimum k^{*} interpolating between the cautious (k = 1) and bold (k \to \infty) agents that Chapter 2’s Exercise 3 could price only at the poles: name the classical failure mode waiting at each end of the menu.
Exercise 7. A repository watchdog receives, once a minute, a percept with two fields: build_red (a boolean) and disk_free_gb (a number). It is to be built as a subsumption stack (Section 4.2) of three behaviours, bottom layer first: idle, always applicable, action sleep; rerun-build, applicable when build_red, action rerun; free-disk, applicable when disk_free_gb is below 5, action prune. A higher layer, when applicable, subsumes every layer beneath it. (a) Implement the stack in standard-library Python: a behaviour pairs an applicability predicate with an action, and the agent acts as its highest applicable layer directs. (b) Run it on the six percepts (build_red, disk_free_gb) = (False, 20), (True, 20), (True, 3), (False, 3), (False, 9), (True, 4), in that order, and record the six actions. (c) Delete the free-disk layer and rerun: confirm the agent still acts on every percept, and name the advertised virtue of the reactive family this exhibits. (d) The team now asks that, after any prune, the watchdog verify on its next tick that pruning actually worked — freed space the disk did not already have, which no single reading of disk_free_gb can attest — before it permits a rerun. Prove that no stack over this percept can comply — exhibit two situations that demand different actions yet are indistinguishable to the architecture — then name the minimal thing that must be added, and the architectural family the addition quietly leaves behind.
Exercise 8. DocBot handles every issue labelled docs in the team’s tracker with one pipeline: a flagship-model call drafts a plan (nine steps on average), a second call critiques the plan, an execution loop then makes one edit per step with a model call each and replans after every step with another call each, and a final call reflects on the outcome — 21 model calls and roughly 45,000 tokens per issue. The issue history shows that 90 per cent of docs issues are single-file wording or typo fixes. Continuous integration runs the documentation build and link checker on every pull request, and nothing DocBot produces merges without a green check and a human approval. (a) Place DocBot on the two knobs of Section 4.6, and say which classical family its designers have, perhaps unknowingly, committed to. (b) Dismantle it: propose the cheapest architecture that still serves the remaining 10 per cent, state the escalation trigger you would rely on, and justify each part you remove by naming the external safeguard that makes its removal safe. (c) Your reflex path costs one call and 2,000 tokens, succeeds on 90 per cent of the single-file issues, and escalates its failures — caught by the CI build — to the unchanged full pipeline; multi-file issues go straight there, and the tracker’s labels make the trigger free. Compute the expected calls and tokens per issue under the redesign, and the saving over DocBot. (d) Which single safeguard in the scenario, were it removed, would forbid your dismantling, and why?
Exercise 9 (lab). Choose a model family that ships a flagship and a small fast sibling (Section 4.4). Assemble forty short tasks in two strata you can score automatically: twenty easy and twenty hard instances of one kind of problem — single-step against multi-step code fixes, say, or one-hop against multi-hop questions over a repository you know well (Appendix C describes the setup for labs of this kind). (a) Measure each tier’s success rate and cost per task on each stratum. (b) Feed the measurements into the analysis of Exercise 5 — your own hard fraction, success rates, and costs — and determine which of its three policies your numbers favour, under both the reliable-detection and the leaky-detection assumptions. (c) Report which stratum shows the wider gap between the tiers, and test the chapter’s claim that the model menu is System 1 and System 2 ready-made: near parity on the easy stratum, a widening gap with depth. Record the model identifier and the date beside your results.