2 The Language Model as a Cognitive Substrate
The previous chapter closed with the missing piece arriving; this chapter looks at it more closely, and more sceptically. The title gives the conclusion away. A substrate is something other things are built upon — the silicon beneath a circuit, the soil beneath a forest: necessary, enabling, and emphatically not the thing itself. A foundation model is a cognitive substrate in precisely this sense: an extraordinary, general-purpose capacity for producing fluent, knowledgeable, contextually apt text, upon which one can build something that reasons, remembers, acts, and pursues goals — but which, left to itself, does none of those things. The chapter takes inventory of the gifts and the gaps, and almost every property the model lacks turns out to be the subject of a later part of this book: a model that came with memory, goals, grounding, reliability, and a sense of other minds would leave the classical theory very little to do, and it has none of these dependably. The model does not make the old problems go away; it makes them, for the first time, worth solving.
This chapter is deliberately not a tutorial on transformers or the mechanics of training — the engineering of multi-agent systems depends remarkably little on them — and it is the most sceptical chapter in Part I, intentionally so: enthusiasm about what these models can do is not in short supply, and a clear-eyed account of what they cannot is the safer foundation on which to build.
2.1 The Substrate, Characterised
Before saying what the substrate provides and withholds, we need a working picture of what it is — the lean one a person building agents from it can use. At this altitude the foundation model is almost embarrassingly simple to describe, genuinely mysterious to explain, and remarkably easy to mischaracterise; we take the three in order.
2.1.1 A Stateless Function from Context to Continuation
Strip away the marketing and the model is a function. It takes a sequence of text — the context — and returns a probability distribution over which token should come next. Writing x_{1:t} = (x_1, \dots, x_t) for the context, the model with parameters \theta is the conditional distribution p_\theta(x_{t+1} \mid x_{1:t}); one step of generation samples a token from it and appends,
x_{t+1} \;\sim\; p_\theta(\cdot \mid x_{1:t}),
and repeating gives you a system that continues text. At the operational level, that is the whole of it. The dominant implementation is the transformer (Vaswani et al., 2017), but for our purposes the architecture is a detail: a system built on some yet-to-be-invented successor would raise the same coordination problems, and this book is written to outlast the answer.
Two features matter for everything that follows. The model is stateless: each call begins from nothing but the context it is handed — it does not remember the previous call, does not know that a conversation is occurring, and carries no thread of identity from one invocation to the next. And the function is probabilistic: it does not compute the next token so much as sample one, and the same context can yield different continuations on different calls. A great deal of the engineering in later chapters consists of coaxing reliable behaviour out of a component that is, by construction, neither persistent nor deterministic.
2.1.2 The Surprise of Scale
Described that baldly, the foundation model sounds like a glorified autocomplete, and for most of the history of language modelling that is essentially what it was. The surprise — it earns the word, because almost no one foresaw its extent — is what happens when this modest objective is pursued at enormous scale: trained on a sufficiently large fraction of the written record, the system acquires, as a by-product, a startling range of competences it was never explicitly taught — translation, code, rudimentary reasoning — because predicting the next token well enough turns out to require them.
This is a measured regularity, not folklore. Performance improves smoothly as a power law in model size, data, and compute — the scaling laws (Kaplan et al., 2020), whose first-published exponents were substantially recalibrated within two years (Hoffmann et al., 2022) — and on the strength of these curves the field placed the expensive bet that produced models such as GPT-3 (Brown et al., 2020). More contentiously, certain abilities appear to switch on abruptly past a threshold of scale: the claim of emergent abilities (Wei, Tay, et al., 2022), to which a pointed rejoinder answers that much of the apparent emergence is an artefact of discontinuous measurement (Schaeffer et al., 2023). We need not adjudicate; what we should register is that the central empirical fact of the field remains only partly understood by the very people producing it. A substrate whose powers are not fully explained even by its makers is one to build upon with a certain humility.
2.1.3 “Cognitive Substrate”, Defined
The term foundation model was coined to capture the way these systems serve as a common base — trained once, at great expense, then adapted to countless downstream uses (Bommasani et al., 2021). The word is well chosen, for a foundation is something you build on: not itself a building. We will speak throughout of the model as a cognitive substrate — the raw material of cognition rather than cognition itself — and the analogy is to the processor beneath a program. A CPU computes anything computable, yet has no purpose, no memory beyond a handful of registers, and does nothing until a program organises it towards an end. The foundation model stands in the same relation to an agent: it supplies a general capacity for producing apt text, and the agent — through memory, tools, a control loop, and a goal — organises that capacity towards actually doing something. To mistake the substrate for the system — to point at the processor and call it the program, or at the model and call it the agent — is the error every later chapter is, one way or another, written to prevent.
2.2 What the Substrate Provides
What does the substrate actually give you? The list is short, but each gift is large, genuinely new, and exactly what the hollow agent of Section 1.7 could never be handed. The gifts appear here in their best light; the qualifications follow.
2.2.1 A Compressed Model of Human Knowledge
The first gift is knowledge. In learning to predict text drawn from a substantial fraction of the written record, the model comes to encode an enormous amount of what that record contains, from the shape of a sonnet to the syntax of Python. It is illuminating to view this as compression: a predictor that assigns probabilities well is, by the source-coding theorem, an efficient compressor of what it predicts — and a system that compresses the written record this well must have captured a great deal of its regularity to do so (Delétang et al., 2024). The model is, in effect, a lossy, queryable compression of human knowledge — interrogable for facts as though it were a knowledge base (Petroni et al., 2019), with no database, no schema, and no query language — part of what makes it feel, on first contact, like magic. Two words in that description are load-bearing, and Section 2.3 will lean on them hard: lossy and compression. A system that reconstructs plausible-looking facts from a compressed store will, on occasion, reconstruct plausible-looking facts that were never true. For the moment, though, the gift is real: competence on tap, across an extraordinary breadth of subject matter.
2.2.2 Language as a Universal Interface
The second gift is easy to overlook precisely because it is everywhere: the substrate speaks. Fluent command of natural language sounds like table stakes for a language model, but text is the nearest thing computing has to a universal interface: it is how the model takes instruction, returns results, invokes a tool, addresses a human and — decisively for this book — how one agent addresses another. The insight that an enormous range of tasks can be cast uniformly as text-in, text-out predates the current models (Raffel et al., 2020); the foundation model is what made the casting pay — the quiet structural fact that makes multi-agent systems buildable at all. If every agent, tool, and human shares a common medium, composing them becomes a matter of routing text to the right place at the right time — hard, but tractably hard, and much of what Part III is about. The classical dream of agent communication languages (Section 1.7) was a shared tongue for autonomous software; the foundation model supplies one, and it happens to be English (or Mandarin, or Python). What is gained and lost by pressing a natural language into that role is the business of Chapter 5.
2.2.3 Programmable by Description
The third gift is that the substrate can be programmed by description. Conventional software does what it is coded to do; a foundation model can be steered by a statement of what you want, in the same natural language it reads and writes. Two mechanisms deserve naming. The first is in-context learning: shown a handful of examples in its context, the model will often infer the pattern and continue it, with no change to its weights — programming by example, at inference time (Brown et al., 2020). The second is instruction-following: a model post-trained to do as it is told will carry out a task set in plain prose — an ability deliberately instilled, most influentially by training against human preferences (Ouyang et al., 2022), for the raw base model that falls out of pretraining merely continues text. The reader who recalls Section 1.7 will feel a flicker of recognition: Shoham proposed programming machines by declaring their mental states and commitments rather than their procedures. The system prompt is that declaration, and the foundation model is the machine that, at last, acts upon it — agent-oriented programming’s wish granted three decades late, with the catch, taken up in Section 2.3, that a substrate steered by description is also a substrate mis-steered by description, exquisitely sensitive to how the description is phrased.
2.2.4 Reasoning, of a Kind
The fourth gift is the one that earns the model its cognitive adjective, and the one to handle with the most care: the substrate can reason — after a fashion. Asked to work towards an answer step by step, models of sufficient scale produce intermediate reasoning that markedly improves their performance on arithmetic and logical problems; the technique, chain-of-thought prompting, is as simple as inviting the model to think aloud (Wei, Wang, et al., 2022). The current generation builds the coaxing in: reasoning models, post-trained to produce and check long private working before committing to an answer, turn deliberation into a dial — the caller chooses how much inference-time computation a question deserves (2025) — though the dial changes how much working the model does, not what kind of thing the working is. How much of this is reasoning and how much a convincing performance of reasoning is a genuine and unresolved question, taken up with the sceptical apparatus it deserves in Section 3.7. For this chapter the modest claim suffices: the substrate provides a usable, if unreliable, capacity to break a problem down and make progress on the parts — the capacity that lets an agent built upon the model deliberate rather than merely respond. That is the deliberative core the hollow agent never had. What it conspicuously still lacks is the subject of the next section.
2.3 What the Substrate Withholds
Now the bill. The gifts are real, but each arrives wrapped around absences so consequential that an agent is, in the end, mostly the machinery built to compensate for them. We take the four most important in turn; one cannot build reliably on a foundation whose weak points one has not located.
2.3.1 No Memory, No State
Begin with the absence we have already met. Because the model is stateless, it has no memory of its own: every apparent thread of continuity — a conversation’s history, a persona, a goal carried across time — is sustained from outside it, by gathering the relevant past into text and feeding it back in as context. This is not a temporary limitation awaiting a larger model; it is what the substrate is. The coder agent of our running example that has just written a function has, by its next call, no recollection of having done so: the continuity is manufactured by the harness, never brought by the model.
The obvious patch — simply put everything in the context — runs into two hard walls. The context is finite, nowhere near large enough for an agent’s full history; and, more insidiously, the model does not use what it is given equally well. Information placed in the middle of a long context is attended to markedly less reliably than information at the beginning or end: a fact can be present in the prompt and yet, for practical purposes, be missed (Liu et al., 2024). The context window is not a memory but a small, lossy, curiously uneven working buffer; what to retain, recall, and forget is a problem the substrate hands the agent designer entirely unsolved. The deliberate assembly of each call’s context has acquired its own name, context engineering; it, together with memory, is the subject of Section 4.8.
A fair objection: the platform through which you meet the substrate recalls the conversation perfectly well. So it does; but the statefulness is the platform’s, not the model’s. Recent model APIs keep the running transcript server-side and let you continue it by reference rather than resending it — OpenAI’s Responses API began the fashion, and a vendor-neutral Open Responses specification (2026) has since begun to standardise the shape across providers — a genuine convenience that changes nothing underneath: the model is still handed a context and still begins from nothing but it; only the apparatus that remembers has moved behind the API.1 Statelessness, in short, is a property of the substrate, not of the interface you happen to meet it through.
2.3.2 No Goals, No Will to Act
The second absence is more easily missed: the substrate has no goals, and no means of pursuing them. It does not want anything. Handed a context, it returns a continuation and falls silent; it does not act, does not persist, does not check whether the world has changed, and does not come back tomorrow. There is no loop, and nothing that would turn one. This is the precise sense in which, as Section 1.2 insisted, the model is not an agent. Agency is a property of the system wrapped around the model: the control loop that calls it repeatedly, the tools through which its words become deeds, the state that lets it pursue something across many calls. The substrate supplies the capacity to judge what a competent response would be; the agent supplies the standing intention to keep producing them until a goal is met — the business of Chapter 3 and Chapter 4. The model is the engine, not the driver, and a foundation model left to idle does precisely nothing, very fluently.
2.3.3 A Closed World of Text
The third absence is that the substrate lives in a closed world of text — where text should be heard broadly, as the trace reality leaves, since current models read images and audio too, and the closure survives every widening of the letterbox. The substrate’s entire acquaintance with reality is that trace: it knows the word stove, and ten thousand sentences in which stoves appear — shown a photograph, it may nowadays even recognise one — but it has never been burned, and it cannot, unaided, turn one on. This is the symbol grounding problem in modern dress — the old worry that symbols manipulated only in relation to other symbols are never quite fastened to the things they are about (Harnad, 1990); the contemporary form is pointed — a system trained only on linguistic form has, in principle, no direct route to meaning (Bender & Koller, 2020) — and a diet of pixels widens the record without changing its nature. What the substrate lacks is not another medium but a loop: acting on a world that pushes back, on the world’s schedule rather than the training set’s. However absolute the limit, the practical consequence stands: without tools, the model can neither perceive the world nor change it, so connecting it to reality is not an optional enhancement but the very thing that turns a generator of plausible sentences into something that can get anything done. And even once connected, the gap between a symbol and its referent does not close; it merely becomes a hazard to be managed. This is the subject of Chapter 4.
2.3.4 Capable but Not Trustworthy
The fourth absence is the likeliest to do you harm, because it hides behind the substrate’s greatest strength. Fluency reads, to a human, as competence and confidence; but fluency is neither accuracy nor honesty, and a system that generates statistically plausible text will, when the plausible and the true diverge, cheerfully generate the plausible. This is hallucination: confident, well-formed, wholly fabricated content — not a bug awaiting a patch but a direct consequence of what the substrate does (Ji et al., 2023). The model does not know that it does not know; its expressions of certainty are themselves merely more generated text, tuned to sound right rather than to be right. The coder agent that reports “tests pass” without having run them is the canonical instance, of exactly the kind the rest of this book is built to catch. The unreliability has further faces: the same request, rephrased or merely reformatted, can yield materially different answers (Sclar et al., 2024); the model’s knowledge is frozen at the moment its training stopped, leaving it confidently out of date about anything since; and it remains, at bottom, a stochastic parrot in the precise sense its critics intended — extraordinary facility with the form of language, no guaranteed purchase on its content (Bender et al., 2021). None of this makes the substrate useless; it makes it a component whose outputs must be checked, bounded, and corroborated rather than trusted — reliability is something the surrounding system manufactures, a thread that runs through Chapter 20 and much of the book besides.
2.4 Why the Classical Theory Becomes Newly Relevant
We can now collect the chapter’s two halves into the claim the preface promised. Set the gifts beside the gaps and a striking thing comes into view: the shape of what the foundation model leaves undone is, almost line for line, the shape of the classical theory of agents and multi-agent systems. The arrival of the substrate does not make that theory redundant. It makes it, for the first time, applicable.
2.4.1 The Hollow Agent, Filled
Recall the hollow agent of Section 1.7: classical architectures for deliberation, communication, and commitment, with no general competence to put inside them. The foundation model is precisely the missing filling (Figure 2.1): drop a general, promptable competence into the cavity the classical architectures were shaped to hold, and structures that had been elegant but inert — agent-oriented programming, the belief–desire–intention loop — become, all at once, things one can actually run. This is Section 1.7’s verdict — premature, not wrong — arrived at from the other side: the substrate does not replace the classical body of work; it completes it.
2.4.2 Every Absence Is a Chapter
Now look again at Section 2.3, but as a reading list: no memory — the matter of Section 4.8; no goals and no loop — the matter of Part II; no grounding — the matter of Chapter 4; no reliability — the matter of Chapter 20 and Part VII. Each thing the model withholds names a chapter that supplies it; the book’s table of contents is, in large part, a catalogue of the substrate’s omissions.
One omission towers over the rest — the one this whole book exists for. The substrate knows nothing of other agents: no inkling that the text it is shaping might be addressed to, contested by, or dependent upon another mind; no notion of coordination, negotiation, commitment, or collective choice. These are exactly the concerns of the classical multi-agent literature — the canon preserved, against precisely this day, by Wooldridge (2009) and by Shoham and Leyton-Brown (2009). The foundation model fills the single agent; the moment you have two, everything the substrate cannot supply about their interaction is waiting, already worked out, in thirty years of theory. That is the wager of this book, and the substrate is what finally makes it collectable.
2.4.3 Substrate, Not Solution
A closing caution, and the handoff to what follows. The temptation is to treat the model as the system rather than the substrate — to assume that a sufficiently good model, suitably prompted, simply is an agent, and that several of them in a group chat simply are a multi-agent system. This is the error the whole chapter has been arming you against. The model is a layer: the agency, the memory, the grounding, the reliability and — above all — the coordination are not in it but built on it. An agent is a foundation model plus the apparatus that makes it act — a loop, tools, memory, a goal (Weng, 2023) — what the practitioner literature has taken to calling the harness,2 in a framing — Agent = Model + Harness — that is becoming common shorthand. A multi-agent system is that, several times over, plus everything that happens between; and before wiring several substrates into a group chat, the question this book keeps returning to — and asks of the software-engineering team most of all — is whether you needed more than one substrate at all.
Forget the distinction and you build the most modern version of the oldest mistake in the field: a hollow agent wearing a foundation model — dazzling in a demonstration, baffling in production, and hollow in exactly the place the classical theory was written to fill. Remember it, and you hold the right picture for everything that follows: a remarkable substrate, a great deal of careful engineering, and older-than-it-looks theory, stacked on top. With the raw material understood, Part II turns to the first thing we build from it — a single agent.
2.5 Summary
- The foundation model is a substrate, not an agent. It is the raw material of cognition — enabling and necessary, but not the finished mind — and confusing the two is the field’s costliest single error.
- At the book’s level of abstraction it is a stateless function from a context of text to a continuation, trained by prediction at a scale that produced an unexpected, and still only partly understood, general competence.
- What it provides is real and new: a compressed, queryable model of human knowledge; fluent language as a universal interface; programmability by description rather than by code; and a genuine, if limited, capacity to reason.
- What it withholds is just as consequential: no memory between calls, no goals or loop of its own, no grounding in or means of acting on the world, and no reliability — fluent by construction, truthful by none.
- The bridge has two halves: the provision fills the hollow agent’s missing core, and every absence is a problem the classical theory already studied — which is why a thirty-year-old body of theory becomes newly applicable, rather than newly redundant, the moment the substrate arrives.
- The substrate is a layer, not a solution. The agency, the reliability, and the coordination must all be built above it — which is the work of the rest of this book.
2.6 Exercises
Exercise 1. A reviewer agent’s model, handed a context x_{1:t} containing a diff, emits a two-token verdict by the sample-and-append loop of Section 2.1: a judgement drawn from “approve” or “revise” with probabilities 0.7 and 0.3, then a severity code — after “approve”, the code is “clean” with probability 0.8 or “minor” with probability 0.2; after “revise”, it is “minor” or “major” with probability 0.5 each. (a) Compute the probability of each of the four complete verdicts. (b) The orchestrator calls the model twice, independently, on the identical context. What is the probability that the two verdicts agree exactly? (c) The orchestrator treats disagreement between the two calls as evidence that the diff is contentious, and escalates it to a human. What is the probability of an escalation on a perfectly ordinary diff? If the harness switches to greedy decoding — always take the most probable token — the two calls agree with certainty; what information, present under sampling, has the escalation signal now lost?
Exercise 2. Statelessness is paid for in tokens. A coder agent’s session runs to k = 40 model calls. On every call the harness sends a system prompt of s = 1500 tokens plus the entire transcript so far; each turn adds u = 300 fresh tokens of instructions and tool results before the call, and the model replies with r = 200 tokens. (a) Write down the number of input tokens the model processes on call t, and derive a closed form for the total input processed over the session. (b) Evaluate that total, and compare it with the number of distinct tokens the session ever produces — the system prompt, every instruction, every reply. (c) The team moves to a provider that keeps the transcript server-side, in the style noted in Section 2.3, so that the harness transmits only each turn’s fresh material. Which of your quantities changes, which does not, and what exactly has moved? (d) The model’s context window holds 20,000 tokens. On which call does the assembled context first exceed it, and what is the harness forced to begin doing from then on — the business of Section 4.8?
Exercise 3. A tester agent’s context is assembled from seven documents, and the probability that the model actually uses a fact depends on where the fact sits: for positions 1 through 7, take the probabilities of use to be 0.95, 0.85, 0.70, 0.55, 0.70, 0.85, 0.95, independently across facts. (a) An oblivious assembler places the one fact the task needs uniformly at random; with what probability is it used? (b) A task needs two facts, both essential. Compare an assembler that lands them in positions 3 and 4 with one that deliberately places them at positions 1 and 7: compute both success probabilities and their ratio. (c) The context must carry one critical instruction — “never push to main” — and six reference documents, two of which you expect to matter far more than the rest. Assign the seven slots, justify the assignment with the numbers, and state in one sentence what your answer implies about the advice to just put everything in the context.
Exercise 4. Under an ideal code, a token to which the model assigns probability p costs -\log_2 p bits. (a) Model A assigns the six tokens of a status report the probabilities 1/2, 1/4, 1/8, 1/2, 1/16, 1/4; model B assigns the same six tokens 1/4, 1/8, 1/8, 1/4, 1/32, 1/8. Compute each model’s cost in bits for the whole report, and explain why A predicts better and A compresses better are the same claim. (b) A harness compacts the tester agent’s report — pass or fail for each of 20 test files, one bit each — into a 12-bit memory record, from which a later call must reconstruct the full status vector. Show that at least two distinct reports must share a record, and conclude that no reconstruction scheme can be right about every test for both. (c) Show further that, whatever the scheme, some report comes back with at least three of its twenty statuses wrong. (Count the reports that must share a single record, and compare with the number of status vectors lying within Hamming distance two of one fixed reconstruction.) (d) Every reconstruction is nonetheless a syntactically perfect twenty-status report. Use (b) and (c) to defend, in two or three sentences, the chapter’s claim that hallucination is a direct consequence of what the substrate does rather than a bug awaiting a patch.
Exercise 5. A vendor’s one-page pitch for Athena, “an autonomous staff engineer”, makes five claims: (1) “Athena remembers every design decision your team has ever made, across sessions and staff changes.” (2) “Athena watches your CI overnight and opens a fix before you wake.” (3) “Athena always knows the current release of every library you depend on.” (4) “Athena’s reviews can be trusted: it explains its reasoning step by step before every verdict.” (5) “Athena has confirmed the patch: all tests pass.” For each claim, identify (i) which withheld property of Section 2.3 the claim silently assumes away, (ii) the machinery a harness would have to supply for the claim to be honest, and (iii) the cheapest experiment a sceptical buyer could run against the live product to test it.
Further exercises for this chapter continue in the web edition’s exercise bank.
The serving stack does keep state beneath the semantics: the attention keys and values already computed for a context’s prefix are cached — the KV cache — so that calls sharing a prefix can share the work; providers now price cached tokens separately. It is an inference-time economy, not a memory: it changes what a call costs and how quickly it returns, never what it returns.↩︎
The term was popularised by Anthropic’s engineering literature; see Effective Harnesses for Long-Running Agents, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents. It has since acquired an engineering literature of its own: Weng (2026) surveys harness design patterns and the emerging practice of treating the harness itself as the thing one optimises, a development Chapter 14 weighs.↩︎