Exercise Bank — Implementing a Multi-Agent System from Scratch

The printed Chapter 17 keeps Exercises 1–4; the bank continues from Exercise 5.

Exercise 5. Section 17.5 printed the successful first flight’s journal in abridged form: the ModelCalled and BudgetDebited lines were dropped for the page, and the run had begun with an allowance of 5,000 tokens. (a) From the abridged trace alone, reconstruct what fold from foundations/journal.py returns for the full journal — total spent, who finished, how many messages — and explain why dropping every BudgetDebited line cost the reconstruction nothing. (b) The reducer has no per-agent resolution. Write a bill view — tokens per agent, from the same events — re-stage the flight against the companion runtime, and report each agent’s bill, the fraction of the team’s spend that is the coder’s thread, what remains of the allowance, and which single event kind of Table 17.1 your view consumed. (c) Each model call appends ModelResponded carrying usage and, on the next line, BudgetDebited carrying an equal amount. Is the second event dead weight? Make the strongest case for keeping it, and explain why no two views of one journal — bill, conversation thread, audit — can ever disagree, however many are added later. (d) Section 17.3 calls resumption a re-read; make it one. Write resume_point(evts), returning the stage the flight stands at — fix, review, verify, or complete — and verify it on two truncations of the journal: cut immediately after the coder’s DONE is sent, and immediately after the reviewer’s ACCEPT is sent.

Exercise 6. Crash four of Section 17.5 cost the coder three model calls — 70 tokens to run the tests into the flaky runner, 70 more to try again after reading the returned error, 40 to finish — because the loop’s stated policy hands a transient tool error back to the model and lets it decide to retry. Extend run in foundations/agent.py (in your working copy; the repository stays as printed). (a) Add a retries parameter: when an observation carries an error key, re-dispatch the same call up to retries further times before the model sees anything, journalling every ToolReturned. (b) Re-run crash four with retries=1 and account for the difference — model calls, tokens, and the saving as an exact fraction — then say what the loop-level retry spends that the budget cannot see, and why the flake nonetheless remains visible in the journal. (c) Add graceful interruption: an operator’s stop signal checked at the turn boundary beside the budget, a new journal event kind, and a status that resumes cleanly — Section 17.1 excluded a human-approval interface, but an off switch is not an interface. (d) Measure the work: report lines added to the module and to the 43-line run for each extension, reconcile the count with the Summary’s claim that such upgrades “add lines rather than architecture”, and name the class of tools for which the automatic retry of (a) is unsafe, together with the cheapest guard.

Exercise 7. Section 17.4 claims a second agent is the same construction as the first and that constructing it is one function call; price the claim at five. Add a docs agent that records every shipped change in docs/changelog.md, without editing a single line under foundations/. (a) Write the harness extension: once run_team ships, a REQUEST on a fresh task thread carries the shipped result to the docs Worker; its run is journalled like everyone else’s; its report returns as DONE or ERROR. (b) Give the new agent a jurisdiction rather than a job title, from existing parts alone: it sees only what its REQUEST payload carries (the diff and summary, not the orchestration history); it may write only under docs/ — enforce that in the toolset, and demonstrate the enforcement by scripting one straying write, which must come back as an error observation, not an exception; and rule on its standing — should shipping gate on documentation the way integration gates on ACCEPT? — defending the ruling in a sentence. (c) Fly it scripted and audit the run: the fold’s finished list, the message count, the per-agent bill of Exercise 5, and the number of changed lines under foundations/ — then name the design decision of Section 17.1 that made that last number zero.

Exercise 8. The coordinating harness is the Contract Net minus the competitive bidding round, and Section 17.4 marks the bidding as an exercise (Chapter 12 explained what it buys and what it costs); this is that exercise. (a) Write announce_and_award(task, bidders, journal_path, budget) from existing parts alone: a REQUEST announcing the task to each bidding coder; one cheap model call per bidder returning its estimate as a done payload with a bid field, where a missing bid declines; a PROPOSE per bid to the orchestrator; an ACCEPT to the lowest bidder and REJECTs to the rest; the winner set to work on the task proper, reporting DONE. (b) Fly it with two scripted coders — coder-a bids 900 in a 25-token call, coder-b bids 700 in a 30-token call, and the winner’s working thread costs 275 tokens — then read the journal back: write out the full performative thread in order, price the auction from the bill, and explain why the six auction messages cost nothing although the auction as a whole does: what is the true currency of an announce–bid round in this runtime? (c) Count the foundations/ modules your function edited, note what it means that PROPOSE has sat unused in the enum since Section 17.4 introduced it, and say which sentence of Section 17.1’s exclusions paragraph this exercise just tested — and whether it survived.

Exercise 9. Section 17.5’s wish list includes a reflection loop — the coder critiques its own diff before the reviewer sees it — as a pattern Chapter 18 names and the runtime can carry in an afternoon; build it, and decide when it earns its tokens. Let a review pass cost u_v = 200 tokens, a rework of a rejected diff cost w = 150, and a self-critique call cost u_r = 30; a fraction p = 0.4 of the coder’s diffs are defective, the reviewer rejects every defective diff it sees, and the critique catches a fraction f = 0.8 of defects before submission. (a) Derive the expected extra cost per task beyond the happy path, without reflection and with it, and reduce the choice to a single inequality — being careful to notice which cost reflection does not avoid, and why. (b) Evaluate both expectations, the saving, the break-even critique price u_r^{*}, and the break-even defect rate p^{*}, exactly; then confirm the two expectations by simulation. (c) Implement ship_with_reflection as a harness step — critique, at most one revision on a “revise” verdict, then review — and validate it with scripts: the reviewer must be called exactly once and must never see the defective diff. (d) The model flatters the pattern; name the two omissions that matter most — one about who the critic is, one about what the reviewer misses — and give the direction of each.

Exercise 10 (lab). The design brief promised the vendor swap in one file, and the claim has so far been tested only against a fake; test it against a vendor (the companion repository’s setup guide describes the set-up; keys live in a git-ignored .env). Write a LiveClient(ModelClient) for a provider you hold keys for: translate the runtime’s message dicts and Tool schemas into the vendor’s shapes and back; return ModelResponse with text, tool calls, and a non-zero usage on every call; and put retry-on-rate-limit inside the client, explaining in a sentence why that retry lives here while Exercise 6’s lives in the loop. Stage a real flight: a scratch repository containing a parse function that fails the flight’s empty-input test; real read_file, edit_file, and run_tests tools over that directory, the last shelling out to pytest; the flight’s three roles each on a LiveClient; an allowance of 30,000 tokens. Fly it at least five times and report three things: (i) the seam audit — every file you created or changed, a grep for the vendor’s name under foundations/ that must come back empty, and the companion test suite green with no edits; (ii) per flight, the outcome and the per-agent bill of Exercise 5; (iii) at least one behaviour the scripts of Section 17.5 never exercised — a plain-prose turn and the nudge, a budget scare, an inconclusive. Record the model identifier and the date beside the table: the durable finding is that the seam held and the tests never noticed, not any token figure or success rate, which are perishable.